Identifying Risk Factors for S-Risks

Original text Luciano Carlos Cunha · Volume IX, p. 58 of the PDF edition · English translation

8.1. What risk factors are and why it is important to identify them

One of the most common objections to concern for the long-term future⁷⁰ is the difficulty of estimating the long-term consequences of each decision. For example, it is hard to assess how a given intervention would affect s-risks (that is, risks of suffering of astronomical magnitude in the future⁷¹), since there are countless possible s-risks and the future is highly uncertain. One possible way out of this difficulty is to try to investigate risk factors⁷², which are not themselves s-risks but greatly increase their likelihood or severity.

The concept of a risk factor is already widely used in medicine. For example, smoking, lack of exercise, and poor diet are not health problems in themselves, but they can lead to many diseases. To know that it is good to prevent these factors, we do not need to analyze specific diseases or know what each person's health trajectory will be. Analogously, if we discover what the risk factors for s-risks are, we do not need to know what the future will be like to know that it is good to prevent them.

As with risk factors in medicine, this does not mean that risk factors for s-risks would necessarily cause one or more s-risks to materialize. It means only that they would greatly increase their likelihood. Thus, if we identify such risk factors, we will be able to plan effective interventions to reduce a wide range of s-risks, without needing to know what the future will be like.

Baumann (2002.p. 49-56) suggests several examples of risk factors for s-risks. In the following sections we will discuss each of them separately, and in section 8.9 we will discuss the interaction among these various risk factors.

8.2. Advanced technology

Many s-risks are possible only because of new technologies. However, every technological

advance is worrying to a greater or lesser degree. Among the most worrying are technologies that could make it very easy to create large amounts of suffering. Here are two examples.

• The ability to create digital sentient beings⁷³ could result in enormous amounts of suffering because of the disregard for nonbiological sentient beings, something we addressed in section 1.8.3.
• Space colonization could vastly multiply the population of sentient beings, who would be subject both to the decisions of agents and to natural processes⁷⁴. There are between 100 and 400 billion stars in our galaxy, and between 100 and 200 billion galaxies in the universe⁷⁵. Without space colonization, suffering would be limited to that on Earth (which is already enormous, but tiny compared with what could arise).

8.3. Aversion to technology

Technology also offers opportunities to reduce s-risks. For example, if technological progress were halted, this would prevent the emergence of s-risks caused by humans, but it would not prevent natural s-risks (on Earth or beyond it) or those caused by other civilizations, should these exist (and it would also mean that there would be no technology to prevent these other s-risks).

Technology is a tool. As such, it will be used to achieve the ends of those who use it. Thus, whether a technology will increase or reduce suffering depends on the goals of those who will make use of it. For example, space colonization could multiply s-risks beyond Earth, but it could also make it possible to reduce s-risks, if there are s-risks beyond Earth.

Of course, it could be said that, with respect to certain technologies, such as space colonization, the risk they pose of causing s-risks is so great that it might still be worth never using them, even if this means failing to prevent suffering

as well. However, even if it had been shown that, because of this risk, it is better never to develop certain technologies, this would not support the claim that it is better to halt technological progress as a whole (since it would not apply to other technologies), much less to abandon already existing technology.

Moreover, we must take into account that much of the aversion to the use of technology stems from an unrealistically positive view of nature. For example, the prevailing belief is that, if humans stop harming wild animals, these animals will usually have at least minimally positive lives as a result of natural processes. This view is the opposite of reality. Natural processes tend to maximize the number of animals who are born to experience mostly suffering and to die prematurely. However, the romanticized view of nature is widespread, and it is one of the beliefs that contributes most to the mistaken idea that, by abandoning technology, suffering will decrease.

8.4. Lack of consideration for nonhuman sentient beings

Most s-risks, as well as the little attention given to preventing s-risks, exist only because the level of moral consideration given to nonhuman sentient beings is low. We saw in section 1.6.1 that a lack of consideration for the individuals affected by decisions is enough for there to be an s-risk (agents need not aim at suffering for its own sake). The overwhelming majority of the potential victims of s-risks are nonhuman (whether nonhuman animals or possible future nonorganic sentient beings). Thus, the lack of consideration for nonhuman sentient beings is one of the main risk factors for s-risks. Moreover, besides being a risk factor itself, this lack of consideration also gives rise to another risk factor: the lack of efforts to reduce s-risks, which we will address next.

8.5. Lack of efforts to reduce s-risks.

Progress in preventing s-risks is possible only if enough people care about this question. For example, future generations may come to have technology that could be used to prevent or minimize s-risks, but choose not to do so because they lack sufficient moral consideration for all sentient beings.

This does not mean that, for there to be efforts to reduce s-risks, there must necessarily be a very large number of people focusing on this. Even if few people focus on preventing s-risks, they can seek compromises with people who have other focuses (but who are not opposed to preventing s-risks) in order to implement low-cost measures to prevent them. Moreover, one can focus on trying to get people who have more resources, power, or influence to want to prevent s-risks. The positive impact of these people may be much greater than the combined positive impact of many people who have few resources, little power, or little influence.

8.6. Misguided or ineffective efforts to reduce s-risks

Wanting to reduce s-risks is not enough. Efforts need to be well planned so that they are not of little effect, ineffective, or even counterproductive.

Some risks related to these possibilities are:

• If the cause of reducing s-risks becomes associated with highly controversial ideas, this may generate strong opposition to the proposal to prevent s-risks.
• The relevant decision-makers may want to prevent s-risks, but political institutions may be ineffective or there may be cooperation problems⁷⁶.
• It may also be that decision-makers fail to notice certain s-risks in their early stages and, by the time they do, it is already too late.

The question of efforts that are of little effect, ineffective, or counterproductive will be addressed in detail in Chapter 13.

8.7. Polarization, conflict, and hostility

S-risks are more likely to occur if there is a high degree of hostility among future agents. Large-scale polarization leaves little room for reflection (about s-risks or anything else) and for mutual compromise. It also has the potential to result in conflicts and even wars, which increases the chances of agential s-risks.

Polarization, and how to reduce it, will be the subject of Chapter 11.

8.8. Lack of security against agents who aim to cause suffering

Another risk factor is insufficient security against agents whose goal is to cause suffering. At present, some of these agents are restrained to some extent by moral norms and laws, but there are possible future scenarios in which such forms of restraint could be ineffective or insufficient (for example, in a future with powerful, autonomous AIs and/or with space colonization).

Individuals with personality traits such as psychopathy, sadism, narcissism, and Machiavellianism increase the probability that s-risks will materialize when they attain positions of power⁷⁷. And it is not uncommon for them to be in such positions, since they usually have a strong desire for power and their personality gives them many advantages in that pursuit (and this desire and advantage are greater the more of these traits are found in the same individual). These risks are increased if such malevolent agents have access to advanced technology (which illustrates that the more risk factors are combined, the greater the chances of an s-risk materializing, a point we will discuss in the next section).

8.9. Interaction among various risk factors and among various ways of preventing them

The various risk factors interact with one another. For example, polarization increases the chances of individuals with malignant traits coming to power. If this happens, they will probably obstruct efforts to prevent s-risks. Advanced technology, in turn, could multiply the harms caused by malignant agents, and so on. Just as in the medical context, catastrophes are more likely to occur if several risk factors coincide.

Analogously, a risk factor can be mitigated by a set of favorable circumstances. For example, advanced technology is much less worrying if consideration

for all sentient beings is more widespread and there are adequate efforts to prevent s-risks. Likewise, without space colonization, the suffering caused by a malignant dictator would be limited to Earth, and so on.

8.10. Conclusion

In this chapter we have seen what risk factors are, as well as some examples of risk factors for s-risks. We have also seen that the probability of an s-risk materializing is greater when several risk factors coincide, and that the chances of preventing it are also greater if several forms of prevention coincide.

In the next chapter we will look at a classification of the kinds of strategies for preventing risk factors for s-risks.