Shaping Emerging Technologies to Prevent Future Suffering

Original text Luciano Carlos Cunha · Volume IX, p. 81 of the PDF edition · English translation

12.1. Introduction

S-risks are risks that practices will arise in the future generating suffering of astronomical size, many orders of magnitude greater than the suffering that exists today¹⁰⁵. Several of these risks concern the increase in technological power and the high probability that those who control these technologies will have no moral consideration for the individuals affected by them.

In light of this, one might think that a solution is to try to stop technological development. However, there are several reasons why this strategy would not be very promising. Baumann (2022, p. 77) lists the following reasons:

• Technology could also be used to reduce suffering. In the absence of technology, there would be no way to prevent suffering resulting from natural causes¹⁰⁶, for example (and perhaps not even suffering caused by humans that does not result from the use of advanced technology). We have already discussed this question in more detail in section 8.3.
• It is not realistic to suppose that, even if we succeeded in stopping technological development, this would be guaranteed forever.
• A campaign against technological development could generate conflict with those who seek to reap the benefits of technology.
• It is not realistic to expect that we will be able to prevent the development of certain harmful technologies in time.

These reasons seem to indicate that trying to shape the development and use of technologies is a more promising strategy for trying to reduce s-risks. In this discussion, the focus has usually been on how to shape AI (artificial intelligence). However, as we will see, there are other s-risks related to the advance of other kinds of technology (such as the possibility of

space colonization and of the emergence of new forms of sentience), which may be more worrying than AI.

Below we will discuss some suggestions made by Baumann (2022, p.77-84) regarding shaping emerging technologies. We also add a discussion of technologies that could give rise to other forms of sentience in the future.

12.2. Arguments for and against focusing on shaping artificial intelligence (AI)

Let us begin by discussing the points for and against focusing on trying to shape AI¹⁰⁷.

12.2.1. Why try to shape AI

The main reason to try to shape AI, if our goal is to prevent s-risks, is this: if AI comes to have superhuman intelligence and becomes autonomous, it could shape everything that happens from then on according to its values. Thus, influencing this process could be central to influencing the long-term future. Advanced AI has the potential to cause s-risks, but it could also prevent s-risks arising from other sources. Everything will depend on what values this AI will have.

Work to reduce s-risks from AI is highly neglected. Few people work on ensuring that the goals of autonomous AI are aligned with those of its creators (what is called AI safety). However, even alignment with the goals of its creators is not enough to prevent s-risks, since its creators may be indifferent to the suffering of nonhuman sentient beings (or may even aim to cause them suffering). For example, the few people who work on AI safety usually take into account only the risks to humans. This high degree of neglect allows us to have considerable influence¹⁰⁸.

12.2.2. Risks of focusing on shaping AI

One risk involved in focusing on trying to shape AI is the following: since we do not know what future technologies will be like, there is a high risk that attempts to shape AI will be ineffective, or even a waste of time. We may be mistaken about which future technologies will have the most influence. We may focus on AI while the most influential technologies turn out to be others that have yet to emerge. Just as in the past there was not just one technology that determined the course of history, there is probably not just one lever for shaping the future and reducing s-risks.

12.2.3. How great is the influence of technologies?

Even if there were a single technology with the greatest influence in determining what the future will be like, and even if we knew which one it is, it could still be that improving values, political culture, and institutions would be the most effective way to shape the future (rather than trying to shape technology directly), since technological development is never divorced from the political and sociocultural context (it is this context that will determine which goals are pursued through the use of technology).

12.3. An example of AI alignment focused on s-risks

Since advanced AI systems would be very powerful agents, it is quite possible that conflicts between such systems would have disastrous consequences. So efforts to achieve cooperative AI are promising for reducing s-risks.

For example, one could research how to mitigate the risks if something analogous to traits such as psychopathy, sadism, narcissism, and Machiavellianism were to emerge in AIs¹⁰⁹. These traits arose in humans over the course of evolution. So something equivalent could arise in AI training environments. By studying the incentives reinforced in these environments, we could try to ensure that such tendencies do not emerge.

We can also create safety measures that would kick in if there is a failure in aligning AI with the values of its creators. However, as we have seen, the values of its creators are crucial in determining what the outcome will be. We will discuss this next.

12.4. Avoiding indirect risks related to AI

Baumann (2022, p. 80-4) suggests the following ways of avoiding indirect risks related to AI:

Treaties to ensure cooperative AI development could prevent an arms race among nations for control of AI. International bodies could oversee AI development, and rules could be adopted on how to proceed if an AI comes to have certain capabilities. Of course, there is no reason to be overly optimistic about this, since any such agreement could be very difficult to implement.

The emergence of advanced AI could endanger the rule of law and democratic institutions. So we need to investigate how to avoid political turmoil during a period of transition to advanced AI, and how to ensure that laws still apply in that context.

It is also vital to limit the ability of malevolent agents to make use of AI. One possible way would be to try to distribute future technological capabilities so as to make it difficult for any single agent to cause great harm.

In addition, we can establish standards on how to proceed if we realize that an AI has met the criteria for sentience¹¹⁰.

12.5. Space governance

As mentioned before, s-risks due to technology are not limited to AI. Many are related to space colonization¹¹¹, since with such activity suffering could be vastly multiplied across the universe, to the point of making the suffering that exists on Earth "a grain of sand" by comparison.

When discussing this risk, animal advocates usually have in mind the possibility of humans living on other planets and reproducing animal exploitation

there (as, for example, the use of animals for food). However, the risk related to the possibility of space colonization is probably much more closely tied to the possibility that it will spread across the universe the suffering and deaths caused by natural processes, since when (and if) space colonization becomes feasible, animal exploitation may already be obsolete. And, even if it is not, natural processes tend to be more efficient than animal exploitation at maximizing suffering¹¹². We will look at this below.

Space colonization could multiply the suffering caused by natural processes simply by spreading life to other places (even nonsentient life, since it could, after a long time, give rise to sentient beings). This is so because, given the way natural selection operates, natural processes tend to maximize the number of sentient beings who are born only to have a life full of suffering and to die prematurely. For example, the characteristic of reproducing by having gigantic broods (with thousands or millions of offspring per brood) is a trait that is very likely to prevail, since there is a high probability that at least some offspring will survive and reproduce, passing this trait on, even if all the others are born only to suffer and die. Indeed, this is what can be inferred to occur in stable populations (if a population remains stable over several generations, this is a sign that, on average, only one descendant per parent survived). This is one of the reasons why the number of animals who suffer and die as a result of natural processes is so gigantic that it makes even the figures for animal exploitation almost vanish by comparison.

There are currently no regulations on space exploration. It is therefore worth creating regulations to achieve positive outcomes when (and if) space colonization becomes feasible, both in terms of avoiding conflicts and in terms of not indirectly spreading suffering.

One might think that the easiest way to avoid these s-risks would be to oppose space colonization. This is certainly a possible strategy. However, if space colonization is inevitable, it is also important to emphasize that we should undertake it only after

we have done everything possible to ensure that the outcome is positive.

12.6. The possibility of creating other forms of sentience

Like space colonization, the possible creation of new forms of sentience in the future has the potential to multiply suffering enormously. This could happen through genetic modification of nonhuman animals and also through the creation of nonorganic sentient beings (for example, in digital media¹¹³).

Especially in the latter case, these beings could be very different from the sentient beings we already know (they might not have faces, might not show that they are suffering, and so on), and so most people would probably not empathize with them. Nonhuman animals are already very similar to humans in terms of expressing suffering and, even so, all kinds of atrocities are committed against them every day. Moreover, it may become very cheap to create digital sentient beings on a large scale, and this could be useful for various purposes, such as creating a universe inside a computer.

As in the case of space colonization, there is also no regulation of this, and no broad discussion taking place. It is therefore important to begin advocating the need for discussion about regulating this practice before it emerges.

There is also the option of trying to campaign for these forms of sentience never to be created. However, as in the case of space colonization, it may be impossible to prevent their emergence, given that technological progress is usually much faster than the progress of ethical debate in society about the practice in question (what is called differential progress). However, differential progress is a problem not only for attempts to prevent the emergence of practices such as space colonization, the creation of new forms of sentience, or other uses of technology that could generate s-risks: it is also a problem for attempts to regulate the use of these technologies (since they will very probably begin to be used before there is any regulation).

In any case, there is a strategy we can adopt that would help increase both the chances that such technologies are never created and the chances that, if they are created, the resulting harm will be smaller: spreading moral consideration for all forms of sentience as efficiently as possible. This seems to be a very promising (and essential) strategy for increasing the chances that efforts to shape technologies will be directed toward preventing suffering rather than increasing it.