Stop technology or shape it?
Since many s-risks depend on new technologies, someone might think: "The solution is to stop technological development." The author shows why that doesn't seem like the best way out:
- technology can also reduce suffering;
- even if we managed to stop it for a while, that wouldn't last forever;
- a campaign against technology would create conflict with people who want its benefits;
- it isn't realistic to think we'll manage to stop dangerous technologies in time.
That's why trying to shape how technologies are created and used seems more promising.
The case of artificial intelligence
Why worry about AI: if one day it becomes more intelligent than humans and acts on its own, it could shape everything that happens afterward according to its goals. If those goals don't take sentient beings into account, the result could be terrible. And this work is still very neglected.
The risks of this focus: since we don't know what future technologies will look like, efforts to shape AI might not work or might even be a waste of time. Also, other technologies, like space colonization and new forms of sentience, might be even more worrying.
In short
- Stopping technological progress doesn't seem realistic or desirable.
- Shaping technologies is a more promising strategy.
- AI is an important case, but there are other technological risks.