Abstract:
Although Voss' views about AIs are relatively sunny, other participating technosavants weren't so sure. For example, computer scientist Stephen Omohundro argued that self-improving AIs would be ultra-rational economic agents, basically examples of homo economicus. Such AIs would exhibit four drives; efficiency, self-preservation, acquisition, and creativity. Regarding efficiency AIs optimizing their resource use would turn to nanotechnology and virtualization wherever possible. Self-preservation involves protecting its utility function from death which it would do by building in redundancy and embedding itself in mutually defensive social relations. The drive to acquire more resources means that AIs could be dangerously competitive with humans. If Omohundro is right, there are good reasons to doubt that an AI that is a relentless utility maximizer will be friendly to less than perfectly efficient humanity. The drive for creativity enables AIs (and us) to explore new possibilities for transforming and satisfying our utility functions. Omohundro's solution to for making AIs human-friendly? Try to teach AIs our highest human values, e.g., happiness, love, compassion, beauty and so forth.