Scalar reward is not enough: a response to Silver, Singh, Precup and Sutton (2021)

Author(s):Show less -

Vamplew, Peter ; Smith, Benjamin J. ; Kallstrom, Johan ; Ramos, Gabriel ; Radulescu, Roxana ; Roijers, Diederik M. ; Hayes, Conor F. ; Heintz, Fredrik ; Mannion, Patrick ; Libin, Pieter J. K. ; Dazeley, Richard ; Foale, Cameron

• Document type: Journal article
• Source: AUTONOMOUS AGENTS AND MULTI-AGENT SYSTEMS; v. 36, n. 2, p. 19-pg., 2022-10-01.

Abstract

The recent paper "Reward is Enough" by Silver, Singh, Precup and Sutton posits that the concept of reward maximisation is sufficient to underpin all intelligence, both natural and artificial, and provides a suitable basis for the creation of artificial general intelligence. We contest the underlying assumption of Silver et al. that such reward can be scalar-valued. In this paper we explain why scalar rewards are insufficient to account for some aspects of both biological and computational intelligence, and argue in favour of explicitly multi-objective models of reward maximisation. Furthermore, we contend that even if scalar reward functions can trigger intelligent behaviour in specific cases, this type of reward is insufficient for the development of human-aligned artificial general intelligence due to unacceptable risks of unsafe or unethical behaviour. (AU)

• FAPESP's process: 20/05165-1 - Communication and machine learning in urban mobility: a multiagent and multiobjective approach
• Grantee: Ana Lúcia Cetertich Bazzan
• Support Opportunities: Regular Research Grants