THE ROBOT WITH A HIDDEN AGENDA

BRIAN KNUTSON

Associate professor of psychology and neuroscience, Stanford University

广告：个人专属 VPN，独立 IP，无限流量，多机房切换，还可以屏蔽广告和恶意软件，每月最低仅 5 美元

Why should people think about machines that think (or anything that thinks, for that matter)? One tipping point might involve considering others as agents rather than automata. Automata act at the behest of their creators (even if removed in space or time). Thus, if automata misbehave, the creator gets the blame. Whereas agents act based on their own agendas. When agents misbehave, they themselves are to blame.

While agency is difficult to define, people naturally and rapidly distinguish agents from nonagents and may even use specialized neural circuits to infer others’ feelings and thoughts. In fact, a designer can co-opt features associated with agency (including physical similarity, responsiveness to feedback, and self-generated action) to fool people into thinking they’re interacting with agents.

What is necessary to endow an entity with agency? While at least three alternatives present themselves, two of the most popular and seductive possibilities may not be necessary:

1.  Physical similarity. There are infinite ways to make machines similar to humans, both in terms of appearance and behavior, but ultimately only one is accurate. It’s not enough to duplicate the software—you also have to implement it on the underlying hardware, with all of its associated affordances and limitations.
One of the first automata, de Vaucanson’s duck, appeared remarkably similar to a duck, right down to its digestion. But while it may have looked like a duck and quacked like a duck (and even crapped like a duck), it was still not a duck. Nonetheless, maximizing physical similarity is an easy way to trick others into inferring agency (at least initially).
2.  Self-awareness. Many seem concerned that if machines consume enough information they’ll become self-aware and will then develop their own sense of agency, but neither logic nor evidence supports these extrapolations. While robots have apparently been trained to recognize themselves in mirrors and sense the position of their appendages, these trappings of self-awareness haven’t led to laboratory revolts or surgical lapses. Perhaps conveying a sense of self-awareness would cause others to infer that a machine had greater agency (or would at least entertain philosophers), but self-awareness alone doesn’t seem necessary for agency.
3.  Self-interest. Humans aren’t mere information processors. They’re survival processors. They prefer to focus and act on information that promotes their continuance and procreation. Thus, humans process information based on self-interest. Self-interest can provide a unified but open framework for prioritizing and acting on almost any input.

Thanks to a clever evolutionary trick, humans don’t even need to be aware of their goals, since intermediate states like emotions can stand in for self-interest. Armed with self-interest and an ability to flexibly align responses to changing opportunities and threats, machines might develop agency. Thus, self-interest might provide a necessary building block of agency and also could powerfully evoke agentic inferences from others.

Self-interest might transform machines that act on the world (aka robots) from automata into agents. Self-interest also flips the ordering (but not the content) of Isaac Asimov’s prescient Laws of Robotics:

1.  robots mustn’t harm humans or, through inaction, allow a human being to come to harm,
2.  robots must obey humans (unless this violates the first law), and
3.  robots must protect themselves (unless this violates the first two laws).

A self-interested robot would protect itself before helping or averting harm to humans. Constructing a self-interested robot would thus seem straightforward: Endow it with survival and procreation goals, allow it to learn what promotes those goals, and motivate it to continually act on what it learns.

Still, we should think twice before building self-interested robots. Self-interest can conflict with others’ interests. Witness the destructive effect of viruses’ simple drives to survive. If self-interested robots did exist, we’d have to think about them more seriously. Their presence would raise basic questions: Should these robots have self-interest? Should they be allowed to act on it? Should they do so without awareness of why they were acting that way?

And don’t we have enough of these robots already?