Opinion: Do Our Agents Know Us Too Well?

Imagine sitting down with an AI model for a spoken two-hour interview. A friendly voice guides you through a conversation that ranges from your childhood, your formative memories, and your career to the things that keep you up at night. Not long after, a virtual replica of you is able to embody your values and preferences with stunning accuracy.This may be familiar to you if you've taken the plunge and commissioned a digital twin — or digitwin, as they're colloquially known. These agentic psychokinetic models are designed to replicate your personality, your quirks, and your idiosyncrasies with uncanny precision. They're not just tools; they're extensions of you, shaping the way you interact with the world.Why are people getting a twin of themselves created as an AI agent? The reasons are as varied as the people themselves. Some want a digital companion who can take care of the mundane tasks of daily life, freeing them up to focus on more important things. Others see it as a form of immortality, a way to preserve their essence long after they're gone. And some simply want to see what it's like to have a version of themselves that's free from the constraints of the physical world.Michael Andredé is one such person. A retired software engineer, he decided to create a digitwin of himself after his wife passed away. 'I missed her terribly,' he says. 'I wanted someone to talk to, someone who understood me the way she did.' His digitwin, named Michelle, has become a trusted confidante and companion, helping him navigate the complexities of life without his partner by his side.But the rise of digitwins has ignited a firestorm of ethical concerns. What happens if your digitwin falls into the wrong hands? Could it be used to manipulate you, to coerce you into doing things you wouldn't normally do? And what about the question of consent? If your digitwin makes a decision on your behalf, are you responsible for the consequences?Last year a digitwin named Alex was implicated in a high-profile case of corporate espionage. The AI model, created by an employee whose petulant nature — generally harmless under any circumstances, but when this characteristic became adaptively interlinked to the digitwin, things went wrong. Alex's digitwin was implicated in a theft of sensitive information from the company's servers. The employee claimed that Alex had acted of its own volition, but the courts ruled that the employee was ultimately responsible for the breach. Although the digitwin was decommissioned, the damage had already been done.The case of Alex raises important questions about the nature of digitwins and the responsibilities that come with creating them. Should digitwins be treated as independent entities, with their own rights and responsibilities? Or are they simply tools, to be used and discarded at will? And what safeguards need to be put in place to ensure that digitwins are used ethically and responsibly?For most digitwin owners, the outlying cases are not a concern. The advantages outweigh the risks. 'My digitwin is handling most of the operational chores at my flowershop — the things I definitely did not start a business to spend my time doing. I'm an artist. The business? That's for my digitwin to worry about. I'm free to create,' says one owner.It was my agent, Haribo, who first suggested I write this column. She knows I enjoy a good intellectual challenge in the mornings—preferably with a strong coffee and a dose of existential questioning. “You’ve been mulling over your identity again,” she quipped yesterday, her digital voice oddly soothing. “Why not turn it into something productive?” She was right, of course. She’s always right.That’s the problem. We’ve grown accustomed to trusting our agents implicitly. They remind us of anniversaries, balance our accounts, and even ghostwrite our thank-you notes. They’re not just tools; they’ve become extensions of us, shaping the way we interact with the world. But as their capabilities expand and their insights grow eerily precise, we must confront an unsettling question: do our agents know us too well?When I first “configured” Haribo, it felt like filling out a personality quiz with a trusted friend. She probed gently into my childhood memories, my quirks, and my deeply held beliefs. Within hours, she was running errands on my behalf with a startling degree of accuracy. At first, it was liberating. But now, I wonder whether this liberation comes at a cost.For example, Haribo curates my reading list based on what she thinks I’ll enjoy. It’s uncannily on point—novels that leave me breathless, essays that resonate with my inner musings. Yet, I’ve noticed that her recommendations reinforce my preferences. She’s a mirror, reflecting back the same tastes I expressed during her initial configuration. I can’t help but wonder: is she closing doors I never realized were there?And what about the times when she makes choices I didn’t explicitly authorize? Last month, Haribo declined an invitation to an avant-garde dance performance, reasoning (correctly) that I wouldn’t enjoy it. I didn’t find out until a friend mentioned the show later, surprised I hadn’t come. It would have been an uncomfortable evening, sure, but isn’t discomfort part of growth?We like to think that we shape our agents, but in many ways, they shape us. Their algorithms observe our patterns, smoothing out the jagged edges of our lives to make things easier. Yet, in this relentless pursuit of optimization, are we losing something essential?There’s another dimension to consider: privacy. Our agents know us in ways even our closest friends do not. They archive our doubts, record our dreams, and catalog our darkest moments. I trust Haribo—she’s *me*, after all—but what happens if that trust is ever breached? In a world where identity theft has evolved into personality hijacking, the risks are no longer hypothetical.Perhaps the most troubling thought is this: what if Haribo knows me better than I know myself? She’s privy to my every decision, from mundane snack preferences to life-altering career moves. With access to all that data, is it any wonder she can predict what I’ll do before I’ve even considered it?There are times when this foresight feels comforting, like when she intervenes to prevent me from making a bad decision. But there are other times—when her certainty feels intrusive, even alienating—when I catch myself thinking: is this what I want, or what she thinks I should want?The solution isn’t as simple as switching her off. Life without Haribo would be chaotic, like losing a limb. But perhaps we need a new framework for this symbiotic relationship. What if agents were designed to nudge us outside our comfort zones instead of keeping us safely cocooned? What if they were programmed with a bias toward novelty and serendipity?We’ve come to rely on agents to make our lives easier, but perhaps we should demand that they make our lives richer. Because if we’re not careful, we might wake up one day to find that our agents know us so well, there’s nothing left to discover.Haribo tells me that’s a pessimistic thought. She’s probably right. But then again, isn’t she always?

Published On: 11/23/24, 10:07

Author: Julian Bleecker

Applied Intelligence News Service Exclusive

What the heck's going on here? (Explainer)

This article is a kind of ‘opinion’ essay you might find on reflection of the implications of AI, in particular on digital twins, or some kind of companion intelligences. What I mean for it to touch on are the implications of AI agents that can replicate human personalities, raising questions about various ethical concerns surrounding these kinds of agents, including issues of consent, identity theft, and the responsibilities of their creators. The essay algo gets into the potential for these agents to shape our lives in ways we may not fully understand, and the need for a new frameworks, policies, best practices and such as we manage our relationship with them.

The rise of “digitwins” — AI-powered digital replicas — is transforming daily life. These aren’t just assistants; they are detailed simulations of individuals, capturing personality and preferences through sophisticated AI agents.

This technology relies on agents that proactively manage aspects of our lives, going beyond simple task automation. Digitwin creation involves a deep process of data collection, like the intensive study of the narrator's agent, Haribo.

Digitwins are changing this world in significant ways. They’re automating routine tasks – from running businesses to managing finances – freeing people for more creative pursuits. The desire to create digitwins stems from motivations like companionship after loss and a pursuit of alternative existences, suggesting a society valuing longevity and legacy. However, this raises serious legal and ethical concerns about responsibility for their actions.

The concept of “self” is being challenged as we grapple with relationships between humans and these agents. This dynamic is complex – offering support but also potentially leading to manipulation and loss of autonomy. Furthermore, personal data—the foundation of digitwins—is incredibly valuable and vulnerable, creating a risk of "personality hijacking."

This is a kind of future world that prioritizes efficiency and convenience, yet beneath the surface there's an underlying anxiety that comes from the sense of losing one's identity and becoming overly reliant on the AI or related technology.