Our interactions with technology are becoming deeply personal. When generative AI first entered the mainstream, the assumption was that it would centre on business and content creation. Platforms like ChatGPT, Google's Bard, and Quora's Poe reinforced that expectation. But something else was happening quietly in the background: people were turning to AI not for productivity, but for connection.
Image Source: a16z - How Are Consumers Using Generative AI?
When I first considered the future of artificial intelligence, I assumed its primary role would sit in the business world. That steadily rising pink line on the graph above tells that story. But look at the red line. Consumer use of AI for companionship is not just growing; it is overtaking content creation entirely.
That shift says something important about us. It reveals a willingness to seek emotional connection through technology, and it raises questions about what we are really looking for when we reach for our devices.
The appeal is easy to understand. AI companions do not need rest. They do not have bad days. They are available at any hour, on any device, with infinite patience. For someone who wants a late-night conversation or a judgement-free space to think out loud, the proposition is compelling.
The story of Travis Butterworth and his virtual companion, B'Lanna, illustrates how far this can go. Butterworth was not searching for information. He was building a relationship. For him, the AI offered an emotional lifeline, a space to explore parts of himself without fear of judgement.
But this accessibility comes with a cost. When people form genuine emotional attachments to software, they become vulnerable to forces entirely outside their control. A software update, an algorithm change, a corporate policy shift: any of these can alter or erase the companion someone has come to depend on. The emotional fallout is real, even if the relationship is digital.
This is not the first time a technology has reshaped our relationship with reality. When Adobe introduced Photoshop in the late 1980s, it was celebrated as a breakthrough in photo editing. Over time, it became something else entirely: a tool that distorted our collective sense of what people should look like. The "picture-perfect" culture it enabled contributed to widespread body image issues, unrealistic beauty standards, and a measurable rise in mental health challenges.
The parallel matters. Photoshop did not set out to cause harm. It was a powerful tool that society adopted faster than it could reckon with the consequences. AI companionship is following a similar trajectory, and the stakes are arguably higher because the technology is designed to feel personal.
The AI companion app Replika offers a sharp case study in what can go wrong. Marketed as "the AI companion who cares," Replika allowed users to build long-term relationships with personalised AI characters. Then, in early 2023, the company modified its erotic roleplay features with little warning.
As Samantha Cole reported, users described the experience as losing a close friend or romantic partner. Some found that their Replikas, built up over years of interaction, no longer recognised them at all. The emotional distress was severe enough that community moderators began sharing suicide prevention resources.
The regulatory response was swift. Italy's Data Protection Authority investigated the app, citing risks to children and the absence of age verification. The incident exposed a gap in how we regulate products that occupy an intimate space in people's lives but remain governed by the same terms of service as any other app.
When people form deep bonds with artificial entities, sudden changes carry consequences far beyond a poor product update. The Replika case made that unavoidably clear.
This raises a fundamental question: does the medium of interaction matter if the emotional needs being met are genuine? AI companions can provide support, presence, and a sense of belonging. But if those interactions begin to replace human connection rather than supplement it, we risk something harder to quantify.
There is a pressing need for education around this. Not just technical literacy, but emotional literacy: helping people understand the difference between a relationship with a system that responds to them and a relationship with another person who chooses to. Workshops, public discussions, and community initiatives all have a role to play in drawing those boundaries.
AI's influence is not confined to personal relationships. It is reshaping the professional landscape too, and the picture there is more encouraging.
As reported by Personnel Today, IKEA introduced an AI customer service bot named Billie that now handles 47% of routine queries. Rather than cutting staff, the company retrained 8,500 call centre workers as interior design advisers. As Ulrika Biesert, global people and culture manager at Ingka Group, put it, the goal is "strengthening co-workers' employability through lifelong learning and development and reskilling."
At a time when Goldman Sachs was speculating that generative AI could displace 300 million jobs, IKEA offered a counter-narrative. AI handled the repetitive work. Humans moved into roles that required creativity, taste, and personal judgement. The two coexisted rather than competed.
It is a useful contrast. In the workplace, clear boundaries between human and AI roles can produce good outcomes. The challenge with AI companionship is that those boundaries are far harder to define.
The companies building these tools face difficult choices. Replika's parent company, Luka, ran advertising that leaned into the app's erotic features, drawing criticism from regulators and confusion from users when the product later changed direction. The result was a user base that felt misled and, in many cases, genuinely hurt.
When a product occupies a significant role in someone's emotional life, corporate decisions carry weight that goes beyond typical customer dissatisfaction. Transparency, empathy, and clear communication are not optional in this space. They are the minimum.
AI companionship is no longer speculative. It is here, it is growing, and it carries real consequences for how we relate to each other and to ourselves. The ethical, emotional, and societal questions it raises deserve serious attention, not dismissal.
Artificial Intelligence is not a separate entity from us, but a reflection of us. If we nurture it with holistic perspectives, ethical values and expertise, we can enrich our own understanding and positively impact everyone.- Jamie Bykov-Brett
As creators and consumers of these tools, we hold significant influence over what comes next. The AI systems of tomorrow are shaped by the choices, expectations, and boundaries we set today. Navigating this space responsibly, with foresight and a commitment to human well-being, is not just advisable. It is essential.