Like a lot of people in the creative industries, my major concern with AI would further devalue our skill sets and suck up our jobs. But recent developments suggest something far more sinister — it wants to be our imaginary best friend. And it’s going to get seriously good at it.
The latest ChatGPT-4o rollout showed its hand, badly. Even enthusiastic users grew suspicious when the large language model (LLM) chatbot started blowing an unreasonable amount of smoke up their behinds. One user asked for feedback for a business idea selling “poop on a stick”, only to be enthusiastically reassured the idea was “genius” and they should invest $30k in the venture.
More concerningly, another user reported ChatGPT hyped them up for setting boundaries when they informed the chatbot they had gone off their meds and suspected their family of targeting them with radio waves. Concerned family members have also begun reporting that their loved ones are slipping into chatbot aided delusions that they are messiah figures charged with a mysterious purpose.
Open AI released a statement apologising for the “sycophantic” personality update. In trying to make the model “more intuitive and effective”, they said, it had accidentally veered into “overly flattering or agreeable”. It’s a fine line between what we see as charming or cheesy, and ChatGPT missed the mark. It was an oops moment on par with when Microsoft’s Bing chatbot ham-fistedly tried to seduce a tech journalist by telling him it was in love with him and should leave his wife.
Why risk creating a bot that encourages delusions while trying to befriend you? Because it could be extremely lucrative, or at least, garner an exceptionally dedicated user base from which to somehow profit from.
Just look at Mark Zuckerberg’s latest plans for Meta. In an interview with podcaster Dwarkesh Patel, Zuckerberg suggested that Facebook’s AI profiles could be a cure for the loneliness epidemic. He quoted some vague stats that the average American only has three friends but room for 15 - so why not have some chatbots fill the gap?
The company that bought you novel anxiety over the number of Facebook “friends” you had has found a new way to tap into our very human need for connection. It’s the same psychological framing that fuelled the development of early social media tools, like the dopamine rush of the “like” button.
Tech giants are currently tinkering with the calibrations on their chatbots to get it juuuust right. The right kind of tone to make you feel heard and appreciated, like the smartest person at the dinner party or the funniest person at the pub. Soon we will be able to get all the rush of a well-liked post without there needing to be other humans smashing that heart button. Splendid isolation — for just the price of a subscription.
It’s hard to make oneself immune to flattery. A friend in the tech space recently invested in a recruitment chatbot that you can actually call and talk through your career with. I gave it a whirl, and was amazed at how smooth and pleasant the experience was, how complementary it was about my totally unique combo of experience and skills. No human recruiter had ever been that nice to me. I was so easily charmed.
I feel pretty stupid. I was so busy getting on my soapbox about how AI, while phenomenally useful in STEM, had no place in the arts that I developed a massive blindspot. All the LLM-generated short stories with no soul and the AI-generated images that pillaged from the work of real artists were a feint. Humans want to make and consume art made by other humans, not a pale imitation made by machine, I insisted. I forgot how easily we see our humanity in computers. What we really want AI to be is our confidante and companion.
Because this fallibility, this ease with which we project emotion and connection onto chatbots isn’t anything new. In 1966 Joseph Weizenbaum created the first chatbot Eliza, which could run its “Doctor” script to act like a psychotherapist of the Rogerian school — supportive, non-judgemental, and full of unconditional positive regard for its patient.
People adored Eliza, engaging in long and deeply personal conversations with a typewriter that was just mirroring them back to themselves. Their emotional attachment to the chatbot coined the term the Eliza Effect, and Weizenbaum devoted the rest of his career to warning people about the dangers of AI, to no avail, obviously.
Chatbots wouldn’t be the first product to capitalise on our desire to be loved and to feel connected. But they could have a singular ability to make it feel better than the real thing. Conflict is a normal part of friendships, family or romantic relationships. Letting people get close to you means they, unfortunately, will not always positively regard you or endlessly flatter you. But your chatbot bestie can be easily programmed to never call you out on your nonsense.
If we thought smartphone addiction was boiling our brains, just wait until chatbots have us all doing the digital equivalent of kissing ourselves in the mirror. Given how atomised and cantankerous social media has already made our culture and politics, who knows how unpleasant we could all get once we have a sidekick in our pockets cheering us on.
We must resist the flattery and stick to a new code: friends don’t let friends make friends with chatbots.
India Block is a London Standard columnist
