It's no great shock that AI models scrape information from just about everywhere to train their data sets, but some users have been surprised to find their chats with ChatGPT turning up in Google searches.
As Fast Company reports, "Nearly 4,500 conversations come up in results for the Google site search, though many don’t include personal details or identifying information." In ChatGPT, if you chose to share a chat or prompt with someone else, you were hit with a tick option, which gave it permission to 'make this chat discoverable'.
Underneath this tickbox, in smaller text, sat the disclaimer, "Allows it to be shown in web searches". Fast Company reports that, though no identifying information came through, one of those publicly accessible chats "described in detail their sex life and unhappiness living in a foreign country, claiming they were suffering from post-traumatic stress disorder (PTSD) and seeking support."
This same chat reportedly included details of family history, friends, and interpersonal relationships.
In a conversation with Fast Company, AI ethicist at the University of Oxford, Carissa Veliz said: “As a privacy scholar, I’m very aware that that data is not private, but of course, ‘not private’ can mean many things, and that Google is logging in these extremely sensitive conversations is just astonishing.”
We just removed a feature from @ChatGPTapp that allowed users to make their conversations discoverable by search engines, such as Google. This was a short-lived experiment to help people discover useful conversations. This feature required users to opt-in, first by picking a chat… pic.twitter.com/mGI3lF05UaJuly 31, 2025
You might be wondering, 'What's the big deal? Users had to opt in to share this," but it's important to note that a checkbox as you go to share a chat may look like something else to a user. It may just look like a terms of service prompt, and 'making the chat discoverable' could be read as a necessary step for a friend or family member to see the prompt you chose to share.
According to Ars Technica, ChatGPT's opt-in discoverability box was initially defended as "sufficiently clear". Yet just a day after the publication of Fast Company's report, OpenAI CISO Dane Stuckey announced that the feature to share chats in web searches would be removed from the ChatGPT app.
"Ultimately we think this feature introduced too many opportunities for folks to accidentally share things they didn't intend to," says Stuckey, "so we're removing the option. We're also working to remove indexed content from the relevant search engines."
I don't much like Google's AI overview, but I think I'd like it even less if it gave me back chats people had with ChatGPT. It's a good reminder that, even with this tickbox now gone, sharing any information online can sometimes lead to that data being used for training AI models, or end up being displayed to the public in its entirety. So, as always, be careful what you share.
