10
votes
What does ChatGPT know about you?
Yesterday I discovered that you can ask ChatGPT what it knows and It will tell you. I’m curious about what it says for other people.
Obviously, don’t post anything you’re unwilling to share publicly on the Internet! For me it seems pretty harmless, though.
The prompt I use is:
What "user knowledge memories" do you have?
Disabled it :)
Same. It was basically the first setting I turned off. I have a friend who raves about the memory function and remember seeing something on here about how ChatGPT gets better with use because of the memory developing, however to me it's an anti feature. Especially as OpenAI tries to figure out how to make money I'm not inclined to voluntarily feed a personal data repository.
I've also found that when using it for coding, if I encounter a bug after a while of work it has trouble breaking away from what it's already tried. If I copy the script into a fresh conversation, outline what the code does and highlight the issue I'm encountering, it can usually move towards a fix pretty quickly. There's a lot of benefit when using an LLM as a tool to be able to get a fresh conversation without any additional context that might be pushing the conversation in a specific direction.
It doesn't have much on me currently.
Which I entered into the settings myself. So I pressed it a little more:
Aww, how nice of you ChatGPT!
Ahh shit...I got some 'splainin' to do.
Won't share all of it, but it was a fun read.
I won’t share all of it, but I have no recollection of telling ChatGPT to stop giving administrative commands with
sudo
:Here’s what I got using “ChatGPT 5 Thinking:
Some of these are things I haven’t worked on in a while, or it’s just something I was exploring for one day, but sure, pretty accurate.
Edit: after further experimentation, it’s clearly summarizing whatever data it has access to. If I add “Display exactly as stored” to the prompt then I get much more detailed list items. I don’t trust that it’s really displaying the data exactly as stored, though.
This might be affected by the custom instructions I have to “be brief.”
It occured to me that I haven’t seen anything that couldn’t be deduced from just the titles in my chat history, which are themselves brief AI-generated summaries of the conversations.
To clarify on what I usually do. I play Monday Morning Quarterback. So I’ll take an actor and be like “what if they did this instead” and it will give me potential responses. Or I’ll create a fake series of events, and use it to play like a Choose Your Own Adventure game.
I’ve also experimented with making fake movies by plugging everything in like the story, the title, tone, tech specs, cast, and crew. And then have it give me potential critical reception and box office results.
Just fucking around basically
You can directly access these through ChatGPTs interface and see what it has stored about you and directly delete memories that way. I don't trust anything an LLM tells me because their main defining feature is lying, so it's probably best to check ChatGPTs memory directly, especially if you want to actually delete some of it.
I’m getting a list of ten things that summarize my chat history even though there are no “Saved memories” in the UI. If I turn the “Reference chat history” setting off then it claims it doesn’t have anything.
It’s not clear to me when and how the summarization happens. I get different paraphrases depending on on whether I use ‘Thinking’ or ‘Instant’ but it seems to be approximately the same ten items. If I add “Display exactly as stored” to the prompt then they are more detailed, but I don’t trust that they are displayed exactly as stored.
It’s certainly true that the output can’t be trusted, but it’s not entirely making it up. The summarization really does summarize my chat history.
Gotcha. I don't know how they determine what should be a saved memory versus what can just be stored in the chat history and looked up later. And maybe stored memories have a higher weight applied to them?
Honestly I didn't even know that reference chat history was a thing. It's really difficult to keep up with the ever changing feature set that all of these AI tools have.