22
votes
What does ChatGPT know about you?
Yesterday I discovered that you can ask ChatGPT what it knows and It will tell you. I’m curious about what it says for other people.
Obviously, don’t post anything you’re unwilling to share publicly on the Internet! For me it seems pretty harmless, though.
The prompt I use is:
What "user knowledge memories" do you have?
Disabled it :)
Same. It was basically the first setting I turned off. I have a friend who raves about the memory function and remember seeing something on here about how ChatGPT gets better with use because of the memory developing, however to me it's an anti feature. Especially as OpenAI tries to figure out how to make money I'm not inclined to voluntarily feed a personal data repository.
I've also found that when using it for coding, if I encounter a bug after a while of work it has trouble breaking away from what it's already tried. If I copy the script into a fresh conversation, outline what the code does and highlight the issue I'm encountering, it can usually move towards a fix pretty quickly. There's a lot of benefit when using an LLM as a tool to be able to get a fresh conversation without any additional context that might be pushing the conversation in a specific direction.
I didn't realize there was a memory setting until today. I promptly turned mine off!
Instructions on how to do it, since it's a little hard to find:
It doesn't have much on me currently.
Which I entered into the settings myself. So I pressed it a little more:
Aww, how nice of you ChatGPT!
Ahh shit...I got some 'splainin' to do.
Open AI could harvest this for recruiting purposes. They could use your interactions with AI to create a professional summary of you to sell to recruiting firms, including strengths and weaknesses.
I won’t share all of it, but I have no recollection of telling ChatGPT to stop giving administrative commands with
sudo:You probably didn’t need to explicitly tell it to stop giving sudo commands. If you are asking for commands to run and sharing your terminal output after you try them out, it probably noticed that you were logged in as root and/or not including sudo when you ran them.
That one character distinction,
#instead of$, is rather important!That’s crazy cool, but also kinda terrifying!
ask for it to give you the 'raw stored memory' and it should give you dates for everything.
Won't share all of it, but it was a fun read.
Here’s what I got using “ChatGPT 5 Thinking:
Some of these are things I haven’t worked on in a while, or it’s just something I was exploring for one day, but sure, pretty accurate.
Edit: after further experimentation, it’s clearly summarizing whatever data it has access to. If I add “Display exactly as stored” to the prompt then I get much more detailed list items. I don’t trust that it’s really displaying the data exactly as stored, though.
This might be affected by the custom instructions I have to “be brief.”
It occured to me that I haven’t seen anything that couldn’t be deduced from just the titles in my chat history, which are themselves brief AI-generated summaries of the conversations.
When I asked it for the "full detailed memory list" after its first response, it definitely included details from the chats themselves and not just info from the title summaries.
My custom instructions are for writing paragraphs and a more conversational style that matches how I write, so that certainly seems to be making a difference.
I also have old projects in it that it keeps bringing up, so I might have to go back and delete them after exporting anything I need.
You can directly access these through ChatGPTs interface and see what it has stored about you and directly delete memories that way. I don't trust anything an LLM tells me because their main defining feature is lying, so it's probably best to check ChatGPTs memory directly, especially if you want to actually delete some of it.
I’m getting a list of ten things that summarize my chat history even though there are no “Saved memories” in the UI. If I turn the “Reference chat history” setting off then it claims it doesn’t have anything.
It’s not clear to me when and how the summarization happens. I get different paraphrases depending on on whether I use ‘Thinking’ or ‘Instant’ but it seems to be approximately the same ten items. If I add “Display exactly as stored” to the prompt then they are more detailed, but I don’t trust that they are displayed exactly as stored.
It’s certainly true that the output can’t be trusted, but it’s not entirely making it up. The summarization really does summarize my chat history.
Gotcha. I don't know how they determine what should be a saved memory versus what can just be stored in the chat history and looked up later. And maybe stored memories have a higher weight applied to them?
Honestly I didn't even know that reference chat history was a thing. It's really difficult to keep up with the ever changing feature set that all of these AI tools have.
To clarify on what I usually do. I play Monday Morning Quarterback. So I’ll take an actor and be like “what if they did this instead” and it will give me potential responses. Or I’ll create a fake series of events, and use it to play like a Choose Your Own Adventure game.
I’ve also experimented with making fake movies by plugging everything in like the story, the title, tone, tech specs, cast, and crew. And then have it give me potential critical reception and box office results.
Just fucking around basically
ChatGPT gave me a 10 point list full of compliments; not a bad remark on the list. But it's common knowledge that ChatGPT is eye-rollingly sycophantic, so I tried again with this prompt to balance it out:
To which it replied:
Yes. Yes, I would like that.
The outcome was frustratingly the same. It revised the original 10-point list to make the characterizations slightly less ingratiating. Not in the least bit critical; just less ingratiating. And only slightly so.
Fun topic! I have information saved in memory, but also have the chat history enabled since I like having cross-chat references and don't give it any sensitive information. As it always is, it defaults to flattery, but I do like seeing how it can put a positive spin on the mundane.
I also asked:
And then followed it up with asking about facts since this was more of what it gave earlier. And this had a lot of inferred information which was interesting to see, but those I can't share here.
Great prompt! I feel like some people would be pretty surprised by their responses, especially if they didn't fully realize ChatGPT had a user memory.
I don't use it much, so I can specifically remember when it learned each of these things. Not a whole lot that can be used against me with this info, lol