Related to the post a few days ago about criminals crafting ChatGPT searches to get data out of the engine, apparently they've decided to skip and step and just start handing it out without...
Related to the post a few days ago about criminals crafting ChatGPT searches to get data out of the engine, apparently they've decided to skip and step and just start handing it out without prompting. As part of an unrelated conversation with an unrelated user, ChatGPT divulged credentials and secure links (with even more credentials embedded in the URLs) to a pharmacy management system. It really causes some wonder about how the internals of ChatGPT function with respect to user data - it looks like there are no rails at all, anything that anyone ever enters impacts and is potentially available verbatim to every other user.
That’s not the issue at all, though. What happened in this case is that someone else’s chat log appeared in a users list of conversations. That’s a bad bug from ChatGPT if true, but it has nothing...
That’s not the issue at all, though. What happened in this case is that someone else’s chat log appeared in a users list of conversations. That’s a bad bug from ChatGPT if true, but it has nothing to do with “AI” or training material, it’s just a bug in the website.
It’s no different than if gmail showed you someone else’s email by accident. That has nothing to do with “AI”.
It is a surprisingly bad blunder from a very well funded company. I’m honestly kind of shocked at how slow the product development side of things has been for ChatGPT. Over the last year it’s...
It is a surprisingly bad blunder from a very well funded company. I’m honestly kind of shocked at how slow the product development side of things has been for ChatGPT. Over the last year it’s steadily improved. But really basic features have taken months to include. And then on top of it they’re making basic user access mistakes? “Move slow and break things” isn’t one of the good options.
Not trying to take for granted the LLM itself. They’re definitely going fast on the AI side of things.
I wonder if this is the classic science/engineering breakdown at play. Maybe OpenAI filled their rosters with computer scientists, academics who are at the bleeding edge of new AI theory and...
Over the last year it’s steadily improved. But really basic features have taken months to include. And then on top of it they’re making basic user access mistakes?
I wonder if this is the classic science/engineering breakdown at play. Maybe OpenAI filled their rosters with computer scientists, academics who are at the bleeding edge of new AI theory and practice, ie, scientists. Then they neglected to hire enough of those who actually know how to make a usable product, engineers or developers.
I've seen some of this first hand where I currently work. It's a 3 person (soon 4 person) company. The CEO has a fancy pants PhD and hired 2 ML engineers - one PhD drop-out and one PhD from his...
I've seen some of this first hand where I currently work. It's a 3 person (soon 4 person) company. The CEO has a fancy pants PhD and hired 2 ML engineers - one PhD drop-out and one PhD from his alma mater. His hiring criteria is mostly based around IQ and how far people get in academia. I'm the only product engineer and will be for the next 6 months. That should be fine given our expected growth curve. But I keep getting reminded by how little the others know about product development. If OpenAI didn't have any founded team-members that were engineering-first then I could see it being a huge culture shift to give the engineers the reins over their most popular product.
Yep, posted too quickly without reading deeply enough and conflated the two issues. There are problems with ChatGPT consuming data and spitting it out to other users, but that is not what is being...
Yep, posted too quickly without reading deeply enough and conflated the two issues. There are problems with ChatGPT consuming data and spitting it out to other users, but that is not what is being seen in this case, thank for the correction.
Yep, looks like a classic caching bug. Caching is critical but surprisingly hard to get right at scale. Valve did the same thing some years back and revealed customers billing information. I wish...
Yep, looks like a classic caching bug. Caching is critical but surprisingly hard to get right at scale. Valve did the same thing some years back and revealed customers billing information.
I wish the title were a little more clear as to the cause of the bug, but I guess that doesn't really matter for those affected.
Related to the post a few days ago about criminals crafting ChatGPT searches to get data out of the engine, apparently they've decided to skip and step and just start handing it out without prompting. As part of an unrelated conversation with an unrelated user, ChatGPT divulged credentials and secure links (with even more credentials embedded in the URLs) to a pharmacy management system. It really causes some wonder about how the internals of ChatGPT function with respect to user data - it looks like there are no rails at all, anything that anyone ever enters impacts and is potentially available verbatim to every other user.
That’s not the issue at all, though. What happened in this case is that someone else’s chat log appeared in a users list of conversations. That’s a bad bug from ChatGPT if true, but it has nothing to do with “AI” or training material, it’s just a bug in the website.
It’s no different than if gmail showed you someone else’s email by accident. That has nothing to do with “AI”.
It is a surprisingly bad blunder from a very well funded company. I’m honestly kind of shocked at how slow the product development side of things has been for ChatGPT. Over the last year it’s steadily improved. But really basic features have taken months to include. And then on top of it they’re making basic user access mistakes? “Move slow and break things” isn’t one of the good options.
Not trying to take for granted the LLM itself. They’re definitely going fast on the AI side of things.
Funds almost never go towards security until it's an issue.
I wonder if this is the classic science/engineering breakdown at play. Maybe OpenAI filled their rosters with computer scientists, academics who are at the bleeding edge of new AI theory and practice, ie, scientists. Then they neglected to hire enough of those who actually know how to make a usable product, engineers or developers.
I've seen some of this first hand where I currently work. It's a 3 person (soon 4 person) company. The CEO has a fancy pants PhD and hired 2 ML engineers - one PhD drop-out and one PhD from his alma mater. His hiring criteria is mostly based around IQ and how far people get in academia. I'm the only product engineer and will be for the next 6 months. That should be fine given our expected growth curve. But I keep getting reminded by how little the others know about product development. If OpenAI didn't have any founded team-members that were engineering-first then I could see it being a huge culture shift to give the engineers the reins over their most popular product.
Yep, posted too quickly without reading deeply enough and conflated the two issues. There are problems with ChatGPT consuming data and spitting it out to other users, but that is not what is being seen in this case, thank for the correction.
Yep, looks like a classic caching bug. Caching is critical but surprisingly hard to get right at scale. Valve did the same thing some years back and revealed customers billing information.
I wish the title were a little more clear as to the cause of the bug, but I guess that doesn't really matter for those affected.