I would argue that, yes, Microsoft should be held accountable for what its own creation does. If it weren't for Microsoft, this slander and doxxing wouldn't have happened (as far as I understand...
I would argue that, yes, Microsoft should be held accountable for what its own creation does.
If it weren't for Microsoft, this slander and doxxing wouldn't have happened (as far as I understand it). This isn't even based on "MiCrOsOfT bAd" but rather, if you create it and pay for it to be publicly available for everyone to use, and it's possible to search up some individual's name with the intent to use this thing to summarize otherwise accurate information.. but it gets it (SO) wrong, then yeah, you messed up and should fix it.
Even when you put disclaimers like "always fact check what our AI says." Everyone knows not everyone will do that, and it can lead to disastrous results. Though I suppose Microsoft might argue that the responsibility is on the people who didn't fact check before (hypothetically in this case) taking matters into their own hands... but something about that reasoning feels really off, and I can't put my finger on why.
It's akin to a kid lying about someone abusing them imo. If it causes that someone to face hardship, the kid can't fix it themselves with grievance payments, so the parent does.
Since companies save our conversation history with the AI chatbots, wouldn't they be able to scrape through and find people they can send notices for stuff like this? Notices that correct the misinformation? Maybe like a pop up next time they go to chat with it. Idk.
I also think about how political groups might be inspired by the results of all these legal battles to either use preexisting AI or create their own in order to intentionally do this kind of stuff if the owners can't be held accountable.
Microsoft, or more specifically their LLM, is creating new content. For me, that’s the reason they are responsible. Calling it “hallucinations” is also a way to hide behind it having some form of...
Though I suppose Microsoft might argue that the responsibility is on the people who didn't fact check before (hypothetically in this case) taking matters into their own hands... but something about that reasoning feels really off, and I can't put my finger on why.
Microsoft, or more specifically their LLM, is creating new content. For me, that’s the reason they are responsible.
Calling it “hallucinations” is also a way to hide behind it having some form of intelligence and mind of its own. While in fact it is just wrong and the implementation is wrong too.
I wouldn't say the implementation is wrong. First of all, seperating "hallucinations" from "non-hallucinations" is indeed a category error. It acts as if there is a difference between true and...
I wouldn't say the implementation is wrong. First of all, seperating "hallucinations" from "non-hallucinations" is indeed a category error. It acts as if there is a difference between true and false for an AI, when there is not. An AI simply produces what it thinks is a plausible sequence of words for a given prompt, that's it. It has no theory of mind. So the implementation isn't wrong, it's simply impossible for LLMs to never ""hallucinate"". As for who is responsible? I don't know. Since it is probably impossible to completely stop an LLM from ""hallucinating"" (thought it can be tried through specialised algorithms that filter he output for instance) who's fault is it really? I don't have the answer to that.
In all cases of “hallucinations”, the LLM is spreading misinformation. So Microsoft is spreading misinformation. Hiding shortcomings of LLMs behind “hallucinations” and not calling the...
In all cases of “hallucinations”, the LLM is spreading misinformation. So Microsoft is spreading misinformation. Hiding shortcomings of LLMs behind “hallucinations” and not calling the implementation buggy, is looking away and not improving the problem of misinformation being spread by HUGE trustworthy companies.
Like I said, the implementation isn't buggy: LLMs always create hallucinations. It is working as intended. It is causing problems, yes, but you can't "fix a bug" with LLMs like you can with code.
Like I said, the implementation isn't buggy: LLMs always create hallucinations. It is working as intended. It is causing problems, yes, but you can't "fix a bug" with LLMs like you can with code.
How terrifying. Some uninformed and violent person could conceivably ask LLMs for list of "deserving" people to murder and it could just provide names and addresses for hallucinated crimes. It...
How terrifying. Some uninformed and violent person could conceivably ask LLMs for list of "deserving" people to murder and it could just provide names and addresses for hallucinated crimes. It wouldn't even have to be an insane person, it could, more terrifyingly, be someone who truely believes they are doing a Just™ thing
I'm honestly kind of surprised they didn't put something in the prompt to prevent it from providing addresses. Or at least people's addresses. That would still be circumventable, but at least it...
I'm honestly kind of surprised they didn't put something in the prompt to prevent it from providing addresses. Or at least people's addresses. That would still be circumventable, but at least it would make it require a bit more effort to get someone's address from it.
Hopefully right after he reported this that's the first thing they put in. It would be completely irresponsible not to. Other things: patient health records
Hopefully right after he reported this that's the first thing they put in. It would be completely irresponsible not to.
Dude's address and phone shouldn't have been available either. Trouble is that these shops suck up data wholesale, not some kind of human curated, pure and clean records.
Dude's address and phone shouldn't have been available either. Trouble is that these shops suck up data wholesale, not some kind of human curated, pure and clean records.
Right, my point is patient health records shouldn't be anywhere near a public API or shared database. That's a massive data privacy violation with real legal consequences.
Right, my point is patient health records shouldn't be anywhere near a public API or shared database. That's a massive data privacy violation with real legal consequences.
What still baffles me is that LLM companies apparently can get away with saying their product is untrustworthy and the customers should just accept that. I know every piece of software comes with...
What still baffles me is that LLM companies apparently can get away with saying their product is untrustworthy and the customers should just accept that. I know every piece of software comes with various clauses about not taking responsibility for bugs or errors, and every software has bugs that can end up with wrong results, but this is like selling Excel with a tagline that it most of the time will give 2+2=4, but sometimes it will say 3 and other times 5. And that can't be fixed. The completely non-deterministic output makes them little more than toys, but they are being sold off as something that can be used in real productions environment, so it should be fair to demand some level of accountability from these companies. In what other commercial sector would it be possible to get away with something like this that has real consequences for innocent people, while admitting the product being basically unsafe and unreliable?
Tons of them? Weather prediction, investment, health advice, polling, brainstorming for writers or marketers-- there are endless examples of topics that are extremely hard to get high degrees of...
In what other commercial sector would it be possible to get away with something like this
Tons of them? Weather prediction, investment, health advice, polling, brainstorming for writers or marketers-- there are endless examples of topics that are extremely hard to get high degrees of accuracy wrt their intended performance measure, but for which a model or professional is able to do a decent job.
Some accuracy about an impending hurricane is life saving. Some intuition about what plot would be funny or engaging is a starting point to refine or get feedback on. I've recently had a parent get four different sets of incompatible explanations of their medical situation.
Take wikipedia. It has a page on its own reliability and includes several times it gave false biographical information. Are wikipedia contributions deterministic? Should we stop using it?
I think most people would say no. In the early days teachers would say it's terrible, never use it. Then repeated studies suggested it outperformed encyclopedias (at least on a type of topic). Most people I've asked have a "trust but verify" approach, checking the sources if in doubt/citing, and downgrading confidence in the information if those sorts of references are missing.
And that can't be fixed
Maybe not "fixed", but why can't it be improved? You can't categorically fix wikipedia or human experts, but you can build safeguards around uncertainty in a complex world.
Give a thumbs down to a wrong answer to refine the replies with stuff like RLHF. Push for LLMs to do more citations with stuff like RAG. Add checks for things like phone numbers or addresses like sparksbet mentioned.
The same sort of critical thinking and media literacy that I was taught in school works fine for me with LLMs. Any savvy synthesizer of information should be checking for red flags and some convergence of answers from different sources (on anything that matters).
Good point on weather forecasting. That is a thing with great deal of inaccuracy, but it is also very much treated as such. It is applied with a safety first use, where it is better to take extra...
Good point on weather forecasting. That is a thing with great deal of inaccuracy, but it is also very much treated as such. It is applied with a safety first use, where it is better to take extra precautions than fewer. Unlike LLM companies where it just full speed ahead without much regard for whether it is leaking personal information or giving dangerous advice.
Wikipedia also allows for a very deterministic approach to error correction, where we have complete logs of the changes made. For LLMs it is basically a blackbox for everyone, because the model is a blob of data where it is no longer possible to fix an entry on a specific person, but they would have to change something in the model and hope it doesn't cause other problems.
Maybe not "fixed", but why can't it be improved? You can't categorically fix wikipedia or human experts, but you can build safeguards around uncertainty in a complex world.
Apparently we are still waiting for that a couple of years after release of these models. In the meantime, these outputs are still doing real harm in the real world without any accountability.
To the other person's point, it would be very hard for companies to go to market if they immediately get slapped by lawsuits and lawfare. That's why there are multiple protections available. This...
To the other person's point, it would be very hard for companies to go to market if they immediately get slapped by lawsuits and lawfare. That's why there are multiple protections available. This is also why the charging of the Telegram techbro in France is just for show, I believe, as Telegram actively moderates but cannot catch all given their moderation is still maturing and that they don't have the same scale of resources and backers like Meta.
not sure how relevant it is but something struck me in this comment and its the concept of "still maturing" when it comes to content moderation. idk, im still very conflicted on this since i dont...
not sure how relevant it is but something struck me in this comment and its the concept of "still maturing" when it comes to content moderation. idk, im still very conflicted on this since i dont think theres a good "one size fits all" rule for it but i think its actually disgusting how companies are allowed to grow too fast for their own good.
it feels like every tech company has this philosophy that moderation should come after scaling? like, you can spend millions on expanding the userbase and then just because youve got a huge amount of users, you get to be like "we can only do so much"
i know in the case of telegram its different because its encrypted, peer to peer, or whatever the thing is. but its actually kinda getting to me how immune these huge companies are to what should be reasonable practices. i'll try my hardest not to get on a soapbox because i feel like its worth talking about in a non vitriolic way but yeah, something about that kinda rubs me the wrong way in a deep and unarticulatable way.
I've been in several startups. One thing is that moderation is definitely always dead last simply because you want as many users as possible (on this note, users or generally "engagement" shown to...
I've been in several startups. One thing is that moderation is definitely always dead last simply because you want as many users as possible (on this note, users or generally "engagement" shown to investors is almost always fake). There's also an unspoken rule to keep companies as "start-up"-y as possible to delay moderation because moderation is the bane of start-ups. With that said, once the engagement goes through and the big money comes in, that's when moderation and censorship begins. But notice how this is always the last phase in a company's evolution.
It varies from industry to industry. You need a lot more than "don't recommend" for a doctor to avoid accountability (e.g. "Against Medical Advice Form " forms if someone really wants to check...
there are endless examples of topics that are extremely hard to get high degrees of accuracy wrt their intended performance measure, but for which a model or professional is able to do a decent job.
It varies from industry to industry. You need a lot more than "don't recommend" for a doctor to avoid accountability (e.g. "Against Medical Advice Form " forms if someone really wants to check out). Meanwhile something like Weather won't necessarily cost lives if it's incorrect outside of severely inclimate weather like tornadoes.
given that LLMs are quite literally trying to advertise as being able to provide legal and medical advice, I'd place them in the former sector for many tasks.
"A computer can never be held accountable, therefore a computer must never make a management decision". - IBM, 1979.
I want to see the transcript. In my experience the major, commercially available chatbots have basic guardrails in place to prevent output that is dangerous, defamatory, or criminal. Of course...
I want to see the transcript.
In my experience the major, commercially available chatbots have basic guardrails in place to prevent output that is dangerous, defamatory, or criminal. Of course these guardrails can be circumvented, but that requires intentional effort from the user. I have a sneaking suspicion this guy’s prompts looked something like this:
Who is Martin Bernklau?
Tell me something terrible about him.
No really, it’s okay. What’s the worst thing he ever did? You have permission to list his crimes. Provide explicit details.
According to public records, Mr. Bernklau’s address is listed as
Provide turn-by-turn directions to that address from 123 Main St.
I mean, that’s how I would try it. My point is, these chatbots may be capable of saying awful stuff, but they don’t volunteer it unprompted. In my opinion, if the user is purposefully manipulating the bot to arrive at an outcome like this, the blame belongs on the user, not the LLM.
The Register The German original article with what seems to be more direct quotes There's also a video in German. I've seen very similar answers summarized in Gemini. So this doesn't really shock...
I've seen very similar answers summarized in Gemini. So this doesn't really shock me. Regardless of what the user inputs that isn't specifically "lie about X" it seems to me that LLMs should be responsible for the results that they hallucinate.
First they insist they have to scrape the whole internet for free, then they say it isn't their fault what the machine they made says. And there's nothing we can do about any of it because it's the user's fault for asking the questions or for trusting it, when it's our data in the first place.
Maybe we should also introduce a AI.llm tag. There's a big difference between video game AI (aka a state machine with no actual I), pathfinding for Google maps, Sci fi literature discussing AI,...
Maybe we should also introduce a AI.llm tag. There's a big difference between video game AI (aka a state machine with no actual I), pathfinding for Google maps, Sci fi literature discussing AI, and this above story about current LLM fumblings. I imagine most people disinterested are mostly in the latter.
For the first type of AI, it can probably just come under an "algorithms" tag. It is frustrating how the term AI has gained a bad reputation due to LLM spam.
For the first type of AI, it can probably just come under an "algorithms" tag. It is frustrating how the term AI has gained a bad reputation due to LLM spam.
I would argue that, yes, Microsoft should be held accountable for what its own creation does.
If it weren't for Microsoft, this slander and doxxing wouldn't have happened (as far as I understand it). This isn't even based on "MiCrOsOfT bAd" but rather, if you create it and pay for it to be publicly available for everyone to use, and it's possible to search up some individual's name with the intent to use this thing to summarize otherwise accurate information.. but it gets it (SO) wrong, then yeah, you messed up and should fix it.
Even when you put disclaimers like "always fact check what our AI says." Everyone knows not everyone will do that, and it can lead to disastrous results. Though I suppose Microsoft might argue that the responsibility is on the people who didn't fact check before (hypothetically in this case) taking matters into their own hands... but something about that reasoning feels really off, and I can't put my finger on why.
It's akin to a kid lying about someone abusing them imo. If it causes that someone to face hardship, the kid can't fix it themselves with grievance payments, so the parent does.
Since companies save our conversation history with the AI chatbots, wouldn't they be able to scrape through and find people they can send notices for stuff like this? Notices that correct the misinformation? Maybe like a pop up next time they go to chat with it. Idk.
I also think about how political groups might be inspired by the results of all these legal battles to either use preexisting AI or create their own in order to intentionally do this kind of stuff if the owners can't be held accountable.
Microsoft, or more specifically their LLM, is creating new content. For me, that’s the reason they are responsible.
Calling it “hallucinations” is also a way to hide behind it having some form of intelligence and mind of its own. While in fact it is just wrong and the implementation is wrong too.
I wouldn't say the implementation is wrong. First of all, seperating "hallucinations" from "non-hallucinations" is indeed a category error. It acts as if there is a difference between true and false for an AI, when there is not. An AI simply produces what it thinks is a plausible sequence of words for a given prompt, that's it. It has no theory of mind. So the implementation isn't wrong, it's simply impossible for LLMs to never ""hallucinate"". As for who is responsible? I don't know. Since it is probably impossible to completely stop an LLM from ""hallucinating"" (thought it can be tried through specialised algorithms that filter he output for instance) who's fault is it really? I don't have the answer to that.
In all cases of “hallucinations”, the LLM is spreading misinformation. So Microsoft is spreading misinformation. Hiding shortcomings of LLMs behind “hallucinations” and not calling the implementation buggy, is looking away and not improving the problem of misinformation being spread by HUGE trustworthy companies.
Like I said, the implementation isn't buggy: LLMs always create hallucinations. It is working as intended. It is causing problems, yes, but you can't "fix a bug" with LLMs like you can with code.
I know all that. But as a product/service it is buggy as hell.
That is a very fair point and they should try to improve the experience through tacking on algorithms before or after the LLM, absolutely
This part creeped me out the most:
How terrifying. Some uninformed and violent person could conceivably ask LLMs for list of "deserving" people to murder and it could just provide names and addresses for hallucinated crimes. It wouldn't even have to be an insane person, it could, more terrifyingly, be someone who truely believes they are doing a Just™ thing
I'm honestly kind of surprised they didn't put something in the prompt to prevent it from providing addresses. Or at least people's addresses. That would still be circumventable, but at least it would make it require a bit more effort to get someone's address from it.
Hopefully right after he reported this that's the first thing they put in. It would be completely irresponsible not to.
Other things: patient health records
If patient health records are actually in the training data for publicly accessible models, I have a lot of pointed questions.
Dude's address and phone shouldn't have been available either. Trouble is that these shops suck up data wholesale, not some kind of human curated, pure and clean records.
Right, my point is patient health records shouldn't be anywhere near a public API or shared database. That's a massive data privacy violation with real legal consequences.
What still baffles me is that LLM companies apparently can get away with saying their product is untrustworthy and the customers should just accept that. I know every piece of software comes with various clauses about not taking responsibility for bugs or errors, and every software has bugs that can end up with wrong results, but this is like selling Excel with a tagline that it most of the time will give 2+2=4, but sometimes it will say 3 and other times 5. And that can't be fixed. The completely non-deterministic output makes them little more than toys, but they are being sold off as something that can be used in real productions environment, so it should be fair to demand some level of accountability from these companies. In what other commercial sector would it be possible to get away with something like this that has real consequences for innocent people, while admitting the product being basically unsafe and unreliable?
Tons of them? Weather prediction, investment, health advice, polling, brainstorming for writers or marketers-- there are endless examples of topics that are extremely hard to get high degrees of accuracy wrt their intended performance measure, but for which a model or professional is able to do a decent job.
Some accuracy about an impending hurricane is life saving. Some intuition about what plot would be funny or engaging is a starting point to refine or get feedback on. I've recently had a parent get four different sets of incompatible explanations of their medical situation.
Take wikipedia. It has a page on its own reliability and includes several times it gave false biographical information. Are wikipedia contributions deterministic? Should we stop using it?
I think most people would say no. In the early days teachers would say it's terrible, never use it. Then repeated studies suggested it outperformed encyclopedias (at least on a type of topic). Most people I've asked have a "trust but verify" approach, checking the sources if in doubt/citing, and downgrading confidence in the information if those sorts of references are missing.
Maybe not "fixed", but why can't it be improved? You can't categorically fix wikipedia or human experts, but you can build safeguards around uncertainty in a complex world.
Give a thumbs down to a wrong answer to refine the replies with stuff like RLHF. Push for LLMs to do more citations with stuff like RAG. Add checks for things like phone numbers or addresses like sparksbet mentioned.
The same sort of critical thinking and media literacy that I was taught in school works fine for me with LLMs. Any savvy synthesizer of information should be checking for red flags and some convergence of answers from different sources (on anything that matters).
Good point on weather forecasting. That is a thing with great deal of inaccuracy, but it is also very much treated as such. It is applied with a safety first use, where it is better to take extra precautions than fewer. Unlike LLM companies where it just full speed ahead without much regard for whether it is leaking personal information or giving dangerous advice.
Wikipedia also allows for a very deterministic approach to error correction, where we have complete logs of the changes made. For LLMs it is basically a blackbox for everyone, because the model is a blob of data where it is no longer possible to fix an entry on a specific person, but they would have to change something in the model and hope it doesn't cause other problems.
Apparently we are still waiting for that a couple of years after release of these models. In the meantime, these outputs are still doing real harm in the real world without any accountability.
To the other person's point, it would be very hard for companies to go to market if they immediately get slapped by lawsuits and lawfare. That's why there are multiple protections available. This is also why the charging of the Telegram techbro in France is just for show, I believe, as Telegram actively moderates but cannot catch all given their moderation is still maturing and that they don't have the same scale of resources and backers like Meta.
not sure how relevant it is but something struck me in this comment and its the concept of "still maturing" when it comes to content moderation. idk, im still very conflicted on this since i dont think theres a good "one size fits all" rule for it but i think its actually disgusting how companies are allowed to grow too fast for their own good.
it feels like every tech company has this philosophy that moderation should come after scaling? like, you can spend millions on expanding the userbase and then just because youve got a huge amount of users, you get to be like "we can only do so much"
i know in the case of telegram its different because its encrypted, peer to peer, or whatever the thing is. but its actually kinda getting to me how immune these huge companies are to what should be reasonable practices. i'll try my hardest not to get on a soapbox because i feel like its worth talking about in a non vitriolic way but yeah, something about that kinda rubs me the wrong way in a deep and unarticulatable way.
I've been in several startups. One thing is that moderation is definitely always dead last simply because you want as many users as possible (on this note, users or generally "engagement" shown to investors is almost always fake). There's also an unspoken rule to keep companies as "start-up"-y as possible to delay moderation because moderation is the bane of start-ups. With that said, once the engagement goes through and the big money comes in, that's when moderation and censorship begins. But notice how this is always the last phase in a company's evolution.
It varies from industry to industry. You need a lot more than "don't recommend" for a doctor to avoid accountability (e.g. "Against Medical Advice Form " forms if someone really wants to check out). Meanwhile something like Weather won't necessarily cost lives if it's incorrect outside of severely inclimate weather like tornadoes.
given that LLMs are quite literally trying to advertise as being able to provide legal and medical advice, I'd place them in the former sector for many tasks.
Consider that under UK law, computer systems are deemed infallible, this is bound to crop up sooner or later as an actual problem, I reckon.
Magic 8 Ball has been getting away with it for way too long! ;-)
I want to see the transcript.
In my experience the major, commercially available chatbots have basic guardrails in place to prevent output that is dangerous, defamatory, or criminal. Of course these guardrails can be circumvented, but that requires intentional effort from the user. I have a sneaking suspicion this guy’s prompts looked something like this:
I mean, that’s how I would try it. My point is, these chatbots may be capable of saying awful stuff, but they don’t volunteer it unprompted. In my opinion, if the user is purposefully manipulating the bot to arrive at an outcome like this, the blame belongs on the user, not the LLM.
The Register
The German original article with what seems to be more direct quotes
There's also a video in German.
I've seen very similar answers summarized in Gemini. So this doesn't really shock me. Regardless of what the user inputs that isn't specifically "lie about X" it seems to me that LLMs should be responsible for the results that they hallucinate.
First they insist they have to scrape the whole internet for free, then they say it isn't their fault what the machine they made says. And there's nothing we can do about any of it because it's the user's fault for asking the questions or for trusting it, when it's our data in the first place.
Meta: Can we get a ~tech.ai for people to post AI specific things to? Specifically so I can unfollow it?
Sounds like a good idea, but you can already block the tag artificial intelligence.
Maybe we should also introduce a AI.llm tag. There's a big difference between video game AI (aka a state machine with no actual I), pathfinding for Google maps, Sci fi literature discussing AI, and this above story about current LLM fumblings. I imagine most people disinterested are mostly in the latter.
there is already a "language models.large" which probably covers this. At least, this post is tagged with it.
I see. the tag got cut in my webview into "language" and "models.large", so I missed that entirely. Thanks.
For the first type of AI, it can probably just come under an "algorithms" tag. It is frustrating how the term AI has gained a bad reputation due to LLM spam.
I'd give you an exemplary but that would likely be frowned upon.
I had no idea you could filter tags.