This is an interesting argument. Since you asked for new perspectives, I would argue that the biggest issue with this piece is that it is maximally positive on human efficacy and maximally...
This is an interesting argument. Since you asked for new perspectives, I would argue that the biggest issue with this piece is that it is maximally positive on human efficacy and maximally negative on AI efficacy. Under those bounds, of course implementing more AI is going to look like a bad idea. But reality is more complicated.
Here's what I mean. The author cites the current accuracy of pilots (>99.999%) to demonstrate the accuracy of human-run systems and then writes, in the context of government services,
Even if the system is designed generously and cautiously, you're still replacing an accountable human decision with an automated system. Is it worth it if it's faster for the average user, but some percentage of people end up having to laboriously appeal a bad LLM decision? Again: thinking this works requires making huge assumptions about improvements in accuracy.
But government services aren't currently at 99.999% accuracy. Right now, more than 10% of Medicare claims are appealed and more than 80% of those appeals are accepted. That is abysmal in every direction and makes any claim that AI agents can only worsen the system immediately suspect. The whole piece is kind of like that.
And I agree with you re: maliciousness. LLMs are demonstrably less likely to be malicious than humans.
to add to that, pilots don't actually do very much piloting anymore. and the other example, nurse, was very funny to me. I love all nurses but I really wouldn't rate their accuracy as anywhere...
accuracy of pilots (>99.999%)
to add to that, pilots don't actually do very much piloting anymore.
and the other example, nurse, was very funny to me. I love all nurses but I really wouldn't rate their accuracy as anywhere close to 99%. We just don't advertise that to the patient, it would scare them off...
I would rephrase this to say that LLMs are capable of being demonstrably less malicious than humans. My expectation is that as we refine and polish the technology, we will also refine and polish...
LLMs are demonstrably less likely to be malicious than humans.
I would rephrase this to say that LLMs are capable of being demonstrably less malicious than humans. My expectation is that as we refine and polish the technology, we will also refine and polish how we can weaponize it for purposes of selling things and controlling information.
I think we're in the "build trust" phase of a new industry. I don't think it's long before LLMs are asked to misrepresent undesirable facts or serve as incognito sales consultants.
This blog post ( not mine ) expresses that even assuming LLM do become reliable enough, both side of any dealings will be using them (customer and companies, administration and citizens). As LLM...
This blog post ( not mine ) expresses that even assuming LLM do become reliable enough, both side of any dealings will be using them (customer and companies, administration and citizens).
As LLM are choosen to act in favor of their respective user only, this would encourage all dealings to be maximally malicious in a race to the bottom. And using LLM will become necessary for no (or even negative) utility compared to when no LLM existed.
This sounds somewhat convincing to me, but appart from some "magical" human touch, whatever stopped humans from being maximally malicious so far could very well also apply to our LLM? Couldn't we see some kind of reputation system emerge to be wary of agents working for humans known to employ unusually "malicious" agent?
I'm hoping to learn some new perspective here.
... proceeds to predict what the AI traffic jam will look like. If you have ever tried to contest a parking ticket, you will know it is already "a traffic jam." The first request is always...
It is easy to predict an automobile in 1880; it is very hard to predict a traffic problem.
... proceeds to predict what the AI traffic jam will look like.
If you have ever tried to contest a parking ticket, you will know it is already "a traffic jam." The first request is always rejected with a form letter. You have to appeal, and sometimes put some skin in the game with a payment, in order to get an actual genuine human to look at your appeal. If governments wanted to reject unemployment claims, they just would. The process is hard because it's a government process. Even their procurement processes are nightmarish.
Support organizations aren't rolling out agents to fuck with their customers. They are trying to minimize cost, while still providing some level of customer support.
The most likely issues with Gen AI will become us becoming more stupid, less able to think critically, and more awash with advertising and spam on the internet.
It's way to soon to predict what issues Agentic AI will produce, except perhaps a bunch of unemployed programmers, artists, and maybe business folks.
I think it’s quite hard to “imagine the traffic jam” accurately and in enough detail to do much in the way of planning, even if you’re pretty sure there will be traffic jams in some vague sense....
I think it’s quite hard to “imagine the traffic jam” accurately and in enough detail to do much in the way of planning, even if you’re pretty sure there will be traffic jams in some vague sense.
But it does seems likely that our AI minions will largely try to do what they’ve been told to do. They don’t have enough context to know when to act against their owners. At best you can build in some ethical rules about things the AI should never do, for anyone. But even these could probably be overridden by giving the AI some misleading context.
So maybe we will end up with a system where everyone has their own AI lawyer advocating for their interests? And we’ll take for granted that this is an adversarial process.
Perhaps in such a world, the reliable reporting of verifiable facts becomes more important? We take it for granted now, but the overwhelming success of Wikipedia was surprising at the time. I wonder what the equivalent will be in this new era?
I would not be surprised if the Internet becomes a far less valuable place to run operations like gov. benefits after LLMs. Theoretically, paying to open up in-person benefit centers to take...
Right now every open service on the internet is attack surface for loosely-directed agents noodling around. What happens when large numbers of people spin up an LLM agent and tell it to "try to get me as many government benefits as you can"?
I would not be surprised if the Internet becomes a far less valuable place to run operations like gov. benefits after LLMs. Theoretically, paying to open up in-person benefit centers to take requests would be cheaper and more efficient if you assume a vast majority of online requests are malicious. I’m sure there will be other solutions, too (like going in-person to submit the initial request and then managing the benefits online, or comparing request IP addresses to application details as an initial filter), but I’m not sure what the tradeoffs would be.
Online government systems are capable of requiring you to submit documentation in advance that tends to verify you're a real person. Indeed, if they do not already require this, you don't require...
Online government systems are capable of requiring you to submit documentation in advance that tends to verify you're a real person. Indeed, if they do not already require this, you don't require a bot remotely as sophisticated as an LLM to send malicious requests. For instance, until they updated the system here in Berlin, it was pretty much impossible to get an appointment at the Foreigner's Authority without buying one from a scalper. Even now getting appointments for some government services basically necessitates a web scraper because they get taken so fast. But none of that really requires an LLM, and your system would have to be really dysfunctional and incredibly easy to exploit with simpler tools if it can't identify an LLM trying to fake official documents like a passport or other official ID.
This is an interesting argument. Since you asked for new perspectives, I would argue that the biggest issue with this piece is that it is maximally positive on human efficacy and maximally negative on AI efficacy. Under those bounds, of course implementing more AI is going to look like a bad idea. But reality is more complicated.
Here's what I mean. The author cites the current accuracy of pilots (>99.999%) to demonstrate the accuracy of human-run systems and then writes, in the context of government services,
But government services aren't currently at 99.999% accuracy. Right now, more than 10% of Medicare claims are appealed and more than 80% of those appeals are accepted. That is abysmal in every direction and makes any claim that AI agents can only worsen the system immediately suspect. The whole piece is kind of like that.
And I agree with you re: maliciousness. LLMs are demonstrably less likely to be malicious than humans.
to add to that, pilots don't actually do very much piloting anymore.
and the other example, nurse, was very funny to me. I love all nurses but I really wouldn't rate their accuracy as anywhere close to 99%. We just don't advertise that to the patient, it would scare them off...
I would rephrase this to say that LLMs are capable of being demonstrably less malicious than humans. My expectation is that as we refine and polish the technology, we will also refine and polish how we can weaponize it for purposes of selling things and controlling information.
I think we're in the "build trust" phase of a new industry. I don't think it's long before LLMs are asked to misrepresent undesirable facts or serve as incognito sales consultants.
This blog post ( not mine ) expresses that even assuming LLM do become reliable enough, both side of any dealings will be using them (customer and companies, administration and citizens).
As LLM are choosen to act in favor of their respective user only, this would encourage all dealings to be maximally malicious in a race to the bottom. And using LLM will become necessary for no (or even negative) utility compared to when no LLM existed.
This sounds somewhat convincing to me, but appart from some "magical" human touch, whatever stopped humans from being maximally malicious so far could very well also apply to our LLM? Couldn't we see some kind of reputation system emerge to be wary of agents working for humans known to employ unusually "malicious" agent?
I'm hoping to learn some new perspective here.
... proceeds to predict what the AI traffic jam will look like.
If you have ever tried to contest a parking ticket, you will know it is already "a traffic jam." The first request is always rejected with a form letter. You have to appeal, and sometimes put some skin in the game with a payment, in order to get an actual genuine human to look at your appeal. If governments wanted to reject unemployment claims, they just would. The process is hard because it's a government process. Even their procurement processes are nightmarish.
Support organizations aren't rolling out agents to fuck with their customers. They are trying to minimize cost, while still providing some level of customer support.
The most likely issues with Gen AI will become us becoming more stupid, less able to think critically, and more awash with advertising and spam on the internet.
It's way to soon to predict what issues Agentic AI will produce, except perhaps a bunch of unemployed programmers, artists, and maybe business folks.
I think it’s quite hard to “imagine the traffic jam” accurately and in enough detail to do much in the way of planning, even if you’re pretty sure there will be traffic jams in some vague sense.
But it does seems likely that our AI minions will largely try to do what they’ve been told to do. They don’t have enough context to know when to act against their owners. At best you can build in some ethical rules about things the AI should never do, for anyone. But even these could probably be overridden by giving the AI some misleading context.
So maybe we will end up with a system where everyone has their own AI lawyer advocating for their interests? And we’ll take for granted that this is an adversarial process.
Perhaps in such a world, the reliable reporting of verifiable facts becomes more important? We take it for granted now, but the overwhelming success of Wikipedia was surprising at the time. I wonder what the equivalent will be in this new era?
I would not be surprised if the Internet becomes a far less valuable place to run operations like gov. benefits after LLMs. Theoretically, paying to open up in-person benefit centers to take requests would be cheaper and more efficient if you assume a vast majority of online requests are malicious. I’m sure there will be other solutions, too (like going in-person to submit the initial request and then managing the benefits online, or comparing request IP addresses to application details as an initial filter), but I’m not sure what the tradeoffs would be.
Online government systems are capable of requiring you to submit documentation in advance that tends to verify you're a real person. Indeed, if they do not already require this, you don't require a bot remotely as sophisticated as an LLM to send malicious requests. For instance, until they updated the system here in Berlin, it was pretty much impossible to get an appointment at the Foreigner's Authority without buying one from a scalper. Even now getting appointments for some government services basically necessitates a web scraper because they get taken so fast. But none of that really requires an LLM, and your system would have to be really dysfunctional and incredibly easy to exploit with simpler tools if it can't identify an LLM trying to fake official documents like a passport or other official ID.