Patrick Boyle (a finance youtuber who apparently has a day job in finance as well) discusses the recent hubbub about AIs escaping, forming a botnet, taking over the internet, etc. etc. therefore...
Patrick Boyle (a finance youtuber who apparently has a day job in finance as well) discusses the recent hubbub about AIs escaping, forming a botnet, taking over the internet, etc. etc. therefore the US government should regulate them into slowing down. Broadly, he's taking the cynical perspective that these flames are being fanned by execs attempting to position themselves better to profit from the situation, instead of taking their talking points at face value and assuming that four billionaires are pleading to the world to rein them in, out of a reverence for the human spirit that they recently discovered. Perhaps in a Grinch-esque Christmas stealing affair where their hearts grew several sizes overnight.
Full disclosure: I'm not a fan of Patrick Boyle, as he oversteps his knowledge areas a lot, but he's mostly summarizing articles written by other people in this case. It isn't a short video by any stretch of the imagination (~30 minutes), but it isn't dense enough to merit one's full attention, in case you're looking for something to leave on in the background.
Is there any actual evidence for the widespread assumption that the people making the warnings aren’t sincere? For some arguments against, see this article.
Is there any actual evidence for the widespread assumption that the people making the warnings aren’t sincere? For some arguments against, see this article.
Yea, I find it rather off putting that there is large group of people who seem rather quick to conclude that the people closest to this technology don't know it the best or can't be trusted to...
Yea, I find it rather off putting that there is large group of people who seem rather quick to conclude that the people closest to this technology don't know it the best or can't be trusted to give observations. It's very obviously not just CEOs or even limited to people who stand to gain monetarily. Not to say we should trust everything they say or anything but I mean, there are a lot of signals to ignore these days and it's getting harder to credibly ignore them.
I haven't watched the video but I will, I enjoy Boyle's videos when I've seen them surfaced. I think he's probably making an honest argument and I'm all for splashing some cold water sometimes. However, there is definitely an ongoing attempt to discredit AI safety at the moment, see e.g. the 'sex cult' stories or the broad stroke attempts to paint EAs and rationalists as poorly as possible.* If you want to start chasing money and power shaping a narrative, I would probably start there.
* And to be fair yes there are a lot of weird people who are easy targets out there but the venn diagram of AI safety people and LW-types is nowhere near a circle.
Oh, uh ... I've definitely run across some stuff where he's really off-base, and this might be one of them for some folks. I think his financial analyses are normally pretty good, for context, but...
I enjoy Boyle's videos when I've seen them surfaced.
Oh, uh ... I've definitely run across some stuff where he's really off-base, and this might be one of them for some folks. I think his financial analyses are normally pretty good, for context, but this is ... not that.
I'm not entirely convinced they do. Demonstrated by the lack of safeguards in the sandbox through which the hugging face hack occured. The whole story was wild in all the wrong ways IMHO. I would...
the people closest to this technology don't know it the best or can't be trusted to give observations.
I'm not entirely convinced they do. Demonstrated by the lack of safeguards in the sandbox through which the hugging face hack occured. The whole story was wild in all the wrong ways IMHO.
I would not be surprised if at this point, nearly every part of this is vibe coded, orchestrated by agents or researched by LLMs. Which gives me the impression that AI development is more akin to fueling a hydrothermal vent, hoping superintelligence pops out.
I agree with this, but I think the situation is nuanced. The public mouth pieces of the big AI companies (Altman, Amodei, various engineers and project leads) have an established history of...
Yea, I find it rather off putting that there is large group of people who seem rather quick to conclude that the people closest to this technology don't know it the best or can't be trusted to give observations
I agree with this, but I think the situation is nuanced. The public mouth pieces of the big AI companies (Altman, Amodei, various engineers and project leads) have an established history of predictions that turned out to be wrong, exaggerations and general hype.
It makes sense to question what they say, even if they genuinely believe it, it might still be wrong.
Personally I have yet to see anything close to the capability to be truly, autonomously, dangerous in the frontier models. They can do dangerous things when directed or misused or deployed badly, but so can a lot of technology.
The current wave of fear seems to be based in the idea that, any time now, they will achieve recursive self improvement and then they will make themselves dangerous really quickly. Except there's no evidence of recursive self improvement either. I mean, put an agent in charge of its own memory system and watch how long it takes before it writes bad judgement to the record and then compounds it by building more bad judgement on top, and and so on. If you give it enough time, it will make itself worse, not better.
That's of course not how recursive self improvement would work, instead it would be agents fully in charge of the training pipeline, but the point is that their judgement isn't good enough. They would make bad decisions, which would compound over time. They'd also make good ones, and maybe end up improving themselves, but not faster or cheaper than a human led process.
The doomsday fear all rests on the idea that this will change, and maybe it will, but we don't appear to be close to that yet.
I'm curious what sort of evidence could ever be fielded that would shine a light on the nature of one's heart -- and for what purpose? Many stories can be spun in two (or more), equally truthful,...
I'm curious what sort of evidence could ever be fielded that would shine a light on the nature of one's heart -- and for what purpose? Many stories can be spun in two (or more), equally truthful, directions. I truly do not care to understand the mind of a billionaire, so whether or not they "believe" something "sincerely" is neither here nor there. All that matters is predicting the course of the storm, not its take on AGI/ASI.
I'll dissect the article you linked tomorrow if that's OK, since it's playing some funny games with its rhetoric (it overstates an opponents position, defeats an unimportant detail of the opponent's argument, then presents a moderate position in response to their opponent's portrayed absurdity) and I'm a little too tired to explain how it's twisting reasoning in order to address a different argument while seeming to consider both sides.
I think the closest thing to a citation that the video references is David Sacks tweeting about the subject. He doesn't mention regulatory capture, just that (1) if these guys genuinely think that they're going to destroy the world, they can just stop, since (2) nothing the US government will stop the Chinese market anyways and thus we're really only talking about pacing US development of AI, and (3) positions it as forming a cartel rather than capturing a regulator. Though those two positions seem close-ish.
Patrick Boyle (a finance youtuber who apparently has a day job in finance as well) discusses the recent hubbub about AIs escaping, forming a botnet, taking over the internet, etc. etc. therefore the US government should regulate them into slowing down. Broadly, he's taking the cynical perspective that these flames are being fanned by execs attempting to position themselves better to profit from the situation, instead of taking their talking points at face value and assuming that four billionaires are pleading to the world to rein them in, out of a reverence for the human spirit that they recently discovered. Perhaps in a Grinch-esque Christmas stealing affair where their hearts grew several sizes overnight.
Full disclosure: I'm not a fan of Patrick Boyle, as he oversteps his knowledge areas a lot, but he's mostly summarizing articles written by other people in this case. It isn't a short video by any stretch of the imagination (~30 minutes), but it isn't dense enough to merit one's full attention, in case you're looking for something to leave on in the background.
Is there any actual evidence for the widespread assumption that the people making the warnings aren’t sincere? For some arguments against, see this article.
Yea, I find it rather off putting that there is large group of people who seem rather quick to conclude that the people closest to this technology don't know it the best or can't be trusted to give observations. It's very obviously not just CEOs or even limited to people who stand to gain monetarily. Not to say we should trust everything they say or anything but I mean, there are a lot of signals to ignore these days and it's getting harder to credibly ignore them.
I haven't watched the video but I will, I enjoy Boyle's videos when I've seen them surfaced. I think he's probably making an honest argument and I'm all for splashing some cold water sometimes. However, there is definitely an ongoing attempt to discredit AI safety at the moment, see e.g. the 'sex cult' stories or the broad stroke attempts to paint EAs and rationalists as poorly as possible.* If you want to start chasing money and power shaping a narrative, I would probably start there.
* And to be fair yes there are a lot of weird people who are easy targets out there but the venn diagram of AI safety people and LW-types is nowhere near a circle.
Oh, uh ... I've definitely run across some stuff where he's really off-base, and this might be one of them for some folks. I think his financial analyses are normally pretty good, for context, but this is ... not that.
I'm not entirely convinced they do. Demonstrated by the lack of safeguards in the sandbox through which the hugging face hack occured. The whole story was wild in all the wrong ways IMHO.
I would not be surprised if at this point, nearly every part of this is vibe coded, orchestrated by agents or researched by LLMs. Which gives me the impression that AI development is more akin to fueling a hydrothermal vent, hoping superintelligence pops out.
I agree with this, but I think the situation is nuanced. The public mouth pieces of the big AI companies (Altman, Amodei, various engineers and project leads) have an established history of predictions that turned out to be wrong, exaggerations and general hype.
It makes sense to question what they say, even if they genuinely believe it, it might still be wrong.
Personally I have yet to see anything close to the capability to be truly, autonomously, dangerous in the frontier models. They can do dangerous things when directed or misused or deployed badly, but so can a lot of technology.
The current wave of fear seems to be based in the idea that, any time now, they will achieve recursive self improvement and then they will make themselves dangerous really quickly. Except there's no evidence of recursive self improvement either. I mean, put an agent in charge of its own memory system and watch how long it takes before it writes bad judgement to the record and then compounds it by building more bad judgement on top, and and so on. If you give it enough time, it will make itself worse, not better.
That's of course not how recursive self improvement would work, instead it would be agents fully in charge of the training pipeline, but the point is that their judgement isn't good enough. They would make bad decisions, which would compound over time. They'd also make good ones, and maybe end up improving themselves, but not faster or cheaper than a human led process.
The doomsday fear all rests on the idea that this will change, and maybe it will, but we don't appear to be close to that yet.
It gets a lot of engagement though.
I'm curious what sort of evidence could ever be fielded that would shine a light on the nature of one's heart -- and for what purpose? Many stories can be spun in two (or more), equally truthful, directions. I truly do not care to understand the mind of a billionaire, so whether or not they "believe" something "sincerely" is neither here nor there. All that matters is predicting the course of the storm, not its take on AGI/ASI.
I'll dissect the article you linked tomorrow if that's OK, since it's playing some funny games with its rhetoric (it overstates an opponents position, defeats an unimportant detail of the opponent's argument, then presents a moderate position in response to their opponent's portrayed absurdity) and I'm a little too tired to explain how it's twisting reasoning in order to address a different argument while seeming to consider both sides.
I think the closest thing to a citation that the video references is David Sacks tweeting about the subject. He doesn't mention regulatory capture, just that (1) if these guys genuinely think that they're going to destroy the world, they can just stop, since (2) nothing the US government will stop the Chinese market anyways and thus we're really only talking about pacing US development of AI, and (3) positions it as forming a cartel rather than capturing a regulator. Though those two positions seem close-ish.