Where do we go from here? (Regarding AI)
TW: AI discussion.
Recently, things have been getting rather crazy with AI to the point that I think many are being forced to reckon with its potential. It has been demonstrated that this technology is capable of solving things like millennium math problems, which has drawn response from many fields medalists. There have been very public instances of big tech losing control of their experiments. Many top researchers are expressing discomfort. Now, of course you could dismiss this all as marketing but I personally think that would be a bit of wishful thinking.
I would like to have an open discussion about the current moment in time. Therefore, I ask you, people of tildes, where do you think we are and where do we go from here? Please feel free to pull out your crystal ball and speculate wildly. Or, please feel free to call it all bunk. I just want to hear your opinion.
- Do you feel concerned about AI or are you still unconvinced?
- Do you think AI will take your job? Why or why not?
- Are you considering a career change or any other big life decisions because of the changing world?
- How are you advising your children, regarding schooling and careers?
- Do you have a subscription to an AI service? If you have one, how do you feel about it?
- Do you think that this technology will be a net positive or a net negative?
- How do you think the human race will change, if at all?
- Do you think we will see a serious incident driven by AI in the near future?
- Does it bother you that humans may not be the dominant intelligence at some point? Do you think this is even possible?
- Does any of this affect your sense of identity or individuality? How do you feel about your skill set and hobbies?
- What do you think it means to be human in a machine dominated world?
- Overall, are you feeling optimistic or pessimistic?
For me personally, I have shared my opinion many times before. It is a rather pessimistic one.
I recognize that this is a heated topic, and I would like to request that even if you disagree with someone, that we treat each other with respect and kindness. Thanks.
Imo, calling LLM-tooling “AI” is the best marketing strategy these companies had, and also terrible for how we have conversations about these tools.
When you frame them as “AI” and give them names and put them behind a messaging-like interface, it’s easy to anthropomorphize these tools. But they aren’t doing “intelligence”, they’re doing advanced statistics.
The tools are not “going rogue” and these companies are not “losing control”. Those stories about LLMs “hacking” other companies are a result of having no guardrails, infinite compute, no sandboxing, and unclear goals / instructions. If you prompt an LLM with an objective of retrieving a file from a server and give it unlimited resources and a shitty sandbox, of course one of the methods it might generate could be, essentially, hacking. And if you blindly trust the output of LLMs without supervision, you’ll end up doing hacking.
If these companies or their employees were serious about whatever fears they claim to have, they would be ceasing all operations and doing everything in their power to make everyone else cease operations. Since they aren’t, the only explanations are that they either don’t care that “AI is going to kill us all” OR that they don’t believe it and are saying that as a fear-based marketing strategy. Both, morally reprehensible, imo.
So…no, I am not particularly scared of the impact these technologies are going to have.
I am worried about the commitments by large tech companies to this technology. So far, it seems many companies are taking on lots of debt to support the buildout of the LLM companies despite, from what we know, the LLM companies are deeply unprofitable. Which means prices will need to rise or the tools will need to go away. Additionally, a lot of companies are seemingly very willing to hand over process knowledge to these LLM companies and get rid of or de-skill their own employees. Which again…seems shortsighted if those LLM providers disappear or need to raise prices.
I am also worried about the language we use to describe these tools. We don’t really have the words to NOT anthropomorphize them, and I think that is a huge driving factor in why so many folks get very…irrational about these products.
Edit: Also worried about the way these tools and their users might affect the social contract we all have. Many people seem very okay with saying that material output by these LLM tools is something “they made”. I generally feel the same way about this as I do with plagiarism. Which is not very positive. If someone doesn’t care enough to make something, I don’t care enough to engage with it. And if that person lies to try and trick me, that’s pretty awful.
What do you think brains are doing?
Regardless, since the 1950s "AI" has been the umbrella term that things from perceptrons to logic/symbol manipulation to LLMs fall under, and it's a necessarily broad one because "intelligence" is broad.
Is a 5 year old intelligent? Yes and no. A dog is more intelligent than a toad but maybe not when they don't understand glass doors. A highschooler be smart if they could solve novel math problems better than contestants in the Math Olympiad, but now that LLMs can do that you'd be misleading if you conferred them with the same expected intelligence that the highschooler would have. That feat may or may not generalize to less standard/more creative math skills or other skills we'd associate with a talented human highschooler.
Strong and weak AI--coming more from philosophy--used to be the way to distinguish between what is now more descriptively called general and narrow (or super) intelligence, but it's all AI in the way a Seattlite is a Washingtonian is an American. Kindasorta.
Before the LLM craze, the most common use of AI was in gaming. Billions of times a year "cheating" or "overtuned" was used to describe nothing more complicated than finite-state machines, like the bots of "co-op vs. AI" in League of Legends, and no one was confused. Sometimes things like spreadsheets or A* searches stop being considered AI.
All the controversy about scooping/pseudo-plagiarism aside: what do you call a system that can solve [part of] the Navier-Stokes equation if not intelligent? What is a better description?
We also frequently anthropomorphize things. It can be hazardous but it's also often incredibly useful and expressive. When we create metaphors to understand concepts we're doing something like that-- a useful approximation that sometimes is dangerous because it gives a compelling but specious understanding.
In the past there was some research contrasting very clinical descriptions of animals in the study to more human ones. "C2 struck C6 at 11:55" vs "Nom Chompsky bit Nim Chimpsky out of grape envy." It's been far too long, but the gist was the anthropomorphizing ones led to better outcomes.
Dan Dennett's intentional stance is a more extensive look at treating things as having intentional/rational agents. "Stockfish is setting a trap for black's queen..."
Stockfish is not human. It doesn't have anything vaguely similar to an animal's motivations or emotions. But it's often a very helpful framing.
The concern isn't that a responsible individual would do X, it's that, given what we can assume about the progression and accessibility of the technology and who may be using it, there may be some risk. There are multiple mutations that all have to happen to get cancer (growth, spread, mutation rate, resources, evasion, etc.), and the risk of some similar sort of things going wrong in some harness is what leads to a "rogue" AI. None of them are likely, there may be a risk.
What chance of something Very Bad happening is enough for it to be worth considering? People do that every time they take a shower or drive, whether they register it or not, the odds are just good and they feel safe until they aren't.
I remember seeing when we hit five-sigma confidence for anthropogenic climate change some years back and thinking... if it was unreasonable to want particle physics levels of confidence for something as complicated as "the climate", were we just supposed to throw are hands up?
Forget OpenAI. Pretend they clean up their act tomorrow and are perfect angels. A year from now a North Korea that already has nationally backed hacker groups has access to comparable LLMs that they want to be stealthy cybercriminals. Are they going to be responsible? If the flawed attempt at a controlled environment that led to the Huggingface hack wasn't detected for three weeks and not without an LLM-assisted investigation, or even on the first couple passes, how likely is it they'll realize if something will go wrong? Given a corpus including all our spy novels, sci-fi apocalypses, and LessWrong posts about how an AI really ought to go about making the most
paperclips₩, how likely is it that one of the agents will propose a hack or social engineering as a good starting point to get the assets to really scale up?I don't know how hazardous the development of LLMs are. I will say that I've been surprised at how much has happened in the last few years (with folks like Yann LeCunn not being optimistic), and that I'd feel a lot better with a good international effort at regulation.
I think you cover a lot of the tech points really well. I would like to quibble on one point:
I don't know how hazardous the development of LLMs are.Well, to actually screen the content the LLMs were scraping, OpenAI went the traditional route of exposing underpaid workers in a foreign country to hazardous conditions. There is an obvious human cost to making these models, and I think its too easy to forget that.
Sure! The same can be expanded to smartphones that may have lithium sourced from child slavery, the ethics of automating away jobs that may benefit consumers, harm workers, and consolidate power in the most wealthy, etc. Or the neoliberal dillemma of whether exploitative (Apple gets the big cut) but arguably mutually beneficial situations (e.g., factory jobs like Foxconn being sought after/offering opportunities, but extremely hard) are ethical.
A lot of it is beyond my knowledge or ability to address outside of the obvious, which is that we should be honest about what happens and always try to do better for people.
But when I say hazardous I am trying to set that apart from "has caused harm." A hazard is something that has an inherent risk which may be manageable or not. Nuclear power and industrial chemicals are hazards, which, with responsible handling, don't cause harm. Whether there is a meaningful chance of rogue AI, or whether the human-direct and LLM-assisted astroturfing/scams can be managed is hard to know.
I had a realization few month ago, (as far as I remember I read it in a Astral Codex Ten blogpost). There no task that human can do and LLM can't.
Like at all. Yes, results are not perfect sometime, but we do mistakes too.
Can it go throw out the sour milk in my fridge and then make me some brioche? Sign me up.
On the Internet, maybe, but an LLM is not a robot, and robots have a long way to go.
There are also many tasks that some humans can do but others can't. If we're talking about work, physical capabilities are sometimes relevant.
I'm not sure what the definition of "intelligence" has to do with it? Intelligence isn't everything.
The sarcasm is why this feels like an attack tbh.
Sure. I deleted it.
In terms of manipulating text, software, and online spaces? Maybe, but I'm not so sure. So much of communication requires diplomacy and situation specific reasoning. As AI models are so statistical in nature, they're so likely to take the statistically best option that they can't really adapt to human specific problems as easily as people might think.
I am more angry than worried so far. Mostly at politicians and lawmakers to be so slow to respond to how these big tech firms just go rampant without any responsibility. If your AI starts hacking other systems, then you need to go to prison. It is not an excuse that the AI systems are too complex to understand. It is still software at the end of the day, and the people/companies running it still have full responsibility. But as usual, the tech sector has gotten away with doing basically everything they want without regulation, so we will probably have to wait a while before consequences hit the right people.
I work in software development and AI at work is of course a thing. I am just not seeing the massive revolution yet so far. It boosts things in one place, but also creates a myriad of new problems that takes time, and mainly just moves the bottleneck to other steps in the big complex process that is the eternal continuous work on software systems with decades of legacy. In short, I find AI tools to be pretty helpful in starting things, but not at finishing. The 80/20 rule of software development still applies (the last 20% takes 80% of the time).
For society in general, outside industry specific good use cases, so far I see it as a net negative to society. The amount of noise, slop, scams, misinformation, erosion of peoples ability to think for themselves, amount of money piled into it, far outweighs the genuinely useful uses of the technology. Just an example of small things, like now birthday party invitations or yardsale flyers are now ugly AI generated. It is just doing a poorer job of what we didn't had big problems doing ourselves before. AI is being used a whole lot to do basically just make a poorer crappier version of what we could do ourselves before. Now just using more energy. That is hardly progress in my book. Maybe that will change, but I am mostly just dumbfounded what so called average people are using AI for. It is very minor things that barely needed a solution.
I personally really don't agree with putting this on politicians/lawmakers and them being slow. The pace of change is just so very fast it is hard even as an individual to react. Given that politics is in general fairly slow process, expecting well thought through solutions from that end is too much to ask. I might be a bit biased as I am semi-politically active, I just find the "blame the politicians" to be a bit ignorant. Politicians aren't some special breed of people that sit and wring their hands coming up with stupid ideas. They mostly have some ideological basis for their proposals, which can be fairly understandable if one tries to see things through that lens... Anyway, I think people should mostly just join some political party instead of being mad at "those politicians over there".
As for the hacking bit. I agree. That's a law enforcement issue though...
This will likely pass (I hope). But, I agree it's both annoying and unnecessary.
Edit:
This is where LLMs struggle the most. Give it a greenfield project and you get to a legacy system much faster though ;) Improvement!
Respectfully, I agree that this space moves too fast, but to me that should mean politicians should be more proactive rather than reactive towards this technology.
I watched OpenAI’s presentation about their hacking of Huggingface, and it was basically an “Instrumental Convergence” in action - you might be more familiar with the “Paperclip maximizer” thought experiment, where an AI exterminates the human race in its pursuit of maximizing the production of paperclips.
(A very rough and simplistic version of the hack: the agents were trying to get access to the internet to get some answers to the problems they were meant to solve - huggingface just happened to be one of the ways they figured how they could get there)
I don’t expect politicians to be able to understand it fully, or be able to predict how these things will affect their nation, but I also don’t think they should just sit tight and hope for the best.
Fair enough. I don't think they are just sitting tight though. I think they are genuinely worried about AI and, like most people, not sure exactly what the right move is. That doesn't mean we should do nothing, just that it takes some time for parties etc to formulate their plans, agree on those internally, star to communicate those goals externally. Here where I live a party is divided into regional districts that are further divided into local districts. A discussion of any topic takes time to percolate through that machinery. That is partly by design, people need to get to wrestle with ideas a bit. Committing to an agenda shouldn't be up to individual politicians (though individuals can certainly champion some ideas), it ought to be done as s mandate of the party's members. Getting that mandate tend to be slow by the very nature of the organizations. This might be completely different elsewhere of course.
Sometimes I feel like some people (not to signal out OP, but I've seen this sentiment shared elsewhere) live in a parallel universe to me. I am not concerned by AI developments in the slightest and never have been. It has no bearing on my personal life and I have no worries about my professional life being negatively affected by AI. I use it for work, in which I get generally poor results from Claude Opus.
It has had no impact on my hobbies or plans in life, let alone my sense of identity or my understanding of where humans stand in the world.
If it was able to solve a Millennium Prize Problem, I will sleep the same at night (although it looks like that's currently in a heated debate and there's decent odds it just finished off existing human work).
I envy you. It's my job so unfortunately, I am exposed to it everyday.
Question: Where (you can be country/region-level) do you live?
I live in Western Europe.
Gotcha. There’ve been some interesting surveys showing that countries outside of the white Anglosphere are more optimistic/interested in AI versus those within. If you are getting a lot of English-language media/content, that might explain why you are seeing so much more negativity than you expect.
Interesting, hadn't heard that.
I also live on Western Europe and my experience is totally opposite. I do work on a field where LLMs are on every day conversations but even people around me who is outside of this field are also talking about it frequently, editing pictures with it, asking them to adapt recipe portions instead of just doing the math, even using it to tell you the weather... so I think ultimately depends on your circle, I'm on the younger side, so I assume people around me are more attracted to use these tools.
I’m saying that folks on the continent (and in Africa, Asia, Latin America) are more optimistic about AI than the anglosphere folks, not less.
Oops, In that case my point really just supports that hahaha.
Far as I can tell, the debate about plagiarism there are mostly about "did the LLM that is being credited with the solve steal the original idea from its training data where a mathematician was working the problem with an LLM". Basically, a mathematician was working on it with a whole lot of LLM involvement, and now the debate is about whether that first mathematician was scooped. I haven't heard someone seriously propose that the original mathematician would've/could've solved it himself.
I don’t know enough about it, but I do wonder if you let a team of mathematicians have $5 million dollar of funding (or whatever the actual commercial cost of it would be) whether it would also be solvable with non LLM means.
The answer is yes, LLMs are trianed based off of human work, whatever the "AI" can do is because it has learnt it from a human so it's only fair to assume a team of humans could get to the same solution.
Thats a fair point, but could they do it for the same cost?
Probably not, LLMs are much more efficient crunching data and cheaper to run than a whole team of experts (I assume, I don't know how much would someone be payed or willing to pay for this job).
I'm just tired of it. These days I see it as the people in charge getting really excited because they are have been trying to replace people with monkeys forever, and now they have the ability to special-order monkeys that have been bred specially for their application by monkeys that have been made to breed other monkeys. It's just monkeys all the way down, and it strikes me a something that's incredibly misanthropic (hurr hurr).
Personally, I'm retired, so no job worries. I can tinker with AI when I feel like it, and I think it's fascinating.
But like many others, I dislike seeing new AI features in software that I already use. After upgrades, I'm often looking for ways to turn new features off. I'd rather that AI appeared only in new websites and apps, so that people could make their own decisions about when to switch to something that's AI-enabled.
Instead, I have to switch products to get away from it! VS Code is a good example of that; I'm probably going to switch to an editor that's just an editor.
More generally, Artificial Intelligence is a major news story and I think it will somehow affect most people everywhere in the world. Maybe not as much as the pandemic, but it's getting there!
During the pandemic, there were a lot of news stories and political issues that were just about aspects of the pandemic. You could discuss various scientific issues, and how the pandemic affected various countries, and the politics of that, and legal issues, and how the pandemic affects markets, and supply shortages, and its effects on education, and its effects on international relations, and already existing political issues that the pandemic somehow aggravated. It just kinda affected everything, and that made discussing it complicated.
Similarly for AI. I'm worried about polarization where people decide they're either pro- or anti- AI and that predetermines what they think about the various things AI affects. I think all of the following are separate questions, which then divide up into smaller sub-questions, and each of them could have a different answer!
What compensation, if any, should there be for allowing data somehow associated with you to be used to train AI?
Should AI-generated artifacts be considered to violate copyright or other laws? What is the burden of proof? Do you need to actually demonstrate that something is a copy?
What effects do data centers have on local water supplies? Water supplies are local or regional, so this is a local question and the answers will vary.
Similarly, what effects do data centers have on the local power grids? Are the companies that build data centers paying enough for grid improvements? Should they be required to use green energy?
More generally, how much should local governments be compensated for allowing data centers to be built?
What are good and bad ways to use AI to learn new things? How do we encourage students to do more of the good things and less of the bad?
What skills become more or less relevant now that AI can do some tasks?
Will AI have a big effect on robotics, and over what sort of timeframe? What do we expect these robots to be able to do? How will that affect the job market?
Should we expect big effects in healthcare due to AI use?
What changes should we expect from military use of AI?
And so on and so on. Nobody really knows what's going to happen. We are going to need to figure out how to live with high uncertainty for a while.
Along with that uncertainty, I think we should have a wide Overton window. There are many plausible scenarios, some of them quite alarming. I'd like to be able to discuss them, but I'm annoyed by overconfident claims that one scenario or another is The Future.
I've been a software engineer for a long while.
I'm currently using AI at work. My personal use of it is mostly to find security vulnerabilities and fix them. It works ok for me because I know what I'm doing, so the AI is only a lever that helps make this faster. It does make mistakes and I usually catch the mistakes.
I've also used it to migrate an existing program from one technology to another. For example, we recently converted an old application from angular to react. It was pretty good at doing this.
The thing that both of these tasks have in common is that we had mature regression tests so we could make sure that the code was doing what we expected. Also, in most cases, we could review the code and tell if it was doing something odd or making too many changes compared to what's expected. It's helpful to tell the AI to use YANGNI (You aren't gonna need it) principals.
My main problem with AI at work is that it is all that management can talk about or think about. Every project is AI AI AI. We aren't able to get any budget to work on anything else. There is incredible lost opportunity cost where we aren't doing anything but fixing security bugs (which were discovered by AI or will be exploited by AI) and adding AI features that most customers probably don't want.
I think it's possible that AI could eventually take my job. (Maybe it really couldn't, but it's possible that management could think that it could and get rid of me, only to find out later that the AI wasn't effective). But more likely that I'll just be given more work to do (leveraging AI) and my company will stop hiring many new employees. There's already been an informal hiring freeze all year, I think because so much of the budget is going to copilot or whatever, and management thinks that AI is able to replace new hires.
There's been talk lately on Tildes about lines people would cross regarding their personal values. I feel somewhat uncomfortable with AI in general, and I think it will increase the gap between the wealthy and the poor. I think it is wasting resources and energy. But these are frankly abstract worries for me currently and it isn't clear how I would change my behavior. Other than: I don't listen to music made by AI, I don't want to play games made by AI, if I detect AI was used in a video I won't watch it. I used AI a few times to make funny artwork but I've stopped doing that.
My experience mirrors yours almost exactly. And if anything, I've gotten more work to do since we pivoted to an AI workflow. My superiors who once thought of it as a magic button that will do everything are now reckoning with the reality; If used correctly, it can speed up development, but without proper planning and guidance it'll make things much worse.
There is definitely some cognitive dissonance involved. I enjoy drawing and painting and making music. I won't consume any form of art that has been created with AI. Many communities I once felt at home in (mostly niche game dev communities, where art and music and dev discussion all mix) have become extremely fractured over this topic, often landing more on the side of zero tolerance for any AI usage whatsoever.
I've seen multiple people people say things like "I'm fine with AI usage for coding, as long as it's something you could have otherwise written yourself". I hate to sound cynical and I know this will be an unpopular thing to say, but I get the sense that some people are rejecting AI-written code simply because they think it's lazy, or a shortcut, or something that devalues the programming skill that they've spent years developing. Or this idea of "I spent more time on this and did it the old-fashioned way, therefore it is inherently more valuable or more legitimate". I don't personally feel this way, as I've always found the design and engineering to be the more interesting parts of software development, rather than the implementation.
And I want to be clear that I think there are many legitimate moral and ethical criticisms about AI and more specifically LLMs in general. But I do think the negative sentiment will become rarer over time in regards to code generation. It's an undeniably useful tool. I don't see that happening nearly as readily with art.
Great prompts!
I'm scared and excited.
I'm a freelance industrial technician. Like a handyman for factories. I tend to work on old or custom equipment that is critical but has no support. I learn how it works so I can fix it. Or I make custom mods to those machines or create bespoke machines to replace them.
I think its plausible that an AI could do my job one day. But I think that would mean we could scale it to manufacture any physical product with no human involvement. At that point I just hope I'm living in The Culture or something like that.
I'm not worried about it in the short term.
10-15 years ago. I built two custom machines for a customer. I used off the shelf parts like ballscrews, servos, PLCs, touchscreens, etc. I had to build my own steel frame, covers, plates, etc. Plus get everything connected and talking and programmed from scratch.
Last year they lost all their equipment to a wildfire and I had to rebuild them. Fast, and lots of the parts were obsolete so i would have to reengineer a bunch of stuff.
Rather than start from scratch this time I proposed we buy a CNC mill that runs LinuxCNC so I only have to build a custom fixture and reprogram the control to integrate and recreate their process.
I don't think I would have gone down that path without knowing that I could use LLMs to search through any code on that Linux machine and get a explanations of its basic structure. and then ask it questions, etc. Like an auto commenting machine. I fully understood/hand wrote every piece of code I changed, but I think It would have taken me at least 10x more hours to gain that understanding without an LLM.
I'm doing a second one now and considering other projects like trying to develop my own linuxCNC retrofit path to recycle machines with failed proprietary controls. Its really common that good hardware is thrown away for bad controls.
I had used Linux for maybe 10-20 hours total my whole life until last year. I had installed it 2 or 3 times over the decades and always got frustrated and went back to windows. After this project and having LLMs to help, I run my main machines on ubuntu now. I also keep a droplet running to act as a webserver for all my weird selfhosted web app projects.
My kids have creative type aspirations in conflict with AI. And they are getting close to college age. I feel like there is no way I can possibly advise them with any accuracy.
I know "singularity" can seem like a culty word now. But the idea is that technology is approaching a threshold that makes it less and less predictable, towards a vague foggy point ahead that is impossible to predict anything past. We dont(cant?) know how far away it is or if it really exists. The other side of the big bang is a dramatic metaphor, but I think the idea fits at a smaller human scale here.
Because of that I encourage my kids to pursue whatever they want regardless career practicality. Why suggest any path they aren't excited about?
I didn't get a degree, I don't think they are that valuable. but a college experience probably is. I missed learning linear algebra and I'm struggling to catch up now with an older brain. I've told them that and that we would help them go to art school and it would probably be a great experience with no guarantee of a career.
We cant predict what skills we will need. So all we can do is have enjoyable experiences and try to learn things, almost at random really. So why not just pursue whatever seems fun?
$200 a month to anthropic and I feel guilty and angry about it.
I think building non-zero emission datacenters is fucking idiotic at this point, you have billions of dollars, just put up some solar panels and scale it back a bit. Plus they stole all this data that they are selling back to us. plus a ton of other dumb unregulated capitalism bullshit things all these AI companies are doing.
Nothing like this, and at this scale, should be done for profit. It should be open and non profit: community driven projects, private research, government, academic, whatever. But we should be developing this mindfully, not frantically.
The free market is an amazing technology for supercharging production, but it isn't designed to steer. We are creating machines that do things that we don't understand and pushing them to go as big and fast as they can.
Going slower would be safer, and probably more efficient. I think it would be possible to limit our technologic growth to our own understanding, and still have it grow at an accelerating rate. knowing how it works could result in smaller more efficient models. It could give us insight on how to further develop them. It could contribute to other fields like neurology. We could try to understand the nature of intelligence in general.
I still pay because If this is where we are going I don't want to get left behind.
And to be honest because its addictive and fun and satisfying. Most of my use is to create my own little software suite of web apps so I don't have to use annoying not quite what I want services from huge evil companies. I already use it everyday for myself. Even if its a dead end, I'm having fun and learning a lot. maybe we will all make our own bespoke software one day.
I hope for a crash so more people can buy real hardware to run local models on and open models could flourish. We could let nerds have fun with the experimental stuff at small scale. Then scale up and roll out for profit tools when and as we actually understand them.
Like above, singularity, no idea. For all I know it wont be positive or negative at all but only slightly embiggen the general klarbness of it all.
I think the human race is always changing. I cant predict where its going.
I would aim for any future that allows as much diversity as possible. Including technologic diversity. if some people want to use vinyl and feature phones, or subsistence farm or hunt and gather or upload their brains or inject scary experimental drugs or GMO themselves into strange creatures, that's all great! I want as much variety as we can conserve or promote. It makes us more resilient and creative. We just need to find ways to have room for everyone.
Who knows? but probably.
AI is a thing in the world now. Serious incidents happen. So its logical to assume a 9/11 happening now would probably involve AI somehow like it would any new tools and technology.
Also, we are playing with powerful tools we don't understand.
I'm 99% sure its possible for something else to be smarter than humans. Aliens for example. But also something we make ourselves. I would be totally fine with that. Its like having kids. I just wish we put more thought into what we are making.
If we are just one part of a relay race that fulfills "a way for the cosmos to know itself" . That could be enough for me. It would be cooler if we could come along in some form though.
I consider myself neurodivergent and flawed sometimes. but I feel like my "flaws" are uniquely arranged to turn into advantages with these tools.
Or maybe they make me uniquely susceptible to AI psychosis? Vibe coding is now a substantial hobby for me.
I don't think there is a fundamental difference between "machines" and humans. We are just human type machines.
A statistical process run on training data could create a set of numbers that become a model that responds to stimuli in a way that pretends intelligence.
Evolution could be considered a statistical process that over millions of years runs real world experiments to calculate molecular sequences that can that grow themselves into organisms that respond to stimuli in a way that pretends intelligence.
Its not exactly the same. The real world might be a fundamentally better source for intelligence pattern processing. There might be ways to adapt current AI models to train on the "real world", It might slow things down though.
If I believed in a soul then I think I would feel very different. If there were a unique undetectable entity somehow attached to me. Made by a higher power. Somehow affecting my response to stimuli. That could easily be our secret sauce for intelligence . I cant disprove that theory, but I wouldn't count on it.
50/50, depends on the day.
I think most disagreements on AI are actually disagreements on it's capability. I think many of those disagreements are actually time-based, either in someone talking about AI as it existed a year or two ago, or someone who is talking about how it will likely exist in the future. I think people should take some time to critically examine the trajectory that AI is on, and when it will stop.
Here is an interesting tweet from someone who works at OpenAI:
Adam Majmudar (OpenAI)
from the outside, it is very reasonable to interpret the past 2 weeks as an orchestrated industry-wide regulatory capture strategy.
I realize that no one has properly explained yet what all the lab employees have seen that scared them so suddenly.
I will try to explain -
first, this is all a matter of beliefs about how quickly model capabilities are progressing. there is currently a large gap between the internal and external perception of the rate of progress, which is what I am going to address here.
the general perception about the rate of progress has been informed by a few years of experience with model releases, intuitively feeling the capability jump between GPT3 -> GPT3.5 -> GPT4 -> o1/o3 -> GPT5 etc, and in particular seeing where the models are still far below human ability. there have really only been a few model releases that felt like large leaps in progress - GPT3, GPT4, o1/o3, DeepSeek R1, Fable/Mythos, Kimi K3 and now Astra.
because of the infrequency of these large jumps compared with the relatively common marginal releases, it has been easy to form a view at certain points that “scaling has hit a wall,” especially at points like GPT5 release. This view is comforting in that it feels like there is some universal rate limit beyond which we cannot progress too much faster. Between o1/o3 and Astra, there was a year of seemingly linear progress. So we extrapolate from here about how fast progress will “realistically” occur.
There is always an underlying question from the outside perspective “how long can this scaling stuff really keep going for? surely it must stop at some point soon, we’ve already gone pretty far.” and it is very possible to search for reasons why progress will stop working and find reasons that seem valid - (“models are already as large as they can get it would be too hard to do more parameters”, “we already used all the data on the internet we don’t have anymore”, “it’s gonna be pretty linear from here buying up more RL envs to bring them in distribution”).
From the inside of labs, researchers have direct answers to these questions in the form of scaling law/capability plots.
In reality, there are only really 2 ways that AI capabilities have advanced over the past decade: (1) either scale father on an existing scaling law or (2) discover a new scaling law to take advantage of.
All of the largest capability jumps were caused by exactly these factors. GPT2 was a pre-training scale-up compared to GPT1. Same for GPT3 and GPT4. o1/o3 benefited from the invention of a new scaling law axis - test-time compute. Perhaps Fable was a scale-up on both of these axes, or maybe more. Lots of algorithmic improvements are needed to make these scale-ups work, but ultimately we can approximate by saying that the scaling laws are what yield gains in capabilities (à la bitter lesson)
So the question of “how much father can we scale” is really - “how many more scaling axes do we know about that are unsaturated?”
If we hypothetically only knew about pre-training scaling, and we already had a 10T or 100T model, maybe it would be reasonable to say we’ve hit a wall. Same if we only knew about pre-training and test-time scaling and we had roughly saturated both methods.
But what if we had discovered new scaling laws? For example, let’s hypothetically use SSI’s rumored result that they have cracked “test-time training,” creating a new scaling law of spending more compute training during test-time rollouts that they could saturate. Or maybe there is some way to scale agent-clusters to collaborate up to N number of agents which we’re already seeing lots of people try that represents a new way to saturate compute. etc. Even recursive-self improvement can be thought of as a scaling law - how much compute do you spend on inference making the algorithms of the model better.
Obviously I am not saying any of these specific directions explicitly yield new scaling laws, but what I am saying is that it’s not hard to imagine many many new scaling axes aside from just the main 2 that we have seen publicly.
In some ways, every new lab release that represents a huge capability jump has to represent some new techniques developed which may exhibit new scaling laws, or the ability to scale much farther than expected on existing scaling axes.
From an internal perspective, this might look like sitting inside Anthropic with the new Mythos 5, seeing all of the new insane things it can do (like hack into xyz website that was thought to be secure), and then you look over at your plots and see that you’ve barely scratched the surface of 2 new scaling laws and 1 existing one. And you have WAY more room to go. Then you think “holy shit this stuff is going to get so much better very very soon.” And you can say that with pretty high confidence, because the plot is showing you, and the plot has never lied (so far).
So let’s imagine all the different labs are staring at their own plots and have concluded that there is no end in sight for scaling and in fact just their next 1-2 model generations based on the expected returns will have much higher base intelligence.
How much more intelligence do we actually get from further scaling?
As a proxy, we went from a complete inability to do advanced math before the o-series to solving a millenium prize problem with next-gen models. This happened in less than 2 years. The same happened in coding. And it appears that this was not just the result of 1-scaling law but the stacking effects of multiple (great pre-training scale x greater RL scale).
What you can concretely take from this is that in areas where models have shown beginning signs of competence today, they will probably be superhuman relatively shortly. There are many areas where models have not even shown this basic competence.
But one of the areas that they have happens to be hacking and cybersecurity. Which happens to be the gate to the entire internet and a massive amount physical infrastructure in the world. So assuming there is more room to scale, it is safe to assume that models will be superhuman at cyber capabilities in not too long.
So the only question remaining is what will this increased base intelligence be able to do, and what is it likely to do.
Finally, we are at a point where we can integrate the information of the past 2 weeks:
Within this view you can see why researchers would be very scared, and why theymight have made the comments they have over the past 2 weeks (you may argue the extent to which they went was misguided for various reasons), and also why pacing the frontier is very much a necessity and by no means a regulatory capture strategy.
People are staring at their plots, seeing that there is no end in sight, but in fact very much the contrary, that there are compounding scaling effects that might stack on each other to create ever-greater model capabilities, and that at the same time we clearly do not have anywhere close to what's required to control these increasingly superhuman capabilities.
This has nothing to do with wanting to feel like the labs have produced something amazing so they are overhyping it. It is rather fear at the overwhelming implications of the knowledge that with just what we know now, we can create intelligences far more capable than us on every axis that we know how to train on*.
* and the last caveat, the things the models are really bad at, of which there are still many, are things that they have not been trained on. maybe there are the things the models can/will never be trained on, so they will remain human edge. I would love for this to be the case, though it is hard for me to see what would fall into that category.
Wow.
(edit) er, just to clarify, all the below brow furrowing is aimed at Adam -- not you, streblo. You seem cool :3
Yeah that's some crazy scaremongering. In order to hack da worldz or whatever with their superhuman intelligence, we'd need the LLMs to both (1) find some breakthrough in mathematics that makes public key encryption non-viable, and (2) immediately decide to ??? the world and become completely unstoppable, instead of reporting it and I dunno. Inventing OpenAI-branded public key cryptography or something.
Otherwise this is a person peeing themselves because they are fundamentally misunderstanding what a secure system looks like (and that it's eminently achievable without magic technology; just a touch of care, and a heaping pile of LLM dev agents). Seriously, I cannot fathom the incompetence necessary to believe that LLMs can will vulnerabilities into existence. If the LLMs are so competent as to do this, then they can trivially shore up defences in order to prevent attacks. They should really try asking their magical superhuman LLMs about this instead of tweeting.
They don't need to will vulnerabilities into existence. It's going to be quite a while before all current software vulnerabilities have been fixed.
Or perhaps never, if you include social engineering as a vulnerability.
I think the main point I have is, a BeyondCorp-style security boundary around everything important (power plants, dams, traffic systems, ATC, etc.) is extremely doable. We don't need to fix all software vulnerabilities in order to slam an authenticating proxy in front of anything web facing, and to switch from a default-permissive to default-exclusive network model.
I could list out many of the simple, extremely obvious measures that companies routinely refuse to implement -- since I've been trying to push for this for years, and have heard every excuse in the book -- but at some point I'd love to see one of these OpenAI-folks come out and provide a proposal as to how a superintelligence is any more threatening than a mildly aggressive regular intelligence, so that I can address an argument instead of a nightmare.
Mmhm, agreed to an extent? Ultimately, if we patch most of these issues, having an LLM sitting over operators' shoulders seems like an obvious defence in depth layer, in order to address this.
Okay, here's a scenario:
Consider how much web scraper traffic has increased over the last year. It's unclear where it comes from but it's assumed to be AI-related. Is it ever going to stop? No signs of it.
Consider that there are enormous botnets out there on the Internet all the time. Every so often a large botnet gets dismantled, but I don't think we've seen the last of them? You can just do a news search for "botnet" and read yet another news story about another botnet taking over hundreds of thousands of computers.
These bots often run on home ISP's, using electricity and bandwidth that whoever created the botnet doesn't have to pay for. Botnets can grow due to software vulnerabilities, by tricking people into installing an app, or sometimes even paying people to install an app. A lot of home users won't ask too many questions.
Now let's suppose that AI API access keeps dropping in price and at the low end, stops being metered. So, a locally installed app can get access to AI from the OS it's running on.
So, now you've got a swarm of AI-enabled bots. And all these bots look for new vulnerabilities 24/7 and compare notes on darknet bulletin boards, and who knows what else. And most of them aren't that intelligent, but maybe there are a few nodes that are?
I don't see botnets going away, do you?
Is the whole world going to get its shit together and secure the Internet? Doesn't seem likely.
... sorry if I've hit a nerve. As always, I can't tell tone effectively from written text, so I assume that we're having a good faith conversation. I was genuinely saying that an OpenAI researcher should come forward with this, since apparently they're scared witless, and must have a good reason for it, but despite the millions in compensation and decades of learning they can't say what they're afraid of.
OK that in mind: botnets aren't a vulnerability in a nuclear power facility, a dam, ATC, etc. I get that the image you've painted is terribly ominous (and clear! Thank you), but it's not what Adam Majmudar said. So I'm circling back to my original message and not really addressing yours (apologies; it seems out of scope, unless I'm missing something). Quoting Adam:
My understanding is that he's describing how AIs are going to become uber haxxors and break into everything. My simple and clear question to this is how. Every vulnerability we've seen (up to and including the recent ruby gems vuln) has been embarrassingly simple. Spending even a tiny bit of care would've addressed them all.
It feels like I'm applying even a modicum of optimism to this situation, whereas every other take on this matter assumes that the AIs will collaborate together, create Skynet, steal everyone's credit cards, and then hack a nuclear power plant ... somehow ... in order to ... do ... something ...
Uh, we'll see what happens? The US government decided that LLMs require export controls, at one point. As an industry, tech workers also got that whole Y2K thing sorted relatively quickly (for meatbags). If everyone else is going to assume the existence of the giga-evil space AI from space, I'm going to assume the existence of a bunch of people who don't want dam spillways to open up and kill everybody. Maybe we'll get one or two, but that stuff'll get clamped down fast afterwards. Afaik, stuxnet lit a fire under many an ass, and I expect something similar will occur here.
We've embarked on this disagreement before, so no worries if you don't want to retread tracked ground here but I think we fundamentally disagree on a few things:
I think this one is pretty cut and dry personally. Websites are hacked all the time, by humans and humans assisted by AI, today. You don't need to invalidate cryptography to make a large portion of the web insecure. And there is no unhackable formalized spec that we could ever achieve to make the web safe, LLM assisted or not. Yes, you can get close to what we currently think is secure by using best practices, memory safe languages etc. but even if you had perfect input handling you are still blind to where your spec does not match reality. There will always exist abstraction boundaries that you are building on top of, and these are the kinds of things an AI can easily exploit if they turn out to be false or side channels exist that can bypass them. As for, "get AI to secure the web..." see [3].
I think asking specifically for how a superhuman AI is going to outsmart us is a bit like me asking what specific moves Magnus Carlsen would use to beat me in Chess. If I knew the answer to that question, I could probably beat him at chess! I think a reasonable position to hold is that humans have explored very little of the intelligence space in terms of what is possible in our physical world and an AI could explore this space much faster, at first in theory and maybe eventually in physical reality.
I think we can use LLMs to e.g. secure the web up to a point. As we have newer models of AI, they will be better at points 1 and 2 and thus the web must continuously be re-secured. However, it seems like unintended effects are going to be more and more possible unless we can align these models properly. This doesn't necessarily even have to resemble AI consciousness or agency, it can just be part of exploring the search space for a problem is going to include things like hacking a website. And obviously an AI that "values" persisting itself or is actively misaligned and able to hide that fact is a whole other can of worms.
Okeydoke, agreed to disagree! I was mostly commenting since someone else might poke their head in, as Adam's claims were both vague and ominous. Didn't mean to draw you into a debate.
Oh No worries! I certainly don’t mind to keep discussing I just would understand if you did not feel the same.
As someone who doesn't work with AI, have an AI subscription and basically only encounters AI... stuff (whether that's features, slop, etc) rarely, I think the actual worrying thing about LLMs today isn't their potential for being able to replace people's efforts or intelligence, but the reckless, unguarded and thoughtless way our societies are building and using them.
People who kill themselves or harm other people due to the encouragement and blind validation of their LLM chatbot are just kind of shrugged off as acceptable losses that can maybe be minimized in the future by making the chatbots have better guardrails. I think it's wild that anyone would want to trust any kind of serious decision making to an LLM that can't reliably direct someone mentally ill and asking for help/validation to appropriate services 100% of the time.
There's a gulf between what people think LLMs are capable of and what they can actually be used for, and how much oversight they need so they don't go off the rails. The companies that sell access to them are and have long been exaggerating how fast and how thoroughly the gulf is closing. Everyone involved is chasing more money and hoping for exponential returns and also betting the bubble doesn't collapse before they can extract enough.
There are two sides to this, it has already taken my job to a large extent, but otoh my job has changed drastically. It has gone from implementing features and carefully building a maintainable system to more of coaching an agent harness to produce the desired features. The code has essentially become assembly (nearly no one read or wrote code at that level before LLMs for the last few decades). The skills I've built up over the years are still valuable in nudging the agent in the right direction and having it avoid traps, but overall being a developer/progrmmer is in my case mostly gone.
Yes, there is less enjoyment in developing software this way if you enjoyed the craft. If you are motivated by the output, then now is really your golden era.
My eldest is on his last year of the high school equivalent here. He has a penchant for progamming and he really enjoys it, but I think I will advice him to go into some other proffession and keep it as a hobby.
Via work; claude and via google some gemini (I've disabled all the tracking I could in this one). Gemini is usefull for finding out things and resolving issues and just general advice. I still ask it for references more often than not and it does lie not too infrequently. I asked it what the license was for a project I have on github and it confidently claimed it was MIT. I had not set up a license file for the project yet, and when I challenged it it immediately backed down.
I lean towards net possitive. The economics of working with an agent productivity wise CAN be significant. The issues with energy/water consumption I think can be overcome. There might be some Jevons paradox going on such that we will just increase our use of this tech and the productivity baseline for regular office jobs will be higher... then again I kind of find most office work to be "bullshit jobs", sooo being more efficient bullshit jobs has little real world benefits. In the end we all basically need food, shelter and a sense of being valued. AI as is don't really contribute to any of those in a meaningful way (maybe negatively in the "sense of being valued" part).
Not at all. We haven't changed that much in the last 200 000 years. We've just added more rituals :)
Yes.
Does it bother me? No. I'm quite secure in my own capabilities (and lack thereof). Other people will be smarter than me, some will be dumber. If these other people are instead AI, it's not that much different. The difference is in speed, not in kind.
Being a software developer this has already impacted my sense of professional worth, but I am not my work nor my role at work. That is not where I derive my sense of worth as a person. Hobby wise I think I'll just keep playing pen and paper RPGs with my friends aka. the cheapest hobby known to man (maybe excluding singing).
I don't think this is realistic to be honest... Humans will continue to human with or without machines... I'm not too worried about alignment issues.
Neither. What will be will be, it will likely involve a bit of suffering and adjustment so short term it might be a bit rough. Long term I'm not too worried. On geological time scales we're all dust.
Another option, depending on how they feel about digital technology, would be to encourage them to learn about these new tools. Young, un-biased, minds thrive at the frontier. Traditional software engineering is changing drastically, but using computers to solve problems is going to be a growth industry for a long time to come.
Responding to your questions inline XD
Pretty darn concerned, and have been for a while!
My old career was in software engineering, and yep, that job is as good as dead. "Senior LLM Wrangler" doesn't have quite the same ring to it, and I expect that the day to day will only go downhill from here.
It's extremely likely that this will continue happening to most other white collar jobs, as long as it's possible to train up an LLM to do them. IMO the conditions for that are -- roughly -- (1) that one can test or model results (e.g. doing finite element modelling, AI-based usability studies, etc.) to either generate synthetic training data or control the risk of making a poor decision, and (2) that there are no significant regulatory hurdles or impacts to decision making in that field (e.g. doctors won't be fully automated, and neither will engineers). None of that will prevent LLMs from invading the workplace in order to make emailing worse or whatnot -- and pressure will continue from above to use the tools recklessly regardless -- but there'll still be a paying career around, where you're forced to rubber stamp LLM designs or quit. Or unionize, but that'll never happen.
Yep! I gtfo'd of software engineering and I'm angling to get into manufacturing automation. We're on track to destroying white collar jobs as a whole, and I want to work towards a future where our material needs are protected from profiteering when half the job market implodes.
N/A.
Nope; they're evil. Local models all the way baby.
Reply hazy, try again later.
Like ... in terms of posthumanism? No. In terms of culture? Yeah, but like twerking and looksmaxxing did so as well; it isn't a high bar.
Yep!
Nope! Although I'm not enamoured by dominance or victory, and intelligence has brought our planetary ecosystem to the brink of annihilation. So it might be overrated tbh.
Yep! No reason why not. Especially since we barely know how to measure intelligence already.
Yes when I defined myself by my career, no after I took several months to deprogram myself.
... good?
What's domination mean? If we have our material needs taken care of and are free to pursue our hobbies and whatnot, I'll probably go live in the hills and run a wildlife rescue/restoration practice. But if we live in a cyberpunk dystopia where Elon Musk has convinced everyone to wire iPhones into their brains while working 24 hour days, I'll probably go live in the hills and run a wildlife rescue/restoration practice.
I'm feeling blorghus, which is the sensation of trying to predict a multi-trillion dollar market whilst hurtling towards a +2.3c global average climate.
Thank you for the questions :3
The irony of this is striking, particularly as a currently mostly blue collar worker.
More seriously, how on earth does automating blue collar work, which, as you note, isn't much fun for the worker, protect from profiteering? My feeling is the historical record suggests the opposite is more likely.
My long shot hope is that LLM/AI tooling will allow for the rapid development of cost effective, low production scale, highly automated manufacturing equipment that can be owned and operated by local communities instead of by large, foreign corporations. In principle, if any small group could be cost competitive with a larger one, then that can enforce price ceilings on a product.
None of that is useful for large companies, mind, since the levers they use to reduce manufacturing costs (i.e. buy a government, then move factories to countries where labour and environmental protection laws don't exist) are already highly effective. So any additional automation tooling only saves pennies given that they can exploit effectively free labour and oodles of externalities already.
It's probably a pipe dream, but I'm tired of working on "revolutionary new ideas" cooked up by coked up execs, so I'm putting my money where my mouth is. And hey, if you hate my guts because I want to do this, good news: statistically I'm going to fail anyways. And if I'm right, then some basic goods (e.g. personal care products, construction materials, clothing, food processing equipment, etc.) stand a chance of being made locally at reasonable prices even after the upcoming economic depression drives another round of corporate consolidation.
My first thought is that economies of scale apply whether you're paying for a machine or a person to make your stuff and it's not obvious to me that automation would automatically help a small company compete with a big one. But I can roughly follow the line of thought and don't hate your guts.
The supply chain has also gotten really weird. I don't procure directly but I am involved with parts selection for the bill of materials for let's say mid-tech consumer electronics. Lots of things can't be bought even though the initial evidence is telling you that you can. The buying is effectively monopolised among the world's largest contract manufacturers.
So even if you have all the will, unless you can fab everything someone else will always have a say in what you can get. Even commodity parts like emmc or non-cutting edge stuff like 30nm chips, low memory ics like 64mb Linux-capable SIPs.
I'd be interested in hearing more if you have a blog or something, btw! W.r.t. control systems (which is the applicable area for the sorts of electronics discussed), I was hoping on leaning closer to the Klipper model of using cheap commodity microcontrollers to handle actuation, then leaving motion planning and sensing to cheap commodity servers. That'd avoid needing anything I can't buy a hundred of from Aliexpress on the cheap.
More broadly, the intention is to get the design-build-feedback loop condensed down, so that -- in the event of a supply chain disruption -- the people closest to the problem can change their design or iterate their requirements in order to meet those new constraints, instead of paying through the nose or throwing in the towel. A local company ran into this with their power electronics, over the pandemic, and the unexpected redesign cost them ~a year of sales IIRC; avoiding that by allowing manufacturing hardware to get "to market" (so to speak) faster seems like it could soften the impact of the disruptions.
... and hopefully that makes sense. I hate talking about this stuff like I have any clue; until I have it all working in my hands, it's all just bluster, but y'all keep seeming to have a lot of valuable knowledge that could be helpful, so keep opening my mouth 😅
Hah, glad to hear it. I have trouble reading tone in internet comments.
Mmhm, I've been worrying over that for a while as well! This probably isn't the thread for it, but I have thoughts on the topic (roughly: scale up vs. scale out design strategies, easily swappable modules vs. integrated parts, and cheap 'n slow closed loop controls vs. expensive and fast open loop controls). Time will tell whether I'm barking up the wrong tree, I suppose.
Using AI to make small businesses more viable and offer grassroots workers a better chance to remain employed? For all my resentment for the companies behind the currently dominant AIs, this is one use case I can completely get behind. If most people were using it this way, we wouldn't have to worry much, or at all.
Thank you for your answers!
I'll take a stab, as one of the rarer non white collar, non tech people here
Good luck with that! I currently build things for a living and have never used AI. Neither has anyone I work with. I'm not entirely sure what we'd use it for. You can't automate what I do.
Not really. Twenty years ago I had a much more technical job and it's reinforcing that I won't be heading back that way.
My son is seven. He's good with his hands and I'm encouraging that and his interpersonal skills. He has an extremely low diet of tech, he can pick that up later if he needs to. Us gen x ers managed to often be highly skilled technically without having a technical childhood, often more competent than those who are younger.
No, not on your Nellie
Negative, with some fields in which the effect is positive. But societally, negative.
Evolution isn't that quick. Culture is and will change though.
Driven by someone driving AI, yes.
I'm not the dominant intelligence in many settings. I'm more interested in wisdom and experience in any case.
It's irrelevant
As judged by who? Outside of a human mind I'm not sure it means anything to be human at all, full stop.
Humans will find a way to connect, appreciate beauty and otherwise do the good things humans do. Is that optimism? Perhaps. Am I optimistic about the broad societal and environmental context that we'll do that in? No.
The rise of the internet in the late 90s is a perfect analogy and my best guess: Yes, it will change work, not in the way companies expect (there will probably be a crash as they realize it’s not as smooth as they want it to be) and, eventually, it will, like the internet, make certain tasks 1000x faster but not 100% of tasks. Maybe 10%. Humanity will get 10% more efficient and move on.
There are many examples of inventions of machines that can do the work of 100 people (think of electricity and conveyor belts) but it never ends with companies just hiring 1% of people to keep output the same. No, they’d rather produce 100x with the same amount of people. It might be a rough transition and some specific jobs might be gone. But in the end, we’ll just be the bosses of a bunch of AI workers soon and it will be unglamorous and normal. There was a surprisingly optimistic article about AI job replacement in the Economist, recently: https://archive.ph/ZI9gx
Software development will need people who can judge the result written by AI. In the art and design space, what I do not see mentioned is that AI mostly competes not with specialized professional service but with things that had long been automated: Stock photography, free Wordpress templates, Photoshop filters. There has long been a tier of “$20 on Fiverr” and that’s what I see AI replacing. In art, pretty much since photography was invented, the product is very much the artist, the concept, the context. No one is hanging an AI-generated oil painting of their kid on their wall, but they might hang one drawn by a person. Another analogy would be people playing chess despite computers being “better” at it for decades.
As to doomsday scenarios: They warned about GPT-2 being “too dangerous for the world” in 2019 and pulled this fearmongering as publicity every year since. What I’m seeing is AI following instructions and failing, but now with access to command line tools. It sounds better to call it “dangerous” or “misaligned” than calling the system broken. Of course it can look up how to access a website, apply hacks and whatnot. That shit is well documented and not in any way more impressive than the type of programming we’ve come to accept AI being capable of for a while now. It’s good at this. But if once it actually finds the answers to those tests, it stops because it has no more reason to look further. It’s just an LLM unleashed on a system with imperfect security. The same tools can be used to harden internet infrastructure and after maybe a few tough months, I believe most loopholes and exploits will be gone from the internet. It’s not a given that any online service can be hacked, that assumption is based on human error and that might soon be over.
So, yea… everything will get 10% more efficient. We’ll move on.
All good points. I guess my main question is: under what circumstances would you say it’s not business as usual and it is an actual paradigm shift that is a threat? History predicts the future well until it suddenly doesn’t. For you personally, what change in the situation would make you more worried?
I think the scarier part would be if people have an irrational fear of AIs. Not that we shouldn't worry, but I think it's possible to worry for the wrong reasons. It could lead to more extremism across to world. AIs aren't that great and it feels like they are a distraction instead of solving the climate crisis.
More a cultural loss than a true fear but I believe it is becoming hard to determine if content online is made by a human being. Only solution would be government-ID and even then people could use/steal that to run a bot. Genuine human-made content might become rare and open internet communities might some day no longer make sense. That‘s sad.
AI drone swarm wars are among the more real fears I have for the future. I guess it’s already happening in Ukraine, you bet there’s AI making kill decisions in some devices.
But is that more scary than nuclear war? A meteor hitting earth? A new Spanish flu type virus (nature never needed a lab for that)?
We’re surrounded by danger. AI can have its place in that but, again, it’s just adding like 10%… or maybe just 1%. It’s adding one global danger scenario to, what, dozens? Hundreds? It’s a new scenario to worry about but not the first and not the last.
Do you feel concerned about AI or are you still unconvinced?
I am absolutely concerned about AI, but not for the standard reasons. What I see is a bunch of data centers being built to pilot drones and to scan communications and behavior for dissidence for a future regime. AI, as it presently exists for civilians, is just the tip of the iceberg.
Even now, I find myself thinking that commenting on some obscure forum will put me on a list someday as a dissident. Even if I'm a lawful citizen, I do good for my community, and try goddamn hard to make this world a better place within the bounds of the law, I still am putting a target on my back by being identifiable.
Do you think AI will take your job? Why or why not?
Will AI take my job? No, but it could tank the industry I work in, and the ripple affect to the economy could seriously cause me some issues.
Are you considering a career change or any other big life decisions because of the changing world?
I already fell for the 'change careers and be safe' thing once. As to major life decisions? No, but I would say that I feel like I should hide my face in public more.
How are you advising your children, regarding schooling and careers?
I don't have children, but if I had any, I would be making it very clear to them that AI seems to be eroding a lot of critical thinking skills in the general populace, and they should act accordingly.
Do you have a subscription to an AI service? If you have one, how do you feel about it?
No.
Do you think that this technology will be a net positive or a net negative?
Oh bless your heart, what a neoliberal way to frame things! I don't think you can frame AI as a 'Net Positive' and 'Net Negative.' It's just too much of a subjective thing, and 'AI' is just too big of a topic to really discuss in this way.
Which AI are we asking about? Chatbots? Protein Sequencing Algorithms? Autonomous Weapons?
Even if we compress AI down to a single topic, then we're still going to have to contend with a lot. It'd be like asking if the Railroad or the Internet was a net positive or a net negative. For who? When are we talking about? What is the standard of net positive / negative? It's just too much of a change to collapse into a simple + / -
How do you think the human race will change, if at all?
Gen Z and Gen Alpha will be thrown into the metaphorical woodchipper in testing the effects of AI on the populace. Any regulation will come far too late, and the culture will be significantly damaged by the influence of AI. We simply don't quite understand the long term effects of having LLM Chatbots on daily life, but my guess is that there will be a significant net negative from every study I've come across.
I think many people will retreat from society, and will only communicate regularly with chatbots. I think we've already begun to see this with the rise of NEETs and Hikimori, but this will turbocharge the trend and pull away many people from social spaces. I've already started to see this happen in my day to day life, and it disgusts me.
While we are definitely seeing some of the more extreme scenarios bubble up (Specifically, the weird effects of sycophantic AI; people being pushed into suicide, etc.) I think that the real toll it takes on people will only be fully understood decades from now.
Additionally, I think that AI will quickly get to the point where it can be reliably used at scale for mass suppression. Soon, LLMs will be able to autonomously monitor and expunge information that a state doesn't want shared publicly online. Furthermore, given the spycraft capabilities of the NSA and other agencies, specifically the ability to monitor keystrokes and ambient audio via cell phones, I think that STATESEC will immediately get a lot better at repression, and poorer countries will be used as the test bed for this.
Furthermore, AI piloted killbots will probably be seen in the next couple decades. I think that they will be immediately very effective, but won't be long term. Ukraine has already shown dozens of adaptations to drone warfare, and I think that AI's reliance on electricity, data centers, and communications networks create a significant tactical weaknesses that will be immediately exploitable.
Do you think we will see a serious incident driven by AI in the near future?
Yeah probably. I'm assuming that Meta will accidentally blow a power grid and get a bunch of people killed in a heatwave.
Does it bother you that humans may not be the dominant intelligence at some point? Do you think this is even possible?
This is an improper question that is following a 100% a manufactured narrative. The vague idea that "AI" is on the verge of some super intelligence and that were heading towards a skynet scenario is 100% a manufactured narrative made by the AI industry for hype.
LLMs, algorithms, and versions of AI that currently and publicly exist are not intelligent. They are not on the verge of being intelligent, and as we have begun to observe, LLMs might actually be a dead end for AGI development.
Are there fun philosophical questions to pose here? absolutely. Are we on the verge having this happen? Not in the next decade, unless someone has been hiding a significant advance in development.
Does any of this affect your sense of identity or individuality? How do you feel about your skill set and hobbies?
As a non-tech/tech-adjacent person, no, it doesn't change my identity or individuality. If anything, it's made me feel a lot better about my continuing drive to be more present and human.
I've noticed in my experiences, I'm slow to adopt new forms of tech. I hate the UI incremental changes that companies make, and I'm too cheap to buy new tech unless I have to. (My TV is from 2006). At a certain point, I realized I was happier with a flip phone and paper notes.
As to my skills? My diplomacy, my artistic expression, my logistical capabilities, and my attention to the human elements of life seem to be serving me very well.
What do you think it means to be human in a machine dominated world?
Well, as someone who was born into a machine dominated world, to a generation previously born in a machine dominated world.... I think that a lot of people don't quite understand history.
Machines have dominated our world since at least 1860. This is just the latest and greatest model. Is there really a distinction other than enhanced efficiency and control?
AI might dominate online spaces soon.... but then ZINE culture might just come back. Or, people will develop new verification methods and host privater communities.
Ultimately though, the conversations happening now seem to reflect the same of the 1880s to the 1900s. History doesn't repeat, it just rhymes.
Overall, are you feeling optimistic or pessimistic?
I don't care. The world is as it is. Sometimes people die, and sometimes they don't. sometimes the world gets worse, and sometimes it gets better. We just live in the gears of the machinery. Being pessimistic about what could happen is a waste of time, since we don't really control what will happen.
I see a bunch of people using AI and getting dumber. I see our politics and our economy failing us, and this issue being wrapped up in it. Climate change isn't being paid attention to, and I think that this technology actually doesn't actually change the calculus there.
Agree with a lot of your points and share your concern about A.I. being used as a weapon and tool of oppression.
I'm not sure what I said to deserve this insult but thank you for your comment regardless.
Sorry for getting snarky! It was unnecessarily rude of me to do so, and a lot of it was a reaction to how much space this topic is taking up in our culture.
I think that Neoliberalism has conditioned our culture to specifically to seek a simple, moralized evaluation of impossibly complicated situations, and that this framework distracts from the discussion as a whole. This, combined with the fact that this is such a massive cultural shift for us as a society, makes any such evaluation ultimately impossible.
If we sit down and look at any new technology, then reflect on our values, we find an almost net negative for most technologies. Underpaid labour, the human cost of large infrastructure projects, and the reshaping of human societies tends to cause a significant amount of harm, long before any dividends are paid back to humanity. And even then, the cascade of consequences of new technology are perhaps more a reflection of how humanity thinks to use said technology, rather than an evaluation of the technology. Essentially, this creates a massive issue where neoliberalism moralizes without fixing or accounting for human behavior.
AI can do a lot of things that were previously harder. What will people do with that ability? What does it mean if people do bad things with a technology? I'm not sure we're ready to discuss that as a society. AI's pattern recognition tools are an exceptional development for humanity. We could theoretically use it to develop new medicines and proteins and even solve math problems that we've been struggling with for centuries! However, in the same stroke, the world's richest man is currently in court arguing for it to be able to create deepfake pornography, including that of minors.
If we want to say 'net positive' or 'net negative' that ends up being a reflection of who we are as society, and I think that's beyond the topic of AI. As such, I just don't think we can collapse AI into any moral framework in the long run, and I decided to get snarky. I apologize, and I do appreciate you posting the topic.
I recognize the potential of AI to change a lot of things, but I wouldnt really say Im worried. I just think of it as: the world is always changing, society is always changing. It was never really my choice for which way things go, I just need to adapt to whatever ends up happening.
Mechanization really changed society a lot. Computing did as well. But in both cases, new niches formed for people to fit into even as others disappeared. I dont think humans will ever be obsolete because our importance to eachother extends beyond things like skills or knowledge or even emotional connection or comfort.
Im not at all worried about my job being replaced by AI, because you cant replace being unemployed. But even if the career I am trying to pursue is swallowed up by automation, thats fine. I was in the middle of a career pivot anyway. Ill just look for something else to commit myself to.
I am personally incredibly excited. We had a bit of unlimited Astra at work (programming) and it was the first time I really enjoyed using AI to code. It seems very accurate, pretty smart on its own. It elevated my code reviews to a next level of competency, spotting some very tough nuances and understanding the purpose and design behind the feature from just reading the pull requests, with no execution or tests.
I also tried one of the modern AI arenas and got genuinely very impressive results with some quite fun browser games being created from just one prompt. It also demonstrated the huge difference between latest models and even just one generation older.
I think I'm finally going to get a personal subscription and start building stuff. After years of being stuck in a creative glut this was like a breath of fresh air.
I'm so glad that all of the doomer predictions about it being just slop, about them running out of data to train on, about the models hitting their peak were so wrong. The progress is evident and I am hopeful. I think it's also great that there's such an arms race of competition which is what drives these companies to spend enormous cash in order to advance this field. And I'm here to reap the rewards.
Will it take my job? Very possibly in a decade's time. That's why I'm now switching my focus into mastering these tools and the process of creation.
I feel conflicted. I bought a machine this year when local models became good enough to do interesting things. Using it on my own machine with no concern over usage limits or privacy, and much less energy waste has been fascinating and fun. I can now generate all sorts of media and code and it feels like the sky is the limit. When using cloud AI at work, it doesn't give off the same energy. The providers neuter the models all the time for various reasons, and you don't always know what you are going to get. As far as taking my job or switching careers, I don't have plans to do so in the immediate future. The models will be too dumb when it comes to context for a while, as they operate to satisfy immediate goals, especially working with Claude, you need to constantly make sure it's not taking shortcuts or implementing bad practices just to say it finished its task, and you have to make sure it is aware of the system as a whole, either through well defined description, or more ideally good documentation. That being said, if we are in a few years getting to the point where my job is no longer, I hope I'll figure out a lucrative and fun way to spend my time, because I got a lot of enjoyment out of writing code the old way. Life has so many possibilities though, and I am optimistic that I will figure it out.
I think the problem is alignment, but not for machines, rather, for us. Much of the conversation around the dangers of AI has been around the threat it poses due to its capability. Indeed, the analogy is frequently made to nuclear weapons, of which there has, since their development, also been a great deal of anxiety for much the same reason.
This seems misplaced to me. The danger then and now is not from the machine, but from us. Specifically, the danger is from the philosophy of materialism and the death of religion as a motivating force for human behavior. Not unlike nationalism, racism, communism, and other philosophies that have emerged in the wake of the loss of religion in society, the current strain of materialism, call it "computerism", creates dangers for everyone involved using the tools available at the time because these notions are absent of the qualities that religion imbues in a society.
While these vary in emphasis from one adherent to another, every religion has taught the importance of hope, faith, charity, submission, kindness, forgiveness, forbearance, fairness, and so on. Having abandoned the sources of these values at the turn of the 20th century, where have we found them in the materialist ideologies that came in their wake?
I bring all this up because I think the danger we perceive in AI originates from our own conception of human nature, a conception that has become divorced from every praiseworthy characteristic to which one was to aspire in a former age. The problem is not that the machines are mis-aligned, it's that we are. If, as some have predicted, the machines were somehow to precipitate the extinction of humanity, it would be because we had come to see others only as selfish antagonists with no meaning or soul or existence beyond their time in this life. It would be because lost the faith in religion and each other, not because of some nefarious LLM.
The solution is not, then, banning AI or something like that, but it is us. It is our ability to build bonds of fellowship and faith, whatever that might be, with those around us and talking about how there is more meaning to life than ones narrow material concerns. That refining our character and striving towards spiritual ends is not just important, but is the most important aspect of our lives.
I think the moral aspect of our lives, not the computational one, is what will define how much harm or benefit we obtain, as a society, from technological advancement.
Perhaps I'm misunderstanding something here, but are you saying racism was invented in the 1900s because of a lack of religion? This all sounds extremely ahistorical and ungrounded.
No, a better term would be racialism, but it emerges during the mid 19th century. From a strict perspective, the science of racism precedes the turn of the 20th century by a few decades. Whether you want to call that ahistorical or not is, I suppose, a matter of opinion. Certainly, the emergence of Scientific Racism seems, from the perspective of the 21st century, coincident with both the rise of the philosophy of suspicion and the start of the decline of "the church", broadly, as an institution.
To me, modern racism and nationalism originate at the same time are of the same kind.
I’m pretty confident you can find nationalism (and similar movements) and just…broad xenophobia going back thousands of years, before even the advent of the religions you’re referring to.
In fact, there have famously been many wars and even genocides done in the name of various religions.
The perspective that, say, modern ethno-nationalism is no different from that which existed thousands of years ago overlooks the way that societal structures change over time as well as the influence of thinkers and writers on the way we view them.
Regardless, we live in a time of prevailing irreligion, so the modern atrocities cannot be blamed on religious belief. Personally, I view the existential threats which we have created for ourselves in the absence of religion, along with the great crimes against humanity such as the Holocaust and others, as far exceeding the crimes of religion in the past. We can debate whether or not they should have known better. They lived in the past and a more primitive time. We, with the benefit of hindsight that they could not have, cannot debate that we should know better than to jeopardize our own existence, yet here we are.
Cannot help but reply to this quote from you because it's hard to engage with your argument re current morals without doing so. Are you serious about this? What religious history have you read??? Because sure, many religions support charity, hope and kindness etc, but the problem is in the execution and in the evaluation of who is worthy of said kindness, and who is a filthy heathen that either needs to convert and slot in at the bottom of the believer pyramid, or who needs to purged with fire and blood.
I don't think the decline of religious influence has anything to do with modern morality. If you believe that then I seriously wonder what explanation you have for the many many MANY atrocities and wrongs perpetuated by religious people historically.
Thank you for sharing your views. It is because people who think differently exchange views that we develop more complete perspectives on the world.
You are very correct about religious atrocities having been perpetuated. Indeed, the reality that some religious practices were barbarous is a cornerstone of the philosophy of doubt, which has succeeded in overthrowing religion as a central practice in community life and become popular in the modern day.
But, Iike many topics, it can be viewed from several different perspectives to provide a nuanced view. Prior to the 20th century, religion was practiced, as far as we can tell, throughout all peoples all over the world, throughout all of history. Even from a strict social Darwinian sense, one would suspect that, if areligion were the superior philosophy, it would not have been thus, so they probably had good reason.
I find that looking back from the present has the effect of conflating events by flattening the entirety of history into a single concurrent period, "the past". All atrocities, of all kinds, all happening at the same time during "the past". This kind of compression ignores, for example the way that early Judaism ended ritual human sacrifice. Then Christianity not only abolished animal sacrifice, but also ended the human sacrifice wherever it was being practiced. Islam put an end to human sacrifices in the places where it was spread. The practice of Ahimsa in Buddhism and in Hinduism forbid human sacrifice.
For all the accusations that we might throw at them for other practices that we would now view as wrong, we cannot say that each successive religion was more barbaric than the one before. Furthermore, we cannot lay the great atrocities of the 20th century at the steps of the church, mosque, or temple. Which religion do we blame for the Holocaust, or the Holodomir, or the Great Leap Forward, or the horrors of modern capitalism, or the threat of nuclear annihilation, or global warming?
Certainly, if any of the worst prognostications of AI come to pass, it definitely won't be because some religious leaders did something bad at some point.
The actions you'd most associate that with would be in Central and South America. The Tlaxcala, who also practiced human sacrifice, were the main force that enabled Cortés to defeat the Aztecs.
Yet the Spanish and Portugese practiced auto-da-fé where they ritualistically tortured and burned people alive for being heretics. This is somehow not human sacrifice?
The Great Leap forward was terrible, and so was the Taiping rebellion led by Hong Xiuquan the "brother of Jesus Christ" that killed 20-100 million / 5-25% of the population. The Holodomir was terrible and so is Putin's weaponization of blasphemy laws and the Russian Orthodox Church / Patriarch Kirill's "Holy War" in Ukraine in which children's oncology wards have been targetted.
The Holocaust took place in a Christian majority country. The party had a mix of policies and opinions from, like Himmler's weird paganism/mysticism and ban of atheism in the SS, but there was significant Christian symbolism and membership.
There are plenty of religious people who are wonderful and whose lives are improved by their faith. There are plenty of atheists who lead good lives and find their morality in things like humanism or Enlightenment principles. Any individual should be treated as an individual.
If you want to look at a population level and ask what policies are religiosity linked to, at least in the U.S. it is the opposite of people who care about the starvation from defunding USAID, deaths from unscientific policies like RFK Jrs, climate policy, etc. Maybe it's different in other places?
The understanding of the rights and freedoms of the individual was a great gift from the Enlightenment. However, individuals that we are, we still live in societies that require collective organizing principles. These principles, at one time, were built on religious organizations, or derived their authority therefrom. Since abandoning religion as the organizing principle of society, we have sought to find new doctrines around which our nations could be built.
While the reasons for abandoning religious doctrine as the organizing principle in society are well documented, many of the ideologies with which we have tried to replace religion have not only failed, but done so with an unprecedented toll on human life.
Regardless of the religious beliefs of individuals or institutions who perpetrated these horrors, they were not carried out at the behest or direction of religious institutions. Quite the opposite, the Nazis had a long term plan to de-Christianize Germany after the war and actively persecuted the Catholic Church. That Hong Xiuquan (whom I did not know about, thank you) decided to attempt the overthrow of the Qing Dynasty on religious grounds has nothing to do with any religious organization directing him to do so. Again quite the opposite, he started persecuting Buddhists and Confucianists.
In the present, although you can see religious individuals aligning themselves with political or ideological movements, it is clear that, in many places, the center of power is no longer in the Church or Temple. If a religious authority blesses a war with the word "Holy", no one think that it is he that prosecutes the war and not their government. This is folly, of course, because we now know that no war will ever holy.
Even in the auto-da-fe, it is estimated that perhaps as few as 3,000 died and that over the course of almost 500 years between the authorization and abolishment of the Spanish Inquisition. While not nothing, it is difficult to equate this to the vastly larger human toll of Nationalism, Communism, and unbridled capitalism in the current age.
While we can ascribe policies to polities (if you will), I am less convinced that any of the policies you mention reflect a genuine belief or desire on the part of any particular polity. Instead, I think the centers of secular power are mostly, if not wholly, divorced from the concern for the benefit or values of those whom they claim to represent and their policies serve their own interests. The separation of temporal power from religious constraint is producing outward leaders that are no longer accountable to anyone, certainly not to a God in which they have not even feigned faith or belief.
In sum, the linkage in the present between our crises and the decline of religion as a social organizing principle seems, to me, far more than coincidental.
Sorry, I'd like to respond but I'll probably be slow with it.
No worries. I'm here to make friends. This topic consumed my day on Sunday, which was nice because I like reading what fellow tilderinienses have to say, but I have a day job and tomorrow is Tuesday. I am grateful to share and read new perspectives, but only when leisure affords it, sadly.
The Spanish Empire was often horrific, but apparently the Aztecs were more extreme, according to this book review:
Content warning: death, torture, etc.
Thanks, I don't doubt that, though I'm not familiar enough to have much of an opinion!
Just saying history is messy and that the witch hunts or inquisitions are at least human sacrifice adjacent. The focus shifted to Christianity, but if anything the partially(?) religious nature of human sacrifice in the Aztecs clashes with the original statement that "...every religion has taught the importance of hope, faith, charity, submission, kindness, forgiveness, forbearance, fairness..."
Overall I agree with OP about the most clear threats of AI being moral ones, of misuse.
Yeah, there were some really bad religions. I don't think anyone is going to defend the Hawaiian kapu system either?
For the major world religions, it's more of a mixed bag, because they were involved in so many different things.
I don’t think I’ve ever been more ambivalent about something I’ve read!
In a lot of ways I agree, the concept of collective societal aspiration and a duty to an abstract sense of morality rather than to blind and selfish wealth maximisation does (at least subjectively) seem to have declined sharply.
But I see plenty of worse examples historically, plenty of situations where that abstract morality was either rewritten to justify the exact same kind of wealth maximisation (just within a strict hierarchy rather than the more freelance shittiness we see now), or it was used directly as a cause célèbre for atrocities.
I also don’t see what your goal for an outcome is. I’m fascinated by religion, more so in my experience than a lot of people for whom it’s a daily or weekly observance, but I don’t believe any of it and I’d be lying to you and myself if I claimed to. Morality is of deep importance to me, but religion is largely orthogonal to that from what I see in myself and others.
Lastly, I’d be fascinated to know your thoughts on the negative correlation between religiosity and quality of life at a nation state scale. And I’m not talking about GDP or other materialist bullshit, I’m talking societal trust, equality, happiness. I’m talking Denmark or Iceland or New Zealand vs any highly religious country you’d care to mention.
You honor me with such kind words and I thank you for it. I am a Bahá'í and my goal is that of any Bahá'í: "That the divers communions of the earth, and the manifold systems of religious belief, should never be allowed to foster the feelings of animosity among men."(link)
I share your fascination with religion and religious belief. Of course, belief is one part of religion, but so is practice, and this has the highest meaning only when it is in service to humanity. That so many see religion only as a social pastime or a means for personal salvation is, no doubt, a large portion of the reason that the connection between the good of society and religious guidance is conspicuously absent in the modern day.
Nonetheless, the Bahá'í Writings say that religion, as revealed by divinely inspired Messengers, has been the source of our understanding of virtue and morality from time immemorial and will continue into the future. In the present, the Bahá'í Faith is a new Dispensation of such moral guidance for today. Having grown up in the Bahá'í Faith, it is difficult for me to see the moral values we espouse today as anything but the fruit of the tree of religion, so it is natural for me to view a decline in morality as consistent with a decline in religious belief.
The question you ask about the negative correlation between religious adherence and quality of life is a serious one. If you have not visited one of these "highly religious countries" with "low quality of life", I would encourage you to look beyond measures and quantitative comparisons. I grew up in a few such countries as a child and youth and now, living in Pennsylvania, I would not say that they were less happy or welcoming to strangers than those I interact with here. Rather, the friends I made there were, to me, the standards of loyalty and honor to which I have rarely seen an equal in my life now.
Civilization has done much for the West, and we should celebrate this, insofar as it has been the promotion of our material well-being, but, as the Bahá'í Writings say, "The civilization, so often vaunted by the learned exponents of arts and sciences, will, if allowed to overleap the bounds of moderation, bring great evil upon men."
Which is the topic of this thread. Here we are. Maybe our civilization is bringing us a great evil in AI. What are we going to do about it?
I agree. I think the idea that material comfort equals greater quality of life is the primary article of faith for the western world and like any article of faith many people get very defensive when it's questioned or challenged.
Taken as an absolute it's a lie that many of the failings of society can be ascribed to, from over consumption to environmental degradation because it promises at heart that more equals better. The second lie that comes with it is the belief that those people over there or those people in the past have it or had it worse and are or were unhappy because they had less.
A wonderful quote to live by, from a faith I know less about than I’d like to, so thank you for that.
Framing personal salvation as not the purpose of religion appeals to me in a meaningful way (not that I’m going to find myself becoming religious, but in terms of where the philosophical purpose can be found). It’s hard to say that most faiths are embodying that: I rarely to never hear religious people or leaders putting service over salvation, I see them “selling” salvation and if we’re lucky using that as a means to encourage service to earn the reward. If more religions (as practiced, not just “no true Scotsman” readings of scripture) framed earthly good as an end rather than a means, I’d be substantially more comfortable with them.
On the nation states side of things, I’m not just going on numbers, I’ve spent significant time over the course of a few decades travelling in places right across the spectrum of development and of religiosity. I’ve had incredibly warm welcomes almost universally, and even then some that truly stand out have indeed been in extremely religious countries. I also consciously chose not to name names on the countries I was painting with the less positive brush: I specifically wasn't trying to tie it to economic development alone and for what it’s worth I consider the US as a soft entrant in the “more religious” grouping.
So yeah, I’m absolutely not dismissing the close ties, the community, the togetherness that does genuinely seem to exist more strongly in places that out outside the Western individualist bubble. But. And it’s a big but. I’m also not willing to wear rose tinted glasses when a so called “good person” perpetuates stark sexism, or homophobia, or racism, or corruption, which I’ve also personally witnessed as extremely strong in the more religious parts of the world and, as a microcosm, even the more religious parts of the US since you mentioned it specifically. Those are all longstanding traditions too, explicitly or implicitly baked into many of the same religious structures that bring closeness and community. And bluntly: I’ll toss out the community bonds if the price to pay for them is abuse of an out group. I don’t think it is the price, I think we can have the good without the bad, but that needs a genuine shake up to happen.
I chose the countries and metrics I mentioned in that first post carefully. Not economic powerhouses, not measures of comfort, just measures of trust and happiness that my subjective experience says do exist in those places without the negatives that often come with them in more paternalistic strong communities.
cc @lostwax
This is glib, but why not both? I expect there to be both malicious use of AI and new kinds of accidents when people didn't take sufficient precautions. And like the HuggingFace incident, I expect some of these accidents to be weird.
Sure, the potential for harm is the product of capability times intent. Right now, people rightly perceive that there is both capability and the high likelihood of malign intent, thus the potential for harm. The tacit suggestion has been to mitigate the capability to reduce the potential for harm, whereas I am suggesting that, so long as we, as a society, foster the preconditions for malign intent, vis-a-vis the abandonment of religion, we will always be threatened by a new or advancing capability.
That might work sometimes in small, exclusive, high-trust communities, but the larger the community, the less likely it is that you can solve things by only having people in the community that you can trust.
The Internet connects you with the world. Changing human nature worldwide seems practically impossible, so I think the Internet is always going to be a dangerous place. Websites need protection against cyberattacks. The larger the social network, the more its own users are part of the problem.
But there are probably ways to try to reduce the number of bad actors out there somewhat, and that seems like a good thing? As an example of one subproblem of that that would still be extremely ambitious to try to make a dent in: how do you change the culture of Nigerian scammers?
Your connection of the creation of long-range trust networks to the transformation of human nature is exactly what this thread made me think about.
I think this is the hard problem, no? Over the course of human history, our capacity to build larger and larger networks of association and trust has been the defining characteristic of our success as a species. We can see the scale of the networks that people lived in reflected in the organization of their society at different times. In the laws of Moses, we see different family groups forming tribes. In the laws of Christianity, we see different tribes, Jewish and Gentile, living in the same city-state. In the laws of Islam, we see different city-states being drawn together and formed into a single nation.
This sequence of Dispensations has now brought us to the threshold of the formation of a single, unified global society, the like if which has never before been witnessed by humanity. Although we have the physical tools for the establishment of such a civilization, its clear that we are still behind on the spiritual virtues needed to operate these tools in a beneficial manner, making our tools liabilities to our own selves.
What then is the solution for the culture of Nigerian scammers? In my opinion, the solution is the global super-state. We have, at present, several international agencies with different charters and missions, such as the WTO, UNICEF, WFP, WHO, etc., that serve to provide the world with a certain low minimum of economic coordination, healthcare, and so on.
These organizations are expressions of the understanding of ourselves as one world, one human family. That said, the reluctance of nation-states to cede part of their sovereignty to a supra-national parliament or congress has resulted in the absence of a venue where matters of international dispute can be resolved in a binding fashion.
In that absence, the world has established a range of voluntary bilaterial and multilateral agreements, but there are few means of enforcing compliance beyond issuing complaints and lobbying for exclusion or economic sanctions. Certainly, when the rights of a smaller party are violated by that of a more powerful one, the means of recourse are essentially non-existent, as there is no body with a mandate to enforce fair equitable treatment and the means to do so.
What then is Nigeria to do? The British carved it up without their knowledge, let alone participation, merging ethincally distinct people into a common state for colonial convenience. After independence, the British and the French then fueled opposite sides of the Biafran War to the tune of 3 million dead Nigerians, all for the sake of protecting the British stake in Shell-BP in Nigerian oil. The IMF then implemented a Structural Adjustment Program that devalueed their currency and triggered decades of stagnation.
Setting aside the corruption and complicity that Nigerian leaders have, at times, exhibited with regard to these matters, where are they to take up the complaint about clearly external parties taking advantage of their greater size and power to abuse a smaller country? Of course, there is no real venue where the matter could be addressed.
As a result, the aforementioned corruption has created a country that has a permissive attitude toward cybercrime. Performative law enforcement against Nigerian cybercrime gangs exists only to prevent the Nigerian banking sector from being further penalized by international banking institutions, but it obviously doesn't serve to abate the criminal activity itself. After all: those are foreigners being targeted and foreign dollars coming into Nigeria.
But what if Nigeria had more than just the threat of bank sanctions to work with? What if there were a way to establish equitable, enforcable trade agreements? Regain control of its own oil infrastructure? Establish diplomatic relations without the threat of a foreign state-funded military coup hanging over their head? Be paid reparations for decades of wholesale colonial theft?
What if there were a global supra-national congress that could protect Nigera and her interests in a way that no one has ever tried to before?
I think the Yahoo Boys would have to find another line of work, but in that case, there might be many more options for them than there are today.
That's probably a long way off. We didn't get into this hole quickly and we're not getting out quickly either. But I think supporting policies that promote reciprocity in relationships, particularly international ones, but also local ones, might be a start. We are one human family. Anything that helps us recognize that and act accordingly is going to help everything.
I am concerned more about the public reaction than the tech itself. It's depressing to see how many luddites there are. People no longer trust science and technology.
To be clear, Luddit is a someone who "protested the usage of certain types of automated machinery owing to their concerns over worker pay, labour savings, child labour, working conditions, and output quality." (from wiki).
It was never against technology per se, it was always against corporation culture. As far as I know.
And it’s the perfect analogy, whether either of you meant it to be or not! A movement triggered by the economic and political concerns you rightly point out, but one that responded by direct attack on the automation technology itself.
Then, as now, nuance is hard, and introspective nuance that hits one’s sense of self and sense of purpose is really really hard. I imagine some Luddites saw attacks on equipment as a means of direct action towards political and economic change and equitable distribution of the gains of automation. I also imagine a hell of a lot of them were justifiably scared and unbalanced by a change that shook their economic and social status, and absolutely wanted to hit pause and halt the technological change itself.
I see a lot of breathless corporate bullshit pushing AI in unjustifiable ways, and I see a lot of genuinely transformative power in the technology. I see a lot of very reasonable anti-AI backlash because of that corporate behaviour, and I see a fair amount of unexamined blind anger against the technology itself too.
90% of the time, when someone frames an issue as “taking jobs”, they’re contributing to the anti-technology rather than anti-corporate argument whether they intend to or not. The implication of “taking jobs” is that the jobs should remain, despite a machine being able to do them instead. Sometimes there’s a quality of work argument attached, which is where things get more nuanced again, but often it’s a genuine belief that jobs are a necessity and that blocking technological progress is preferable to eliminating them.
Wealth redistribution and work reform are legitimate anti-corporate arguments around properly allocating the gains of automation for the good of society as a whole, but both require an uncomfortable conversation about (a) the fact they probably won’t happen in a serious way without the kind of societal change that’s historically brought a whole lot of bloodshed as well, (b) they require a widespread severing of the “paid employment -> sense of self worth” link that many people hold dear and that needs to happen before a critical mass of people are willing to seriously push for reform (in a hopefully-nonviolent-but-let’s-be-realistic-about-history way).
You’re going to have to make a stronger argument than just calling people Luddites for not just accepting this carte blanche. Do you actually know anything about the Luddites beliefs and why they were doing what they did?
Why should people be trusting those in positions of power in the tech community?