This is the same song and dance the games industry has been doing since at least Spore with the same results. Turns out good content is curated content. If you look at every facet of game...
This is the same song and dance the games industry has been doing since at least Spore with the same results. Turns out good content is curated content.
If you look at every facet of game development, it's all gotten easier/more productive. You couldn't make the kinds of graphics todays games have in the 1980s, and you'd struggle to make the same kind of systems, and it'd be a massive undertaking to make it cross platform and on and on.
Except for plot. At the end of the day, someone has to write the script, someone has to tie it all together, and if you want voice acting, well someone needs to do all that too. This section of a game takes basically as long in 1980 as it does today in 2024. You have more flexibility thanks to the internet allowing remote collaboration, and there are better story tools for tracking your plot and its timelines, but there haven't been the leaps and bounds in writing a story that we've seen with graphics/processing/dynamic content/whatever.
The main things I see AI actually being used for in this list of problems:
Writers block. Bounce some scenarios off the AI, see what it comes up with, hopefully it gets you moving again.
The downside here is that you might wind up getting the same answers as other people, so if you're looking for something really unique this may not be a good idea.
Voice Acting. This is, potentially, the big one. Being able to rune an entire script through AI and have it voice all of it would save a huge amount of time and money (and likely nuke an industry of voice actors but lets not go there now). You could rough draft it, EA the game, then get feedback from players on where the tone/intonation/pronunciation is off and then go workshop those bits.
The downside being, i'm not sure how easily you can get consistency...right now. Obviously if someone sounds happy when you wanted sad, you'd need to rerun the prompt, but you need it to sound like the same character. I think this, and the initial problem of intonation, are the most solvable problems facing mass use. I wouldn't be surprised if we see better tooling for this side of things (symbols/methods for "Huh you think?" vs "Huh you think?/s" and all similar issues with language).
Annnd that's about it. It might help generate the massive "boilerplate" text of "oh so you're the chosen one", but the simple fact is that randomly generated content will ALWAYS feel jarring. There's nothing stopping skyrim from having bandits cower in fear and run rather than try to jump you because you're wearing demon armor and known throughout the country, except time and effort. Actually putting that in is the issue. Having them do the EXACT same mugging but with "Ahh the chosen one, GET HIM!" will still feel out of place.
In relation to the recent piledriver article, fix your shit first? New Vegas is still one of the best to ever do this and it's because they spent a lot of time and effort on getting it right, and that was under a gross timetable. There are all sorts of interesting writing strategies out there to handle these massive narratives (much like how you change your coding infrastructure so you can handle a larger code base), but no one is bothering.
Instead we're going to slap AI on the box, claim in can make infinite content, and get the exact same quality we saw out of Spore or any other game claiming any other of a zillion "dynamic content" technologies. The smart studios WILL use AI to help in the ways I mentioned, and hell I'm sure there's more I haven't thought of, but it's not a marketing thing. It'd be like advertising the brand of tools your car manufacturer uses.
I was recently preparing a curriculum and rather than start from scratch, I created a large prompt to create one slide per bulleted topic, and in each bullet I have a detailed description of the...
I was recently preparing a curriculum and rather than start from scratch, I created a large prompt to create one slide per bulleted topic, and in each bullet I have a detailed description of the content. When I asked it to include equations and explain variables, it did, it was able to include theorems and preconditions. It saved me some time.
But it still took real work to lay out and describe what I wanted, and it still required careful editing. For stories and conversation arcs it would be the same thing. You will get out what you put into it, which still requires time and care, and editing by good writers.
Right, and I think the harder part is that unlike a "self contained" curriculum, there's MANY parts to a proper narrative that need to reference each other, and it's going to go through revisions....
Right, and I think the harder part is that unlike a "self contained" curriculum, there's MANY parts to a proper narrative that need to reference each other, and it's going to go through revisions. That's where I see the major hurdle.
I do think we'll see a lot of AAA studios trying to cut corners with this crap wherever they can, and a lot of indie studios "punching above their weight" to an extent by using this. Of course i'm going to be a lot more willing to ignore the issues the indie game might have than the AAA game that fired a legion of writers.
Yeah, getting generative AI to track diverging and converging plot lines sounds like it would be as much work as hiring some good writers in the first place. Like you say, I think indie studios...
Yeah, getting generative AI to track diverging and converging plot lines sounds like it would be as much work as hiring some good writers in the first place. Like you say, I think indie studios will be able to create more filled out games, if not always perfectly harmonized, and that is ok I think because it engaged what they could have done rather than a AAA cheapening their product.
To some extent, but it's not as if procedural generation is dead. Many of the most popular games have algorithmic generation at their heart. Minecraft? The twenty billion roguelikes that come out...
This is the same song and dance the games industry has been doing since at least Spore with the same results. Turns out good content is curated content.
To some extent, but it's not as if procedural generation is dead. Many of the most popular games have algorithmic generation at their heart. Minecraft? The twenty billion roguelikes that come out on steam every day?
It might help generate the massive "boilerplate" text of "oh so you're the chosen one", but the simple fact is that randomly generated content will ALWAYS feel jarring.
I don't think this is necessarily the case, you just have to be selective. LLMs aren't suited for making PLOT decisions, but it's excellent for adding chrome to a story. For example, imagine a plane jane NPC that just talks about their life; their wife is dying of some disease, say. You write their baseline text.
Then you take an embedded LLM, and prompt it: Given this base dialogue, rewrite with the context that: (insert all the flags the player has activated). If you just killed the king, he'll talk about how he can't get medicine because of the chaos. If you've won a war, he'll talk about how it's much easier to buy medicine for his wife. And so forth.
No plot changes, but people can change the flavor around what they're saying based on dynamicity. This is all technically possible today with just brute forcing and decision trees, but isn't practical.
Those 20 billion roguelikes mostly suck because of it. The top of the crop of roguelikes are almost all curated content. Just because STS randomly picks your map to some extent doesn't mean it's...
To some extent, but it's not as if procedural generation is dead. Many of the most popular games have algorithmic generation at their heart. Minecraft? The twenty billion roguelikes that come out on steam every day?
Those 20 billion roguelikes mostly suck because of it. The top of the crop of roguelikes are almost all curated content. Just because STS randomly picks your map to some extent doesn't mean it's not combing through an manually curated and balanced pile of content and rules.
Minecraft is procgen done ok, but also is suuuuuch a small % of what was claimed. We've had random map gen long before minecraft, especially for creative games, and minecraft is just taking that a step further. Spelunky is another good example of "doing it right", but it still isn't changing the face of AAA gaming, or applying to the area they always claim it will, which is plot.
As for you example:
I'm really not sure that curating that to not sound schizophrenic or arbitrary (in the same way 40,000 skyrim guards have taken arrows to the knee), isn't going to be as much work as doing it manually. How do you make sure this maps to Plain Jane? Do all Plain Jane's get the same script? If so we're right back to "i used to be an adventurer like you....". If not then how do you keep the consistent? Unique IDs for EVERY Plane Jane? How do you track memory usefully?
The whole problem is that LLMs in most contexts have all sorts of edge case flaws, and that's exactly the kind of immersion breaking stuff that the original article claims they'll solve.
I mean, you can think what you like of procedural games in terms of personal taste but it’s undeniable that games like Balatro become hits over and over again. Many people like procedural...
I mean, you can think what you like of procedural games in terms of personal taste but it’s undeniable that games like Balatro become hits over and over again. Many people like procedural generation because it’s procedural and not baked in.
I don’t really see what edge case you’re talking about. These NPCs aren’t particularly related to each other. There’s not consistency that needs to be held. It’s chrome to make them more responsive.
Exactly. Belatro, FTL, Slay The Spire...none of these are procedurally generated content. They're randomly-combined crafted content for a given run. Its a lot of a different playfeel than full...
Exactly. Belatro, FTL, Slay The Spire...none of these are procedurally generated content.
They're randomly-combined crafted content for a given run. Its a lot of a different playfeel than full procedural generated content, namely map design.
Playing shooters on procedurally generated maps kinda sucks. A well-designed map, that plays to strengths and weaknesses of different aspects of a given loadout in different areas makes all the difference in the world.
No Mans Sky is the most recent example I've played, and it still suffers from the fairly universal 'mile wide, inch deep' problem. After you've been to like 10 planets in each biome type, you can start spotting "Oh the creatures from 1, the textures from 2, the plants from 5, the hills from 8".
Binding of Isaac, Terraria, Spelunky, and No Mans Sky make great use of procedural generation. No Mans Sky became significantly better over time with more updates. LLMs, Stable Diffusion/Image...
Binding of Isaac, Terraria, Spelunky, and No Mans Sky make great use of procedural generation. No Mans Sky became significantly better over time with more updates.
LLMs, Stable Diffusion/Image Generation, and procedural generation likely have a place in entertainment and gaming. You can tightly curate the interactions or usage of these concepts.
I'm a big fan of constructing NPCs as agents with motivations, interactions, relationships, etc... and letting the sandbox happen within a set of guard rails.
To use Binding of Isaac as an example, while procedural generation puts together the level from a library of options, the rooms and bosses are all handcrafted.
To use Binding of Isaac as an example, while procedural generation puts together the level from a library of options, the rooms and bosses are all handcrafted.
I think that's the best way to use Stable Diffusion / LLMs for games. Tightly control the interactions, but allow the tools to introduce a little bit of novelty and flavor.
I think that's the best way to use Stable Diffusion / LLMs for games. Tightly control the interactions, but allow the tools to introduce a little bit of novelty and flavor.
Increase the amount of variability while decreasing the effort to achieve it. If various systems are connected it may allow entirely novel scenarios per play through as well. I could see it being...
Increase the amount of variability while decreasing the effort to achieve it. If various systems are connected it may allow entirely novel scenarios per play through as well.
I could see it being useful in adding relevant damage to a player avatar for example. The developers can handcraft the avatar and various basic states and then use stable diffusion or LLMs to apply relevant damage to relevant areas without designers needing to make a thousand combinations by hand.
What’s really gross is that Minecraft could have easily 10-15 years ago claimed that it was using “AI” to the same level almost anyone is today, if that buzzword had been in vogue at the time. AI...
What’s really gross is that Minecraft could have easily 10-15 years ago claimed that it was using “AI” to the same level almost anyone is today, if that buzzword had been in vogue at the time. AI in 2024 = “the cloud” from like 2016–it’s just “someone else’s computer.”
I've been toying with the idea of creating a GM AI hooked up to a voice chat since chatgpt 3. The main issue in my very limited trials was finding a prompt that didn't cause it to direct the...
I've been toying with the idea of creating a GM AI hooked up to a voice chat since chatgpt 3. The main issue in my very limited trials was finding a prompt that didn't cause it to direct the actions of the players too much. I.e. it usually went
Me: I try to open the door.
GM: You succeed at opening the door. You walk in. There is a troll there which you engage in combat. Finally it flees down a dank tunnel....
And it would go on describing the outcomes and essentially rob the player of agency. This part is a bit of a balancing act and I believe with time and sufficient promot engineering a better more dynamic output could be achieved. It seems however that current LLMs are bad at short answers preferring longer expositions (might just be my prompts).
The second part is something i call the "yes and" disease. LLMs are too accommodating. If I tell it I open a portal to the realm of ferries. It won't protest even if the player character has no magical abilities or portal in sight. Nor if there is no realm of ferries in the setting. I'd want it to be able to deny such requests
Ideally it would even be able to handle false information in the setting, I.e. there is a widespread belief in the setting about dragons that is actually false. This however leads to the main problem of LLMs: Consistency.
Going through a door and then going back will likely lead to completely different descriptions of the room. Those could be awesome in a magical labyrinth, but in regular circumstances it doesn't fly. I think this could be solved by using an assembly of specialised LLMs in a chat; a chronicler that mostly observes and just outputs facts about the world as has been previously stated ny others, a rules laywer that interjects when different tests are required, faction and major npc representatives who would output potential actions these undertak, and finally a mouthpiece that is responsible for narrating the actual play and drives the dialogue with the players. There could of course be more of these specialised agents to handle some element of the game...
w/r/t intonation, we're already there. We have emotional style transfer algorithms that can be applied to voice clones and markup languages for annotating TTS inputs. If you want to check out some...
w/r/t intonation, we're already there. We have emotional style transfer algorithms that can be applied to voice clones and markup languages for annotating TTS inputs. If you want to check out some pretty cool open source stuff, https://github.com/netease-youdao/EmotiVoice
Maybe I’m short sighted for thinking so but as long as dialogue doesn't lead to mechanics or some sort of other novel interaction, conversing with AI in a game doesn’t seem like a value add. The...
Maybe I’m short sighted for thinking so but as long as dialogue doesn't lead to mechanics or some sort of other novel interaction, conversing with AI in a game doesn’t seem like a value add. The article mentions Elden Ring in the same sentence as Starfield, but that game is basically just a handful of lines per NPC that amount to “go this way”, which ultimately cuts down on fluff for the sake of gameplay. Even if your game is story driven ideally there’s some sort of intent behind your dialogue and script, so I’m really not seeing the appeal here.
I think it might work best in a "cozy" game with a focus on interacting with NPCs rather than an overall story, like Animal Crossing or Stardew Valley. Dialogue can get stale in those games pretty...
I think it might work best in a "cozy" game with a focus on interacting with NPCs rather than an overall story, like Animal Crossing or Stardew Valley. Dialogue can get stale in those games pretty easily, so having AI generate conversations could add to it.
To add to that: there's a mystery game called Vaudeville that uses AI to generate conversations with the NPCs. The AI isn't really needed honestly since it has a solidly defined story with one answer, and apparently actually works against solving it since they can lie or mislead you with wrong information (and not intentionally). Lots of frustration from people trying to solve the mystery.
But if you ignore the actual story and just goof around with the AI, you can apparently have a lot of fun. Generally, positive reviews come from people who ignore the murder mystery and just mess around with the AI. Seriously, sort by positive reviews, people have some pretty funny stories. I first learned about it when CallMeKevin played it, and that was the hardest I'd laughed at a video in a while.
My takeaway is that AI would be best for an immersive simulation game without a solid end goal, or at least one that isn't dependent on NPCs since... Well, generative AI conversations can go in a lot of directions.
I found that odd as well. You have ER, which is 90% environmental story telling/worldbuilding with some small amount of actual dialogue, and then you have starfield, the poster child for phoned in...
I found that odd as well. You have ER, which is 90% environmental story telling/worldbuilding with some small amount of actual dialogue, and then you have starfield, the poster child for phoned in false choice.
For regular non-story games, that might have what's essentially throwaway dialog, LLM tech adds very little. I wouldn't say it adds nothing, but basically little. For example, if it's a shooter,...
For regular non-story games, that might have what's essentially throwaway dialog, LLM tech adds very little. I wouldn't say it adds nothing, but basically little.
For example, if it's a shooter, and the NPCs are going to yell shit at you like "Duck" or "this way" or "the secret cabal is holed up in there but we have to find a way in" then yeah that dialog adds very little. Rather than scripting the one line, that fans of that game are going to get to know very well as they play through repeatedly, the game engine could throw to an LLM routine with prompts that request back a sensible, believable, variable and always changing on each playthrough 'realistic' line.
So instead of "the secret cabal is holed up in there but we have to find a way in" the LLM could very easily find dozens of ways to say the same thing. For gameplay, all that matters is the player is told "go in there to keep playing." But for texture and cosmetic, art reasons, there are obviously many, many, many ways that could be said.
Finding different ones isn't something a game company is going to sit down and script out, much less pay for a voice actor to say ... all so there are dozens of options on deck to keep the sameness of current games from hitting the player smack in the face.
I'm replaying Cyberpunk 2077 right now, and let me tell you a properly worked up LLM tool could easily add tremendous levels of realism and texture to the story. Even without touching the core story scenes. The randoms around the city, for example; they could have a functionally unlimited amount of responses to the player when they're interacted with (which would include having fights or explosions or whatever erupting next to them; not just when you walk up and push action to trigger them).
You play a tabletop RPG, most GMs can come up with human responses to inject into their NPCs. Sure it's just the bartender or a beggar on the street, but unlike in a computer RPG, those roles don't have just the one line they always have. Maybe this time the beggar is cheerful, next time they're scared. Maybe they saw you fight that dude ten minutes ago, and have a comment. There's thousands upon thousands of examples of why any actor in a story might come up with something to say. Variable subject, tone, emotion, etc.
But to get them to say those things, you need a writer to script it all out. And a VA to speak it. And a dev team that goes to that effort, stores it all in the game data, and has it ready to be called.
No one does that. None of them think it adds any value. They don't want to take the cost (financial as well as game space) hit, so they don't bother. They figure gamers will just know that's part of the way it has to be, and ignore how the only scenes that feel like a real story are the ones that actually got scripted and acted out.
But even those play out the same way, every time. In Cyberpunk 2077, once you get into the second act with the meat of the story, there's at least half a dozen major story threads going at any given point. You might go talk to Rogue to trigger Panam's quest, or call Judy looking for Evelyn, or go sit down with Takemura, and so on.
But you can tackle all that in any order you want. And most of those missions have little sub-steps and chains, where it's scripted in that there's a pause or a waiting period. During which you probably go off and do other stuff until that mission's ready again. You might rush through the story missions, but you might leave them waiting for a long time while you do other stuff.
An LLM overseen by a proper writer (not a coder who dabbles, but an actual writer) could lay out a structure and framework for how various missions and NPCs in them might interact. How they might react to waiting, or what you have or haven't done yet. Maybe Judy heard about you talking to Rogue, or knows you have been laying waste to vast sections of Watson's underworld, or whatever. Those things could be used the way a human GM would, and tweak how Judy interacts with you. With some framing, an LLM could spin off dynamic (at the least, much more dynamic) responses to you when you call Judy or finally go sit down with Takemura.
So a shooter or RTS player might not care about the value add of a more realistic computer "GM" helping tell the story, but some games are story.
On Cyberpunk 2077, I love it. Such a great story, told well, plus the interactive aspect; it's fantastic. Would make an amazing movie. Hell, if someone ran through and did a "cinematic" playthrough and knew how to edit, they could put the whole thing together as a full on movie as is and it'd be just as believable and fun as any movie you might have to so sit in a theater to watch.
But after you've seen the story scenes, you've seen them. You know them. Nothing changes. It's basically a very expensive movie. The interactive aspects give it some replayability, but you're mostly just replaying to play the game at that point, not see the story. So you skip a lot.
What if the story presentation was more dynamic? What if it took what you do and don't do into consideration on a more granular level? Are you ignoring stealth, or carrying no weapons, or always polite, or whatever? There's a bunch of stuff that could be used as triggers, the same as a writer would use those same things to inform dialog and tone.
The story might be different every time. And, nothing says the devs couldn't release with their one bespoke human written and crafted storyline from start to finish .... and give you the option of turning on the LLM enhancements for your replays. So the first time through, you get the creative team's vision of that story, but after that as you stay with a game (and story) for a while, it could find all sorts of ways to keep that story interesting to you.
LLM is just an engine that knows how to assemble words into human communication. It's pretty good at it right now, and presumably will get better. It is orders of magnitude closer to human communication than any computer program ever has been before. Is it a full on human Gamemaster, sitting there scripting your story game for you?
No.
Not yet. But it's close. It seems quite realistic that it'll get there, and not at some mysterious far off moment in time that's never going to happen; it's just a matter of years it feels like. Not that long at all.
I just truly don't get who these articles are for. Yes, for a dev company, the idea of making an eternal universe that never ends sounds great. But for any player, why would they interact with...
I just truly don't get who these articles are for. Yes, for a dev company, the idea of making an eternal universe that never ends sounds great. But for any player, why would they interact with your AI boilerplate characters when they could just play another game?
Say LLMs are better in 10 years. So what? Why would a player stick around when there's 10 other games in their backlog with curated stories? What, because the barmaid sound a bit more realistic?
I think this sort of tech can be used for older RPGs like Daggerfall or Morrowind to 'enhance' their content and make their world immersive. I don't know how well it can be used for modern-day...
I think this sort of tech can be used for older RPGs like Daggerfall or Morrowind to 'enhance' their content and make their world immersive. I don't know how well it can be used for modern-day RPGs, but out of all the AI stuff I hear in the day this can be very interesting.
Maybe if someone wanted to make another game of Daggerfall's size that felt more realized it could make sense, but Morrowind was good BECAUSE it was curated (even if I was furious at what was lost...
Maybe if someone wanted to make another game of Daggerfall's size that felt more realized it could make sense, but Morrowind was good BECAUSE it was curated (even if I was furious at what was lost vs Daggerfall at the time).
This is the same song and dance the games industry has been doing since at least Spore with the same results. Turns out good content is curated content.
If you look at every facet of game development, it's all gotten easier/more productive. You couldn't make the kinds of graphics todays games have in the 1980s, and you'd struggle to make the same kind of systems, and it'd be a massive undertaking to make it cross platform and on and on.
Except for plot. At the end of the day, someone has to write the script, someone has to tie it all together, and if you want voice acting, well someone needs to do all that too. This section of a game takes basically as long in 1980 as it does today in 2024. You have more flexibility thanks to the internet allowing remote collaboration, and there are better story tools for tracking your plot and its timelines, but there haven't been the leaps and bounds in writing a story that we've seen with graphics/processing/dynamic content/whatever.
The main things I see AI actually being used for in this list of problems:
Writers block. Bounce some scenarios off the AI, see what it comes up with, hopefully it gets you moving again.
The downside here is that you might wind up getting the same answers as other people, so if you're looking for something really unique this may not be a good idea.
Voice Acting. This is, potentially, the big one. Being able to rune an entire script through AI and have it voice all of it would save a huge amount of time and money (and likely nuke an industry of voice actors but lets not go there now). You could rough draft it, EA the game, then get feedback from players on where the tone/intonation/pronunciation is off and then go workshop those bits.
The downside being, i'm not sure how easily you can get consistency...right now. Obviously if someone sounds happy when you wanted sad, you'd need to rerun the prompt, but you need it to sound like the same character. I think this, and the initial problem of intonation, are the most solvable problems facing mass use. I wouldn't be surprised if we see better tooling for this side of things (symbols/methods for "Huh you think?" vs "Huh you think?/s" and all similar issues with language).
Annnd that's about it. It might help generate the massive "boilerplate" text of "oh so you're the chosen one", but the simple fact is that randomly generated content will ALWAYS feel jarring. There's nothing stopping skyrim from having bandits cower in fear and run rather than try to jump you because you're wearing demon armor and known throughout the country, except time and effort. Actually putting that in is the issue. Having them do the EXACT same mugging but with "Ahh the chosen one, GET HIM!" will still feel out of place.
In relation to the recent piledriver article, fix your shit first? New Vegas is still one of the best to ever do this and it's because they spent a lot of time and effort on getting it right, and that was under a gross timetable. There are all sorts of interesting writing strategies out there to handle these massive narratives (much like how you change your coding infrastructure so you can handle a larger code base), but no one is bothering.
Instead we're going to slap AI on the box, claim in can make infinite content, and get the exact same quality we saw out of Spore or any other game claiming any other of a zillion "dynamic content" technologies. The smart studios WILL use AI to help in the ways I mentioned, and hell I'm sure there's more I haven't thought of, but it's not a marketing thing. It'd be like advertising the brand of tools your car manufacturer uses.
I was recently preparing a curriculum and rather than start from scratch, I created a large prompt to create one slide per bulleted topic, and in each bullet I have a detailed description of the content. When I asked it to include equations and explain variables, it did, it was able to include theorems and preconditions. It saved me some time.
But it still took real work to lay out and describe what I wanted, and it still required careful editing. For stories and conversation arcs it would be the same thing. You will get out what you put into it, which still requires time and care, and editing by good writers.
Right, and I think the harder part is that unlike a "self contained" curriculum, there's MANY parts to a proper narrative that need to reference each other, and it's going to go through revisions. That's where I see the major hurdle.
I do think we'll see a lot of AAA studios trying to cut corners with this crap wherever they can, and a lot of indie studios "punching above their weight" to an extent by using this. Of course i'm going to be a lot more willing to ignore the issues the indie game might have than the AAA game that fired a legion of writers.
Yeah, getting generative AI to track diverging and converging plot lines sounds like it would be as much work as hiring some good writers in the first place. Like you say, I think indie studios will be able to create more filled out games, if not always perfectly harmonized, and that is ok I think because it engaged what they could have done rather than a AAA cheapening their product.
To some extent, but it's not as if procedural generation is dead. Many of the most popular games have algorithmic generation at their heart. Minecraft? The twenty billion roguelikes that come out on steam every day?
I don't think this is necessarily the case, you just have to be selective. LLMs aren't suited for making PLOT decisions, but it's excellent for adding chrome to a story. For example, imagine a plane jane NPC that just talks about their life; their wife is dying of some disease, say. You write their baseline text.
Then you take an embedded LLM, and prompt it: Given this base dialogue, rewrite with the context that: (insert all the flags the player has activated). If you just killed the king, he'll talk about how he can't get medicine because of the chaos. If you've won a war, he'll talk about how it's much easier to buy medicine for his wife. And so forth.
No plot changes, but people can change the flavor around what they're saying based on dynamicity. This is all technically possible today with just brute forcing and decision trees, but isn't practical.
Those 20 billion roguelikes mostly suck because of it. The top of the crop of roguelikes are almost all curated content. Just because STS randomly picks your map to some extent doesn't mean it's not combing through an manually curated and balanced pile of content and rules.
Minecraft is procgen done ok, but also is suuuuuch a small % of what was claimed. We've had random map gen long before minecraft, especially for creative games, and minecraft is just taking that a step further. Spelunky is another good example of "doing it right", but it still isn't changing the face of AAA gaming, or applying to the area they always claim it will, which is plot.
As for you example:
I'm really not sure that curating that to not sound schizophrenic or arbitrary (in the same way 40,000 skyrim guards have taken arrows to the knee), isn't going to be as much work as doing it manually. How do you make sure this maps to Plain Jane? Do all Plain Jane's get the same script? If so we're right back to "i used to be an adventurer like you....". If not then how do you keep the consistent? Unique IDs for EVERY Plane Jane? How do you track memory usefully?
The whole problem is that LLMs in most contexts have all sorts of edge case flaws, and that's exactly the kind of immersion breaking stuff that the original article claims they'll solve.
I mean, you can think what you like of procedural games in terms of personal taste but it’s undeniable that games like Balatro become hits over and over again. Many people like procedural generation because it’s procedural and not baked in.
I don’t really see what edge case you’re talking about. These NPCs aren’t particularly related to each other. There’s not consistency that needs to be held. It’s chrome to make them more responsive.
Maybe i'm missing something. What about balatro is procedurally generated?
Exactly. Belatro, FTL, Slay The Spire...none of these are procedurally generated content.
They're randomly-combined crafted content for a given run. Its a lot of a different playfeel than full procedural generated content, namely map design.
Playing shooters on procedurally generated maps kinda sucks. A well-designed map, that plays to strengths and weaknesses of different aspects of a given loadout in different areas makes all the difference in the world.
No Mans Sky is the most recent example I've played, and it still suffers from the fairly universal 'mile wide, inch deep' problem. After you've been to like 10 planets in each biome type, you can start spotting "Oh the creatures from 1, the textures from 2, the plants from 5, the hills from 8".
Binding of Isaac, Terraria, Spelunky, and No Mans Sky make great use of procedural generation. No Mans Sky became significantly better over time with more updates.
LLMs, Stable Diffusion/Image Generation, and procedural generation likely have a place in entertainment and gaming. You can tightly curate the interactions or usage of these concepts.
I'm a big fan of constructing NPCs as agents with motivations, interactions, relationships, etc... and letting the sandbox happen within a set of guard rails.
To use Binding of Isaac as an example, while procedural generation puts together the level from a library of options, the rooms and bosses are all handcrafted.
I think that's the best way to use Stable Diffusion / LLMs for games. Tightly control the interactions, but allow the tools to introduce a little bit of novelty and flavor.
What do LLMs/ML add here over the status quo of picking from a list with an RNG as happens in BoI though?
Increase the amount of variability while decreasing the effort to achieve it. If various systems are connected it may allow entirely novel scenarios per play through as well.
I could see it being useful in adding relevant damage to a player avatar for example. The developers can handcraft the avatar and various basic states and then use stable diffusion or LLMs to apply relevant damage to relevant areas without designers needing to make a thousand combinations by hand.
What’s really gross is that Minecraft could have easily 10-15 years ago claimed that it was using “AI” to the same level almost anyone is today, if that buzzword had been in vogue at the time. AI in 2024 = “the cloud” from like 2016–it’s just “someone else’s computer.”
I've been toying with the idea of creating a GM AI hooked up to a voice chat since chatgpt 3. The main issue in my very limited trials was finding a prompt that didn't cause it to direct the actions of the players too much. I.e. it usually went
Me: I try to open the door.
GM: You succeed at opening the door. You walk in. There is a troll there which you engage in combat. Finally it flees down a dank tunnel....
And it would go on describing the outcomes and essentially rob the player of agency. This part is a bit of a balancing act and I believe with time and sufficient promot engineering a better more dynamic output could be achieved. It seems however that current LLMs are bad at short answers preferring longer expositions (might just be my prompts).
The second part is something i call the "yes and" disease. LLMs are too accommodating. If I tell it I open a portal to the realm of ferries. It won't protest even if the player character has no magical abilities or portal in sight. Nor if there is no realm of ferries in the setting. I'd want it to be able to deny such requests
Ideally it would even be able to handle false information in the setting, I.e. there is a widespread belief in the setting about dragons that is actually false. This however leads to the main problem of LLMs: Consistency.
Going through a door and then going back will likely lead to completely different descriptions of the room. Those could be awesome in a magical labyrinth, but in regular circumstances it doesn't fly. I think this could be solved by using an assembly of specialised LLMs in a chat; a chronicler that mostly observes and just outputs facts about the world as has been previously stated ny others, a rules laywer that interjects when different tests are required, faction and major npc representatives who would output potential actions these undertak, and finally a mouthpiece that is responsible for narrating the actual play and drives the dialogue with the players. There could of course be more of these specialised agents to handle some element of the game...
w/r/t intonation, we're already there. We have emotional style transfer algorithms that can be applied to voice clones and markup languages for annotating TTS inputs. If you want to check out some pretty cool open source stuff, https://github.com/netease-youdao/EmotiVoice
Maybe I’m short sighted for thinking so but as long as dialogue doesn't lead to mechanics or some sort of other novel interaction, conversing with AI in a game doesn’t seem like a value add. The article mentions Elden Ring in the same sentence as Starfield, but that game is basically just a handful of lines per NPC that amount to “go this way”, which ultimately cuts down on fluff for the sake of gameplay. Even if your game is story driven ideally there’s some sort of intent behind your dialogue and script, so I’m really not seeing the appeal here.
I think it might work best in a "cozy" game with a focus on interacting with NPCs rather than an overall story, like Animal Crossing or Stardew Valley. Dialogue can get stale in those games pretty easily, so having AI generate conversations could add to it.
To add to that: there's a mystery game called Vaudeville that uses AI to generate conversations with the NPCs. The AI isn't really needed honestly since it has a solidly defined story with one answer, and apparently actually works against solving it since they can lie or mislead you with wrong information (and not intentionally). Lots of frustration from people trying to solve the mystery.
But if you ignore the actual story and just goof around with the AI, you can apparently have a lot of fun. Generally, positive reviews come from people who ignore the murder mystery and just mess around with the AI. Seriously, sort by positive reviews, people have some pretty funny stories. I first learned about it when CallMeKevin played it, and that was the hardest I'd laughed at a video in a while.
My takeaway is that AI would be best for an immersive simulation game without a solid end goal, or at least one that isn't dependent on NPCs since... Well, generative AI conversations can go in a lot of directions.
I found that odd as well. You have ER, which is 90% environmental story telling/worldbuilding with some small amount of actual dialogue, and then you have starfield, the poster child for phoned in false choice.
For regular non-story games, that might have what's essentially throwaway dialog, LLM tech adds very little. I wouldn't say it adds nothing, but basically little.
For example, if it's a shooter, and the NPCs are going to yell shit at you like "Duck" or "this way" or "the secret cabal is holed up in there but we have to find a way in" then yeah that dialog adds very little. Rather than scripting the one line, that fans of that game are going to get to know very well as they play through repeatedly, the game engine could throw to an LLM routine with prompts that request back a sensible, believable, variable and always changing on each playthrough 'realistic' line.
So instead of "the secret cabal is holed up in there but we have to find a way in" the LLM could very easily find dozens of ways to say the same thing. For gameplay, all that matters is the player is told "go in there to keep playing." But for texture and cosmetic, art reasons, there are obviously many, many, many ways that could be said.
Finding different ones isn't something a game company is going to sit down and script out, much less pay for a voice actor to say ... all so there are dozens of options on deck to keep the sameness of current games from hitting the player smack in the face.
I'm replaying Cyberpunk 2077 right now, and let me tell you a properly worked up LLM tool could easily add tremendous levels of realism and texture to the story. Even without touching the core story scenes. The randoms around the city, for example; they could have a functionally unlimited amount of responses to the player when they're interacted with (which would include having fights or explosions or whatever erupting next to them; not just when you walk up and push action to trigger them).
You play a tabletop RPG, most GMs can come up with human responses to inject into their NPCs. Sure it's just the bartender or a beggar on the street, but unlike in a computer RPG, those roles don't have just the one line they always have. Maybe this time the beggar is cheerful, next time they're scared. Maybe they saw you fight that dude ten minutes ago, and have a comment. There's thousands upon thousands of examples of why any actor in a story might come up with something to say. Variable subject, tone, emotion, etc.
But to get them to say those things, you need a writer to script it all out. And a VA to speak it. And a dev team that goes to that effort, stores it all in the game data, and has it ready to be called.
No one does that. None of them think it adds any value. They don't want to take the cost (financial as well as game space) hit, so they don't bother. They figure gamers will just know that's part of the way it has to be, and ignore how the only scenes that feel like a real story are the ones that actually got scripted and acted out.
But even those play out the same way, every time. In Cyberpunk 2077, once you get into the second act with the meat of the story, there's at least half a dozen major story threads going at any given point. You might go talk to Rogue to trigger Panam's quest, or call Judy looking for Evelyn, or go sit down with Takemura, and so on.
But you can tackle all that in any order you want. And most of those missions have little sub-steps and chains, where it's scripted in that there's a pause or a waiting period. During which you probably go off and do other stuff until that mission's ready again. You might rush through the story missions, but you might leave them waiting for a long time while you do other stuff.
An LLM overseen by a proper writer (not a coder who dabbles, but an actual writer) could lay out a structure and framework for how various missions and NPCs in them might interact. How they might react to waiting, or what you have or haven't done yet. Maybe Judy heard about you talking to Rogue, or knows you have been laying waste to vast sections of Watson's underworld, or whatever. Those things could be used the way a human GM would, and tweak how Judy interacts with you. With some framing, an LLM could spin off dynamic (at the least, much more dynamic) responses to you when you call Judy or finally go sit down with Takemura.
So a shooter or RTS player might not care about the value add of a more realistic computer "GM" helping tell the story, but some games are story.
On Cyberpunk 2077, I love it. Such a great story, told well, plus the interactive aspect; it's fantastic. Would make an amazing movie. Hell, if someone ran through and did a "cinematic" playthrough and knew how to edit, they could put the whole thing together as a full on movie as is and it'd be just as believable and fun as any movie you might have to so sit in a theater to watch.
But after you've seen the story scenes, you've seen them. You know them. Nothing changes. It's basically a very expensive movie. The interactive aspects give it some replayability, but you're mostly just replaying to play the game at that point, not see the story. So you skip a lot.
What if the story presentation was more dynamic? What if it took what you do and don't do into consideration on a more granular level? Are you ignoring stealth, or carrying no weapons, or always polite, or whatever? There's a bunch of stuff that could be used as triggers, the same as a writer would use those same things to inform dialog and tone.
The story might be different every time. And, nothing says the devs couldn't release with their one bespoke human written and crafted storyline from start to finish .... and give you the option of turning on the LLM enhancements for your replays. So the first time through, you get the creative team's vision of that story, but after that as you stay with a game (and story) for a while, it could find all sorts of ways to keep that story interesting to you.
LLM is just an engine that knows how to assemble words into human communication. It's pretty good at it right now, and presumably will get better. It is orders of magnitude closer to human communication than any computer program ever has been before. Is it a full on human Gamemaster, sitting there scripting your story game for you?
No.
Not yet. But it's close. It seems quite realistic that it'll get there, and not at some mysterious far off moment in time that's never going to happen; it's just a matter of years it feels like. Not that long at all.
I just truly don't get who these articles are for. Yes, for a dev company, the idea of making an eternal universe that never ends sounds great. But for any player, why would they interact with your AI boilerplate characters when they could just play another game?
Say LLMs are better in 10 years. So what? Why would a player stick around when there's 10 other games in their backlog with curated stories? What, because the barmaid sound a bit more realistic?
I think this sort of tech can be used for older RPGs like Daggerfall or Morrowind to 'enhance' their content and make their world immersive. I don't know how well it can be used for modern-day RPGs, but out of all the AI stuff I hear in the day this can be very interesting.
Maybe if someone wanted to make another game of Daggerfall's size that felt more realized it could make sense, but Morrowind was good BECAUSE it was curated (even if I was furious at what was lost vs Daggerfall at the time).