Many orders of magnitudes more compute power are used to generate a 2-minute demo of something that is unquestionably worse in every conceivable way than the thing we've already had since 1997.
Many orders of magnitudes more compute power are used to generate a 2-minute demo of something that is unquestionably worse in every conceivable way than the thing we've already had since 1997.
Is it trying to be better though? Research around neural networks is, in the grand scheme of things, still in its early stages, so many projects like this get created as a cool proof of concept...
Is it trying to be better though?
Research around neural networks is, in the grand scheme of things, still in its early stages, so many projects like this get created as a cool proof of concept used to validate some internal research. Sometimes it doesn't lead anywhere, sometimes it does - and if it does, it may not have anything in common with "AI Quake". That was likely just used as an available, useful and widely known source of training data.
I think it's quite cool in the same way that early image generation was - it genuinely feels dream-like, and it's a novel experience.
I think this is a very salient point. Tennis for two was one of the first video games. I am sure people could have said it was meaningless, and you could just go out and play tennis in real life...
I think this is a very salient point. Tennis for two was one of the first video games. I am sure people could have said it was meaningless, and you could just go out and play tennis in real life for a better experience. But that’s not really the point. And now look at where video games are, 60 years later. Will generative AI ever replace normal game engines? I doubt it. Video games never replaced tennis, and I am happy that both exist. It’ll be interesting to see what happens to generative ai over the next 60 years.
As an example of a practical application of the research, see https://developer.nvidia.com/blog/nvidia-rtx-neural-rendering-introduces-next-era-of-ai-powered-graphics-innovation/ What you're doing...
What you're doing there is generating essentially fake details as a post-rendering step. Not the full frame, like in the Microsoft examples, but parts of the frame, so you can have detail but avoid having to meticulously detail every stone in the game.
The research done is important to improving this, as things like making sure the details make sense temporally (e.g a crack on a rock stays in the same place, 3 dimensionally, between frames, or moves logically when the rock moves).
This might be pedantic, but research around neural networks is not remotely in its early stages. Neural networks have been around for decades. You could argue (though imo it's still a bit sketchy)...
Research around neural networks is, in the grand scheme of things, still in its early stages, so many projects like this get created as a cool proof of concept used to validate some internal research.
This might be pedantic, but research around neural networks is not remotely in its early stages. Neural networks have been around for decades. You could argue (though imo it's still a bit sketchy) that research into transformers, the specific type of neural network that LLMs and modern GenAI are based on, is in its early stages, since they were first invented in 2017. But, as a point of comparison, that was while I was in undergrad, RNNs and LSTMs (both well-known and heavily used earlier neural network architectures) were first implemented before I was born.
I overall agree with your main point, that this is much more of an attempt to test the limits of new technology that's on the forefront of research. But there's already too much confusion with people thinking AI automatically means modern GenAI, let's not start the same thing with a term like neural network that still means something!
I was thinking about whether to clarify on that or not. I took a class on neural networks some years ago, I know how old the concept is, but imo it could not be fully realized because we did not...
I was thinking about whether to clarify on that or not. I took a class on neural networks some years ago, I know how old the concept is, but imo it could not be fully realized because we did not have the computing power. A ton of low hanging fruit in research only started to appear some time after we got cheap parallel compute with ordinary GPUs (so I'm not just talking about transformers). Kind of like Wolfram's New Kind of Science, only much more computationally intensive.
I can understand not really counting the very old examples, like the Perceptron, but I wouldn't necessarily call anything after RNNs "low-hanging fruit" in that sense. Certainly access to...
I can understand not really counting the very old examples, like the Perceptron, but I wouldn't necessarily call anything after RNNs "low-hanging fruit" in that sense. Certainly access to computing resources has been a barrier throughout research into neural nets, but the forefront of research has tended to seesaw between coming up with new architectures and seeing what the best architectures can do with more and more data and computing power (we've been solidly in the latter phase for a little while now).
Sure, it's definitely up for discussion and maybe the words I used were not the best chosen, but imo it's clear that we've been in a sort of a new era of research in the last decade or so, however...
Sure, it's definitely up for discussion and maybe the words I used were not the best chosen, but imo it's clear that we've been in a sort of a new era of research in the last decade or so, however we call it, and a lot is yet to come with both more research and more compute capacity.
My reaction to the demo was also "wow, this sucks." They use the words "play" and "game". I can play again, as many times as I like! But am I really playing? There are countless "free" or cheap...
My reaction to the demo was also "wow, this sucks." They use the words "play" and "game". I can play again, as many times as I like! But am I really playing? There are countless "free" or cheap experiences that are infinity times more fun.
But when I read your comment I remembered John Carmack is currently working on AI full time, so I no longer know how I feel about it. Maybe someone will actually make this stuff work at some point?
John Carmack is working on AI full time, but I don't think he's working on making fully AI-generated video games. This is a tech demo that just screams all flash and no substance. The goal is to...
John Carmack is working on AI full time, but I don't think he's working on making fully AI-generated video games.
This is a tech demo that just screams all flash and no substance. The goal is to keep the AI hype train rolling as long as possible before it collapses like the dotcom bubble.
Just like with the dotcom bubble, the underlying technology won't go away, and we will continue to find productive uses for it. But I seriously doubt that this is one of them.
Scroll down to the big “Try now” button to play (or go directly there via this link). The game takes a couple minutes to load and is entirely played via keyboard. Instructions are on the page. On...
Scroll down to the big “Try now” button to play (or go directly there via this link).
The game takes a couple minutes to load and is entirely played via keyboard. Instructions are on the page.
On the one hand, I immediately began imagining the many ways in which generative AI could be a great way to breathe new life into some classic games by creating ROM hacks (I was thinking for example how it could instantly create entirely new Pokémon campaign for gens I through V).
On the other hand, I’m afraid that we’ll soon be bombarded with AI game slop, “vibe coded” garbage, and that developers might lose jobs.
I haven’t tried it but I assumed it’s basically the idea behind the Doom and Minecraft demos that were going around a while back. Basically: technically impressive but not much of an actual game...
I haven’t tried it but I assumed it’s basically the idea behind the Doom and Minecraft demos that were going around a while back. Basically: technically impressive but not much of an actual game due to the utter lack of coherence or logical systems.
Sounds about right. I would argue that it's not even really a game, i.e. a program is not being generated and run here, but a highly interactive video. It's constantly cycling new frames, but a...
Sounds about right. I would argue that it's not even really a game, i.e. a program is not being generated and run here, but a highly interactive video. It's constantly cycling new frames, but a limited selection of user inputs can influence what is drawn next. Enemies are not "spawned", but the complex prompt attempts to draw them where it believes they normally should be based on the given frame set.
In the video feed, Copilot attempts to manage health and armor by drawing a frame with lower numeric values in the same spots, but frequently guesses wrong. As a result, sometimes "taking damage" increases health, but usually the image just cycles between a range of 98/88 & 82 HP. Occasionally, as an enemy is drawn attacking the screen, it will blur enough over time to eventually turn into an inanimate object, which I find delightfully funny.
I'm of two minds on this. From a pure technological progress point of view it's intriguing and amazing that this is even possible but from an ex-gamedev perspective every fibre of my being says...
I'm of two minds on this. From a pure technological progress point of view it's intriguing and amazing that this is even possible but from an ex-gamedev perspective every fibre of my being says throw this junk right in the bin and let it rot.
Very interesting tech demo, thanks for sharing. It's like being in someone's dream to a certain extent. For example, you might walk past an enemy, and look back and the model has forgotten about...
Very interesting tech demo, thanks for sharing. It's like being in someone's dream to a certain extent. For example, you might walk past an enemy, and look back and the model has forgotten about them. Or when you get in a fight it's never really clear what is happening as you shoot them and what will happen. Very surreal. If something like this becomes more accessible, I think in the short term we could see some really neat art projects.
For anyone interested in Carmack's take on this, it is more interesting, and more nuanced, than pro or con. Me personally, I majored in CogSci back in the 90's at UCSD, and have been active in the...
For anyone interested in Carmack's take on this, it is more interesting, and more nuanced, than pro or con.
Me personally, I majored in CogSci back in the 90's at UCSD, and have been active in the machine learning world for decades. The past few years have been a fever dream of never-expected-that-in-a-million-years results. Even if progress stopped tomorrow, we'd have years of work in front of us understanding how these giant matrixes are so capable at so many things.
(Does this mean I think generative tooling is good? Bad? It is both, and neither. The same tool steel that makes a surgical scalpel to save lives makes a bayonet to end them. Most technology is like this. My concerns are of power concentration, and social control along the lines of Sort by Controversial, which was fiction when written a little less than a decade ago. Now, I'm not so sure.)
I got the impression this is "just" generative imagery. Aren't multimodal models all the rage? Why don't they use one that can handle game logic like health and ammo? Perhaps the paper is more...
I got the impression this is "just" generative imagery. Aren't multimodal models all the rage? Why don't they use one that can handle game logic like health and ammo? Perhaps the paper is more specific.
Like others, this seems wrong to me. The Nvidia system has some interest to me, but utilizing LLMs for more natural dialogue is about the extent I want to see AI in video games (and even that I'm wary of).
I think an LLM-powered “narrative engine” could be really interesting, something that tracks story details and characters and generates compelling dramatic arcs in response to player actions. That...
I think an LLM-powered “narrative engine” could be really interesting, something that tracks story details and characters and generates compelling dramatic arcs in response to player actions. That would go hand-in-hand with the dialogue idea you mentioned but take it a step further. I’d be open to that application of AI! Of course If all the games started doing it (if they were very similar, at least) that would probably wear out its welcome fast.
There's an AI D&D bot/thing called Friends & Fables. I tested out the Discord bot version (free trial) and it was pretty cool to have the ability to play D&D without pissing people off because I...
There's an AI D&D bot/thing called Friends & Fables. I tested out the Discord bot version (free trial) and it was pretty cool to have the ability to play D&D without pissing people off because I can't commit X number of hours in a row on a regular basis.
I can't bring myself to pay for it, but just adding this to say the generative storytelling experience is out there and is actually pretty good already.
You sort of describe a more synchronous version of how I've been using chatgpt to create tactical encounters for DND and other table top games. I create a structured set of files that contain the...
You sort of describe a more synchronous version of how I've been using chatgpt to create tactical encounters for DND and other table top games. I create a structured set of files that contain the player sheets, campaign setting, world details, the story of the characters so far, and a file for each scenario complete with conclusion and summary of events. I load these into a project in the chat bot, along with a project instruction about the tone, what it should generate, etc. I then upload updated files after each scenario.
The result is it can create some really nice tactical encounters that includes combat and investigative checks for each player, character specific prompts and flavor text, etc. The stories it suggests tend to be pretty mid, but if given all the data in the right format it can do all the tedious time consuming work of making sure there is something special or a role for every player in every encounter, factoring race, background, class, and major prior choices.
I was thinking more like devs still maintain narrative control, but LLMs would categorize your freeform response and tweak the scripted dialogue to react to you and your tone more realistically....
I was thinking more like devs still maintain narrative control, but LLMs would categorize your freeform response and tweak the scripted dialogue to react to you and your tone more realistically. Being able to literally speak to characters would be lit. While I'm not confident enough in AI to be able to craft unique storylines and engaging characters, it could still be a fun space to explore. But this whole fully generated toy? An abomination.
Boo, made for chrome and no mouse. It doesn't capture the keyboard so it tries to search the page in firefox. :/ Overall, not bad for a proof of concept, but it's got bugs to work out. The enemies...
Boo, made for chrome and no mouse. It doesn't capture the keyboard so it tries to search the page in firefox. :/
Overall, not bad for a proof of concept, but it's got bugs to work out.
The enemies disappear if they get behind you and I think the map changed on me, either that or I did something weird. Enemies died upright (physics issues) and super deformed. I wonder if this'll be the next "simulator" game (Surgeon, Goat, "I am bread", that one with old people, etc...) where half the fun is fighting the game engine and the crazy stuff that happens when you do.
Is everyone else getting horrible lag/slow frame rate? I know it isn't exactly going to be great but the lag and frame rate really screwed it up for me.
Is everyone else getting horrible lag/slow frame rate? I know it isn't exactly going to be great but the lag and frame rate really screwed it up for me.
Yep, that’s the way it runs for everyone. It doesn’t feel like a real game, but rather just very slowly generated frames. I don’t understand anything about generative AI. Maybe that’s exactly what...
Yep, that’s the way it runs for everyone. It doesn’t feel like a real game, but rather just very slowly generated frames. I don’t understand anything about generative AI. Maybe that’s exactly what this is.
Many orders of magnitudes more compute power are used to generate a 2-minute demo of something that is unquestionably worse in every conceivable way than the thing we've already had since 1997.
Is it trying to be better though?
Research around neural networks is, in the grand scheme of things, still in its early stages, so many projects like this get created as a cool proof of concept used to validate some internal research. Sometimes it doesn't lead anywhere, sometimes it does - and if it does, it may not have anything in common with "AI Quake". That was likely just used as an available, useful and widely known source of training data.
I think it's quite cool in the same way that early image generation was - it genuinely feels dream-like, and it's a novel experience.
I think this is a very salient point. Tennis for two was one of the first video games. I am sure people could have said it was meaningless, and you could just go out and play tennis in real life for a better experience. But that’s not really the point. And now look at where video games are, 60 years later. Will generative AI ever replace normal game engines? I doubt it. Video games never replaced tennis, and I am happy that both exist. It’ll be interesting to see what happens to generative ai over the next 60 years.
As an example of a practical application of the research, see https://developer.nvidia.com/blog/nvidia-rtx-neural-rendering-introduces-next-era-of-ai-powered-graphics-innovation/
What you're doing there is generating essentially fake details as a post-rendering step. Not the full frame, like in the Microsoft examples, but parts of the frame, so you can have detail but avoid having to meticulously detail every stone in the game.
The research done is important to improving this, as things like making sure the details make sense temporally (e.g a crack on a rock stays in the same place, 3 dimensionally, between frames, or moves logically when the rock moves).
This might be pedantic, but research around neural networks is not remotely in its early stages. Neural networks have been around for decades. You could argue (though imo it's still a bit sketchy) that research into transformers, the specific type of neural network that LLMs and modern GenAI are based on, is in its early stages, since they were first invented in 2017. But, as a point of comparison, that was while I was in undergrad, RNNs and LSTMs (both well-known and heavily used earlier neural network architectures) were first implemented before I was born.
I overall agree with your main point, that this is much more of an attempt to test the limits of new technology that's on the forefront of research. But there's already too much confusion with people thinking AI automatically means modern GenAI, let's not start the same thing with a term like neural network that still means something!
I was thinking about whether to clarify on that or not. I took a class on neural networks some years ago, I know how old the concept is, but imo it could not be fully realized because we did not have the computing power. A ton of low hanging fruit in research only started to appear some time after we got cheap parallel compute with ordinary GPUs (so I'm not just talking about transformers). Kind of like Wolfram's New Kind of Science, only much more computationally intensive.
I can understand not really counting the very old examples, like the Perceptron, but I wouldn't necessarily call anything after RNNs "low-hanging fruit" in that sense. Certainly access to computing resources has been a barrier throughout research into neural nets, but the forefront of research has tended to seesaw between coming up with new architectures and seeing what the best architectures can do with more and more data and computing power (we've been solidly in the latter phase for a little while now).
Sure, it's definitely up for discussion and maybe the words I used were not the best chosen, but imo it's clear that we've been in a sort of a new era of research in the last decade or so, however we call it, and a lot is yet to come with both more research and more compute capacity.
My reaction to the demo was also "wow, this sucks." They use the words "play" and "game". I can play again, as many times as I like! But am I really playing? There are countless "free" or cheap experiences that are infinity times more fun.
But when I read your comment I remembered John Carmack is currently working on AI full time, so I no longer know how I feel about it. Maybe someone will actually make this stuff work at some point?
John Carmack is working on AI full time, but I don't think he's working on making fully AI-generated video games.
This is a tech demo that just screams all flash and no substance. The goal is to keep the AI hype train rolling as long as possible before it collapses like the dotcom bubble.
Just like with the dotcom bubble, the underlying technology won't go away, and we will continue to find productive uses for it. But I seriously doubt that this is one of them.
Scroll down to the big “Try now” button to play (or go directly there via this link).
The game takes a couple minutes to load and is entirely played via keyboard. Instructions are on the page.
On the one hand, I immediately began imagining the many ways in which generative AI could be a great way to breathe new life into some classic games by creating ROM hacks (I was thinking for example how it could instantly create entirely new Pokémon campaign for gens I through V).
On the other hand, I’m afraid that we’ll soon be bombarded with AI game slop, “vibe coded” garbage, and that developers might lose jobs.
As with all tools that lower the barrier to entry you get lot garbage, and they often produce the best results in the hands of a professional
For those who are going to try it:
Look down, turn around, and look back up. Euclid's nightmare.
I haven’t tried it but I assumed it’s basically the idea behind the Doom and Minecraft demos that were going around a while back. Basically: technically impressive but not much of an actual game due to the utter lack of coherence or logical systems.
Sounds about right. I would argue that it's not even really a game, i.e. a program is not being generated and run here, but a highly interactive video. It's constantly cycling new frames, but a limited selection of user inputs can influence what is drawn next. Enemies are not "spawned", but the complex prompt attempts to draw them where it believes they normally should be based on the given frame set.
In the video feed, Copilot attempts to manage health and armor by drawing a frame with lower numeric values in the same spots, but frequently guesses wrong. As a result, sometimes "taking damage" increases health, but usually the image just cycles between a range of 98/88 & 82 HP. Occasionally, as an enemy is drawn attacking the screen, it will blur enough over time to eventually turn into an inanimate object, which I find delightfully funny.
I'm of two minds on this. From a pure technological progress point of view it's intriguing and amazing that this is even possible but from an ex-gamedev perspective every fibre of my being says throw this junk right in the bin and let it rot.
Very interesting tech demo, thanks for sharing. It's like being in someone's dream to a certain extent. For example, you might walk past an enemy, and look back and the model has forgotten about them. Or when you get in a fight it's never really clear what is happening as you shoot them and what will happen. Very surreal. If something like this becomes more accessible, I think in the short term we could see some really neat art projects.
For anyone interested in Carmack's take on this, it is more interesting, and more nuanced, than pro or con.
Me personally, I majored in CogSci back in the 90's at UCSD, and have been active in the machine learning world for decades. The past few years have been a fever dream of never-expected-that-in-a-million-years results. Even if progress stopped tomorrow, we'd have years of work in front of us understanding how these giant matrixes are so capable at so many things.
(Does this mean I think generative tooling is good? Bad? It is both, and neither. The same tool steel that makes a surgical scalpel to save lives makes a bayonet to end them. Most technology is like this. My concerns are of power concentration, and social control along the lines of Sort by Controversial, which was fiction when written a little less than a decade ago. Now, I'm not so sure.)
I got the impression this is "just" generative imagery. Aren't multimodal models all the rage? Why don't they use one that can handle game logic like health and ammo? Perhaps the paper is more specific.
Like others, this seems wrong to me. The Nvidia system has some interest to me, but utilizing LLMs for more natural dialogue is about the extent I want to see AI in video games (and even that I'm wary of).
I think an LLM-powered “narrative engine” could be really interesting, something that tracks story details and characters and generates compelling dramatic arcs in response to player actions. That would go hand-in-hand with the dialogue idea you mentioned but take it a step further. I’d be open to that application of AI! Of course If all the games started doing it (if they were very similar, at least) that would probably wear out its welcome fast.
There's an AI D&D bot/thing called Friends & Fables. I tested out the Discord bot version (free trial) and it was pretty cool to have the ability to play D&D without pissing people off because I can't commit X number of hours in a row on a regular basis.
I can't bring myself to pay for it, but just adding this to say the generative storytelling experience is out there and is actually pretty good already.
You sort of describe a more synchronous version of how I've been using chatgpt to create tactical encounters for DND and other table top games. I create a structured set of files that contain the player sheets, campaign setting, world details, the story of the characters so far, and a file for each scenario complete with conclusion and summary of events. I load these into a project in the chat bot, along with a project instruction about the tone, what it should generate, etc. I then upload updated files after each scenario.
The result is it can create some really nice tactical encounters that includes combat and investigative checks for each player, character specific prompts and flavor text, etc. The stories it suggests tend to be pretty mid, but if given all the data in the right format it can do all the tedious time consuming work of making sure there is something special or a role for every player in every encounter, factoring race, background, class, and major prior choices.
I was thinking more like devs still maintain narrative control, but LLMs would categorize your freeform response and tweak the scripted dialogue to react to you and your tone more realistically. Being able to literally speak to characters would be lit. While I'm not confident enough in AI to be able to craft unique storylines and engaging characters, it could still be a fun space to explore. But this whole fully generated toy? An abomination.
Boo, made for chrome and no mouse. It doesn't capture the keyboard so it tries to search the page in firefox. :/
Overall, not bad for a proof of concept, but it's got bugs to work out.
The enemies disappear if they get behind you and I think the map changed on me, either that or I did something weird. Enemies died upright (physics issues) and super deformed. I wonder if this'll be the next "simulator" game (Surgeon, Goat, "I am bread", that one with old people, etc...) where half the fun is fighting the game engine and the crazy stuff that happens when you do.
Is everyone else getting horrible lag/slow frame rate? I know it isn't exactly going to be great but the lag and frame rate really screwed it up for me.
Yep, that’s the way it runs for everyone. It doesn’t feel like a real game, but rather just very slowly generated frames. I don’t understand anything about generative AI. Maybe that’s exactly what this is.