In so many ways this feels the antithesis of the can it run doom trope. What makes the trope interesting is what kind of small, lightweight, and extremely low powered things can run doom. Instead,...
In so many ways this feels the antithesis of the can it run doom trope. What makes the trope interesting is what kind of small, lightweight, and extremely low powered things can run doom. Instead, they spent god knows how many watt hours training a model on something else that runs doom to allow a data center also using way too much energy to hallucinate doom for you.
Understanding how AI can hallucinate a game for you is an interesting thought process/endeavor, especially in the context of some kind of adaptive or generative gaming where the world is either new every time or adapts to the player's interaction, but these predictions they are throwing out about it replacing coding are far too hyperbolic and it really feels like they used doom as a prototype just to drum up free press.
My understanding is that is during inference, not during training which tend to be massively more resource intense. On the plus side you don't need to train a model more than once.
My understanding is that is during inference, not during training which tend to be massively more resource intense. On the plus side you don't need to train a model more than once.
That pretty much encapsulates why LLMs in its current state is under heavy lawsuits. They can (and should have) been used to help existing workers develop new Doom-likes faster. Instead, it's...
hey spent god knows how many watt hours training a model on something else that runs doom to allow a data center also using way too much energy to hallucinate doom for you.
That pretty much encapsulates why LLMs in its current state is under heavy lawsuits. They can (and should have) been used to help existing workers develop new Doom-likes faster. Instead, it's being used to take out those workers and tell a CEO "you can make Doom by yourself!", with "by using an existing doom" in the fine print.
Pretty much all sales pitch for the next decade or so of "this can replace your artists" are basically selling snake oil.
Sorry to disappoint, but for anyone looking to make an "it runs DOOM" reference, you've already been beaten to it: Also, I think the most interesting thing about this article isn't the text but...
On Tuesday, researchers from Google and Tel Aviv University unveiled GameNGen, a new AI model that can interactively simulate the classic 1993 first-person shooter game Doom in real time using AI image generation techniques borrowed from Stable Diffusion. It's a neural network system that can function as a limited game engine, potentially opening new possibilities for real-time video game synthesis in the future.
The development of GameNGen involved a two-phase training process. Initially, the researchers trained a reinforcement learning agent to play Doom, with its gameplay sessions recorded to create an automatically generated training dataset—that footage we mentioned. They then used that data to train the custom Stable Diffusion model.
Also, GameNGen only has access to three seconds of history, so revisiting a Doom level seen before by the player would involve probabilistic guesses about the previous game state without any knowledge of that history to go on—in other words, confabulating or hallucinating data, much like other generative AI models do when generating outputs.
Sorry to disappoint, but for anyone looking to make an "it runs DOOM" reference, you've already been beaten to it:
"Turns out the answer to 'can it run DOOM?' is yes for diffusion models," wrote Stability AI Research Director Tanishq Mathew Abraham, who was not involved with the research project.
Also, I think the most interesting thing about this article isn't the text but the embedded video of E1M1. If you're familiar with the map, it hallucinates in a really interesting way.
If you're not: there's an alcove to the west of the starting point that, in the footage, becomes a door. That same alcove then appears to the north of the starting point. The player enters it, turns around and, upon exiting it, is then facing east (appropriate to the original layout), not south (appropriate to the hallucinated layout).
The footage had a surreal, House of Leaves or Antichamber kind of quality to it -- the physical impossibility of the clip struck me. Watching it left me feeling a little odd, though I think that was more the sum total of everything about the clip than just that one detail: DOOM enters the uncanny valley, if you will.
You probably already know about it, but mentioning MyHouse.wad is kind of obligatory at this point so: MyHouse.WAD - Inside Doom's most terrifying mod (Tildes post about it)
The footage had a surreal, House of Leaves or Antichamber kind of quality to it -- the physical impossibility of the clip struck me. Watching it left me feeling a little odd, though I think that was more the sum total of everything about the clip than just that one detail: DOOM enters the uncanny valley, if you will.
You probably already know about it, but mentioning MyHouse.wad is kind of obligatory at this point so:
I figure this was an elaborate ruse to say "AI runs doom" anyway. As far as the feel, it's not as bad as non-Euclidean Doom, but now I'm curious about the possibility of 4th-dimension Doom.
I figure this was an elaborate ruse to say "AI runs doom" anyway.
As far as the feel, it's not as bad as non-Euclidean Doom, but now I'm curious about the possibility of 4th-dimension Doom.
maybe i just spent too much time absorbed in fair folk lore as a child but.......im so incredibly Suspicious of anything that likes to create non-euclidian horrors and convince people to wander...
maybe i just spent too much time absorbed in fair folk lore as a child but.......im so incredibly Suspicious of anything that likes to create non-euclidian horrors and convince people to wander through them ¯\_ (ツ)_/¯
not that i need more reasons to be suspicious of generative AI and the people who profit off of it but hey
I don't think that comparison is fair at all! What is so wrong with wandering into a fairy ring and ending up at the whim of whatever might possibly, hypothetically, theoretically, be on the other...
I don't think that comparison is fair at all!
What is so wrong with wandering into a fairy ring and ending up at the whim of whatever might possibly, hypothetically, theoretically, be on the other side? Compared to current human physical existence?
For a (much simpler and older) example, a developer named Ollin Bohan trained a custom neural net for real time image generation based on pokemon for the Game Boy Color that works in-browser. It's...
For a (much simpler and older) example, a developer named Ollin Bohan trained a custom neural net for real time image generation based on pokemon for the Game Boy Color that works in-browser.
Eventually we will have full on AI graphics pipelines where we send geometry information to a model and it will output a photorealistic rendering of the scene.
Eventually we will have full on AI graphics pipelines where we send geometry information to a model and it will output a photorealistic rendering of the scene.
It would be really interesting to train it on a weird mix of things and have it hybridize them, like two different first-person games. I notice the geometry and contents of the space the player is...
It would be really interesting to train it on a weird mix of things and have it hybridize them, like two different first-person games.
I notice the geometry and contents of the space the player is navigating isn't fully consistent. I wonder if just scaling the model up could fix that, or if techniques will be created to help the model plan and stick to consistent 3d spaces will be developed. I could imagine a system like this might benefit from having a part of the model be specialized for gaussian splatting or another neural net based 3d rendering technique.
It would be cool if a system like this could be made so that you could ask it for things in the game in real-time, like you could make a request about the design of the next room you encounter, or have it create a new enemy on the fly. I feel pretty confident this will exist at some point, at least in a very janky form at first, and the only real question is when.
So I can walk forwards, look behind me, walk backwards, turn around, and the whole thing might be different lol. Like GTA V's traffic
Also, GameNGen only has access to three seconds of history, so revisiting a Doom level seen before by the player would involve probabilistic guesses about the previous game state without any knowledge of that history to go on
So I can walk forwards, look behind me, walk backwards, turn around, and the whole thing might be different lol. Like GTA V's traffic
In so many ways this feels the antithesis of the can it run doom trope. What makes the trope interesting is what kind of small, lightweight, and extremely low powered things can run doom. Instead, they spent god knows how many watt hours training a model on something else that runs doom to allow a data center also using way too much energy to hallucinate doom for you.
Understanding how AI can hallucinate a game for you is an interesting thought process/endeavor, especially in the context of some kind of adaptive or generative gaming where the world is either new every time or adapts to the player's interaction, but these predictions they are throwing out about it replacing coding are far too hyperbolic and it really feels like they used doom as a prototype just to drum up free press.
The article says it's running on a single TPU, and those seem to have similar specs to a high-end GPU.
My understanding is that is during inference, not during training which tend to be massively more resource intense. On the plus side you don't need to train a model more than once.
That pretty much encapsulates why LLMs in its current state is under heavy lawsuits. They can (and should have) been used to help existing workers develop new Doom-likes faster. Instead, it's being used to take out those workers and tell a CEO "you can make Doom by yourself!", with "by using an existing doom" in the fine print.
Pretty much all sales pitch for the next decade or so of "this can replace your artists" are basically selling snake oil.
Sorry to disappoint, but for anyone looking to make an "it runs DOOM" reference, you've already been beaten to it:
Also, I think the most interesting thing about this article isn't the text but the embedded video of E1M1. If you're familiar with the map, it hallucinates in a really interesting way.
If you're not: there's an alcove to the west of the starting point that, in the footage, becomes a door. That same alcove then appears to the north of the starting point. The player enters it, turns around and, upon exiting it, is then facing east (appropriate to the original layout), not south (appropriate to the hallucinated layout).
The footage had a surreal, House of Leaves or Antichamber kind of quality to it -- the physical impossibility of the clip struck me. Watching it left me feeling a little odd, though I think that was more the sum total of everything about the clip than just that one detail: DOOM enters the uncanny valley, if you will.
You probably already know about it, but mentioning MyHouse.wad is kind of obligatory at this point so:
MyHouse.WAD - Inside Doom's most terrifying mod (Tildes post about it)
Quite familiar! I played it not too long ago. Definitely worth a mention.
I figure this was an elaborate ruse to say "AI runs doom" anyway.
As far as the feel, it's not as bad as non-Euclidean Doom, but now I'm curious about the possibility of 4th-dimension Doom.
https://github.com/HackerPoet/Engine4D
Very possible.
maybe i just spent too much time absorbed in fair folk lore as a child but.......im so incredibly Suspicious of anything that likes to create non-euclidian horrors and convince people to wander through them ¯\_ (ツ)_/¯
not that i need more reasons to be suspicious of generative AI and the people who profit off of it but heyI don't think that comparison is fair at all!
What is so wrong with wandering into a fairy ring and ending up at the whim of whatever might possibly, hypothetically, theoretically, be on the other side? Compared to current human physical existence?
Not speaking from personal knowledge, obviously
For a (much simpler and older) example, a developer named Ollin Bohan trained a custom neural net for real time image generation based on pokemon for the Game Boy Color that works in-browser.
It's very trippy; like being a ghost as reality melts away around you https://madebyoll.in/posts/game_emulation_via_dnn/demo/
The accompanying blog post complete with 2022-era musing upon AI and where things will go: https://madebyoll.in/posts/game_emulation_via_dnn/
Eventually we will have full on AI graphics pipelines where we send geometry information to a model and it will output a photorealistic rendering of the scene.
It would be really interesting to train it on a weird mix of things and have it hybridize them, like two different first-person games.
I notice the geometry and contents of the space the player is navigating isn't fully consistent. I wonder if just scaling the model up could fix that, or if techniques will be created to help the model plan and stick to consistent 3d spaces will be developed. I could imagine a system like this might benefit from having a part of the model be specialized for gaussian splatting or another neural net based 3d rendering technique.
It would be cool if a system like this could be made so that you could ask it for things in the game in real-time, like you could make a request about the design of the next room you encounter, or have it create a new enemy on the fly. I feel pretty confident this will exist at some point, at least in a very janky form at first, and the only real question is when.
So I can walk forwards, look behind me, walk backwards, turn around, and the whole thing might be different lol. Like GTA V's traffic
On a similar note, here is AI hallucinating gameplay of Mario Kart DS from a screenshot.