Oh, this is so cool! I'm wondering if you could actually turn this into an actual videogame that is fun to play. The behavior reminds me of "multiplayer" gameplay in a very general sense, you'll...
Oh, this is so cool! I'm wondering if you could actually turn this into an actual videogame that is fun to play.
The behavior reminds me of "multiplayer" gameplay in a very general sense, you'll see this type of rule-breaking exploitation in most multiplayer games that have a large enough playerbase and it can be as frustrating as it can be fun. It is, I would say, the main difference between the appeal of singleplayer games (where NPCs tend to "behave" since their behavior must be scripted in a more basic way) and multiplayer games (where every enemy is a human being who can outsmart you and the very ruleset of the game at any moment). How about something in the middle?
What's happening in singleplayer game AI, really? I haven't seen game AI (especially of an action game) wow me since... Half-Life 1? Can that be? I'm trying to think of another example and can't. There's got to be something fun in using recent AI research for actual videogame design.
I think the real issue is that if you start giving in-game agents any kind of reasonable intelligence, it’s just going to learn to exploit the game rules which becomes very unfun for human...
There's got to be something fun in using recent AI research for actual videogame design.
I think the real issue is that if you start giving in-game agents any kind of reasonable intelligence, it’s just going to learn to exploit the game rules which becomes very unfun for human opponents very quickly. Competition for humans is fun when we think we are on an even playing field, but if we don’t have access to the source code of the game and don’t have access to the virtually infinite time a machine has been given to experiment with the game rules, we won’t really feel like the playing field is even.
Beyond that, if you did manage to find a game whose rules don’t present game-breaking exploits, assuming there is an optimal strategy, AI may simply find it, forcing humans to either achieve frame-perfect inputs using the same strategy the machines came up with, or admit that the machines have solved the game and give up.
Basically, the set of games with rules complex enough to be interesting to humans is likely extremely difficult to create such that it won’t have some subtle flaws that won’t be found by an artificial agent with the right reward function and enough time to try to break it.
Games like Chess or Go are interesting cases where the rules are simple enough that they can be perfectly implemented without being open to exploitation and deep enough that there is no obvious optimal strategy. Such games, however, are so skill-testing for humans that there are very few who learn the game at a high enough level to be able to compete with a well-developed artificial agent who has been trained either from high-level human vs. human games, or lots and lots of games against itself in an adversarial regime (or some mixture).
It's hard for me to believe that there isn't some way of utilizing this technology in a way that's interesting. I agree that there's probably a reason we don't see more of it and that the...
It's hard for me to believe that there isn't some way of utilizing this technology in a way that's interesting. I agree that there's probably a reason we don't see more of it and that the systematic breaking of rules (or superhuman reaction times) would be frustrating. But I remember, for example, the first time I played against these ninja assassin type enemies in Half-Life 1 and them kinda flanking me from the side, forcing me to a side corridor where another of them was waiting and thinking, "damn, they outsmarted me!", in a good way. It probably wasn't even real planning, just enough steps for that to happen organically/randomly, but it worked.
I'm trying to think of a time in a recent action game where I genuinely felt like an enemy had a plan (even if it was a bit faked) and I can't think of any. There's scripted events that do that in a fixed way but I think the very game design of most modern games don't even encourage it. We should totally have the technology to teach an AI to do basic tactics like cornering you or sneaking up from behind in a way that anticipating its moves as if it was a human player would be necessary but I don't see it done, at all. There's various limitations that could be set, in terms of reaction times, aim, plain making mistakes, to not make it frustrating. It's IMO not hard at all to imagine the ideal version of that. So the question is why it's not being done. It's either because it doesn't fit with the modern, linear, "cinematic set piece" design of AAA gaming or because its technologically infeasible. I don't think the latter is still a real barrier, looking at videos like the op's.
The classic example of really player-experience-oriented AI is the enemy soldiers in F.E.A.R, though I'm not sure if that counts as "recent" anymore. The actual logic/"intelligence" of the...
The classic example of really player-experience-oriented AI is the enemy soldiers in F.E.A.R, though I'm not sure if that counts as "recent" anymore.
The actual logic/"intelligence" of the soldiers wasn't really anything particularly novel, just well implemented, and well integrated into the level design with hints for pathing and so on.
What really makes it stand out, even among modern titles, is how the player perceives the AI's behaviors; F.E.A.R uses lots of highly specific voice call-outs in different situations ("Behind the table! Behind the shelf!" etc), letting players know what the AI knows and what it's trying to accomplish. Additionally, the levels are designed in a way that allows for a good mix of flexibility and constraints, you (and the AI) can break a window and dive through it, push over obstacles, crawl through vents, and so on, all while in a (usually) indoor space that is well hinted for the AI to be able to take advantage of those features.
All that, especially when combined with the excellent animation, makes for a very fluid and engaging experience that gives the impression of a surprisingly dynamic opponent, despite the internals being fairly conventional.
There have been many articles written about the game since its release; see here or here for a couple of good ones.
I think maybe the real reason we haven't seen F.E.A.R-level AI (let alone improvements) become more common since then, is that the focus of the industry has shifted away from the kind of intense first person action games where it really shines. A RPG usually needs to focus on telling its own story first, a stealth game seems to work better if the AI has more clearly defined rules and states, and open world games have such a large space of possibilities that the kind of level-hinting F.E.A.R relies on is impractical (this is probably the most likely avenue for improvement), and as a whole there's a focus on multiplayer experiences where you don't need AI at all.
On a more optimistic note, I'll take the opportunity to plug Rain World again: while the creatures are mostly quite simple-minded, the presentation and physics-based animation engine really make them feel uniquely alive compared to the vast majority of AI obstacles. I mean, look at theseguys!
Right, FEAR was the last time I remember this being a bit of a topic! And yea, it was mostly just from them blaring out what they see, not so much groundbreaking AI work (though it was decent)....
Right, FEAR was the last time I remember this being a bit of a topic! And yea, it was mostly just from them blaring out what they see, not so much groundbreaking AI work (though it was decent). Maybe presentation is more important than real tech, here, I dunno. Just saying it's a bit disappointing to see so little of it in modern games. FEAR came out 14 years ago (wow!).
Rain World is also totally on my list, it looks great!
Check out Alien Isolation. The alien AI is really well done. If it finds you hiding in a particular place, it will start looking more for it. If you start sneaking up on him, it will start looking...
Check out Alien Isolation. The alien AI is really well done.
If it finds you hiding in a particular place, it will start looking more for it. If you start sneaking up on him, it will start looking over the shoulders more.
I wonder if you’re maybe being influenced by the AI effect? Because this has already been done in video games for a long time. Did you ever play any FPS games like Perfect Dark, or Halo...
We should totally have the technology to teach an AI to do basic tactics like cornering you or sneaking up from behind in a way that anticipating its moves as if it was a human player would be necessary but I don't see it done, at all.
I wonder if you’re maybe being influenced by the AI effect? Because this has already been done in video games for a long time. Did you ever play any FPS games like Perfect Dark, or Halo single-player (such as the firefights in ODST, or campaign up through Halo: Reach)? The Dark Sims or the Elites exhibit AI where they will try to dodge you and use cover, and they will aggressively move against if you try to hide and avoid them.
I know that for some games, it’s very easy to program a technically perfect agent, so more time is spent making the artificial agents less than perfect so that the difficulty isn’t too high. For instances of that, see, e.g., racing games like Mario Kart where the artificial agents are given "catch-up" incentives if they are too far behind, and "slow-down" incentives if they are too far ahead in order to try to align with the player’s skill level (up to a point).
The topic's linked video is short, technical, and worth watching. I believe it is the quickest way for the average techie user to understand what is cool about this. Here is the OpenAI release,...
The topic's linked video is short, technical, and worth watching. I believe it is the quickest way for the average techie user to understand what is cool about this.
Here is the OpenAI release, which actually contains a more approachable video.
I love 2 Minute Papers, he does such a good job of summing up papers that I as a lay person would have trouble following and does it with such enthusiasm.
I love 2 Minute Papers, he does such a good job of summing up papers that I as a lay person would have trouble following and does it with such enthusiasm.
This is so awesome and weird because at some point i started seeing them as living beings instead of a computer program. I wonder the feelings this gives to the people who created this. Look at my...
This is so awesome and weird because at some point i started seeing them as living beings instead of a computer program.
I wonder the feelings this gives to the people who created this.
Oh, this is so cool! I'm wondering if you could actually turn this into an actual videogame that is fun to play.
The behavior reminds me of "multiplayer" gameplay in a very general sense, you'll see this type of rule-breaking exploitation in most multiplayer games that have a large enough playerbase and it can be as frustrating as it can be fun. It is, I would say, the main difference between the appeal of singleplayer games (where NPCs tend to "behave" since their behavior must be scripted in a more basic way) and multiplayer games (where every enemy is a human being who can outsmart you and the very ruleset of the game at any moment). How about something in the middle?
What's happening in singleplayer game AI, really? I haven't seen game AI (especially of an action game) wow me since... Half-Life 1? Can that be? I'm trying to think of another example and can't. There's got to be something fun in using recent AI research for actual videogame design.
I think the real issue is that if you start giving in-game agents any kind of reasonable intelligence, it’s just going to learn to exploit the game rules which becomes very unfun for human opponents very quickly. Competition for humans is fun when we think we are on an even playing field, but if we don’t have access to the source code of the game and don’t have access to the virtually infinite time a machine has been given to experiment with the game rules, we won’t really feel like the playing field is even.
Beyond that, if you did manage to find a game whose rules don’t present game-breaking exploits, assuming there is an optimal strategy, AI may simply find it, forcing humans to either achieve frame-perfect inputs using the same strategy the machines came up with, or admit that the machines have solved the game and give up.
Basically, the set of games with rules complex enough to be interesting to humans is likely extremely difficult to create such that it won’t have some subtle flaws that won’t be found by an artificial agent with the right reward function and enough time to try to break it.
Games like Chess or Go are interesting cases where the rules are simple enough that they can be perfectly implemented without being open to exploitation and deep enough that there is no obvious optimal strategy. Such games, however, are so skill-testing for humans that there are very few who learn the game at a high enough level to be able to compete with a well-developed artificial agent who has been trained either from high-level human vs. human games, or lots and lots of games against itself in an adversarial regime (or some mixture).
It's hard for me to believe that there isn't some way of utilizing this technology in a way that's interesting. I agree that there's probably a reason we don't see more of it and that the systematic breaking of rules (or superhuman reaction times) would be frustrating. But I remember, for example, the first time I played against these ninja assassin type enemies in Half-Life 1 and them kinda flanking me from the side, forcing me to a side corridor where another of them was waiting and thinking, "damn, they outsmarted me!", in a good way. It probably wasn't even real planning, just enough steps for that to happen organically/randomly, but it worked.
I'm trying to think of a time in a recent action game where I genuinely felt like an enemy had a plan (even if it was a bit faked) and I can't think of any. There's scripted events that do that in a fixed way but I think the very game design of most modern games don't even encourage it. We should totally have the technology to teach an AI to do basic tactics like cornering you or sneaking up from behind in a way that anticipating its moves as if it was a human player would be necessary but I don't see it done, at all. There's various limitations that could be set, in terms of reaction times, aim, plain making mistakes, to not make it frustrating. It's IMO not hard at all to imagine the ideal version of that. So the question is why it's not being done. It's either because it doesn't fit with the modern, linear, "cinematic set piece" design of AAA gaming or because its technologically infeasible. I don't think the latter is still a real barrier, looking at videos like the op's.
The classic example of really player-experience-oriented AI is the enemy soldiers in F.E.A.R, though I'm not sure if that counts as "recent" anymore.
The actual logic/"intelligence" of the soldiers wasn't really anything particularly novel, just well implemented, and well integrated into the level design with hints for pathing and so on.
What really makes it stand out, even among modern titles, is how the player perceives the AI's behaviors; F.E.A.R uses lots of highly specific voice call-outs in different situations ("Behind the table! Behind the shelf!" etc), letting players know what the AI knows and what it's trying to accomplish. Additionally, the levels are designed in a way that allows for a good mix of flexibility and constraints, you (and the AI) can break a window and dive through it, push over obstacles, crawl through vents, and so on, all while in a (usually) indoor space that is well hinted for the AI to be able to take advantage of those features.
All that, especially when combined with the excellent animation, makes for a very fluid and engaging experience that gives the impression of a surprisingly dynamic opponent, despite the internals being fairly conventional.
There have been many articles written about the game since its release; see here or here for a couple of good ones.
I think maybe the real reason we haven't seen F.E.A.R-level AI (let alone improvements) become more common since then, is that the focus of the industry has shifted away from the kind of intense first person action games where it really shines. A RPG usually needs to focus on telling its own story first, a stealth game seems to work better if the AI has more clearly defined rules and states, and open world games have such a large space of possibilities that the kind of level-hinting F.E.A.R relies on is impractical (this is probably the most likely avenue for improvement), and as a whole there's a focus on multiplayer experiences where you don't need AI at all.
On a more optimistic note, I'll take the opportunity to plug Rain World again: while the creatures are mostly quite simple-minded, the presentation and physics-based animation engine really make them feel uniquely alive compared to the vast majority of AI obstacles. I mean, look at these guys!
Right, FEAR was the last time I remember this being a bit of a topic! And yea, it was mostly just from them blaring out what they see, not so much groundbreaking AI work (though it was decent). Maybe presentation is more important than real tech, here, I dunno. Just saying it's a bit disappointing to see so little of it in modern games. FEAR came out 14 years ago (wow!).
Rain World is also totally on my list, it looks great!
Check out Alien Isolation. The alien AI is really well done.
If it finds you hiding in a particular place, it will start looking more for it. If you start sneaking up on him, it will start looking over the shoulders more.
It builds a great atmosphere.
I wonder if you’re maybe being influenced by the AI effect? Because this has already been done in video games for a long time. Did you ever play any FPS games like Perfect Dark, or Halo single-player (such as the firefights in ODST, or campaign up through Halo: Reach)? The Dark Sims or the Elites exhibit AI where they will try to dodge you and use cover, and they will aggressively move against if you try to hide and avoid them.
I know that for some games, it’s very easy to program a technically perfect agent, so more time is spent making the artificial agents less than perfect so that the difficulty isn’t too high. For instances of that, see, e.g., racing games like Mario Kart where the artificial agents are given "catch-up" incentives if they are too far behind, and "slow-down" incentives if they are too far ahead in order to try to align with the player’s skill level (up to a point).
Alien Isolation is the perfect example of this.
Touché, but I'm happy to rephrase my call for better AI to a call for a return of more interesting tactics gameplay.
The topic's linked video is short, technical, and worth watching. I believe it is the quickest way for the average techie user to understand what is cool about this.
Here is the OpenAI release, which actually contains a more approachable video.
Finally, here is the underlying paper: Emergent Tool Use From Multi-Agent Autocurricula.
I love 2 Minute Papers, he does such a good job of summing up papers that I as a lay person would have trouble following and does it with such enthusiasm.
This is so awesome and weird because at some point i started seeing them as living beings instead of a computer program.
I wonder the feelings this gives to the people who created this.
Look at my sons! I'm so proud!