14
votes
How would you feel about companies releasing "game concepts" for you to test?
What is a "game concept":
- visually-unpolished but functional game
- costs little compared to the full product
- only basic UI and UX
- solid, release-worthy mechanics
- released publicly in order to test a particular kind of gameplay (standalone, not part of any other game)
- retracted once the testing period is over
- testers get 50% off purchasing or updating to the polished, complete game (possibly also in-game perks)
Pros:
- game design team gets to test quirkier ideas without the investment of a full game
- mostly prevents flops (idiocy and hubris can still lead on)
Cons:
- players have to pay in order to participate (fewer players will want to join)
- game is retracted after testing is over (may cause player discontentment)
The essence of early access. Relevant to titles anywhere between AAA and indie (though more suited to AAA). Good early tests generate publicity. Bad tests are not as bad a publicity due to disclosed status.
Thoughts?
Requiring payment for a totally unfinished product won't bring any major revenue and will create bad vibes for the customer/gamer/tester.
Releasing an unfinished product, with clear labels, for free to test concepts and ideas is a method to bring in playership and build steam.
There's some precedent for this kind of thing in DoubleFine's Amnesia Fortnight events.
My main worry with this kind of thing would be the studio becoming too dependent or beholden to fan feedback in the design stage, and my reason for this is actually DoubleFine's own Broken Age. It wasn't a product of their Amnesia Fortnight process but was instead a wildly successful Kickstarter (the highest grossing one of all time, at the time that it finished). Nevertheless, even with a large sum of money, the devs ran out of cash before they could fully finish the game, so they released the beginning of the game as "Act 1" and then used revenues from that to make Act 2.
Because of the development gap, they had the opportunity to get feedback on Act 1 while they were developing Act 2, so they were able to change Act 2 as it was made. I'm of the belief that they were, unfortunately, guided by the loudest complainers rather than any sense of their core design, and I think the second half of Broken Age is far, far worse for it. The identity and experience created in the first half doesn't follow through to the second. Their clear vision in the first half gets much muddier in the second.
In a system like this, I think it would be tempting to let players direct design, especially because they paid in for the privilege, but most players aren't game designers by trade. We're more responsive. We can evaluate what's good and bad a lot better than we can talk about how to make something good or bad from the ground up.
Nevertheless, like I said at the beginning, there's certainly precedent for this kind of thing, and I think it could have value less as a design iteration and more as a "temperature check" for certain ideas. Toss some concepts at your players and see what sticks, then build from there!
I, as a player, would be fine with it. But as a developer I have trouble imagining it would be useful at all. The vast majority of fan feedback is kind of derivative and dumb and I would worry that this kind of focus-group driven design process will have you vacillating between the Homer car on one end and Ford’s quote about “if I asked people what they wanted they’d have said ‘faster horses’” on the other.
So I’m not sure what the point would be. Design testing and iteration can be done by soliciting input from friends and colleagues, I would think, without throwing it out for the whole internet to chime in on.
Mirror's Edge was very well-received because of its innovative mechanics. Mirror's Edge: Catalyst was received significantly more poorly because of the sharp turn the developers took in order to make the game more AAA (sandbox is what many games did at the time).
Sandboxing may have worked under a different world structure; this is where player feedback comes in. Most people will give an inkling of insight; by processing it as aggregate data, you may be able to salvage or turn around the feature that would otherwise render your massive project a bomb.
With game concepts, you get to present your idea in action and avoid the "faster horses" pitfall, just like Ford et al. did with actually making cars.
The thing is, I'm not sure just having random people online test a game is going to get you that kind of nuanced feedback. Take Netflix's ratings system as an example. They offered 5 stars, but they found that so many people used it as a binary (5 stars or 1) that it was statistically meaningless and they shifted to thumbs up/thumbs down instead. I don't think developers will be able to parse all the random feedback for what's useful. For every comment that might help, they'll get 5 from people who just don't get it, people who are mad that this is a different game from the one they wanted, people who want insane amounts of options or feature creep, people who want to moan about how the protagonist is female or how the jiggle physics is substandard, etc.
And what's worse is, when people put time into something and give feedback a lot of the time they feel entitled to having their feedback taken seriously and incorporated even if it's out of scope or wasn't part of the developer's intent. So now you're gonna need all sorts of community engagement to manage expectations. This all ends up being a massive distraction from the core work of developing the game that I don't think would be worth the effort. Even big companies are bad at incorporating feedback like this. I'm not sure a small team without the resources to spare could do it at all.
That's what the team behind No Man's Sky did, apparently. After the fiasco that was the initial release, Sean Murrey – the lead designer – started routing all the feedback to himself and sort through it like a proper data scientist.
Which isn't to say that every indie development team is going to do the exact same thing. It's not easy to provide delicate analysis of vague feedback. What you referred to as "dumb", I see as a lack of insight into the way games are developed, which leads to assumptions and presuppositions that aren't supported by reality but proliferate in ignorance all the same.
Besides: 10 reviews of "thing 1 is bad" out of a 100 is enough to have you investigate it. You can ask questions, you can further bring about testing material, you can conduct internal reviews with your design team to see what's up. I'll take some feedback over no feedback any day.
Look at it this way: you're going to have to sieve through criticism one way or another. Would you rather do so while you have the resources to steer the ship in a different direction if you could?
To be fair though, that sounded like a nightmare. I would have hated being that guy and I feel like he was doing this as some kind of masochistic, self-flagellating act of penance.
Something tells me you've never developed a game.
I don't mean that as an insult, or to bring you down in any way. It sounds, from the way you're putting all of it, like it must be a hellish task that one is consigned to rather than takes up on their own.
I love bugfixes. I love getting things to work. Feedback helps focus the efforts, even if they don't necessarily guide you blindly.
When I heard Sean doing all this, I felt like I'd be doing the same thing if I still had any clarity of mind left after all the pressure. This level of data analysis is not something you put up with: it's something you develop, unconsciously, because that's how your brain is wired. Tasks upon issues upon the underlying order.
He's not the only developer out there with that sort of mindset. Neither am I. This is not limited to people of a particular outlook, though: anyone can do it. It's an important part of the workflow, just as much as cleaning your house is: sometimes urgent (when you spilled something on the floor), sometimes as a matter of regular procedure. It's a skill. The more feedback you have about the things you did right and (more importantly) wrong, the clearer your insight, and the taller your point of view, and the more dexterous your efforts.
The hellish task isn't developing the game. It's reading through reams of personally abusive emails to try and get at insights that you could probably have gotten with more focused testing and review. I'm not talking about the utility of feedback and testing, it's about the utility of feedback from a mob of nerds on the internet.
I'm an econometrician by training. I know how data analysis works. That still doesn't mean I think there is anything useful to be gained from a bunch of malformed opinions. I'm a scrum-master too and, for the most part, most of the devs I work with don't have a great grasp of how to qualitatively vet and understanding the validity of the data they're working with. It's a specific skill set that not everyone is good at. And unless that sort of analytics is core to what you're doing I wouldn't even bother hiring for it. There's a whole other discipline around focus-group testing and quality review that doesn't involve the boil-the-ocean approaches that most data analysis takes, and that seems like a much better fit for this kind of thing.
Oh yeah, that must've been one hell of a journey for Sean. Nothing I'd want to live through myself, let alone advise others to. Not saying it was necessary, either; but, it appears he did what he does best: get his head down and work his ass off. (His whole team was like that at least during the initial development.)
Parsing reviews doesn't have to mean having to parse abuse and bullshit. Developers are human beings, too. Reviewing or commenting about a game shouldn't involve fucking death threats, regardless of how upset you are about a fucking game.
Those malformed opinions are the most abundant source of HUMINT you have as a developer of a game of any renown. You can use it, or you can discard it.
Can you hire someone competent to quality-review? – or, can you learn to do it yourself? Maybe. I dunno: I've never been to that stage. Would it be worth it for an indie developer? Maybe. Again, I dunno.
It seems to me that vague feedback is harder than specific feedback? Like, I have a lot of trouble rating anything on a 1-5 scale with any consistency, but I'm better at reporting a bug. I might have trouble saying whether the bug is more or less important than another bug, though, and that is itself easier than giving it a priority compared to every other bug (I don't know every other bug).
In short, don't ask people to put their feedback into context they don't have.
I agree that entitlement is an issue, but I think not directly publishing feedback and using "letters to the editor" approach where you publish and reply to substantive feedback might work better, if only because it encourages people to write feedback good enough to be published.
It's much harder. You gotta do a lot of work as the recipient to untangle what the actual issue is from whatever bugbear the person feels like complaining about. And this is assuming the complaint isn't being laced with abuse, which nobody should be forced to sit through.
Possibly. But then I wonder again if the juice is worth the squeeze. It takes work to read, incorporate, and distill that kind of feedback reasonably after all. Would that work have been better spent just working on the game more?
I think of this kind of like a chef testing out recipes. Part of the "fine dining" experience is that the chef is making certain choices for you about what works and what's interesting and you're kind of there for the ride. There are other restaurants that are more mass market and focus group tested. These places tend not to produce interesting food such much as tasty and predictable/reliable dining experiences. It depends on what you're in for, but I think if you're out to create something novel or interesting that would benefit a lot more from internal testing with a clear sense of auteurship than needing to check everything against the teeming millions online.
This sounds bad, but this is coming from a place of love: itch.io is already selling these games. Minus the time-limited part.
If I were to be completely frank, when it comes to "Triple-A" and "Double-A" games, nobody's really aching for new mechanics. The real investment in those games is in a combination of marketing, the value of the IP, and "polish". That's why indie games seem to have their own universe - vastly different priorities. But if you look at games like Control - which, for the record, I loved - the core gameplay is basically the same as every third person shooter out there.
Except, you know, the whole paranormal abilities Jesse gets to acquire in the process. :P
Which, I agree, isn't exactly cutting edge of game design. It's been done before, in all sorts of situations. I remember playing Psi-Ops when I was young, that had similar abilities to give to the player. Psychonauts did this. Plenty of other games did this, all way prior to Control.
Assuming, however, that you are testing new mechanics – like Disco Elysium, or Dwarf Fortress, or Braid – you might want to test those out.
Hell, even newer generations of familiar, if unexplored, mechanics might receive a game-concept treatment. New parkour/terrain traversal mechanics comes to mind: quite a few games have done it, and many of them have been limited in scope. If you're looking for something bigger than a defined set of maps to run through, you want to test that out to see how it works in the real world.
It really depends on what you mean. I don't like the modern eternal beta with constant feature creep. If you're suggesting a "prototype" pre-alpha which is eventually "fleshed out" into a full game, then no. That sounds terrible. If you mean a paid game-jam style prototype, which then gets remade into a full game with its own development cycle, then that's probably a good idea. See Celeste for a somewhat recent success of the two releases thing.
I don't see why you would revoke the prototype though. If someone wants to play the prototype, let 'em.
What's that, and how does that work?
I'm guessing a prototype that's actually playable, but doesn't have as much content and such. Something like Receiver (game-jam "prototype") & Receiver 2 (full game), I guess?
(note: I'm tired and about to go to bed so I apologize if this doesn't make sense)
From what I can tell that's exactly the kind of thing I was thinking of. It looks like Receiver is a prototype made in a week selling for $5, and Receiver 2 is a 1 year development game going for $20. Obviously specific numbers aren't important, just the concept of "first one's short, second one's long" and "first one's cheap, second one's normal price"
Spend a short time creating a game with a very limited scope. In game jams they tend to spend a few days on the product with a fixed release date, but I could see making your prototype take a couple weeks to a month, and throw it up for a few dollars on itch.io (or even steam these days). If the game doesn't do well, ditch the concept and make another one. If it catches on, then do a full development cycle on it.
It seems like the more traditional way is to make the prototype free, but I guess there's no real reason it HAS to be.
It seems like Receiver did this, as others mentioned.
One of the most popular/famous game jams is Ludum Dare, but many game jams tend to be local events hosted by universities, organizations, etc, with random participants joining in the event, forming small teams, then making a small video game in the short time allotted - often according to a theme.
There are a few examples of the small prototype games that come out of game jams being reworked into a fully developed game that are then sold; Celeste being one that was mentioned earlier and Receiver as mentioned by Crestwave.
I believe the emphasis tends to be speed and creativity rather than the expectation of a group completing an amazing AAA game in 72 hours. The tools used usually tend to be the same ones used in actual game development by big studios, but there are methods/shortcuts used for cutting time and they tend to depend on the actual participants. Many game jams are also formatted into a competition, where at the end participants try out the games the others have made, then vote for their favorites or a panel of judges picks out what they believe to be the best games are. Prizes/money are optional.
I'm all about the shorter experiences, if it's play what we have so far and tell us how we did, I'm down for that. I am a little worried on something like Slay the Spire or Don't Starve, I'd get into it, then lose the taste before the first version comes out.
the leaked HL2 test sold me immediately on the game. so fucking cool what they did with the physics