38% of my bug reports come from the Linux community
My game - ΔV: Rings of Saturn (shameless plug) - is out in Early Access for two years now, and as you can expect, there are bugs. But I did find that a disproportionally big amount of these bugs was reported by players using Linux to play. I started to investigate, and my findings did surprise me.
Let’s talk numbers.
Percentages are easy to talk about, but when I read just them, I always wonder - what is the sample size? Is it small enough for the percentage to be just noise? As of today, I sold a little over 12,000 units of ΔV in total. 700 of these units were bought by Linux players. That’s 5.8%. I got 1040 bug reports in total, out of which roughly 400 are made by Linux players. That’s one report per 11.5 users on average, and one report per 1.75 Linux players. That’s right, an average Linux player will get you 650% more bug reports.
A lot of extra work for just 5.8% of extra units, right?
Wrong. Bugs exist whenever you know about them, or not.
Do you know how many of these 400 bug reports were actually platform-specific? 3. Literally only 3 things were problems that came out just on Linux. The rest of them were affecting everyone - the thing is, the Linux community is exceptionally well trained in reporting bugs. That is just the open-source way. This 5.8% of players found 38% of all the bugs that affected everyone. Just like having your own 700-person strong QA team. That was not 38% extra work for me, that was just free QA!
But that’s not all. The report quality is stellar.
I mean we have all seen bug reports like: “it crashes for me after a few hours”. Do you know what a developer can do with such a report? Feel sorry at best. You can’t really fix any bug unless you can replicate it, see it with your own eyes, peek inside and finally see that it’s fixed.
And with bug reports from Linux players is just something else. You get all the software/os versions, all the logs, you get core dumps and you get replication steps. Sometimes I got with the player over discord and we quickly iterated a few versions with progressive fixes to isolate the problem. You just don’t get that kind of engagement from anyone else.
Worth it?
Oh, yes - at least for me. Not for the extra sales - although it’s nice. It’s worth it to get the massive feedback boost and free, hundred-people strong QA team on your side. An invaluable asset for an independent game studio.
Recently, after reporting a bug, it was fixed by the Ardour team in, literally, one hour from the time of reporting. IRC log below; timestamps included deliberately. I am a Linux user.
Recently, after reporting a bug, it was fixed by the Ardour team in, literally, one hour from the time of reporting. IRC log below; timestamps included deliberately.
01:03:06 <Pistos> I think I found a bug. Steps to repro: Open Import dialogue; start preview playback of an audio file; double click an audio file to immediately import as new track in session; while preview playback is still running, right click on recently imported track; Remove track. Confirmation dialogue appears; attempt to confirm removal. All 3 windows are now unresponsive. (Confirmation, Import, and main window)
01:03:16 <Pistos> (until playback finishes)
01:10:36 <x42> confirmed
01:14:26 <x42> https://paste.debian.net/1211799/ -- only happens if you use > 1 processor.
01:15:52 <las> ouch
01:16:43 <x42> las: audition does not use the process-graph, nor the default process callback.
01:18:54 * las makes dinner
02:03:26 <x42> Pistos: fixed in ardour/git
04:18:45 <Pistos> x42: Wow, cool.
04:19:55 <x42> Pistos: thanks to your recipe to reproduce the bug.
04:20:13 <Pistos> I'm a dev, too, so I know the value of Steps to Reproduce. :)
04:20:46 <Pistos> (I almost consider lack of repro steps to mean a reported bug doesn't exist :) )
04:22:12 <x42> heh
That happened to me this week using MonoGame. I had a pipeline to automate my game builds. It suddenly broke for no obvious reasons. I contacted one of the main contributor that wrote the docs....
That happened to me this week using MonoGame. I had a pipeline to automate my game builds. It suddenly broke for no obvious reasons. I contacted one of the main contributor that wrote the docs. Turns out I was pulling a script that got updated to .net5 on the develop branch. Within 15 minutes the docs were updated. snip1snip2
I do like supporting indie developers, but, for some reason, the core game loop didn't appeal to me. I refunded it while I was still in the refund window.
I do like supporting indie developers, but, for some reason, the core game loop didn't appeal to me. I refunded it while I was still in the refund window.
Oh neat, this is some interesting data! I was actually one of the first pre-Early Access testers of Rings of Saturn, though I haven't played it in ages, so here's hoping I helped start that trend...
Oh neat, this is some interesting data! I was actually one of the first pre-Early Access testers of Rings of Saturn, though I haven't played it in ages, so here's hoping I helped start that trend of good Linux bug reports for them :)
Recently, after reporting a bug, it was fixed by the Ardour team in, literally, one hour from the time of reporting. IRC log below; timestamps included deliberately.
I am a Linux user.
That happened to me this week using MonoGame. I had a pipeline to automate my game builds. It suddenly broke for no obvious reasons. I contacted one of the main contributor that wrote the docs. Turns out I was pulling a script that got updated to .net5 on the develop branch. Within 15 minutes the docs were updated. snip1 snip2
Neat! one of those 400 is me! great game!
I do like supporting indie developers, but, for some reason, the core game loop didn't appeal to me. I refunded it while I was still in the refund window.
Oh neat, this is some interesting data! I was actually one of the first pre-Early Access testers of Rings of Saturn, though I haven't played it in ages, so here's hoping I helped start that trend of good Linux bug reports for them :)