Some HN discussion here. I think the article is a work of art. I'm pro-telemetry but vehemently against idiotic use of it, and this illustrates that point very nicely. The problem with telemetry...
I think the article is a work of art. I'm pro-telemetry but vehemently against idiotic use of it, and this illustrates that point very nicely. The problem with telemetry is the same as with any statistics: You can make it say anything. Correct interpretation of data is very, very difficult.
I'm staunchly against telemetry and for dogfooding. Telemetry is always a slippery slope, always costs the user battery life and bandwidth, and almost never is as useful as one might hope. You...
I'm staunchly against telemetry and for dogfooding.
Telemetry is always a slippery slope, always costs the user battery life and bandwidth, and almost never is as useful as one might hope.
You never understand why the telemetry is what it is. The article demonstrates this with the obvious example of the power button. Of course, anyone would understand why the data looked like that, but it represents the problem well. Dogfooding can uncover the usability or accessibility issues weird telemetry data is stemming from, without the guessing game.
It's very easy to be against telemetry, isn't it? Dogfooding is of course an absolute must, but it's no replacement for it. Now, let's put you in some shoes. Let's say you made this video game, 1...
It's very easy to be against telemetry, isn't it?
Dogfooding is of course an absolute must, but it's no replacement for it.
Now, let's put you in some shoes. Let's say you made this video game, 1 million players play it every single day. Players of all ages, sexes, ethnicities, with tens of thousands of different configurations, and hundreds of potential disabilities.
Next month, you are losing a license to a library which is a core part of your audio stack. The vendor is asking for more than you can afford, and so you investigate alternatives.
You've found three, but all of them require critical reworks of your settings menu, which means you will have to remove some settings. You get to choose which ones to remove when you choose the alternative.
What do you do? (Oh yeah, PS: making the wrong move will cost you your job and your mental health, because working in gaming is the fucking plague. Have fun!)
Telemetry does not meaningfully cost any battery or bandwidth, unless it's really badly implemented, or gathers far far far too much data. In both of those cases, you're dealing with shitware, do you really think telemetry is going to be the one bad component in a sea of well-behaving pieces?
You never understand why the telemetry is what it is. The article demonstrates this with the obvious example of the power button. Of course, anyone would understand why the data looked like that, but it represents the problem well.
Yes, the article is really good, and has some excellent demonstrations. But once again, this applies to all statistics. "Never use statistics" is not an answer to "People are bad at using statistics". Teaching statistical literacy is more useful.
Also, consider channels for user feedback.
Both that and dogfooding are important tools. They should be part of a complete toolbelt, which will include those, telemetry, and more, as a set to understand you software. You don't need the entire toolbelt for very simple products, or those with very few users. Of course it's overkill to use telemetry when you have 100 users: You can literally talk to 10 percent of your userbase in an afternoon.
Something as big and complex as a web browser begs for it, though. That Mozilla sucks at using it is.. yeah, well, they don't really have the brightest people anymore, it's been a while.
First off, I assume there must be some exotic 3D sound library in use that I've never heard of because there are tons of widely used foss cross platform libraries for sound in games. But explain...
First off, I assume there must be some exotic 3D sound library in use that I've never heard of because there are tons of widely used foss cross platform libraries for sound in games.
But explain to me just why some of your settings are now on the crf? Did they have audio lib dependencies? Can you write a shim layer to keep them?
This sounds like a failure to separate concerns. Someone made bad design decisions and now has to eat it. Don't get me wrong, gamedev is absolutely an area where devs make bad designs due to time limitations mostly, but also the shear number of individuals touching the code, and decisions from above, and the natural iterative process.
But at the end of the day this sounds like a problem that didn't need to exist.
As for telemetry not using so much data, of course well written telemetry won't be much at all. But first, consider is it opt in or always on? If always on, the user is on battery and has radios on even if the game is offline, is the telemetry waking up the radios? Is the telemetry logging to the flash memory so that it only has to wake it up once, or waiting for the radios to be enabled? Now you are burning flash cycles. Not to mention if the game was offline you've really managed to increase your surface area. And if the game is online, most of the useful telemetry could probably be cut from the client and implemented on the servers.
Need to cut options from the menu relevant to users? Get feedback from the users most affected. Obviously they will have a better idea of what they want than you or I will.
Edit: I'm not saying telemetry isn't a useful tool, just that it isn't ever as useful as one expects/thinks in my experience. I am always against it. Doesn't mean it can't ever be used.
Edit: not specifically encouraging collecting data on the servers either, although my case against it isn't as strong; that data is arguably more important to the company because it can be used to increase revenue. I argue that making games more profitable via dark patterns and exploiting human behavior isn't exactly admirable. Of course, such data can also be used to improve a game. Slippery slope, and at the end of the day, companies pursue roi.
I think you are saying that telemetry is just another tool, and used properly, offers something other tools do not. And potentially, at scale there is simply no good alternative. Which might be...
I think you are saying that telemetry is just another tool, and used properly, offers something other tools do not. And potentially, at scale there is simply no good alternative. Which might be correct, but I am against it because I feel that what it offers is ultimately at the user's expense.
If you’re willing to indulge me into a bit of an exploratory journey: why do you feel that way? And: Would you be willing to hear that the reason you feel that way has nothing to do with telemetry...
If you’re willing to indulge me into a bit of an exploratory journey: why do you feel that way? And: Would you be willing to hear that the reason you feel that way has nothing to do with telemetry itself?
I feel that way because telemetry, by necessity: consumes resources on users' devices, and/or collects users' data. If telemetry is opt in, I have no moral disagreement with it, but I still have...
I feel that way because telemetry, by necessity: consumes resources on users' devices, and/or collects users' data.
If telemetry is opt in, I have no moral disagreement with it, but I still have reservations about how useful the telemetry was, and feel the development time could have been better spent elsewhere.
These are personal opinions, but are consistent with my general moral principles and have everything to do with telemetry itself. I'm interested in hearing otherwise but if you start saying something like "you are afraid of privacy issues" I'll tune you out.
Disclaimer for context: Long time supporter of the EFF and ORG, who have done incredible work. Used to be a supporter of Mozilla and briefly even worked for them. I've been in open source for 17...
Disclaimer for context: Long time supporter of the EFF and ORG, who have done incredible work. Used to be a supporter of Mozilla and briefly even worked for them. I've been in open source for 17 years, pro free software for just as long, and the founding digital rights I believe in are transparency (open source), privacy (the right to be invisible), and ownership (rights to your own data).
And yet, I'm pro telemetry -- if used well. I'm no stranger to the privacy aspect of it. Like I said, being against it is easy.
Telemetry can absolutely be abused. Of course, in the context of auto-updating apps, you give full control of the code that runs on your machine to the people distributing the software in question. If Mozilla wanted to fuck you over, using telemetry for it is pretty dumb. They could just do it. And they don't need your permission for it either!
Opt-in telemetry is useless and toxic: It's biased. Opt-ins are generally useful when it comes to going above and beyond in terms of feedback. For example: Context around a crash, screenshots, user-input descriptions, etc. But opt in telemetry will for example tell you that 100% of users choose to have telemetry enabled.
One major issue in statistical interpretation is sampling bias. Opt-in telemetry introduces one of the biggest sampling biases ever, so if you want vendors to suck at interpreting the data, congratulations, you've put them in a broken environment from the get-go.
In order to be effective, telemetry needs to be minimal, transparent, and unbiased. What that means is:
All the collected data should be useful. It should be possible to audit what you are sending, how it's collected, and verify the observed results. All of it should be anonymous, and never carry sensitive information.
Of course, that's the ideal scenario. Nothing is ideal, and most telemetry does not adhere to that. But most software does not adhere to any of its ideals, and yet we all use and accept it.
"Telemetry" is not special. There is nothing special about the way it "consumes your resources" -- software you use consumes your resources, period. If telemetry is implemented so badly that it consumes a meaningful amount of resources, rest assured, the rest of that app is implemented just as badly and will also consume your resources needlessly. Telemetry is not special.
And really, concerns over telemetry over the right to "choose what goes into your ethernet cable" is akin to vaccine concerns over the right to "choose what goes into your body": A noble goal, immediately defeated by 98% of the other shit you're running on your device, or 98% of the things you're eating/breathing.
To reiterate: Opt-in is useless; and battery/bandwidth concerns are invalid. The one thing that is valid is in fact the privacy issue you thought I'd be against for some reason. But the outrage there should be against what is collected, not against the concept of collection as a whole. Your screen resolution is not PII.
To give you another example (applies to the US, but please take the root of the argument, not the specifics): The United States Census is mandatory. It is not an infringement on civil rights to make it mandatory. What can be: the questions asked, aka the data that is collected. What can also be problematic is how that data is interpreted, which is often why certain questions can be problematic (because they carry implications about how to interpret the data).
Ha. It tells you you can't make any guarantees about your sample. Always on telemetry doesn't mean 100% of users are reported either. You like to lean on your point that "being against telemetry...
opt-in telemetry tells you 100% of users have opted in
Ha. It tells you you can't make any guarantees about your sample. Always on telemetry doesn't mean 100% of users are reported either.
You like to lean on your point that "being against telemetry is easy" as though holding a controversial opinion lends your argument strength. It IS easy to be against telemetry. I respect you for sticking to what you feel is a good argument for telemetry. But exactly non of your points have swayed me.
Data science is hard, and companies that can't be bothered to have policies about holding user data securely, talk about what happens when the company is purchased, or what data an analytics partner should actually receive, or fix telemetry that logs in order to only send data at startup but actually blasts the servers every 30s anyway, or even keep the telemetry out of band at all, aren't going to apply any measure of rigor to data science.
I'll tell you what happens. An exec starts badgering the devs to collect more data to show board members what the exec wants to show. And then a partner company is brought in to see how the gameloops where users spend most of their time (here's a little secret: it wasn't the most popular segment of gameplay, just the grindiest) could be developed to wring every last drop out of the whales.
I don't care if you have high standards and know what you are doing. You can go ahead and use telemetry. I'm never, ever, going to advocate for it when I could advise someone to speak to a sample of their users.
Points:
data security is a joke
useless data is the majority of what gets collected
almost no one actually uses the data properly
data does get used properly, in ways that definitely do not benefit the user
it wastes resources when it doesn't get put to good use
sometimes there are errors and the data is literal junk and no one even knows because there isn't anyone qualified for this
You're… still missing the point. The concerns you raise are real, but they are not rooted in telemetry itself, they are rooted in data stewardship. Following your arguments: If Firefox stopped...
You're… still missing the point. The concerns you raise are real, but they are not rooted in telemetry itself, they are rooted in data stewardship.
Following your arguments: If Firefox stopped gathering telemetry altogether, and instead started vacuuming whatever data it wants but just called it something else than "telemetry", chances are that with the right marketing you wouldn't mind. Of course, once made aware, you would mind, but that speaks volume to prejudice based on the name of the action alone.
"Telemetry": Bad; ripe for abuse; danger of being snooped on; prone to executive overreach
"Firefox Account": Eh, I trust Mozilla with my data.
Not saying this is exactly you, but I have seen people feel this exact way, and all your rhetoric matches that.
And you're still not accepting of the absolute fact that "speaking to users" is not a replacement for telemetry. You cannot know the following by "just asking", even by going through large amounts:
What percentage of your users have a 4k screen? The answer will guide you whether to prioritize a particular redesign
How often is the help manual accessed while offline? I want to move it to a website as it's a maintenance burden, but I don't know whether it's useful to some amount of people.
How many tabs do users have open on average? 90th, 95th, 99th percentiles? What are those tabs lifetimes? Those numbers will inform performance design decisions.
And keep in mind that talking to user takes time, thus talking to a significant amount of users takes a significant amount of money. Talking to people is a tool, but it's not a sufficient tool, and it needs to be used correctly as well.
I think it's a real shame you are so close-minded to how telemetry can be used for good. I hope you see the light, eventually.
Being pro-telemetry is not a controversial position, by the way, otherwise it would not be used so much. The thing is, I used to play out the exact same arguments as you did, and then after several years realized that by being so close-minded I was not actually promoting what I was on paper fighting for (good data stewardship, statistical literacy, etc), I was just throwing the baby out with the bathwater. Perfect is the enemy of good, as they say.
And that is what I mean by "being against it is easy": Challenging your prejudice is what's difficult.
Argh. Yes. I know why you said being against it is easy. No, I am not so stupid to think telemetry is some sort of magically different type of data collection. I obviously don't use a Firefox...
Argh. Yes. I know why you said being against it is easy. No, I am not so stupid to think telemetry is some sort of magically different type of data collection. I obviously don't use a Firefox account, but privacy is a different topic.
I am against telemetry because I justdon'tfinditbeneficialIRL. It can be. I acknowledged that. Just not enough for me to ever personally endorse it.
Percentage of 4k users: does the percentage matter? You either support it or you don't. Try asking users if they want further resources put into 4k display. If no one ever wanted it, why is it there in the first place? Was it a flex style UI? In that case 4k support should have come pretty much for free, which is why you went with that.
How often is the offline help manual accessed:
This is an automation problem. You are already writing the support docs so fix your pipeline. You can always make sure the website documentation is easy to grab with curl, wget, or ftp.
Browser usage performance targets.
What? This is so dumb. Browsers are used by billions of people in a billion different ways. You have two options. You either keep it lightweight and don't support multitasking at all, or you optimize performance for multiple tabs. Whatever uplift you get will help every single user who multitasks.
NOTE of course you might have a different audience than Chrome. Maybe this is a hypothetical embedded browser. But your niche defines the audience. You won't suddenly need to wonder if your users are using too many tabs and you need to support it better because that isn't the goal of your browser, and anyone using it should have been aware. They can use it however they want. If you want to completely reposition your product, I'm pretty sure telemetry wasn't what drove your decision. END NOTE
There are solutions that just require thinking! Optimizing for many tabs is hard, but it improves perf across the board, so if you support multiple tabs then of course you should be doing it!
Again, you're getting hung up on specifics, which I pulled out of my ass in all of 30 seconds. You are not its user. You do not maintain software that has hundreds of thousands to hundreds of...
Again, you're getting hung up on specifics, which I pulled out of my ass in all of 30 seconds.
I am against telemetry because I just don't find it beneficial IRL.
You are not its user. You do not maintain software that has hundreds of thousands to hundreds of millions of users. I could spend all day giving you thousands of examples, because I have written and maintained such software.
So yes, you do not find telemetry beneficial, in the same way that I do not find my local postal service useful -- I do not use it. But we both reap the benefits of the invisible engineering that surrounds us.
I'm glad you stuck around for this little back and forth, but it's getting unpleasant to go around in circles constantly so I'm leaving you here on that note. If you wish to continue it, I always reply to DMs.
Fact of the matter is that choices need to be made sometimes and you need information for that. You're getting way too caught up in the irrelevant particulars of a throwaway hypothetical. And on...
Fact of the matter is that choices need to be made sometimes and you need information for that. You're getting way too caught up in the irrelevant particulars of a throwaway hypothetical.
And on battery, we're talking about a game. It doesn't have to be, but in this hypothetical even bad telemetry is vastly outclassed by the power consumption of the thing itself, and the vast majority of telemetry isn't that. I don't know of any real-world examples at all, do you?
I did get too caught up in a throwaway hypothetical. I guess I felt it was a bit silly and made assumptions about why telemetry was essential for solving some problem, and I got hung up on the...
I did get too caught up in a throwaway hypothetical. I guess I felt it was a bit silly and made assumptions about why telemetry was essential for solving some problem, and I got hung up on the sense that the problem didn't need to exist, and thus, there was no need to depend on telemetry. My suggestion of a shim layer might have been a perfectly appropriate resolution that wouldn't require altering UI.
As for whether or not the resources consumed by telemetry are vastly outclassed by the application isn't relevant. If you don't need it, don't use it.
I don't know of any real-world examples at all, do you?
Examples of telemetry that has a noticeable problem with power consumption. I've heard of DRM so overreaching that it noticeably slows the rest of the app, but not telemetry.
Sorry, real-world examples of what?
Examples of telemetry that has a noticeable problem with power consumption. I've heard of DRM so overreaching that it noticeably slows the rest of the app, but not telemetry.
Windows telemetry has a pretty bad rep in that regard actually. I think Windows 8 used to have issues where it would keep devices awake / wake them more often than they should, just because of...
Windows telemetry has a pretty bad rep in that regard actually. I think Windows 8 used to have issues where it would keep devices awake / wake them more often than they should, just because of telemetry, which would in turn drain battery. So it's not the telemetry itself that would drain the battery, but the activity resulting from it.
I don't have examples, sorry, but my concern isn't really that a single user's device will appreciably slow. I can say I've seen telemetry that loaded the servers and incurred unexpected bandwidth...
I don't have examples, sorry, but my concern isn't really that a single user's device will appreciably slow. I can say I've seen telemetry that loaded the servers and incurred unexpected bandwidth costs at scale, though. If you don't need to be wasting resources on telemetry, you shouldn't be.
I'm in that very boat. I added a large amount of logging to our system at work around a feature that I implemented. It has been immensely valuable when doing complaint investigations (we're a...
I'm in that very boat. I added a large amount of logging to our system at work around a feature that I implemented. It has been immensely valuable when doing complaint investigations (we're a regulated industry, this is a required task) to determine the root cause of user problems, but unfortunately the systems that this particular feature is used on the most do not upload logs. I now want to remove a limitation in the feature, badly, as I think it's potentially hurting the user experience, but that feature was added to potentially avoid more critical problems for some users. I do not have the data I need to make a decision so I'm stuck. I wish I had information about how frequently this particular edge case scenario is hit and what the outcome was when it did happen.
Made me think of Ella Minnow Pea. I loved the creativity behind it but it's strange how much you have to concentrate to decipher words spelled out with phonetic replacements or substitutes.
Made me think of Ella Minnow Pea. I loved the creativity behind it but it's strange how much you have to concentrate to decipher words spelled out with phonetic replacements or substitutes.
Surely the most optimal keyboard would solely use Morse code. You only need one key. However, the less used time segment would also have to be eliminated.
Surely the most optimal keyboard would solely use Morse code. You only need one key.
However, the less used time segment would also have to be eliminated.
You joke, but the Rockbox alternative firmware for iPod and Sansa devices lets you enter text via Morse code and it's quite well suited for the available input device.
You joke, but the Rockbox alternative firmware for iPod and Sansa devices lets you enter text via Morse code and it's quite well suited for the available input device.
aaa aa aa a aa aaaa aaa a aaa a a aaa aa aaa aaa aa a a aa a a aaa a a aa a aa aaa aa a a aa aa aaa a aa a a aa a aaa a aa a a aaa a aa a a aaa aa aa a a a aa aa aa aaaa aa aaa aaa aa aa a a aaaa...
aaa aa aa a aa aaaa aaa a aaa a a aaa aa aaa aaa aa a a aa a a aaa a a aa a aa aaa aa a a aa aa aaa a aa a a aa a aaa a aa a a aaa a aa a a aaa aa aa a a a aa aa aa aaaa aa aaa aaa aa aa a a aaaa a aa a a aa aaa aa aaa a aaa a aa a aa a a a aaa aa aaaa aa a a aa aaa aaa a a aa a aaa aa a aaa a aa a aa a a a aaa a aa a a aaaa aaa a a aa a aa a a aa aaa aa a a aa aaa aa a a aaa a aa a aaa a aa a a aaa aa
If you are a real minimalist, you could just have one key to transmit bits as long/short presses to encode binary blobs of Unicode text. Just keep a reference of Unicode code points, UTF-8, and a...
If you are a real minimalist, you could just have one key to transmit bits as long/short presses to encode binary blobs of Unicode text. Just keep a reference of Unicode code points, UTF-8, and a hex calculator handy and you should be all set. If telegraph operators could do it with Morse code 100 years ago, so can we. /s
Some HN discussion here.
I think the article is a work of art. I'm pro-telemetry but vehemently against idiotic use of it, and this illustrates that point very nicely. The problem with telemetry is the same as with any statistics: You can make it say anything. Correct interpretation of data is very, very difficult.
Recommended read: How to Lie with Statistics, by Darrell Huff
I'm staunchly against telemetry and for dogfooding.
Telemetry is always a slippery slope, always costs the user battery life and bandwidth, and almost never is as useful as one might hope.
You never understand why the telemetry is what it is. The article demonstrates this with the obvious example of the power button. Of course, anyone would understand why the data looked like that, but it represents the problem well. Dogfooding can uncover the usability or accessibility issues weird telemetry data is stemming from, without the guessing game.
Also, consider channels for user feedback.
It's very easy to be against telemetry, isn't it?
Dogfooding is of course an absolute must, but it's no replacement for it.
Now, let's put you in some shoes. Let's say you made this video game, 1 million players play it every single day. Players of all ages, sexes, ethnicities, with tens of thousands of different configurations, and hundreds of potential disabilities.
Next month, you are losing a license to a library which is a core part of your audio stack. The vendor is asking for more than you can afford, and so you investigate alternatives.
You've found three, but all of them require critical reworks of your settings menu, which means you will have to remove some settings. You get to choose which ones to remove when you choose the alternative.
What do you do? (Oh yeah, PS: making the wrong move will cost you your job and your mental health, because working in gaming is the fucking plague. Have fun!)
Telemetry does not meaningfully cost any battery or bandwidth, unless it's really badly implemented, or gathers far far far too much data. In both of those cases, you're dealing with shitware, do you really think telemetry is going to be the one bad component in a sea of well-behaving pieces?
Yes, the article is really good, and has some excellent demonstrations. But once again, this applies to all statistics. "Never use statistics" is not an answer to "People are bad at using statistics". Teaching statistical literacy is more useful.
Both that and dogfooding are important tools. They should be part of a complete toolbelt, which will include those, telemetry, and more, as a set to understand you software. You don't need the entire toolbelt for very simple products, or those with very few users. Of course it's overkill to use telemetry when you have 100 users: You can literally talk to 10 percent of your userbase in an afternoon.
Something as big and complex as a web browser begs for it, though. That Mozilla sucks at using it is.. yeah, well, they don't really have the brightest people anymore, it's been a while.
First off, I assume there must be some exotic 3D sound library in use that I've never heard of because there are tons of widely used foss cross platform libraries for sound in games.
But explain to me just why some of your settings are now on the crf? Did they have audio lib dependencies? Can you write a shim layer to keep them?
This sounds like a failure to separate concerns. Someone made bad design decisions and now has to eat it. Don't get me wrong, gamedev is absolutely an area where devs make bad designs due to time limitations mostly, but also the shear number of individuals touching the code, and decisions from above, and the natural iterative process.
But at the end of the day this sounds like a problem that didn't need to exist.
As for telemetry not using so much data, of course well written telemetry won't be much at all. But first, consider is it opt in or always on? If always on, the user is on battery and has radios on even if the game is offline, is the telemetry waking up the radios? Is the telemetry logging to the flash memory so that it only has to wake it up once, or waiting for the radios to be enabled? Now you are burning flash cycles. Not to mention if the game was offline you've really managed to increase your surface area. And if the game is online, most of the useful telemetry could probably be cut from the client and implemented on the servers.
Need to cut options from the menu relevant to users? Get feedback from the users most affected. Obviously they will have a better idea of what they want than you or I will.
Edit: I'm not saying telemetry isn't a useful tool, just that it isn't ever as useful as one expects/thinks in my experience. I am always against it. Doesn't mean it can't ever be used.
Edit: not specifically encouraging collecting data on the servers either, although my case against it isn't as strong; that data is arguably more important to the company because it can be used to increase revenue. I argue that making games more profitable via dark patterns and exploiting human behavior isn't exactly admirable. Of course, such data can also be used to improve a game. Slippery slope, and at the end of the day, companies pursue roi.
I'm sorry but you completely missed the point i was making. I don't think I'll be able to relay it to you any better.
I think you are saying that telemetry is just another tool, and used properly, offers something other tools do not. And potentially, at scale there is simply no good alternative. Which might be correct, but I am against it because I feel that what it offers is ultimately at the user's expense.
If you’re willing to indulge me into a bit of an exploratory journey: why do you feel that way? And: Would you be willing to hear that the reason you feel that way has nothing to do with telemetry itself?
I feel that way because telemetry, by necessity: consumes resources on users' devices, and/or collects users' data.
If telemetry is opt in, I have no moral disagreement with it, but I still have reservations about how useful the telemetry was, and feel the development time could have been better spent elsewhere.
These are personal opinions, but are consistent with my general moral principles and have everything to do with telemetry itself. I'm interested in hearing otherwise but if you start saying something like "you are afraid of privacy issues" I'll tune you out.
Disclaimer for context: Long time supporter of the EFF and ORG, who have done incredible work. Used to be a supporter of Mozilla and briefly even worked for them. I've been in open source for 17 years, pro free software for just as long, and the founding digital rights I believe in are transparency (open source), privacy (the right to be invisible), and ownership (rights to your own data).
And yet, I'm pro telemetry -- if used well. I'm no stranger to the privacy aspect of it. Like I said, being against it is easy.
Telemetry can absolutely be abused. Of course, in the context of auto-updating apps, you give full control of the code that runs on your machine to the people distributing the software in question. If Mozilla wanted to fuck you over, using telemetry for it is pretty dumb. They could just do it. And they don't need your permission for it either!
Opt-in telemetry is useless and toxic: It's biased. Opt-ins are generally useful when it comes to going above and beyond in terms of feedback. For example: Context around a crash, screenshots, user-input descriptions, etc. But opt in telemetry will for example tell you that 100% of users choose to have telemetry enabled.
One major issue in statistical interpretation is sampling bias. Opt-in telemetry introduces one of the biggest sampling biases ever, so if you want vendors to suck at interpreting the data, congratulations, you've put them in a broken environment from the get-go.
In order to be effective, telemetry needs to be minimal, transparent, and unbiased. What that means is:
All the collected data should be useful. It should be possible to audit what you are sending, how it's collected, and verify the observed results. All of it should be anonymous, and never carry sensitive information.
Of course, that's the ideal scenario. Nothing is ideal, and most telemetry does not adhere to that. But most software does not adhere to any of its ideals, and yet we all use and accept it.
"Telemetry" is not special. There is nothing special about the way it "consumes your resources" -- software you use consumes your resources, period. If telemetry is implemented so badly that it consumes a meaningful amount of resources, rest assured, the rest of that app is implemented just as badly and will also consume your resources needlessly. Telemetry is not special.
And really, concerns over telemetry over the right to "choose what goes into your ethernet cable" is akin to vaccine concerns over the right to "choose what goes into your body": A noble goal, immediately defeated by 98% of the other shit you're running on your device, or 98% of the things you're eating/breathing.
To reiterate: Opt-in is useless; and battery/bandwidth concerns are invalid. The one thing that is valid is in fact the privacy issue you thought I'd be against for some reason. But the outrage there should be against what is collected, not against the concept of collection as a whole. Your screen resolution is not PII.
To give you another example (applies to the US, but please take the root of the argument, not the specifics): The United States Census is mandatory. It is not an infringement on civil rights to make it mandatory. What can be: the questions asked, aka the data that is collected. What can also be problematic is how that data is interpreted, which is often why certain questions can be problematic (because they carry implications about how to interpret the data).
Ha. It tells you you can't make any guarantees about your sample. Always on telemetry doesn't mean 100% of users are reported either.
You like to lean on your point that "being against telemetry is easy" as though holding a controversial opinion lends your argument strength. It IS easy to be against telemetry. I respect you for sticking to what you feel is a good argument for telemetry. But exactly non of your points have swayed me.
Data science is hard, and companies that can't be bothered to have policies about holding user data securely, talk about what happens when the company is purchased, or what data an analytics partner should actually receive, or fix telemetry that logs in order to only send data at startup but actually blasts the servers every 30s anyway, or even keep the telemetry out of band at all, aren't going to apply any measure of rigor to data science.
I'll tell you what happens. An exec starts badgering the devs to collect more data to show board members what the exec wants to show. And then a partner company is brought in to see how the gameloops where users spend most of their time (here's a little secret: it wasn't the most popular segment of gameplay, just the grindiest) could be developed to wring every last drop out of the whales.
I don't care if you have high standards and know what you are doing. You can go ahead and use telemetry. I'm never, ever, going to advocate for it when I could advise someone to speak to a sample of their users.
Points:
You're… still missing the point. The concerns you raise are real, but they are not rooted in telemetry itself, they are rooted in data stewardship.
Following your arguments: If Firefox stopped gathering telemetry altogether, and instead started vacuuming whatever data it wants but just called it something else than "telemetry", chances are that with the right marketing you wouldn't mind. Of course, once made aware, you would mind, but that speaks volume to prejudice based on the name of the action alone.
"Telemetry": Bad; ripe for abuse; danger of being snooped on; prone to executive overreach
"Firefox Account": Eh, I trust Mozilla with my data.
Not saying this is exactly you, but I have seen people feel this exact way, and all your rhetoric matches that.
And you're still not accepting of the absolute fact that "speaking to users" is not a replacement for telemetry. You cannot know the following by "just asking", even by going through large amounts:
And keep in mind that talking to user takes time, thus talking to a significant amount of users takes a significant amount of money. Talking to people is a tool, but it's not a sufficient tool, and it needs to be used correctly as well.
I think it's a real shame you are so close-minded to how telemetry can be used for good. I hope you see the light, eventually.
Being pro-telemetry is not a controversial position, by the way, otherwise it would not be used so much. The thing is, I used to play out the exact same arguments as you did, and then after several years realized that by being so close-minded I was not actually promoting what I was on paper fighting for (good data stewardship, statistical literacy, etc), I was just throwing the baby out with the bathwater. Perfect is the enemy of good, as they say.
And that is what I mean by "being against it is easy": Challenging your prejudice is what's difficult.
Argh. Yes. I know why you said being against it is easy. No, I am not so stupid to think telemetry is some sort of magically different type of data collection. I obviously don't use a Firefox account, but privacy is a different topic.
I am against telemetry because I just don't find it beneficial IRL. It can be. I acknowledged that. Just not enough for me to ever personally endorse it.
Percentage of 4k users: does the percentage matter? You either support it or you don't. Try asking users if they want further resources put into 4k display. If no one ever wanted it, why is it there in the first place? Was it a flex style UI? In that case 4k support should have come pretty much for free, which is why you went with that.
How often is the offline help manual accessed:
This is an automation problem. You are already writing the support docs so fix your pipeline. You can always make sure the website documentation is easy to grab with curl, wget, or ftp.
What? This is so dumb. Browsers are used by billions of people in a billion different ways. You have two options. You either keep it lightweight and don't support multitasking at all, or you optimize performance for multiple tabs. Whatever uplift you get will help every single user who multitasks.
NOTE of course you might have a different audience than Chrome. Maybe this is a hypothetical embedded browser. But your niche defines the audience. You won't suddenly need to wonder if your users are using too many tabs and you need to support it better because that isn't the goal of your browser, and anyone using it should have been aware. They can use it however they want. If you want to completely reposition your product, I'm pretty sure telemetry wasn't what drove your decision. END NOTE
There are solutions that just require thinking! Optimizing for many tabs is hard, but it improves perf across the board, so if you support multiple tabs then of course you should be doing it!
Again, you're getting hung up on specifics, which I pulled out of my ass in all of 30 seconds.
You are not its user. You do not maintain software that has hundreds of thousands to hundreds of millions of users. I could spend all day giving you thousands of examples, because I have written and maintained such software.
So yes, you do not find telemetry beneficial, in the same way that I do not find my local postal service useful -- I do not use it. But we both reap the benefits of the invisible engineering that surrounds us.
I'm glad you stuck around for this little back and forth, but it's getting unpleasant to go around in circles constantly so I'm leaving you here on that note. If you wish to continue it, I always reply to DMs.
Fact of the matter is that choices need to be made sometimes and you need information for that. You're getting way too caught up in the irrelevant particulars of a throwaway hypothetical.
And on battery, we're talking about a game. It doesn't have to be, but in this hypothetical even bad telemetry is vastly outclassed by the power consumption of the thing itself, and the vast majority of telemetry isn't that. I don't know of any real-world examples at all, do you?
I did get too caught up in a throwaway hypothetical. I guess I felt it was a bit silly and made assumptions about why telemetry was essential for solving some problem, and I got hung up on the sense that the problem didn't need to exist, and thus, there was no need to depend on telemetry. My suggestion of a shim layer might have been a perfectly appropriate resolution that wouldn't require altering UI.
As for whether or not the resources consumed by telemetry are vastly outclassed by the application isn't relevant. If you don't need it, don't use it.
Sorry, real-world examples of what?
Examples of telemetry that has a noticeable problem with power consumption. I've heard of DRM so overreaching that it noticeably slows the rest of the app, but not telemetry.
Windows telemetry has a pretty bad rep in that regard actually. I think Windows 8 used to have issues where it would keep devices awake / wake them more often than they should, just because of telemetry, which would in turn drain battery. So it's not the telemetry itself that would drain the battery, but the activity resulting from it.
I don't have examples, sorry, but my concern isn't really that a single user's device will appreciably slow. I can say I've seen telemetry that loaded the servers and incurred unexpected bandwidth costs at scale, though. If you don't need to be wasting resources on telemetry, you shouldn't be.
I'm in that very boat. I added a large amount of logging to our system at work around a feature that I implemented. It has been immensely valuable when doing complaint investigations (we're a regulated industry, this is a required task) to determine the root cause of user problems, but unfortunately the systems that this particular feature is used on the most do not upload logs. I now want to remove a limitation in the feature, badly, as I think it's potentially hurting the user experience, but that feature was added to potentially avoid more critical problems for some users. I do not have the data I need to make a decision so I'm stuck. I wish I had information about how frequently this particular edge case scenario is hit and what the outcome was when it did happen.
Made me think of Ella Minnow Pea. I loved the creativity behind it but it's strange how much you have to concentrate to decipher words spelled out with phonetic replacements or substitutes.
Surely the most optimal keyboard would solely use Morse code. You only need one key.
However, the less used time segment would also have to be eliminated.
You joke, but the Rockbox alternative firmware for iPod and Sansa devices lets you enter text via Morse code and it's quite well suited for the available input device.
If you are a real minimalist, you could just have one key to transmit bits as long/short presses to encode binary blobs of Unicode text. Just keep a reference of Unicode code points, UTF-8, and a hex calculator handy and you should be all set. If telegraph operators could do it with Morse code 100 years ago, so can we. /s