Trivedi’s criticisms were threefold. First, he believed the software gave too much weight to older age groups, docking your score if you were under 45. The reasoning behind this was the medical assumption that young people could survive longer than older people, although the long-term effects of waiting longer while chronically ill were unknown. “The disadvantage in . . . [getting] a timely liver transplant if you’re young is too great. So that needs to be revised,” Trivedi said.
Second, he believed the premise of the algorithm — trying to reduce absolute mortality five years after a transplant — was flawed. The system did not account for other outcomes, such as the healthy life years lost by young patients kept waiting, their longer-term outcomes or reduced overall life expectancy. Taking these into account might paint a very different picture of whether the algorithm was beneficial and fair.
Finally, Trivedi said the algorithm was trying to equal out the death rate across all ages on the waiting list, rather than reflecting the rate of the general population, where healthy older people are more likely to die than younger ones. Trivedi believed that transplant patients’ risk should be compared with an age- and sex-matched control population, rather than just against each other.
To answer the title question, probably not. I work for a big American contracting company that is all about A.I.. They encourage all of the IT workers, even those with nothing to do with A.I. to...
To answer the title question, probably not.
I work for a big American contracting company that is all about A.I.. They encourage all of the IT workers, even those with nothing to do with A.I. to take an Intro To A.I. course they had made. The course had an interesting video on biases in A.I.. People of color were left out of medical studies and treatments which could have helped. It has already happened in the U.S..
Another interesting example of A.I. bias came from Amazon. They created an A.I. to help with hiring new people. They trained their model on old Amazon resumes from employees who worked out well. The researchers discovered that next to no women were getting called in for interviews that way. The old Amazon resumes used as training data were mostly from men. The model learned to look for text patterns common to men and not women.
Long story short, A.I. amplifies human biases.
If the NHS training data has a bias in them chances are good the decisions about who gets organ transplants will not be fair.
The old 1970s era programmer saying is still relevant:
Note that this isn't an AI and doesn't seem to use training data in the way that machine learning would. At least as far as I can tell, this is basically a formula, designed to convert all the...
Note that this isn't an AI and doesn't seem to use training data in the way that machine learning would. At least as far as I can tell, this is basically a formula, designed to convert all the salient aspects of each case into a single number. Given all the patients in the UK, you can then pick the person with the highest number and get the person who is most in need of a transplant.
That's not to say that this process will be bulletproof, and the humans designing the formula will have biases of their own. But this is different from cases like that of the Amazon resumes or other famous cases of bias in machine learning.
In this case specifically, the argument is that the decision-making process was designed to maximise years of life overall, but some people believe that it should instead put more emphasis on maximising high-quality years of life. In other words, right now, between a 50yo and an 18yo, the NHS will give more weight to the 50yo, because the transplant is more likely to save their life. (An 18yo is theoretically going to be more able to survive longer without a transplant because they're generally fitter, so by giving it to the 50yo, you save more years of life in total.) The article is arguing that instead, doctors should give more weight to younger people, for whom the waiting period is occurring in the prime of their life.
This is the correct definition of an algorithm. I have to point this out, because I have a computer science background and the about of eye-rolling I do when I hear the term "algorithm" used to...
Note that this isn't an AI and doesn't seem to use training data in the way that machine learning would. At least as far as I can tell, this is basically a formula, designed to convert all the salient aspects of each case into a single number
This is the correct definition of an algorithm. I have to point this out, because I have a computer science background and the about of eye-rolling I do when I hear the term "algorithm" used to mean "some kind of magic" probably isn't good for my migraines.
An algorithm is a series of steps, defined with logical rigor, that solves a problem. You can define and carry one out on paper, with no computer involved.
So, the process of deciding whether someone gets a transplant or not is ideally an algorithm, which historically has been performed by humans. You run down an enumeration of potentially disqualifying factors and remove the candidate if any of them is true. Maybe there are vital figures used to weight the chance of success, and an overall score is calculated to rank the candidates. All things you could simplify with a spreadsheet or a dedicated computer program, that the same panel of doctors can use to make the decision. Maybe this offers more accountability, too, since figures could be attached to patient records and there wouldn't be room for a hypothetical unethical doctor to fudge something on papers used during a meeting.
I have a strong distaste for this sort of handwavy fear-mongering in journalism. We already have enough braindead distrust in institutions without fostering neoluddism.
Also, more often than not, the ML "algorithms" have enough approximation and heuristics in them to kill a horse. They are only algorithms if you stretch the definition of that term to its breaking...
This is the correct definition of an algorithm. I have to point this out, because I have a computer science background and the about of eye-rolling I do when I hear the term "algorithm" used to mean "some kind of magic" probably isn't good for my migraines.
Also, more often than not, the ML "algorithms" have enough approximation and heuristics in them to kill a horse. They are only algorithms if you stretch the definition of that term to its breaking point. How parameters and hyperparameters are chosen is often completely undefined, nevermind that it's also nondeterministic.
I don't think most people who know much about ML would equate them with algorithms, to be clear. The failure to distinguish is generally from people with a poor understanding of the difference.
I don't think most people who know much about ML would equate them with algorithms, to be clear. The failure to distinguish is generally from people with a poor understanding of the difference.
For the most part, complete agreement. However, at this point, that colloquialism has crept in so much that it wouldn't surprise me. Nevermind that there's some ML practitioners out there with a...
For the most part, complete agreement.
However, at this point, that colloquialism has crept in so much that it wouldn't surprise me. Nevermind that there's some ML practitioners out there with a poor grasp of ML fundamentals and a poorer grasp of general CS fundamentals. So if I met a ML professional tomorrow who insisted that this or that AI/ML system was an algorithm, but couldn't actually define algorithm, I wouldn't be surprised, just disappointed.
I'm not entirely convinced by this argument, given it's mostly anecdotal and I feel uncomfortable saying that a young person's extended years of life are more valuable than an older person's life....
I'm not entirely convinced by this argument, given it's mostly anecdotal and I feel uncomfortable saying that a young person's extended years of life are more valuable than an older person's life. I also think the use of the word "algorithm", which is one of those words where the lay definition is very nebulous, makes the whole article a lot less clear. In effect, this is just a formula, which are used in all sorts of places within the NHS, and large organisations in general, to make decisions at a mass scale.
But I am surprised at how opaque the details of this formula seem to be. That feels like it needs improvement. I don't know if it's so healthy for people to see their exact place on the waiting list, but it seems like it would make sense to at least be able to communicate how long you're likely going to need to wait — not just in terms of the average time for all patients.
The metric the NHS is supposed to use to ration healthcare is QALYs (quality-adjusted life years) – in this context the lives of young people could matter more (especially in the context of...
The metric the NHS is supposed to use to ration healthcare is QALYs (quality-adjusted life years) – in this context the lives of young people could matter more (especially in the context of transplants) as often treating them will better maximise this metric than treating older people
... and to preemptively defend that choice of metric against plain 'life years': if you have a treatment that cures a debilitating, but non-deadly disease - coma, lock-in-syndrome, pick your...
... and to preemptively defend that choice of metric against plain 'life years': if you have a treatment that cures a debilitating, but non-deadly disease - coma, lock-in-syndrome, pick your poison really - but there's a small chance of death, 'ly' prescribes never to treat, as it could end the life with no measurable upside, while 'qaly' prescribes that if the risk is sufficiently small, we treat, because it can measure the upside.
'ly' wants us to all be nonperishable vegetables with a heartbeat and at least some brain activity. 'qaly' wants us to actually live and be able to do something with it.
This is true, but I assume QALYs have already been taken into account here, and I don't think the article is talking about that. I think the argument from the subject of the article is that her...
This is true, but I assume QALYs have already been taken into account here, and I don't think the article is talking about that. I think the argument from the subject of the article is that her 20s are more meaningful than someone else's 50s - not based on health grounds, but because a person's 20s are some of the most important years of their life. That's the part that I find difficult to agree with.
I don't think it's difficult to argue that from a health perspective, you're enjoying higher QoL in your 20s than your 50s. Perhaps not by a huge margin, the actual dropoff happens past the 50s...
I don't think it's difficult to argue that from a health perspective, you're enjoying higher QoL in your 20s than your 50s. Perhaps not by a huge margin, the actual dropoff happens past the 50s I'd say. But your average 50yo has some acquired disabilities like chronic pain, worn out joints, declining fitness.
That point completely aside, I doubt QALY is sufficiently clear as to make a discussion of what ought or ought not be included or considered a moot point. QALY (as a concept, not as a currently implemented metric) is presumably extremely difficult to quantify, and approximations must be made. Where and how we approximate matters and is up for debate.
Archive link. Excerpts
To answer the title question, probably not.
I work for a big American contracting company that is all about A.I.. They encourage all of the IT workers, even those with nothing to do with A.I. to take an Intro To A.I. course they had made. The course had an interesting video on biases in A.I.. People of color were left out of medical studies and treatments which could have helped. It has already happened in the U.S..
Another interesting example of A.I. bias came from Amazon. They created an A.I. to help with hiring new people. They trained their model on old Amazon resumes from employees who worked out well. The researchers discovered that next to no women were getting called in for interviews that way. The old Amazon resumes used as training data were mostly from men. The model learned to look for text patterns common to men and not women.
Long story short, A.I. amplifies human biases.
If the NHS training data has a bias in them chances are good the decisions about who gets organ transplants will not be fair.
The old 1970s era programmer saying is still relevant:
GIGO - Garbage In, Garbage Out.
Note that this isn't an AI and doesn't seem to use training data in the way that machine learning would. At least as far as I can tell, this is basically a formula, designed to convert all the salient aspects of each case into a single number. Given all the patients in the UK, you can then pick the person with the highest number and get the person who is most in need of a transplant.
That's not to say that this process will be bulletproof, and the humans designing the formula will have biases of their own. But this is different from cases like that of the Amazon resumes or other famous cases of bias in machine learning.
In this case specifically, the argument is that the decision-making process was designed to maximise years of life overall, but some people believe that it should instead put more emphasis on maximising high-quality years of life. In other words, right now, between a 50yo and an 18yo, the NHS will give more weight to the 50yo, because the transplant is more likely to save their life. (An 18yo is theoretically going to be more able to survive longer without a transplant because they're generally fitter, so by giving it to the 50yo, you save more years of life in total.) The article is arguing that instead, doctors should give more weight to younger people, for whom the waiting period is occurring in the prime of their life.
This is the correct definition of an algorithm. I have to point this out, because I have a computer science background and the about of eye-rolling I do when I hear the term "algorithm" used to mean "some kind of magic" probably isn't good for my migraines.
An algorithm is a series of steps, defined with logical rigor, that solves a problem. You can define and carry one out on paper, with no computer involved.
So, the process of deciding whether someone gets a transplant or not is ideally an algorithm, which historically has been performed by humans. You run down an enumeration of potentially disqualifying factors and remove the candidate if any of them is true. Maybe there are vital figures used to weight the chance of success, and an overall score is calculated to rank the candidates. All things you could simplify with a spreadsheet or a dedicated computer program, that the same panel of doctors can use to make the decision. Maybe this offers more accountability, too, since figures could be attached to patient records and there wouldn't be room for a hypothetical unethical doctor to fudge something on papers used during a meeting.
I have a strong distaste for this sort of handwavy fear-mongering in journalism. We already have enough braindead distrust in institutions without fostering neoluddism.
Also, more often than not, the ML "algorithms" have enough approximation and heuristics in them to kill a horse. They are only algorithms if you stretch the definition of that term to its breaking point. How parameters and hyperparameters are chosen is often completely undefined, nevermind that it's also nondeterministic.
I don't think most people who know much about ML would equate them with algorithms, to be clear. The failure to distinguish is generally from people with a poor understanding of the difference.
For the most part, complete agreement.
However, at this point, that colloquialism has crept in so much that it wouldn't surprise me. Nevermind that there's some ML practitioners out there with a poor grasp of ML fundamentals and a poorer grasp of general CS fundamentals. So if I met a ML professional tomorrow who insisted that this or that AI/ML system was an algorithm, but couldn't actually define algorithm, I wouldn't be surprised, just disappointed.
I'm not entirely convinced by this argument, given it's mostly anecdotal and I feel uncomfortable saying that a young person's extended years of life are more valuable than an older person's life. I also think the use of the word "algorithm", which is one of those words where the lay definition is very nebulous, makes the whole article a lot less clear. In effect, this is just a formula, which are used in all sorts of places within the NHS, and large organisations in general, to make decisions at a mass scale.
But I am surprised at how opaque the details of this formula seem to be. That feels like it needs improvement. I don't know if it's so healthy for people to see their exact place on the waiting list, but it seems like it would make sense to at least be able to communicate how long you're likely going to need to wait — not just in terms of the average time for all patients.
The metric the NHS is supposed to use to ration healthcare is QALYs (quality-adjusted life years) – in this context the lives of young people could matter more (especially in the context of transplants) as often treating them will better maximise this metric than treating older people
... and to preemptively defend that choice of metric against plain 'life years': if you have a treatment that cures a debilitating, but non-deadly disease - coma, lock-in-syndrome, pick your poison really - but there's a small chance of death, 'ly' prescribes never to treat, as it could end the life with no measurable upside, while 'qaly' prescribes that if the risk is sufficiently small, we treat, because it can measure the upside.
'ly' wants us to all be nonperishable vegetables with a heartbeat and at least some brain activity. 'qaly' wants us to actually live and be able to do something with it.
This is true, but I assume QALYs have already been taken into account here, and I don't think the article is talking about that. I think the argument from the subject of the article is that her 20s are more meaningful than someone else's 50s - not based on health grounds, but because a person's 20s are some of the most important years of their life. That's the part that I find difficult to agree with.
I don't think it's difficult to argue that from a health perspective, you're enjoying higher QoL in your 20s than your 50s. Perhaps not by a huge margin, the actual dropoff happens past the 50s I'd say. But your average 50yo has some acquired disabilities like chronic pain, worn out joints, declining fitness.
That point completely aside, I doubt QALY is sufficiently clear as to make a discussion of what ought or ought not be included or considered a moot point. QALY (as a concept, not as a currently implemented metric) is presumably extremely difficult to quantify, and approximations must be made. Where and how we approximate matters and is up for debate.