By Marlena Tyldesley
One of the ten most widely-used algorithms in hospitals around the country vastly underestimates the number and severity of illnesses in Black patients in those hospitals. This prioritizes white patients in consideration for a program that would give them more resources for their care within our healthcare system.
In October of 2019, Obermeyer, Powers, Vogeli and Mullainathan published “Dissecting racial bias in an algorithm used to manage the health of populations,” a research article on racial discrimination in a widely-used algorithm in the medical industry. The algorithm is used to screen patients for a program that would grant them additional help in their care. It makes its recommendations based on yearly cost of health services, a label which creates racial bias as to which patients are recommended for additional services (Obermeyer et al., 2019).
Cost of health services should, in theory, be a fair indicator of who would benefit from the program. However, there are many opportunities in the process of interacting with the healthcare system for race to have an effect on the cost of services; these include poverty, access to transportation, access to child care, time off from work, or any number of other issues. The literature reviewed by the authors indicates that because of these socioeconomic situations that disproportionately affect minority populations, Black patients may be visiting hospitals and using medical services at lower rates than White patients. This difference increases the number of White patients paying large sums in health care costs. Thus, when the algorithm uses health care costs as a proxy for health, Black patients who are equally or more sick than White patients are penalized in the screening for a program that would otherwise help them (Obermeyer et al., 2019).
This discrepancy is evident in the data collected from a simulated algorithm created by the authors which attempted to remove racial bias from the system. In it, the authors used a sample of patients from a hospital comprised of 6,079 patients who self-identified as Black and 43,539 who self-identified as White. The original algorithm ranked patients by a health “risk score” based on their yearly medical expenses (the higher the expenses, the higher the risk). The graph below from the study shows the fraction of patients at each risk score who were Black in both the original and the new algorithm. The original algorithm’s data is along the solid purple line. In the simulated algorithm, represented by the dashed line, the authors replaced White patients at each risk score with Black patients who were sicker (based on number of chronic illnesses), but had fallen below the cost threshold in the original algorithm. The authors found that at every risk score above the 50th percentile, the fraction of Black patients increased from the original to simulated algorithm. In fact, at the 97th percentile, the fraction of Black patients rose from 17.7% to 46.5% (Obermeyer et al., 2019).
As a result, the authors find that accurately predicting the costs of healthcare requires being intentionally racially biased by not relying on only medical expense history. Upon discovering the issues caused by “race-blindness” in the algorithm, this study’s authors reached out to the manufacturers of the original algorithm who ran the experiment themselves and found the same results. Together, Obermeyer et al. and the algorithm’s manufacturers are working, unpaid, to improve the algorithm. The top ten most widely-used algorithms (including this one) in the industry calculate risk scores for patients dependent on year cost of medical expenses (Obermeyer et al., 2019).
An improvement in this algorithm could mean similar changes across the entire industry, and a step toward remedying discrimination in our medical system. The racial bias in our healthcare system as is has seeped into our technologies, and these authors are fighting a good fight to remedy that.
References:
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447–453. https://doi.org/10.1126/science.aax2342