Weapons of Math Destruction

In a world governed by WMDs, inequities flourish because they are perpetuated by systems that seem beyond question and target the most vulnerable in society. O’Neil calls out each inequity and injustice as they pertain to WMDs.

The primary target of these inequities is marginalized communities, particularly people of color, women, and those who live in poverty. To a reader familiar with history, the fact that marginalized groups are targets likely doesn’t come as a shock. One example O’Neil highlights is the exclusion of “[a]s many as sixty of the two thousand [medical school] applicants [… who] may have been refused an interview purely because of their race, ethnicity, or gender” (117). This exclusivity also applied to people of color and women trying to access loans or any other privileges typically not afforded to them. Another example O’Neil offers is that hiring companies that use credit checks to determine a worker’s suitability “disproportionately affects low-income applicants of color” (149).

Education is also another area where poorer and middle-class Americans are “saddled with debt that will take decades to pay off” (65). O’Neil even points to an algorithm that seeks out people who are down on their luck to advertise private colleges as a way out of their situation, even though this will only perpetuate the cycle of debt and poverty. In fact, there are almost no domains of life where the odds aren’t stacked against marginalized communities, as O’Neil demonstrates throughout the book.

Of course, the nature of discrimination goes far beyond these barriers of denial and debt, but it’s important to acknowledge they’ve always existed. The key difference here, however, is the extent to which communities of color in particular are being targeted. Before, where a single loan officer or medical school could put up barriers, now there are automated systems that guide daily life, from credit scores to policing rates, which can inflict the prejudiced work of a single model onto millions of people with the execution of a program.

As O’Neil states, “[t]he computer learned from the humans how to discriminate, and it carried out this work with breathtaking efficiency” (116). For the people running these programs, this efficiency looks like success because the toxic feedback loop validates false results and often brings profit. Even if the intentions aren’t nefarious, the data isn’t properly analyzed and used to reformulate the models in a meaningful way by humans. So, these broad-stroke approaches end up isolating and punishing members of these marginalized communities.