Weapons of Math Destruction
Nonfiction | Social Justice
A former Wall Street quant sounds an alarm on the mathematical models that pervade modern life — and threaten to rip apart our social fabric
We live in the age of the algorithm. Increasingly, the decisions that affect our lives—where we go to school, whether we get a car loan, how much we pay for health insurance—are being made not by humans, but by mathematical models. In theory, this should lead to greater fairness: Everyone is judged according to the same rules, and bias is eliminated.
But as Cathy O’Neil reveals in this urgent and necessary book, the opposite is true. The models being used today are opaque, unregulated, and uncontestable, even when they’re wrong. Most troubling, they reinforce discrimination: If a poor student can’t get a loan because a lending model deems him too risky (by virtue of his zip code), he’s then cut off from the kind of education that could pull him out of poverty, and a vicious spiral ensues. Models are propping up the lucky and punishing the downtrodden, creating a “toxic cocktail for democracy.” Welcome to the dark side of Big Data.
Weapons of Math Destruction turned out to be one of those books that will stick with me for a long time… shifting my lens on the world, questioning my Google search decisions, making me even more leery of click-bait articles and ads, forcing awareness of how my online behaviors subtly shape the social opportunities of those that are “like me,” contrasting with those that are not.
I read this book for the “social justice” book club I joined a few months back. We’ve been reading a blend of genres, all somehow tied to a social justice theme (some more loosely than others). Weapons of Math Destruction was selected by the lone biostatistician in the group. Very fitting. I felt somewhat anxious to start, considering the focus on “big data” was something out of my comfortable wheelhouse of expertise (or even remote knowledge). I was nervous it’d be over my head, dry, and a chore to page through.
While it was absolutely ripe with new information, it was laid out so eloquently and with such conviction that this 218-page persuasive essay turned into an engrossing page-turner. I read only a chapter at a time to keep my mind in the game, but it took me a brief week to get through it all.
I found myself gasping outloud, furious with new enlightenment on how algorithms have widened the gap between the “haves” and the “have-nots.” I felt like I was back in my undergrad sociology classes, just rife with anger and frustration with the powers outside of my control. I kept on bringing this book up in conversation the moment I saw any sort of connection to the content. (“Oh, you went to college? Well let me tell you how big data has completely screwed full populations out of equal education and is forcing them into lives of joblessness, loan defaults, and a cruel cycle of despair!” ….what, too much?)
Early on in the book, Cathy O’Neil describes her definition of “Weapons of Math Destruction” (or WMDs). They are mathematical models and algorithms that (as most mathematical models) use proxies to stand in for qualitative information (such as a person’s zip code or language to stand in for their potential to pay back a loan or handle a job.) Now, mathematical models are not inherently evil. Look at baseball statistics. Totally legit. The difference is that WMDs are discriminatory, lacks any feedback loop, is opaque to the impacted person, and is scalable… it grows to hit mass populations.
I’ve tried summarizing one of her arguments in a short and digestible format, but it honestly won’t do it justice here. Just pick up the book – pick up one chapter – and read through her insights. She’ll make you question your assumptions, and force you to look at the injustices woven through our systems.
Go get mad.
- The Guardian, Best Books of 2016
- Boston Globe, Best Books of 2016, Non-Fiction
- New York Times, 100 Notable Books of 2016 (Non-Fiction)
- New York Times, Best Seller
- Long listed for the National Book Award
- Goodreads reviews available here.