Weapons of Math Destruction
Ever since researching my stories on PowerSchool, the data gathering software used by the NS Department of Education and Early Childhood Development, I’ve been curious about the uses and abuses of Big Data.
The goal is not simply to gather and store information but to mine it for deeper revelations by means of algorithms, which data scientist Cathy O’Neill describes as “a toy version — a simplified version” of what our brains actually do all the time:
We use models, we use algorithms in our heads every day to predict the future, to make plans and anything that’s written down in mathematical code is just that except even stupider…It’s not true, it’s not necessarily that sophisticated, it’s a toy version of something that somebody figured out with their brain and wrote down in code, formally.
The great promise of algorithms is that they are objective and neutral, but as O’Neill explains in this excellent London School of Economics lecture (based on her 2016 book, Weapons of Math Destruction: How big data increases inequality and threatens democracy), they are really anything but. They require data and “a definition of success” and “[t]he builder of the algorithm decides what success looks like.”
She opens with a really funny illustration of this, imagining she’s designing an algorithm to measure the “success” of a meal she’s prepared for her family. She herself, she says, would consider factors like, “Did they eat vegetables?” Her son, on the other hand, who loves Nutella, would ask “Did I eat Nutella?” The meal’s rating would clearly vary wildly depending on which of them wrote the algorithm.
If you don’t know what factors are being privileged by an algorithm — and usually you do not, as software manufacturers consider algorithms to be their “secret sauce” — then you can’t know what you’re being judged on or why you’re being found wanting. This judgement can affect you in relatively unimportant ways — as is the case with “queuing” algorithms. Says O’Neill:
This is one of my favorite examples because, even though it’s not that pernicious, it’s very annoying. So, you call up customer service, right? They use your phone number to connect you to a profile and they decide whether you’re a high-value or a low-value customer. And if you’re a low-value customer, you will wait on hold forever.
But that judgement can be pernicious, which is where algorithms become “weapons of math destruction,” for O’Neill. The problem arises when they are used to measure things that cannot be easily measured — like recidivism or teaching ability or a candidate’s suitability for a job. In these cases, she says, algorithms employ “proxy” measures that basically throw wide the door to bias. For example, potential hires are evaluated according to credit scores, although there’s no particular link between poor credit and poor job performance.
It’s fascinating stuff — and don’t worry, it ends with some ideas to improve the situation, so you won’t be left completely demoralized.
Enquiring minds
If you’ve been following the debate about how (or if) the federal government should assist Canada’s struggling newspaper industry, then you’ve no doubt already read, The Shattered Mirror (which is not an Agatha Christie novel but the report on the subject from the Public Policy Forum), or listened to Canadaland’s Jesse Brown discussing that report with Edward Greenspon, former editor in chief of the Globe and Mail, or with Bob Cox — board chair of News Media Canada and publisher of the Winnipeg Free Press.
You may have read National Post columnist Andrew Coyne’s views on it or Halifax Examiner publisher Tim Bousquet’s take on it, or the counter arguments offered by Examiner contributor Stephen Kimber (paywall — unless you’ve taken advantage of the new $15 joint Examiner/Spectator subscription!)
The proposed solution to journalism’s woes is a $350 million/year government fund to subsidize newspapers meeting certain criteria (they must be “registered corporations” so, as Coyne points out, “No non-profits, cooperatives, sole proprietorships etc”). Critics say such a fund would support not so much “journalism” as companies like Post Media.
Nick Fillmore (of The Fourth Estate fame) staked out his own position on government support for legacy print media (he’s opposed to it) on his blog, A Different Point of View…, but he also shared some information about the Post Media board via Facebook. This really caught my eye because it tied into something else I read this week. You can see the full board listed here, I would like to focus on one director:
David Pecker (Director)
Mr. Pecker is Chairman, President and Chief Executive Officer of American Media, Inc., a leading publisher of celebrity and health & fitness magazines in the United States. Prior to that, Mr. Pecker was President and Chief Executive Officer of Hachette Filipacchi Magazines, Inc., one of the world’s largest magazine publishers. In his 30-plus years of publishing experience, he has also overseen the financial divisions of CBS Magazine Group and Diamandis Communications, Inc. and has led the acquisition and divestiture of more than $3.5 billion of magazine brands. Since 2005, he has led over $5 billion of bank and high leverage financing. Mr. Pecker also currently serves as Chairman of iPayment Holdings, Inc., an $800 million credit card processing firm and sits on the board of KSF Acquisition Corp d/b/a SlimFast. He is a Trustee of the Pace University Board of Trustees, a Trustee of Madison Square Boys & Girls Club, and was the co-founder and former President & Director of the Federal Drug Agents Foundation.
American Media, Inc. publishes The National Enquirer which has distinguished itself, of late, by its wholehearted support of President Donald Trump. You can read all about it in this Jeffrey Toobin New Yorker piece which, like the podcast in the last item, I highly recommend.
(I seem to be giving you homework. During the summer. What kind of monster am I?)
