Pluses and Minuses (2018)

Book Review from the January 2018 issue of the Socialist Standard

‘Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy’, by Cathy O’Neil (Penguin. £9.99)

Computers may hold massive amounts of information about individuals, and can process this information to come to decisions which affect people’s lives. But it is often unclear what the basis is for the programs that do this processing, and their consequences can often be quite dire. The mathematical basis of the processing leads O’Neil to call them ‘weapons of math destruction’ (abbreviated, of course, as WMDs). Her book mainly relates to developments in the US, though that does not excuse a reference to ‘the British city of Kent’.

As an example, many job applications and their accompanying CVs are not examined by people at all, but only by WMDs, which among other things make use of personality tests which, for instance, ask applicants whether they are best described as ‘unique’ or ‘orderly’. If you have a bad credit score, that is likely to make you an unreliable worker, as reliable people pay their bills on time. If having a bad credit score stops you getting a job, then naturally that will not help your credit record, so the whole thing becomes a self-fulfilling prophecy. Moreover, what is used is often not a real credit score but what is termed an e-score, a kind of rough estimate that may use information such as where you live, rather than whether you pay bills on time.

This illustrates a general problem with WMDs, that they analyse you by looking at ‘people like you’, rather than truly dealing with you as an individual. Also, rather than dealing with truly relevant data, they rely on proxies, substitute information that may or may not be accurate, such as a supposed correlation between a person’s postcode and their likelihood of paying back a loan. Add to this the fact that much information that is held on computers is inaccurate: you may be in real trouble if someone with the same name as you, or even a similar name, has a police record.

In a slightly different area, Facebook determines what its members see on its social network, though the algorithms that lead to these decisions are opaque. This is particularly important when people get a lot of their news on Facebook. O’Neil says that Facebook is not a political WMD, as its network is not used to cause harm, but ‘the potential for abuse is vast’.

It is clear that WMDs, and much computer processing of data, emphasise efficiency and cost-saving over any concept of fairness or treating people equally. In this respect they are really just the latest extension of how capitalism works. O’Neil suggests that they could in principle be used to, for instance, identify and help problem families, but that is not what the profit system is interested in. 

Paul Bennett