Andreas Weigend: Managing Big Data

Every time Andreas Weigend calls a 1-800 number and gets the standard advisory "this call will be recorded for quality and training purposes" he tells the company representative on the other end of the line that he too will be recording the conversation. Not to be awkward or truculent, but rather to establish what he regards as the very important principle of "data symmetry". With companies amassing enormous amounts of data from consumers through their online browsing and purchasing habits, their use of cell phones, social media postings etc, Weigend told a Global Cafe breakfast meeting of LAWAC on March 2nd that it is time to rectify the balance. "You cannot hide these days, it is too late for that," said Weigend, the former chief scientist at Amazon, "but we should have the right to decide how our data is used".

For example when he is flying in to LAX airport he said it is okay for people to know that he has landed, "but not okay for them to know which bathroom stall I am using." And when he checks into his hotel, "it makes sense for the pizza delivery guy to know which room I am in, but I don't want anyone else knowing which room I am in."

Weigend, who now lectures at Berkeley, is an expert on big data - the enormous amount of information that is being amassed by corporations, governments and non-state actors on everyone who has some digital trail - from using their computers, smart phones, credit cards, Uber, Facebook, Google searches, etc. The amount of information being stored digitally is exploding - according to one recent study, the amount of data produced in the world doubled in 2016 - which means that we produced as much data in 2016 as in the entire history of mankind from the beginning to 2015.

"Data is the new oil," he said. And just like we need refineries to get value out of crude oil, so too we need refineries to deal with the reams of raw data being produced and collected around the world. "You need to refine the data, compare it to other data, and compile it to make it more useful." These refineries are huge companies like Google and Facebook - and it is they who decide what each individual gets to see. Weigend says this is critical. He uses the allegory of Plato's cave, where people see the shadows of ideas projected onto a wall by the light of a fire behind them - "except it is not a fire behind us, but the algorithms of Google and Facebook." So women who confide they are pregnant on a social media post might find they are getting bombarded with advertisements for cribs and baby clothes, even without asking for them.

Weigend says individuals must take back some control over how their data is being used: "We can choose which brand of refrigerator we want to buy - we should also be able to choose which refineries we use for our data." And unlike others in the industry, Weigend says that he believes government does have a role to play: "If companies misbehave, we have a system for correcting this."

At the extreme use of Big Data, Weigend talked about the system that China is experimenting with that would give each Chinese citizen a "social credit score", compiled from multiple sources and activities. A good social credit score would allow individuals access to VIP lanes in airports, for example - but jaywalking or shoplifting would lead to points being subtracted - even associating with someone whom the government judges as subversive could bring a person's score down. It may sound Orwellian, but with data doubling by the year, its uses are incalculable.