Big Data is big bad Big Brother business and should bother you.

Yes, NSA surveillance should worry the law-abiding

The books you read, the emails you send, the TV shows you watch – 'big data' could jump to conclusions about your life

• • Ian Brown
  • guardian.co.uk, Monday 10 June 2013 

Cables at a Google data centre. 'Much more complex patterns are generally being sought out by data analysts, in the way that Google (for example) uses more than 200 "signals” about the quality of a web page to generate its search rankings.' Photograph: Google handout/Camera Press

Many internet users will be feeling slightly bemused by the worldwide reaction to the revelations about US surveillance technology. As President Obama, the UK foreign secretary, William Hague, and many other senior politicians have said, what do the innocent have to fear? Why would the National Security Agency (NSA), or anyone else, care about your search history, Facebook updates, Skype calls, emails, instant messages, and so on?

Data mining tools have developed quickly over the past decade, and a detailed picture can now be painted of people's lives with even small amounts of such information. This picture can ultimately have real-world consequences. Ever had problems getting an electronic visa to travel to countries such as the US and Australia, who pre-screen foreign visitors, or had to go through lengthy additional security at the airport? Thought about getting a job with a government agency or contractor that will do background checks first? Or perhaps you've had difficulty getting medical insurance or credit despite a healthy lifestyle and prompt payment of your bills?

So-called "big data" approaches are revolutionising the way these processes work, in government and the private sector. By crunching through large quantities of data, all sorts of interesting patterns can be found inside people's everyday activities. You might already realise that fatty and sugary foods showing up on your supermarket loyalty card could be interesting to insurers, financial service providers, and even employers concerned about sick days – but did you know significant time spent commuting and watching television are also good predictors of a shorter lifespan?

One phone call to a country such as Pakistan might not in itself say much, but what if three of your own phone and email contacts had made one in the past year, and you also browsed through some quite radical websites protesting at the "war on terror" last month? Or bought three philosophy and history books that have previously been found in the collections of terrorism suspects? Much more complex patterns are generally being sought out by data analysts, in the way that Google (for example) uses more than 200 different "signals" about the quality of a web page to generate its search rankings.

These approaches need huge amounts of both computing power and data. We know that the NSA has both, with its new $2bn Utah Data Centre, due to open in September, reportedly capable of processing trillions of terabytes. Former NSA staff members have explained that the agency has systems that can process trillions of data points to test the strength of connections between them. This explains why Verizon and seemingly other telephone companies have been ordered to hand over complete lists of all telephone calls made – to find suspicious patterns, especially of individuals linked over time to people that have come under suspicion in other ways.

Other such similarly large collections of data about communications ("metadata") are no doubt being analysed by intelligence agencies. Who is talking to whom when, from which location, combined with data about websites visited, can be just as revealing as intercepts of call and email contents, but generally receives much lower legal protection. It is unclear precisely how the Prism system is being used to access data held by companies including Microsoft, Google, Facebook and Apple, but US law regarding foreign surveillance is entirely clear: non-US persons outside the US are fair game, with no constitutional protections. Nor is such monitoring limited to narrow national security and counter-terrorism purposes.

The Guardian's revelations have made clear that we now need a global debate about just how far intelligence agencies should go in undertaking this very broad surveillance and analysis, and what legal protections and oversight mechanisms should be in place – beyond behind-closed-doors scrutiny by officials, secret courts and security-cleared committees. Is it acceptable that one of the main authors of the US Patriot Act, Republican congressman Jim Sensenbrenner, thinks the order to Verizon is not consistent with that act? What should the rest of the world think about their data being held in a country that gives almost no legal protections against government surveillance?

It may no longer be true that "gentlemen don't read each other's mail", as the former US secretary of state Henry L Stimson said. We are certainly unlikely to see a repeat with the NSA of Stimson's 1929 closure of the US Cipher Bureau. But we need much better mechanisms for reassurance that data collection and analysis does not go beyond that required for legitimate government responsibilities, such as preventing future terrorist attacks.

http://www.guardian.co.uk/commentisfree/2013/jun/10/nsa-s
nooping-law-abiding