NSA Goes Shopping at Silicon Valley for Data Mining Tools

Whatever the outcome of Bush's warrantless NSA surveillance program, it seems clear that government surveillance of our communications and even our social networks is only going to increase. The New York Times reports on recent "shopping trips" by NSA officials to Silicon Valley to purchase new data-mining tools.

On the wish list, according to several venture capitalists who met with the officials, were an array of technologies that underlie the fierce debate over the Bush administration's anti-terrorist eavesdropping program: computerized systems that reveal connections between seemingly innocuous and unrelated pieces of information. The tools they were looking for are new, but their application would fall under the well-established practice of data mining: using mathematical and statistical techniques to scan for hidden relationships in streams of digital data or large databases.

Here's a graphic of how it can work with prisoners. Only the NSA isn't interested in prisoners. They are interested in the rest of us.

Data mining is already being used in a diverse array of commercial applications -- whether by credit card companies detecting and stopping fraud as it happens, or by insurance companies that predict health risks. As a result, millions of Americans have become enmeshed in a vast and growing data web that is constantly being examined by a legion of Internet-era software snoops.

Technology industry executives and government officials said that the intelligence agency systems take such techniques further, applying software analysis tools now routinely used by law enforcement agencies to identify criminal activities and political terrorist organizations that would otherwise be missed by human eavesdroppers.

And while we thought the Total Information Awareness program was dead,

...the legislation [terminating it] provided a specific exemption for "processing, analysis and collaboration tools for counterterrorism foreign intelligence."

Then there's AT&T's Daytona Project, which the Electronic Frontier Foundation believes was used in Bush's warrantless NSA program. Located in Kansas, the Daytona Project is a database with more than 1 trillion phone call records, going back over a decade. EFF has sued AT&T over its cooperation with the NSA spying--you can read about the lawsuit's allegations here.

Yet, the NSA has even more sophisticated tools at its disposal.

The National Security Agency has invested billions in computerized tools for monitoring phone calls around the world -- not only logging them, but also determining content -- and more recently in trying to design digital vacuum cleaners to sweep up information from the Internet.

Last September, the N.S.A. was granted a patent for a technique that could be used to determine the physical location of an Internet address -- another potential category of data to be mined. The technique, which exploits the tiny time delays in the transmission of Internet data, suggests the agency's interest in sophisticated surveillance tasks like trying to determine where a message sent from an Internet address in a cybercafe might have originated.

I think it's time we all learned more about data-mining and the warrantless spying the Government is conducting on Americans. Here are some links to get you started:

[Graphic created exclusively for TalkLeft by CL.]

Update: From David Edwards:

Last night on CNBC, Tim Russert interviewed James Risen and Robert O'Harrow. O'Harrow is an expert on the various data mining companies and Risen broke the NY Times NSA spying story. Risen and his sources (see Russell Tice video) have suggested that there is a much larger domestic surveillance program(s) at the NSA than the "limited" program that the President has described.

Last night's Tim Russert show was the first time I have seen a discussion in the corporate media that attempted to explain the potential massive scope of the NSA's domestic "black programs" and their ties to private data mining firms. Here is a Windows Media video with about 24 minutes of clips from the show. I haven't had a chance to blog it anywhere or even make a screen shot.

Here's David's streaming flash video of the whistleblower's hearing on the NSA surveillance.

< We Are The Body Politic | Fletcher Chooses His Own Judges >
  • The Online Magazine with Liberal coverage of crime-related political and injustice news

  • Contribute To TalkLeft

  • Display: Sort:
    This is a superior piece of journalism. I'm not sure how a single post on a weblog becomes a spark that starts an inferno, but if ever one needs to become that spark this one does. Lords knows, it happens all the time for memes a lot less important than this. Of course this is by no means the first time I have come across the idea but this piece is as tight as a grenade. How anybody can miss the point is beyond me.

    Re: NSA Goes Shopping at Silicon Valley for Data M (none / 0) (#2)
    by Edger on Sun Feb 26, 2006 at 06:03:44 AM EST
    There is one small possible bright side to all of the NSA capabilities. By the time the Cheney/Bush administrations ends, whether it just fizzles out like a limp cheney, or explodes into a burning bush, the NSA should have e record of every single byte of vioce, data, and video that went into or came out of the WH, the DoJ, the Pentagon, State, CentCom, FBI HQ, and even in and out of the NSA itself. And let's not forget the computers and telephones of all our friendly trools, too.

    Thank you Hootsbuddy, that's quite a compliment and much appreciated.

    While the boys at the NSA Are looking low down inside your world Osama bin laden's boys are up in the air coming down on this nut house. the nut's in the NSA And the CIA Are fools and will someday get us all killed, but maybe that is what bush wants?

    so the boys are doing "Sweep up info". I have 500 A-Bombs in my home! and right now building a H-Bomb. and just had a great meal with my best friend peejz and bin laden. and guess who just came by to say hi old hitler he will be back tonight? the NSA Is a joke and who is really getting the data and reading it? "No One". oh yes i love my new home Abu Gharib and soon you will be here with me! if you know what i mean?

    Hah. Knowing how well these government minds work, I suspect most of us could learn more about them in five minutes on Google than they can learn about anyone with their special tools.

    Re: NSA Goes Shopping at Silicon Valley for Data M (none / 0) (#7)
    by The Heretik on Sun Feb 26, 2006 at 10:18:22 AM EST
    Well done. Linked you on this at Welcome to the Machine.

    Re: NSA Goes Shopping at Silicon Valley for Data M (none / 0) (#8)
    by aw on Sun Feb 26, 2006 at 10:26:42 AM EST
    They'll probably hire busybodies in every locality to ferret out wrongdoing of any kind. They'll be looking and listening for every little infraction of the law and arresting people and holding them up as examples and frightening or blackmailing everyone else into submission. And listening to phone sex.

    Re: NSA Goes Shopping at Silicon Valley for Data M (none / 0) (#9)
    by aw on Sun Feb 26, 2006 at 12:38:31 PM EST
    Nice, solid overview, Jeralyn. Ever since I first shuddered at the TIA logo and the resurrection of Poindexter, I was convinced that government surveillance would continue but with less public light. All my concerns have proven well-founded. And my remaining ones will - I believe - eventually be proven true. Well beyond the scope of monitoring potential terrorism, such surveillance will be utilized far more for other crimes. It will include spying on lawful groups opposing politicians' initiatives. It will be used by politicians studying their electoral opponents. It will be used to track legitimate whistleblowers. It will be used as the basis for blackmail. It will make J. Edgar Hoover look like an amateur, It'll make the Watergate thieves look ethical by comparison. The only rationale that makes sense for bypassing FISA overview is simply to escape detection of these dangerous realities.