The intelligence director's office is by law exempt from disclosing detailed budget figures, so it's unclear how much money the government has spent on the SPAR project, which is overseen by the DNI's Intelligence Advanced Research Projects Activity office. Birmingham said the research is aimed for use in a "situation where a large sensitive data set is held by one party which another seeks to query, preserving privacy and enforcing access policies."
A Columbia University computer sciences expert who heads one of the DNI-funded teams, Steven M. Bellovin, estimates the government could start conducting encrypted searches within the next year or two.
"If the NSA wanted to deploy something like this it would take one to two years to get the hardware and software in place to start collecting data this way either from phone companies or whatever other entity they decide on," said Bellovin, who is also a former chief technologist for the Federal Trade Commission.
The NSA's surveillance program collects millions of Americans' daily calling records into a central agency database. When the agency wants to review telephone traffic associated with a suspected terrorist - the agency made 300 such queries in 2012 - it then searches that data bank and retrieves matching calling records and stores them separately for further analysis.
Using a "three-hop" method that allows the NSA to pull in records from three widening tiers of phone contacts, the agency could collect the phone records of up to 2.5 million Americans during each single query. Obama this month imposed a limit of "two hops," or scrutinizing phone calls that are two steps removed from a number associated with a terrorist organization, instead of the current three.
An encrypted search system would permit the NSA to shift storage of phone records to either phone providers or a third party, and conduct secure searches remotely through their databases. The coding could shield both the extracted metadata and identities of those conducting the searches, Bellovin said. The government could use encrypted searches to ensure its analysts were not leaking information or abusing anyone's privacy during their data searches. And the technique could also be used by the NSA to securely search out and retrieve Internet metadata, such as emails and other electronic records.
Some computer science experts are less sanguine about the prospects for encrypted search techniques. Searches could bog down because of the encryption computations needed, said Daniel Weitzner, principal research scientist at MIT's Computer Science and Artificial Intelligence Laboratory and former deputy U.S. chief technology officer for the Obama administration.
"There's no silver bullet that guarantees the intelligence community will only have access to the records they're supposed to have access to," Weitzner said. "We also need oversight of the actual use of the data."
Intelligence officials worry that phone records stored outside the government could take longer to search and could be vulnerable to hackers or other security threats. The former NSA deputy director, John Inglis, told Congress last year that privacy - both for the agency and for Americans' whose records were collected - is a prime consideration in the agency's preference to store the phone data itself.
The encrypted search techniques could make it more difficult for hackers to access the phone records and could prevent phone companies from knowing which records the government was searching.
"It would remove one of the big objections to having the phone companies hold the data," Bellovin said.
Similar research is underway by researchers at University of California at Irvine; a group from the University of Wisconsin-Madison and the University of Texas at Austin; another group from MIT, Yale and Rensselaer Polytechnic Institute; and a fourth from Stealth Software Technologies, a Los Angeles-based technology company.
