The Social Laboratory.

In October 2002, Peter Ho, the permanent secretary of defense for the tiny island city-state of Singapore, paid a visit to the offices of the Defense Advanced Research Projects Agency (DARPA), the U.S. Defense Department’s R&D outfit best known for developing the M16 rifle, stealth aircraft technology, and the Internet. Ho didn’t want to talk about military hardware. Rather, he had made the daylong plane trip to meet with retired Navy Rear Adm. John Poindexter, one of DARPA’s then-senior program directors and a former national security advisor to President Ronald Reagan. Ho had heard that Poindexter was running a novel experiment to harness enormous amounts of electronic information and analyze it for patterns of suspicious activity — mainly potential terrorist attacks.

The two men met in Poindexter’s small office in Virginia, and on a whiteboard, Poindexter sketched out for Ho the core concepts of his imagined system, which Poindexter called Total Information Awareness (TIA). It would gather up all manner of electronic records — emails, phone logs, Internet searches, airline reservations, hotel bookings, credit card transactions, medical reports — and then, based on predetermined scenarios of possible terrorist plots, look for the digital “signatures” or footprints that would-be attackers might have left in the data space. The idea was to spot the bad guys in the planning stages and to alert law enforcement and intelligence officials to intervene.

“I was impressed with the sheer audacity of the concept: that by connecting a vast number of databases, that we could find the proverbial needle in the haystack,” Ho later recalled. He wanted to know whether the system, which was not yet deployed in the United States, could be used in Singapore to detect the warning signs of terrorism. It was a matter of some urgency. Just 10 days earlier, terrorists had bombed a nightclub, a bar, and the U.S. consular office on the Indonesian island of Bali, killing 202 people and raising the specter of Islamist terrorism in Southeast Asia.

Ho returned home inspired that Singapore could put a TIA-like system to good use. Four months later he got his chance, when an outbreak of severe acute respiratory syndrome (SARS) swept through the country, killing 33, dramatically slowing the economy, and shaking the tiny island nation to its core. Using Poindexter’s design, the government soon established the Risk Assessment and Horizon Scanning program (RAHS, pronounced “roz”) inside a Defense Ministry agency responsible for preventing terrorist attacks and “nonconventional” strikes, such as those using chemical or biological weapons — an effort to see how Singapore could avoid or better manage “future shocks.” Singaporean officials gave speeches and interviews about how they were deploying big data in the service of national defense — a pitch that jibed perfectly with the country’s technophilic culture.