AOL Released 20 Million Search Queries in 2006, With Names Replaced by Numbers, to Help Academic Researchers. The New York Times Found User No. 4417749 From Her Searches for Landscapers in Lilburn, Georgia.

Share
AOL Released 20 Million Search Queries in 2006, With Names Replaced by Numbers, to Help Academic Researchers. The New York Times Found User No. 4417749 From Her Searches for Landscapers in Lilburn, Georgia.
Photo by Sasun Bughdaryan / Unsplash

In the summer of 2006, AOL's research division put a file on its new research website for the benefit of academics who study how people use search engines. It held about 20 million search queries typed into the AOL client software by roughly 658,000 American users between March and May of that year, together with whether each searcher had clicked on a result, which result it was and where it had appeared on the page. The download was 439 MB compressed and just over 2 GB unpacked. AOL had removed the screen names. Each user was represented instead by a number, so that every search by the same person could be grouped together for study without anyone knowing who the person was.

The file sat there for about ten days without attracting much notice. On Sunday 6 August, Michael Arrington of TechCrunch wrote it up, quoting from the logs of user 491577, user 39509 and user 545605, and noting that it had already been downloaded 809 times. "The utter stupidity of this is staggering," he wrote. Some time around 7pm Pacific time the AOL page came down. The direct link to the file stayed live for a while longer, and by then there were mirrors. Earlier that year, as The New York Times would point out, Google had fought off a Justice Department subpoena for some of its search data, while several other search companies, AOL among them, had complied.

The Apology

On Monday 7 August AOL apologised, through its spokesman Andrew Weinstein, who posted the same statement in the comments of several blogs. "This was a screw up, and we're angry and upset about it," it began. "It was an innocent enough attempt to reach out to the academic community with new research tools, but it was obviously not appropriately vetted, and if it had been, it would have been stopped in an instant." The company had launched an internal investigation, and it told Reuters that the release had not been cleared through official channels.

The statement also put the incident in proportion. According to comScore Media Metrix, it said, the AOL search network had 42.7 million unique visitors in May, so the data covered roughly 1.5 per cent of them, and roughly a third of one per cent of all searches on the network over the period. "Although there was no personally-identifiable data linked to these accounts, we're absolutely not defending this," Weinstein wrote. A further bullet point added that "search queries themselves can sometimes include such information." Jason Epstein, of the law firm Baker, Donelson, Bearman, Caldwell & Berkowitz, told Reuters the release probably did not breach AOL's privacy policy, since no personally identifiable information was included. "This is more of a business snafu than anything else," he said.

User No. 4417749

On 9 August The New York Times published a story by Michael Barbaro and Tom Zeller Jr. about one number in the file. User No. 4417749 had conducted hundreds of searches over the three months, on subjects from "numb fingers" to "60 single men" to "dog that urinates on everything." There were also searches for "landscapers in Lilburn, Ga," for several people with the surname Arnold, and for "homes sold in shadow lake subdivision gwinnett county georgia." The reporters wrote that it did not take much investigating to follow the trail to Thelma Arnold, a 62-year-old widow in Lilburn, Georgia, who loves her three dogs. "Those are my searches," she said, after a reporter read part of the list to her.

The searches were not a perfect portrait. Her history included "hand tremors," "nicotine effects on the body," "dry mouth" and "bipolar," which suggested a woman with a wide range of ailments. Arnold explained that she routinely researched medical conditions for her friends, to calm their anxieties. "I have a friend who needs to quit smoking and I want to help her do it," she said. The data had identified her correctly and diagnosed her wrongly. "My goodness, it's my whole personal life," she told the Times. "I had no idea somebody was looking over my shoulder."

Asked about her, Weinstein said: "We apologize specifically to her. There is not a whole lot we can do." He said he knew of no other cases yet in which users had been identified, but was not surprised. "We acknowledged that there was information that could potentially lead to people being identified, which is why we were so angry." The Washington Post soon tracked down a second searcher, JoAnn Whitman, 55, of Grand Junction, Colorado, who had typed an e-commerce order confirmation into the search box. Arnold told the Times she planned to drop her AOL subscription. "We all have a right to privacy," she said. "Nobody should have found this all out."

The Task Force

On 14 August the Electronic Frontier Foundation asked the Federal Trade Commission to investigate AOL for violating its own privacy policy and the Federal Trade Commission Act, and attached the Times article as Exhibit D. The Register reported that EFF had found searches from 106 users that seemed to contain Social Security numbers, and 4,000 searches that included street addresses. EFF called the episode the "Data Valdez," and asked the FTC to make AOL notify the affected customers and stop logging search data except where absolutely necessary. The World Privacy Forum filed a complaint of its own.

On 21 August AOL's chief technology officer, Maureen Govern, resigned with immediate effect, just short of her first anniversary in the job. She had overseen the division responsible for the release. According to CNet, the researcher who had posted the data and that researcher's supervisor, who reported to Govern, were fired. AOL did not name them. In an email to staff, the chief executive, Jonathan Miller, announced a task force on privacy and data retention, extra restrictions on employee access to data, staff education, and a look at technologies that could flag sensitive information. The company also pledged to appoint its first chief privacy officer.

On Friday 22 September three subscribers, two unnamed Californians and Kasadore Ramkissoon of Richmond County, New York, sued in federal court in Oakland, California, seeking class-action status. "People paid AOL with the belief that their privacy was going to be protected," said their attorney, John Dominguez. "That's not what happened." The litigation outlasted the news cycle by several years. The claims came to rest in a class action, Landwehr v. AOL Inc., in which Judge Claude M. Hilton gave final approval on 24 May 2013 to a $5 million settlement. Members who filed a valid claim could receive $50 or $100. Checks began arriving on 18 November 2013, seven years and three months after the file went up.

The Prize

Two months after the AOL release, Netflix launched the Netflix Prize: $1 million to anyone who could improve its film recommendation system by 10 per cent. Contestants received 100 million ratings from about 480,000 customers, each identified by a number in place of a name. On 18 October 2006, weeks into the contest, Arvind Narayanan and Vitaly Shmatikov of the University of Texas posted a paper titled "How To Break Anonymity of the Netflix Prize Dataset." By comparing the ratings with reviews posted publicly on the Internet Movie Database, they identified the Netflix records of known users, "uncovering their apparent political preferences and other potentially sensitive information." Their conclusion was that an adversary who knows only a little about a subscriber can easily find that subscriber in the data.

The contest ran anyway, attracted more than 51,000 contestants and was won in September 2009 by a seven-member team led by AT&T researchers. At the September news conference where the prize was handed over, Netflix promised a sequel. This time the data would include ZIP codes, ages and gender. Paul Ohm, a law professor at the University of Colorado and a former Justice Department lawyer, called the idea "a privacy blunder that could cost millions of dollars in fines and civil damages." In December 2009 a lawsuit, Doe v. Netflix, was filed in federal court in California on behalf of a lesbian mother who was not out, and who believed that if her sexual orientation became public knowledge it would affect her ability to support her family. The FTC, meanwhile, had begun asking questions.

On Friday 12 March 2010 Neil Hunt, Netflix's chief product officer, wrote on the company blog: "We have reached an understanding with the FTC and have settled the lawsuit with plaintiffs. The resolution to both matters involves certain parameters for how we use Netflix data in any future research programs." The sequel was cancelled before anyone had said how much it would pay. Hunt said Netflix would "continue to explore ways to collaborate with the research community." Scott Kamber, the New York attorney who had filed the complaint, said nothing in the settlement prevented that, provided there were adequate safeguards.

AOL had published its data to help academics understand how people search. Its most widely reported finding came from two reporters at The New York Times, and concerned a widow in Lilburn, Georgia, who had been looking for a landscaper. Netflix published its data to improve its recommendations, and within weeks two researchers had published a paper on how to find the people in it. The sequel, which would have published more, was cancelled to settle a lawsuit brought by a subscriber who sued as Jane Doe.


Filed from public reporting:

Read more

Disney's Guest Assistance Card Gave Guests Who Could Not Wait in Line Faster Access With No Proof of Disability Required. Its Replacement Now Requires a Video Interview, and Disney's Own Court Papers Put Holders at 20 Percent of Guests.

Disney's Guest Assistance Card Gave Guests Who Could Not Wait in Line Faster Access With No Proof of Disability Required. Its Replacement Now Requires a Video Interview, and Disney's Own Court Papers Put Holders at 20 Percent of Guests.

Disney's Guest Assistance Card was, in Disney's later description, an accommodation that asked for nothing. A guest who could not wait in a conventional queue went to Guest Relations, said so, and received a card that opened a side entrance or a shorter line. NBC reported

By The Complaints Department
Westvleteren Abbey Sells Its Trappist Beer Only to Private Buyers Who Agree Not to Resell It. A Dutch Supermarket Put 7,200 Bottles on Its Shelves at 9.95 Euros Each Anyway.

Westvleteren Abbey Sells Its Trappist Beer Only to Private Buyers Who Agree Not to Resell It. A Dutch Supermarket Put 7,200 Bottles on Its Shelves at 9.95 Euros Each Anyway.

The Sint-Sixtus Abbey in Westvleteren, West Flanders, brews three Trappist beers and sells them to one kind of customer: the private individual. The abbey's own conditions of sale, which every buyer accepts, contain a clause of a single sentence, in which the buyer undertakes not to resell

By The Complaints Department
Michigan Refunds 10 Cents a Can, a Trade a 1996 Seinfeld Episode Explained to the Nation. Its Attorney General Said a Flint Man Returned Tens of Thousands From Indiana, and He Was Ordered to Repay $400,000, Which at a Dime Apiece Is Four Million Cans.

Michigan Refunds 10 Cents a Can, a Trade a 1996 Seinfeld Episode Explained to the Nation. Its Attorney General Said a Flint Man Returned Tens of Thousands From Indiana, and He Was Ordered to Repay $400,000, Which at a Dime Apiece Is Four Million Cans.

Michigan voters approved a bottle deposit by ballot in 1976. At 10 cents a container it became the highest refund in the country, and the neighbouring state of Indiana has no deposit at all. The arrangement is a standing invitation, and the Associated Press reported in 2013 that smugglers were

By The Complaints Department
US Airlines Let Emotional Support Animals Fly Free on a Mental Health Professional's Letter. Delta Was Carrying Nearly 250,000 Service and Support Animals a Year by 2018, and the Government Abolished the Category in 2020.

US Airlines Let Emotional Support Animals Fly Free on a Mental Health Professional's Letter. Delta Was Carrying Nearly 250,000 Service and Support Animals a Year by 2018, and the Government Abolished the Category in 2020.

The Department of Transportation added a provision dedicated to service animals on aircraft in 2008, codifying earlier guidance. Emotional support animals and psychiatric service animals counted as service animals, and airlines may not charge for service animals. Airlines were in turn allowed to require a letter from a licensed mental

By The Complaints Department