Techdirt. Stories filed under "data"

Do Dutch Spies Also Have Access To PRISM's Data? And If So, Who Else Does?

Glyn Moody — Thu, 13 Jun 2013 00:57:41 PDT

In the wake of the leaks about NSA's spying activities around the world, one of the interesting subsidiary questions is: who else had access to this stuff? We know that the UK did, and now there are indications the Dutch did as well, according to this report on DutchNews.nl: Justice minister Ivo Opstelten on Tuesday refused to comment on claims the Dutch security service AIVD works together with the US secret services in collecting information from email and social media traffic.

…

Dutch security service AIVD has also received information on email and social media traffic via US spy system PRISM, the Telegraaf reports on Tuesday.
Some pretty dramatic claims are being made: If the AIVD lists an American address as suspicious, it is supplied all the information within five minutes, a source told the paper. The source worked for the department which monitored potential Dutch Muslim extremists, the paper said.

Dutch companies also cooperated with the US authorities' request for information, the source said, claiming that 'there are agents ready to deal with requests for information inside companies and institutions.'

'There are a couple of those secret programmes like Prism active in the Netherlands,' the source is quoted as saying.
There are a few points to note here. First, this is a report about a story in the Dutch newspaper Telegraaf, which draws on unnamed sources. So the chain of information is quite long, and it's likely that details have been lost or mischaracterized along the way. It's also worth noting that PRISM is not the only system mentioned here for gleaning information about people. That's probably muddying the waters yet more, as sources reveal tantalizing information about other spying initiatives that then get subsumed under the general heading of PRISM, simply because it's in the headlines at the moment.

That's not to minimize the shocking nature of these revelations -- the idea that spies around the world may be accessing within minutes any private information they want, is troubling -- merely to note that the picture we have of what is going on remains frustratingly vague. And that, of course, is an argument for more transparency from the authorities, both in the US and elsewhere, about what is really happening to our personal information when we go online, and who has access to it.

Follow me @glynmoody on Twitter or identi.ca, and on Google+

Permalink | Comments | Email This Story

Verizon: We Protect Our Customers' Data... Until The Government Asks For It

Mike Masnick — Fri, 7 Jun 2013 07:31:00 PDT

So, Verizon has finally come out with a statement about the fact that they're handing over all data on every call anyone on its network makes to the government. And the response is just as ridiculous as you'd expect:

You may have seen stories in the news about a top secret order Verizon allegedly received to produce certain calling information to the U.S. government.

We have no comment on the accuracy of The Guardian newspaper story or the documents reference, but a few items in these stores are important. The alleged court order that The Guardian published on its website contains language that:

compels Verizon to respond;
forbids Verizon from revealing the order's existence; and
excludes from production the "content of any communication . . . or the name, address, or financial information of a subscriber or customer."
Verizon continually takes steps to safeguard its customers' privacy. Nevertheless, the law authorizes the federal courts to order a company to provide information in certain circumstances, and if Verizon were to receive such an order, we would be required to comply.

Let's parse that a bit. First, to "not comment" on it is ridiculous. This is the same issue I had with the government pretending that leaked Wikileaks documents had never leaked. It's not reality-based. In the business world, if you sign a non-disclosure agreement, it only applies to information that remains private. If the same information becomes public through other means, it's recognized that the non-disclosure agreement no longer applies. Because that's living in reality. Pretending you can't comment on the document is not reality-based.

Second, the claim that "Verizon continually takes steps to safeguard its customers' privacy" is completely meaningless when they're handing every bit of that data over to the government. Third, the idea that this order "excludes" information like someone's name is pretty silly. Don't you think that the federal government might have a giant database, in the form of a basic phone book that lets them look up the name associated with each number?

But, most importantly, this whole claim that Verizon is compelled to obey is silly and ignores some of the history. When the government asks you to break the law, you have the right to say no. And here's the big thing: even if this is legal today, that only came about because various telcos worked with the government on broad lawbreaking in the past, only to have the government paper that over with new laws that made such things "legal" and included retroactive immunity. And, really, that's all that Verizon really cares about (and you'll note they don't mention it): that they have no liability for coughing up everyone's information.

Permalink | Comments | Email This Story

Oh, And One More Thing: NSA Directly Accessing Information From Google, Facebook, Skype, Apple And More

Mike Masnick — Thu, 6 Jun 2013 15:35:00 PDT

Obviously, the Verizon/NSA situation was merely a small view into just how much spying the NSA is doing on everyone. And it seems to be spurring further leaks and disclosures. The latest, from the Washington Post, is that the NSA has direct data mining capabilities into the data held by nine of the biggest internet/tech companies:

The technology companies, which participate knowingly in PRISM operations, include most of the dominant global players of Silicon Valley. They are listed on a roster that bears their logos in order of entry into the program: “Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, Apple.” PalTalk, although much smaller, has hosted significant traffic during the Arab Spring and in the ongoing Syrian civil war.

Dropbox , the cloud storage and synchronization service, is described as “coming soon.”

This program, like the constant surveillance of phone records, began in 2007, though other programs predated it. They claim that they're not collecting all data, but it's not clear that makes a real difference:

The PRISM program is not a dragnet, exactly. From inside a company’s data stream the NSA is capable of pulling out anything it likes, but under current rules the agency does not try to collect it all.

Analysts who use the system from a Web portal at Fort Meade key in “selectors,” or search terms, that are designed to produce at least 51 percent confidence in a target’s “foreignness.” That is not a very stringent test. Training materials obtained by the Post instruct new analysts to submit accidentally collected U.S. content for a quarterly report, “but it’s nothing to worry about.”

Even when the system works just as advertised, with no American singled out for targeting, the NSA routinely collects a great deal of American content.

I expect we'll be seeing more such revelations before long.

Permalink | Comments | Email This Story

Eric Schmidt Still A Fan Of Figuring Out A Way To Erase The Past

Mike Masnick — Wed, 22 May 2013 15:26:41 PDT

Back in 2010, we wrote about Google's Eric Schmidt suggesting that in the future kids might change their names as they reach adulthood in order to disconnect their present-selves from their youthful indiscretions that were recorded permanently online. That seemed a bit silly to us at the time, but Schmidt is still focused on this basic concept apparently. His latest is the desire for some sort of delete button for the internet, again as a way to cover up some youthful indiscretions:

"In America, there's a sense of fairness that's culturally true for all of us," Schmidt said. "The lack of a delete button on the Internet is a significant issue. There is a time when erasure is a right thing."

Of course, this makes me wonder, what the hell did Eric Schmidt do as a kid that was so bad?

Yes, yes, we erase the criminal records of youthful offenders when they come of age, but I think this is something different. Trying to delete factual information from the internet is a quixotic task, unlikely to yield much that's beneficial.

Perhaps instead of trying to delete the past, society as a whole will become a lot more accepting of the fact that kids do stupid things when they're young. And many of them learn valuable lessons from those stupid things and they grow up to be better people. Plenty of folks have funny tales of their youthful indiscretions and, while these stories may be more difficult to embellish for effect if the details are all sitting on YouTube, does it really make more sense to try to delete that history or just to recognize that kids grow up and things they did as teenagers do not reflect how they're likely to act as adults?

Permalink | Comments | Email This Story

Retired Lt. Col.: Violent Media Has Bred A Generation Of Killers

Timothy Geigner — Tue, 21 May 2013 05:37:00 PDT

I can understand why someone who watches the news on a daily basis might shake their head in dismay. I get how it must feel necessary to put the blame for tragedies like Sandy Hook somewhere, anywhere, even if the eventual target of ire is misplaced. There's probably a certain ill-conceived synergy in pointing at violence in movies for a horrific theater shooting, even if the jury is at best out on the concept of violent media causing violent behavior. But when you hear someone so arrogantly sure that all of this wrong thinking is right and then using that arrogance to pompously try and scare the shit out of everyone, it deserves pushback.

So push back in your own way on Retired Lt. Col. Dave Grossman, whose hobbies amount to telling anyone that will listen that violent media has bred a "generation of vicious, vicious killers."

“This generation is going to give you massacres in the domestic environment and in the work environment,” he predicted, with unrestrained outrage, after ticking off a sobering list of mass murders -- particularly those that have occurred on school grounds and university campuses -- which have plagued American society in recent years.

It's a scare tactic that would be absolutely compelling were it not so absolutely wrong. Nevermind that no conclusive link between violent media and mass violence has been demonstrated, the fact of the matter is that in the long view, mass shootings are decreasing. As is violent crime in general. If one wanted to play the correlation game, they might happily state that violent media is decreasing violent output. I personally won't go that far, but it's fairly difficult to square Grossman's claims with reality.

“There is a generation of vicious, vicious killers out there,” he told his audience. “The video games are their trainers. They’ve been playing Grand Theft Auto every spare moment since they were six years old.”

Oh, please. Who the hell could listen to Niko's voice for years without killing themselves long before they killed anyone else? And besides, if these children are killers and the games their trainers, given the statistics already noted, then the little bastards are really bad at being killers and the games are a poor training manual, since none of this seems to be coming true. But if you thought Grossman had emptied his scare-clip at you already, you're wrong.

His views were no more subtle when he turned his sights to the international scene. A dozen years after the 9/11 terrorist attacks, the energetic and charismatic speaker sees life-threatening threats emanating from Pakistan, Russia, North Korea and Iran. What will the next 9/11 look like, he wondered aloud, before answering his own rhetorical question: “We’re going to get nuked.”

Lovely, and perhaps it will someday happen, but he forgot to mention something: if you look at the number of terrorist incidents in America throughout history, you'll note that the sharp increase in their occurence is non-existent. An enterprising commenter can graph the whole list out from that link if they like, but here's a brief example. We live in the year of our lord, 2013, so let's just start in 2012 and count how many terrorist incidents happened every ten years back:

2012 - 3 incidents, 9 killed
2002 - 14 incidents, 15 killed (note: the beltway sniper is counted as 11 separate incidents, which is stupid. This number should be 3 incidents total.)
1992 - 2 incidents, 1 killed
1982 - 6 incidents, 3 killed
1972 - 7 incidents, 6 killed
1962 - 1 incident, 44 killed
1952 - 0 incidents, 0 killed

Now, my use of 2012 as a starting point certainly is convenient in leaving out 9/11, but it's useful to note that outliers don't dominate the statistics. I should also note that this list ignores Pearl Harbor as the site of a terrorist attack, which seems incorrect in a way. The idea isn't that we can't be concerned about terrorism of any kind, be it Islamic fundementalist or the home-grown Aryan Nation variety, but if you make it your business to go around scaring people like Grossman does, you shouldn't leave out the more comforting facts simply because it is inconvenient to your narrative.

The next generation is not one of killers, this generation is not having to deal with terrorism in a way never before required, but it may be true that the media is more like Grossman and less like myself in that they enjoy scaring you for profit. Beware the man or woman who makes simple proclamations, because they're usually forgetting to tell you something.

Permalink | Comments | Email This Story

Some Data: Big Kickstarter Projects By Famous People Actually Help Other Projects

Mike Masnick — Wed, 15 May 2013 15:56:00 PDT

Last week, in writing about the silly backlash to Zach Braff's successful Kickstarter project, we noted that he claimed he had the data that showed his success did not take away from other Kickstarter projects, but rather it appeared that Braff brought a lot of new people to Kickstarter, many of whom went on to fund other projects. But still, the ridiculous arguments persisted that somehow famous people using Kickstarter take away money from upstarts. It's as if these people don't understand what a non-zero sum game is. They assume, incorrectly, that if one (famous) person is succeeding, it means one (non-famous) person is not. Perhaps the worst example of this was a piece by Reginald Nelson at TheWrap which ridiculously attacks Kickstarter's founders, arguing that these moves harm "the creative class."

To (hopefully) put an end to such ridiculousness, Kickstarter itself has shared the analytics and data that Braff was talking about concerning the impact of his project (as well as the Veronica Mars project, which is the other big one that some people have complained about):

The Veronica Mars and Zach Braff projects have brought tens of thousands of new people to Kickstarter. 63% of those people had never backed a project before. Thousands of them have since gone on to back other projects, with more than $400,000 pledged to 2,200 projects so far. Nearly 40% of that has gone to other film projects.

We’ve seen this happen before. Last year we wrote a post called Blockbuster Effects that detailed the same phenomenon in the Games and Comics categories. Two big projects brought tons of new people to Kickstarter who went on to back more than 1,000 other projects in the following weeks, pledging more than $1 million. Projects bring new backers to other projects. That supports our mission too.

I'd hope this puts to rest the ridiculous claims, but somehow, I doubt it will (and the comments on the Kickstarter blog post suggest people will still complain anyway).

Permalink | Comments | Email This Story

New Data Exposes Scammy Hospital Pricing; Now Let's Crowdsource Some More

Mike Masnick — Fri, 10 May 2013 02:14:46 PDT

A few months ago, we wrote about Steven Brill's amazing article in Time Magazine about hospital billing. As we noted at the time, the story confirmed what many people believed: that healthcare is a giant economic scam, and it's often the hospitals (not necessarily the insurance companies) who are driving the massive increases in costs. A big part of the scam is the fact that hospitals don't reveal their price list -- known as the "chargemaster." It's all a giant secret. There's no such thing as comparison shopping. There's simply no data anywhere.

Well, that may be changing. The US government just released data on what various hospitals charge for various things, along with how much Medicare actually pays in return. This has quickly resulted in people noticing massive differences in pricing for the same treatment in different hospitals (including, at times, hospitals very close to one another). This release definitely provides some significant data about just how massively hospitals are overcharging for things, even if most patients never pay the listed fees.

Still, it's not quite enough. Brill has responded to the release by noting that while this is a big deal and can be quite helpful in highlighting how broken the system really is (and hopefully will lead to a lot more reporting on the subject), it could go much further:

The feds need to publish chargemaster and Medicare pricing for the most frequent outpatient procedures and diagnostic tests at clinics—two huge profit venues in the medical world. But an even bigger step toward transparency would be collecting data that Medicare doesn’t have: exactly what insurance companies pay to the various hospitals, testing clinics and other providers for various treatments and services.

After all, as the hospitals themselves concede in downplaying their chargemasters, these insurance prices are the ones that affect most patients.

And that is one price list where there is close to zero transparency.

While hospitals and insurance providers clearly will not want to give up that information, Bill points out that patients do find out this information, so perhaps we should crowdsource the data:

So even if insurance companies don't want to participate, Brill writes, states could crowdsource price information from patients:
...state pricing centers could gather the information from patients who volunteer, in exchange for a promise that their names won't be used, to submit their Explanations of Benefits. After all, a hospital or insurance company can't claim a patient can be prohibited from talking about or making public his or her own bill.

For a market to work in any effective manner, pricing information must be clear. It's not that way at all in healthcare, and it needs to get that way fast if we're ever to get healthcare pricing under control.

Permalink | Comments | Email This Story

Broad Coalition Of Public And Private Interests Call For Objective Data & Research Concerning Copyright Reform

Mike Masnick — Mon, 6 May 2013 09:59:00 PDT

One of the key things that has been a major concern to us for many, many years is how much of copyright policy tends to be driven by faith-based claims about what must be best (often this falls into the "more must be better" category), rather than any objective analytical look at actual data and evidence. We were encouraged when the UK's Hargreave's Report did start to look at some objective data when it sought to understand how best to reform copyright in the UK. And we've been hearing encouraging things out of Australia as well. With copyright reform back on the table in the US, and Congress seemingly open to the discussion, having reality-based policy discussions will be more important than ever.

That's why it's actually quite encouraging to see a new report from the US National Research Council that has begun the process of calling for more objective data to inform the upcoming copyright reform debate. You can get the full PDF via the National Academies Press for free. They have an embedding widget which we've placed below as well, though it uses Flash, which is a bit annoying. The effort was funded by a broad coalition of organizations with a variety of different views on the issue, so it's not limited to just one particular view. For example, you've got copyright maximalist organizations like the MPAA and the BSA, but also Google and Pam Samuelson, who tend to take a different view on the appropriate level of copyright protection. There is also support from a number of different government and private foundations, including the National Science Foundation, the Alfred P. Sloan Foundation and the Ford Foundation.

The committee who put together this particular work also has a wide range of viewpoints covered, including Mitch Singer from Sony Pictures, former federal judge Marilyn Hall Patel who presided over the case against Napster, Chris Sprigman (law professor who wrote The Knockoff Economy: How Imitation Sparks Innovation and who has been featured prominently on Techdirt in the past) among a number of other big names with various viewpoints.

While the paper itself doesn't have any answers yet, it does highlight the key questions that we should be trying to answer, and indicates the beginnings of some research being done in that direction, with the likelihood of more to follow. I am a little annoyed that they still refer to things like the public's rights to access and use content as "exceptions" to copyright, because that feels unfairly limiting, but overall the direction of the work is quite encouraging. Here's a list of some of the initial questions they note it would be good to answer, if possible, which gives you an idea of the research areas they're interested in supporting and encouraging:

With respect to changing incentives for creators, distributors, and users, research could help determine

how the expenses involved in creative expression and distribution differ across sectors and the role of copyright in generating revenues to offset those expenses;
under what circumstances sources of monetary and/or non-monetary motivation outside of that provided by copyright are effective in motivating creative activity;
the motivations of various types of users and potential users of creative works, including both infringers and lawful users; the effects of enhanced enforcement remedies on promoting creativity, technological innovation, and freedom of expression; and
how the costs of distributing creative content are affected by social media and other new technologies.
With respect to the enablers of and impediments to voluntary licensing transactions in copyrighted works, research would help determine

the significance of transaction costs as barriers to utilization of copyrighted works;
the extent of problems involving orphan works (whose owners cannot be identified), user-generated content, and collaborative and iterative works;
what are successful arrangements for managing transaction costs;
the roles of public and private institutions in facilitating licensing;
the relationship of transaction costs to legal rules such as compulsory licenses; and
changes in transaction costs with new technological and business developments.
With respect to the enforcement challenges, research could help determine

how much is spent by governments and private parties on copyright enforcement;
against whom enforcement efforts are targeted and what remedies are sought and granted;
the results of enforcement efforts in terms of compensation, prevention, education, and deterrence;
how the effectiveness of enforcement efforts is changing with the expansion of digital networks;
the costs and benefits of current enforcement methods vis-a-vis those associated with proposed new enforcement methods;
the relative vulnerability of different business models to infringement; and
the costs and benefits of fair use exceptions and the Digital Millennium Copyright Act (DMCA) safe harbors.
In assessing the balance between copyright protection and the statutory exceptions and limitations to copyright research could help determine

the costs and benefits of copyright exceptions and limitations in terms of the economic outputs and welfare effects of those individuals, businesses, educational institutions, and other entities that rely on them;
how copyright and the various categories of limits and exceptions interact with innovative and/or disruptive technologies and platforms; and
what adverse effects, if any, exceptions and limitations have on copyright holders and their potential to generate economic outputs and welfare effects.
Eventually, research will help inform decisions about key aspects of copyright policy, including

the appropriate scope of copyright protection;
the optimal duration of the copyright term;
the best arrangements for correcting market imperfections that inhibit voluntary licensing;
appropriate safe harbors and fair use exceptions to copyright;
effective enforcement remedies for infringing use and the best arrangements for correcting deficiencies in enforcement mechanisms;
the advisability of reintroducing a formal registration requirement; and
the advantages and disadvantages of reshaping the copyright regime with different rules for different media.

The paper itself points to the concerns raised over things like SOPA and ACTA as reason to have a more empirical based approach to copyright reform, which is a good sign (and goes against those who insist that the SOPA protests had no real impact). The report goes into a lot more details, including a number of other important research topics as well.

One other point that they raise -- which is a key point we've brought up concerning our own Sky is Rising research -- is the need for those who have this data to be much more open about sharing it for the sake of making good overall policy. Since much of the data is considered "proprietary or subject to trade secrecy and privacy protections," the report outlines ways in which the data might be made available "on reasonable terms to qualified investigators." This, alone, would be a huge step forward in looking at many of the key policy questions above. The lack of real data is a huge impediment to being able to create effective policy.

All in all, it's a very good sign that this is underway, as it should really encourage a much more empirically-driven approach to the inevitable upcoming reform process. I hope that the results of future research driven by this particular effort do, in fact, play a role in any future debates on copyright reform. Moving from a faith-based look at copyright to an evidence-based one is a huge step forward, and long overdue.

Permalink | Comments | Email This Story

Anti-Piracy Enforcer Degban Publishes Some Info-less Graphics Stating Atlanta Is File Sharingest Place On Earth

Tim Cushing — Wed, 17 Apr 2013 10:58:00 PDT

Degban, the anti-piracy company that once took down comedian Dave Gorman's Flickr account because it thought photos of distinctive typography were actually its client's porn (and then blamed it on being hacked), has put together some very interesting (that's a colloquialism for "??!?") piracy numbers at its site.

Degban's "State of Piracy" page introduces itself with some rather ominous wording about piracy and the importance (and difficulty) of compiling accurate numbers.

When it comes to statistical data regarding content piracy, the complexity of the data and its abundance is utterly mind boggling. The data channels are scattered across geographical borders and scientific dimensions. The nature of the matter effects the data gathering process, resulting in datasets with chaotic orders and unclean results sets.

It is of grave importance, regardless of the challenge, to understand the trends and fashions behind content piracy as a thriving organism . Furthermore, appreciation of statical [sic] data regarding content piracy can be advantageous from a business intelligence point of view.

And some very fine "statical" data it is. The first chart posted is of the pie variety, breaking down the "market share" of several file storage lockers. Most of the contenders hover somewhere between 4-6% of the "market," possibly giving some credence to the MPAA's claims that Megaupload made the Internet go 'round by driving 99% of the traffic that wasn't Netflix (heavily paraphrased).

The data Degban has collected also shows the US firmly holding the lead in terms of "Origins of Pirate Peers." (USA! USA! USA!) How this number (185,290) is derived remains a mystery, but whatever it is, it's nearly nine times as much as Russia (28,001). So... who's doing all this piracy? Atlantans, apparently.

As can be seen by this chart, Atlanta is more than twice as piratey as Tacoma (?!?), the second place pirate haven, in terms of numbers lying along a Y-axis. Again, no further information is provided as to what these numbers represent, but we can certainly assume that Atlanta is the problem and Tacoma isn't helping. Once we venture beyond these two homegrown pirate bays, we see pirating is a major issue in such metropolises as Chalfont, PA and Iselin, NJ. Other cities/villages/unincorporated townships appear on the list as well, leading to questions of "Where is that, exactly?" and "Didn't that one guy used to live there... the serial killer/general store proprietor?" Cities with large populations -- New York and LA -- are pretty far down the list, which might make a lesser anti-piracy company question the quality of its data.

Degban really drives home the "Atlantans heart pirating" point with its next info-less graphic.

From this graphic portrayal of the United States battling a post-adolescent breakout of Venn diagram remnants, we can only draw one conclusion: if piracy is to be stopped, Atlanta has got to go. No more DMCA requests. No more lousy legislation. No more half-assed, self-serving infographics. You kill something by cutting off its head. The MPAA will be able to return millions of bag boys to their copyright-protected jobs just as soon as someone detaches Atlanta from the internet. And, from what I can tell of the map above, at that point, nearly all piracy will have gone away. If only Degban had made this clear earlier, we could have done away with all these lawsuits and focused in on the real piracy cancer: Atlanta.

Now, the numbers contributing to these SHOCKING bar graphs might possibly be found in this detailed, interactive (click over to Degban's site to experience this in all its glory) Google Map, which pinpoints where these "peers" are located, as well as the number of peers operating at each inverted teardrop.

Yes. There are some numbers in there, alright. Some low numbers. In fact, it's tough to find any location with more than 5 "peers." How these scattered points add up to over 10,000 pirating Atlantans is beyond me, especially considering Atlanta itself has no data point at all. None. Zip. Zero. Clearly, Degban processes data in a far advanced way, in which "0" peers means, "more than double the piracy impact of Tacoma, in which Tacoma is the number two piracy center."

You may also want to check out the dates on this map. It looks like the last data harvest was back in the spring of 2010. "OLD NEWS!!!" I hear you yell like a bunch of irate ~~Digg members~~ Redditors. But, how can that be? "Data is important," Degban tells us in the intro, along with this:

This section is regularly and automatically kept up to date.

From the looks of it, "regularly" means "once per decade." And "automatically" means "please delete this word from this sentence before publishing this page."

So, we have a bunch of numbers that don't add up, aren't current and do little more than randomly drop circles on maps and rub bars affectionately up against sky-high Y-axises. And yet, Degban is a well-known name in the piracy world, one that understands the "grave importance" of accurate and up-to-date statistics. To fight your enemy successfully, you must know them, and from what I'm seeing here, Degban is nearly 100% sure someone's pirating stuff somewhere.

Godspeed, number wranglers! Remind me to keep one hand on my content when passing through the muggy climes of Hotlanta or the suicidal murkiness of upstate Washington! And tell Vkontakte.ru that everyone's extremely disappointed in its lack of effort on the file sharing front.

Permalink | Comments | Email This Story

This Is Not The Cloud Computing We Should Have

Mike Masnick — Fri, 29 Mar 2013 19:39:00 PDT

Even though I was never a big Google Reader user, its death has got me thinking about online services quite a bit lately -- and really reminded me that we've done the cloud wrong. Rather than build true cloud computing, we've built a bunch of lockboxes.

The cloud was supposed to free us, not lock us in

"Cloud computing" went by a variety of other terms in the past before this marketing term stuck, but the key part of it was that it was supposed to free us of worrying about the location of our data. Rather than having to have things stored locally, the data could be anywhere, and we could access it via any machine or device. That sort of happened, and there definitely are benefits to data being stored in the cloud, rather than locally. But... what came with today's "cloud" was a totally different kind of lock: a lock to the service.

I can point many apps to data stored locally

I wrote something related to this a few years ago, concerning music in the cloud. If I have a bunch of MP3s stored locally, I can point any number of music apps at my music folder, and they can all play that music. As long as the data is not in a proprietary format, I can find the app that works best for me and the data is separate from the app. Even when you have proprietary formats like Microsoft's .doc, other apps can often make use of them as well -- so, for example, I can get by with Libre Office, and I don't lose access to all of my old Microsoft Word docs.

This is really useful, because it helps us avoid vendor lock-in in many cases. Even when, say, Microsoft or Apple dominates the market. It's still possible to come in and be compatible. The competition then focuses on building better services, rather than reinventing the data model. That's much more useful to consumers, because the innovation is focused on making their lives better, rather than reinventing the wheel.

Today's cloud brings us back to walled gardens

For the most part, today, however, "cloud" applications bundle the storage and the service as one, and the two are linked inseparably. You check your data into a new cloud service, but the application layer and the data are both held by the same company. Yes, you can often transfer data from one service to the other -- such as Google's "data liberation front" effort, which is fantastic (and goes beyond many other companies' efforts), but just the fact that data needs to be liberated suggests we're taking the wrong approach altogether. Rather than having to "export" all of your feeds from Google Reader and then waiting patiently for 50,000 other people who are trying to upload them to the few small Reader competitors out there, why shouldn't we have each had an OPML file stored somewhere that we control, and that we could easily point any reader application, whether its local or "in the cloud." And, yes, there are some services that attempt to do this, but it's not where the whole "cloud" space has gone.

Separate and liberate the data from the infrastructure

What the cloud should be about is both freeing us from being locked to local data, and also freeing us from having that data locked to a particular service. I should be able to keep my data in one spot and then access it via a variety of cloud clients -- and the clients and the data shouldn't necessarily be directly connected or held by the same party. If I don't want to listen to my music via one app, I can just connect a different app to my personal data cloud and off we go. If Google Reader shuts down, no problem, just point a different app at my RSS data. No extraction, no uploading. Just go.

There are, of course, plenty of players around which sort of do this. DropBox, Box, Amazon's S3 and even Google Drive are setting themselves up as personal data clouds, and there are a growing number of apps that run across them. Projects like the Locker Project are thinking about how we store personal data separated from apps as well. And I know there are a bunch of other projects either around today or quickly approaching release, that also seek to do something in this space.

But, for the most part, all of the stories that people talk about concerning "cloud" computing almost always involve services that tie together the app and the data and all you're really doing is trading the former limitations of local data for the limitations of a single service provider controlling your data. Many service providers want this, of course. It's a form of lock-in. Plus, having some sort of access to your data and your usage can enable them to do other things, such as more accurately data mine you and your usage.

But, as users, we really should be pushing more towards embracing the apps that separate the app from the data and that let you point their "cloud" app at any particular place you store your "cloud" data. Some of this may involve standardizing certain data formats, but that makes sense anyway, as, once again, that's an area where there are tremendous benefits to not reinventing the wheel, so that the innovation and competition can focus on the service level. While some vendors may fear losing lock-in, if they truly believe in their own ability to provide great services, it shouldn't be a problem. At the same time, they should also realize that embracing this kind of world means that it's easier for others to jump in and test their services as well.

The death of Google Reader raised a lot of issues around trust, and while you could "export" the data, that process is still messy and archaic when you think about it. The future of cloud computing should be much more focused on separating the data from the service. That would remove the fear that many are now talking about concerning adopting new cloud services that might not last very long. If the data is stored elsewhere, and entirely in the control of the user, then you don't need to trust the service provider nearly as much, but can dip in and test out different apps operating on the same data, and switch with ease.

If we're going to see the real promise of "the cloud" take place, that's where things need to head. We should be increasingly skeptical of "cloud" apps that also control the data.

Permalink | Comments | Email This Story

The List Of Government Agencies That Can Get Your Data Under CISPA

Mike Masnick — Mon, 25 Mar 2013 15:02:05 PDT

One of the key complaints about CISPA is the fact that it does absolutely nothing to make sure any data of yours that is shared with the government by third parties is sent narrowly to folks working to protect us from cybersecurity threats. Instead, the information can be shared with any agency of the government, so long as they can claim, vaguely, that it's being used for "cybersecurity purposes." But, as the EFF points out, without any limitations on who in the government can see your data, every government agency can see your data. They've even put together a helpful "list."

One question we sometimes get is: Under CISPA, which government agencies can receive this data? For example, could the FBI, NSA, or Immigration and Customs Enforcement receive data if CISPA were to pass?

The answer is yes. Any government agency could receive data from companies if this were to pass, meaning identifiable data could be flowing to the Bureau of Alcohol, Tobacco, Firearms and Explosives, the National Security Agency, or even the Food and Drug Administration.

We've reposted the list below as well, just so you can get an idea of which government agencies could get access to your data on CISPA (and which ones thought that, perhaps, that's not such a good idea).

Permalink | Comments | Email This Story

Apple's Patent For Creating A Leak-Proof Data Pipe, And Why It's Doomed To Fail

Glyn Moody — Thu, 21 Mar 2013 05:33:09 PDT

In 2001, I published a history of free software, called "Rebel Code: Inside Linux and the Open Source Revolution." One of the people I interviewed for the book was Eben Moglen, for many years the General Counsel for the Free Software Foundation, and one of the main architects of the later versions of the GNU General Public License. He had the following interesting thoughts on the delivery of digital media:

Let's think about the Net for a change as a collection of pipes and switches, rather than thinking of it as a thing or a space.

There's lot of data moving through those pipes, and the switches determine who gets which data and how much they have to pay for it downstream. And of course those switches are by and large what we think of as digital computers.

The basic media company theory at the opening of the 21st century is to create a leak-proof pipe all the way from production studio to eyeball and eardrum.

Creating that "leak-proof pipe" has long been the dream not only of media companies, but also of computer companies like Apple that hope to collaborate with and ultimately supplant them. A recent patent application, found through the French title Numerama, seeks to make videos uncopiable during playback by locking down the last section of the pipe -- the part that connects the computer to the screen. Here's how Apple's patent describes it: Securing protected content during video playback. The embodiments provide a system that drives a display from a computer system. During operation, the system writes graphical output, generated from a copyrighted video file, to protected memory and drives the display from the protected memory. If the graphical output lacks copy protection, the system discontinues the driving of the display from the protected memory. In particular, upon detecting a lack of copy protection in the graphical output, the system continues to drive the display from the protected memory during a grace period associated with the lack of protection in the graphical output. The system then discontinues driving of the display from the protected memory if protection of the graphical output does not resume during the grace period.
The rest of the patent describes the details of the process. What's striking -- and sad -- is the effort and ingenuity that has been put into making things less convenient for the end user. After all, introducing a system that automatically shuts down when it thinks security may be absent is a recipe for disaster -- as if current DRM screw-ups weren't enough of problem.

Moreover, Apple's system will fail, just as all the other approaches to "protecting content" have failed. Anything involving copy protection is regarded as a challenge by certain people; it's not a question of "if" the particular scheme employed by Apple will be broken, but "when". And there's another, deeper reason why such attempts won't ever work. As Moglen explained to me all those years ago:

The switch that most threatens that pipe is the one at the end. If the switch closest to your eyeball and your eardrum is under your complete technical control, the whole rest of the aqueduct can be as leak-proof as they like, and it won't do them any good. And the switch is under your control, of course, if the software is free software.

While there's free software, the data pipe will always be leaky.

Follow me @glynmoody on Twitter or identi.ca, and on Google+

Permalink | Comments | Email This Story

Rep. Gohmert's Record For Stunning Technological Ignorance Is Broken By... Rep. Gohmert

Mike Masnick — Wed, 20 Mar 2013 11:06:11 PDT

My goodness. Yesterday we posted about Rep. Louis Gohmert's incredible, head-shakingly ignorant exchange with lawyer Orin Kerr during a Congressional hearing concerning "hacking" and the CFAA. In that discussion, Gohmert spoke out in favor of being able to "hack back" and destroy the computers of hackers -- and grew indignant at the mere suggestion that this might have unintended consequences or lead people to attack the wrong targets. Gohmert thought that such talk was just Kerr trying to protect hackers.

I thought perhaps Rep. Gohmert was just having a bad day. Maybe he's having a bad month. In a different hearing, held yesterday concerning ECPA reform, Gohmert opened his mouth again, and it was even worse. Much, much worse. Cringe-inducingly clueless. Yell at your screen clueless. Watch for yourself, but be prepared to want to yell. The short version of this is that he seems to think that when Google has advertisements on Gmail, that's the same thing as selling all of the information in your email to advertisers. And no matter how many times Google's lawyer politely tries to explain the difference, Gohmert doesn't get it. He thinks he's making a point -- smirking the whole time -- that what Google does is somehow the equivalent of government snooping, in that he keeps asking if Google can just "sell" access to everyone's email to the government. I'm going to post a transcript below, and because I simply cannot not interject how ridiculously uninformed Gohmert's line of questioning is, I'm going to interject in the transcript as appropriate.

Rep. Gohmert: I was curious. Doesn't Google sell information acquired from emails to different vendors so that they can target certain individuals with their promotions?

Google lawyer whose name I didn't catch: Uh, no, we don't sell email content. We do have a system -- similar to the system we have for scanning for spam and malware -- that can identify what type of ads are most relevant to serve on email messages. It's an automated process. There's no human interaction. Certainly, the email is not sold to anybody or disclosed.

Gohmert: So how do these other vendors get our emails and think that we may be interested in the products they're selling.

Okay, already we're off to a great start in monumental ignorance. The initial question was based on a complete falsehood -- that Google sells such information -- and after the lawyer told him that this is not true, Gohmert completely ignores that and still asks how they get the emails. It never seems to occur to him that they don't get the emails.

Google lawyer: They don't actually get your email. What they're able to do is through our advertising business be able to identify keywords that they would like to trigger the display of one of their ads, but they don't get information about who the user is or any...

Gohmert: Well that brings me back. So they get information about keywords in our emails that they use to decide who to send promotions to, albeit automatically done. Correct?

NO. Not correct. In fact, that's the exact opposite of what the lawyer just said. Gohmert can't seem to comprehend that Google placing targeted ads next to emails has NOTHING to do with sending any information back to the advertiser. I wonder, when Rep. Gohmert turns on his television to watch the evening news, does he think that the TV station is sending his name, address, channel watching info, etc. back to advertisers? That's not how it works. At all. The advertisers state where they want their ads to appear, and Google's system figures out where to place the ads. At no point does any information from email accounts go back to anyone. And yet Gohmert keeps asking.

And not understanding the rather basic answers. Unfortunately, the lawyer tries to actually explain reality to Gohmert in a professional and detailed manner, when it seems clear that the proper way to answer his questions is in shorter, simpler sentences such as: "No, that's 100% incorrect."

Lawyer: The email context is used to identify what ads are most relevant to the user...

Gohmert: And do they pay for the right or the contractual ability to target those individuals who use those keywords?

Lawyer: I might phrase that slightly differently, but the gist is correct, that advertisers are able to bid for the placement of advertisements to users, where our system has detected might be interested in the advertisement.

Gohmert: Okay, so what would prevent the federal government from making a deal with Google, so they could also "Scroogle" people, and say "I want to know everyone who has ever used the term 'Benghazi'" or "I want everyone who's ever used... a certain term." Would you discriminate against the government, or would you allow the government to know about all emails that included those words?

Okay, try not to hit your head on your desk after that exchange. First, he (perhaps accidentally) gets a statement more or less correct, that advertisers pay to have their ads show up, but immediately follows that up with something completely unrelated to that. First, he tosses in "Scroogled" -- a term that Microsoft uses in its advertising against Gmail and in favor of Outlook.com -- suggesting exactly where this "line" of questioning may have originated. Tip to Microsoft lobbyists, by the way: if you want to put Google on the hot seat, it might help to try a line of questioning that actually makes sense.

Then, the second part, you just have to say huh? The lawyer already explained, repeatedly, that Google doesn't send any information back to the advertiser, and yet he's trying to suggest that the government snooping through your email is the same thing... and Google somehow not giving the government that info is Google "discriminating" against the government? What? Really?

Lawyer [confounded look] Uh... sir, I think those are apples and oranges. I think the disclosure of the identity...

Gohmert: I'm not asking for a fruit comparison. I'm just asking would you be willing to make that deal with the government? The same one you do with private advertisers, so that the government would know which emails are using which words.

Seriously? I recognize that there are no requirements on intelligence to get elected to Congress, but is there anyone who honestly could not comprehend what he meant by saying it's "apples and oranges"? But, clearly he does not understand that because not only does he mock the analogy, he then repeats the same question in which he insists -- despite the multiple explanations that state the exact opposite -- that advertisers get access to emails and information about email users, and that the government should be able to do the same thing.

Lawyer: Thank you, sir. I meant by that, that it isn't the same deal that's being suggested there.

Gohmert: But I'm asking specifically if the same type of deal could be made by the federal government? [some pointless rant about US government videos aired overseas that is completely irrelevant and which it wasn't worth transcribing] But if that same government will spend tens of thousands to do a commercial, they might, under some hare-brained idea like to do a deal to get all the email addresses that use certain words. Couldn't they make that same kind of deal that private advertisers do?

Holy crap. Gohmert, for the fourth time already, nobody gets email addresses. No private business gets the email addresses. No private business gets to see inside of anyone's email. Seeing inside someone's email has nothing to do with buying ads in email. If the government wants to "do the same deal as private advertisers" then yes it can advertise on Gmail... and it still won't get the email addresses or any other information about emailers, because at no point does Google advertising work that way.

Lawyer: We would not honor a request from the government for such a...

Gohmert: So you would discriminate against the government if they tried to do what your private advertisers do?

No. No. No. No. No. The lawyer already told you half a dozen times, no. The government can do exactly what private advertisers do, which is buy ads. And, just like private advertisers, they would get back no email addresses or any such information.

Lawyer: I don't think that describes what private advertisers...

Gohmert: Okay, does anybody here have any -- obviously, you're doing a good job protecting your employer -- but does anybody have any proposed legislation that would assist us in what we're doing?

What are we doing, here? Because it certainly seems like you're making one of the most ignorant arguments ever to come out of an elected officials' mouth, and that's saying quite a bit. You keep saying "private advertisers get A" when the reality is that private advertisers get nothing of the sort -- and then you ignore that (over and over and over and over again) and then say "well if private advertisers get A, why can't the government get A." The answer is because neither of them get A and never have.

Gohmert: I would be very interested in any phrase, any clauses, any items that we might add to legislation, or take from existing legislation, to help us deal with this problem. Because I am very interested and very concerned about our privacy and our email.

If you were either interested or concerned then you would know that no such information goes back to advertisers before you stepped into the room (hell, before you got elected, really). But, even if you were ignorant of that fact before the hearing, the fact that the lawyer tried half a dozen times, in a half a dozen different ways to tell you that the information is not shared should have educated you on that fact. So I'm "very interested" in what sort of "language" Gohmert is going to try to add to legislation that deals with a non-existent problem that he insists is real.

Gohmert: And just so the simpletons that sometimes write for the Huffington Post understand, I don't want the government to have all that information.

Rep. Sensenbrenner: For the point of personal privilege, my son writes for the Huffington Post.

Gohmert: Well then maybe he's not one of the simpletons I was referring to.

Sensenbrenner: He does have a Phd.

Gohmert: Well, you can still be a PHUL.

Har, har, har... wait, what? So much insanity to unpack. First of all, Gohmert seems to think that people will be making fun of him for suggesting that the government should "buy" access to your email on Google. And, yes, we will make fun of that, but not for the reasons that he thinks they will. No one thinks that Gohmert seriously wants the government to buy access to information on Google. What everyone's laughing (or cringing) at is the idea that anyone could buy that info, because you can't. No private advertiser. No government. It's just not possible.

But, I guess we're all just "simpletons."

Seriously, however, we as citizens deserve better politicians. No one expects politicians to necessarily understand every aspect of technology, but there are some simple concepts that you should at least be able to grasp when explained to you repeatedly by experts. When a politician repeatedly demonstrates no ability to comprehend a rather basic concept -- and to then granstand on their own ignorance -- it's time to find better politicians. Quickly.

Permalink | Comments | Email This Story

Giant Pharma Company Claims Releasing Data On Drug Safety Is Illegal As It's Confidential And 'Commercially Sensitive'

Glyn Moody — Mon, 18 Mar 2013 08:51:16 PDT

One of the initiatives gaining momentum around the world is open data -- the idea that, for example, non-personal data affecting the public should be made freely available. That's partly to improve transparency, so that citizens are more informed about what is happening, and partly to stimulate new kinds of business that build products and services based on that data.

An important category of open data that boosts transparency concerns basic drug safety information. Last month, Techdirt wrote about the AllTrials initiative that seeks to have key information about clinical trials placed in the public domain. As part of a wider move towards greater openness, the European Medicines Agency, the main body that licenses drugs in Europe, is starting to make available information that has hitherto been withheld.

Although doctors and patients are rejoicing at this greater transparency, not everyone is pleased by the move. AbbVie, the pharma company spun out of the Abbott Laboratories at the beginning of this year, for example, is taking legal action to stop it:

AbbVie, a pharmaceutical company has sought an injunction to block Europe's medicines regulator from releasing "confidential" and "commercially-sensitive" information on its blockbuster rheumatoid arthritis drug, a spokeswoman for the U.S. drugmaker confirmed on Sunday.

The Chicago-based company had taken legal action against the European Medicines Agency to stop it from releasing data on the effects in individual patients in clinical trials for its drug Humira, the Financial Times reported earlier on Sunday.

Except, of course, this isn't "confidential" and "commercially-sensitive" information: it's just basic data about its safety and efficacy. Doctors and patients surely have a right to know this before using products that could potentially have serious, even fatal, side-effects.

Another EU body, the European Food Safety Authority (EFSA), is also opening up:

The project is part of EFSA's continuing commitment to openness and addresses recommendations made by an independent evaluation report of the Authority's performance to further enhance transparency in its decision-making processes. EFSA's Science Strategy also highlights the importance of the Authority playing a leading role in making relevant scientific data more accessible to all interested parties.

Here's one particular set of data that it has now released: Given the level of public interest, EFSA will make all data on genetically modified (GM) maize NK603 publicly available on its website today (14 January).
Once more, that seems reasonable, since the public ought to be able to find about what is going into the food chain whose end-products it will consume. But some disagree: according to a story on Bakeryandsnacks.com, Monsanto is threatening to sue the EFSA over the release of this data.

What makes this a little confusing is that the company is quoted in that article as saying that it "firmly supports transparency" -- and yet here it is fighting tooth and nail against precisely that. Apparently, Monsanto also wants the regulatory environment in Europe to be "science-based". Modern science requires experimental data to be made available so that anyone can check the validity of the conclusions that have been drawn from it. If it can't be scrutinized, the conclusions can't be confirmed, and it's not science. So, given its call for "science-based" regulation, why does the company want to keep that data hidden? A cynic might almost suspect that Monsanto and AbbVie have something to hide.

Follow me @glynmoody on Twitter or identi.ca, and on Google+

Permalink | Comments | Email This Story

Why Shouldn't New Legislative Data Flow Directly Into Wikipedia

Mike Masnick — Thu, 14 Mar 2013 16:11:57 PDT

There's an interesting event going on today and tomorrow at the Cato Institute, with a very practical focus: looking at ways to automate the process of getting legislative data into Wikipedia. That is, when new bills are introduced, and as they make their way through Congress and to the President, is there any reason that data doesn't automatically populate to Wikipedia?

Our project to produce enhanced XML markup of federal legislation is well under way, and we hope to use this data to make more information available to the public about how bills affect existing law, federal agencies, and spending, for example.

What better way to spread knowledge about federal public policy than by supporting the growth of Wikipedia content?

There are a bunch of services out there that present such legislative data, but having a straight XML feed from Congress to Wikipedia seems like an all around good idea.

Permalink | Comments | Email This Story

Federal Judge Alex Kozinski Talks About Using Tor To Surf Silk Road & The Armory For Drugs, Weapons And Hitmen

Mike Masnick — Fri, 1 Mar 2013 11:50:00 PST

While I don't always agree with him (who do I always agree with?), like many folks who follow legal issues, Judge Alex Kozinski, the chief judge of the court of appeals for the 9th circuit, is one of my favorite judges. Known almost as much for his ability to entertain as for his clear, well-written (and frequently funny) judicial rulings, one thing that's always been clear is that, unlike some judges, Kozinski is both down to earth and really inquisitive when it comes to understanding how things really work, rather than just accepting common wisdom. Last night, Judge Kozinski gave a lecture at Santa Clara University on "The Two Faces of Anonymity." As I expected, it was entertaining and insightful, with a few Kozinski-esque surprises thrown in.

By far the most entertaining part of the evening was Kozinski sharing (with screenshots) his experience exploring the "hidden web." He claims that when he told his children about the topic of the talk, they told him he needed to explore the hidden web. So, "with some trepidation," he downloaded Tor and dove in, starting out at Silk Road, which still remains the most well known hidden website out there. As we've noted in the past, for all the excitement and press attention Silk Road has received for being a totally anonymous online marketplace used mainly for buying and selling drugs and other illicit goods, it still is a fairly small business. Still, Judge Kozinski detailed his exploration of the market, including checking out various drugs (including many he'd never heard of before). He also looked into the ability to buy forged documents and lots of counterfeit software. From there, he moved over to Silk Road spin-off, The Armory, to see what weapons they had for sale, including 6lbs of C4 explosives. Of course, this is the point that we realize that Kozinski's claims of just having done this recently are probably a fabrication, given that The Armory shut down last summer. It's possible he didn't actually do any of this, but got screenshots from elsewhere online, but there's just something amusing in thinking about Judge Kozinski sitting at home surfing through these sites. He showed a few sites for hiring hitmen, and joked that two of them had such similar language and pricing that he was tempted to report them to the FTC for likely collusion.

He marveled at how much like regular online stores these sites were -- including things like seller ratings -- and compared it to his experiences with eBay. Of course, he also noted that it's entirely possible the whole thing is a front by the feds to track these kinds of things, but if so, he was impressed with the level of detail.

While much of this was entertaining, the point (I think!) was to highlight all of the kinds of things that anonymity enables -- but it wasn't in a necessarily negative or judgmental way (even if he's suggested his concerns in the past). Instead, it was more of a realist approach to what's happening out there and how there are interesting challenges presented concerning both anonymity and privacy -- which he notes are related but not the same thing. To show the difference, he discussed your neighbors across the way, where they may not be anonymous to you, but what they do in their bedroom is kept private from you. Yet, take a random couple in Times Square on New Years Eve doing the same thing -- and they may be "anonymous," but not private at all.

While he did express some concerns about where all this leads, including a dig at anonymous comments online, his biggest concern appeared to be about government abuse thanks to technology. He spent a fair bit of time on the NSA's infamous spy center in Utah, which is supposedly storing a ridiculous amount of information on us all. He pointed out that having that much information in the hands of government is dangerous, and suggested it's likely to be abused. As an example, he pointed to the story from all the way back in 2001 when he and other federal judges discovered that the feds were monitoring their internet usage, something the judges had never been told about.

He explained that the software had been put on the computers to protect the judiciary intranet from being attacked by hackers from China or whatever, but most of the time they weren't doing anything at all, so it wasn't long before the scope began to creep, and someone realized that, hey, if that monitoring software is on those computers, it could also be used to spy on what sites judges were surfing. The judges only found out about it when a judge was called out for his inappropriate surfing habits.

While he didn't say anything explicitly about it, it seems like this should be a pretty clear warning to folks who are supporting laws like CISPA. When you increase information sharing to the government for one purpose, you can almost guarantee that there will be scope creep over time. Someone will point out that "hey, we're already doing this for security, so why not for spying on people...."

Similarly, Kozinski is worried about how all this number crunching and data collection by governments means that people are going to be "targeted" for heightened scrutiny based on some algorithms, even if their activity is perfectly legal. He even noted that he's assuming that his own decision to download Tor and check out Silk Road and other sites probably means that he set off some alarms and may be in for heightened scrutiny. When asked about that later during the Q&A, he admitted that it might just be his own paranoia, but he wouldn't be surprised if it was true.

When asked about how to push back on all this government surveillance, he said that everyone keeps pointing to the courts, and saying that it's their responsibility to limit the government's powers, but suggested that the courts are limited, because it's not clear that anonymity and privacy are really Constitutional issues. Or, he said, if there is a basis for them in the Constitution, it's fairly weak, and could easily be overcome by "other concerns." Personally, I think that he downplayed both the First Amendment's protection of anonymity as confirmed by the Supreme Court, as well as the 4th Amendment's (too often ignored) protection of privacy. Still, he seemed to think that this was really an issue where it was up to Congress to prevent abuses. That's kind of depressing if you remember Congress' recent "debate" and subsequent rubberstamping of the FISA Amendments Act, giving the NSA much more power to spy on Americans with little oversight.

One other bit of useful info: he seemed fairly convinced by Justice Sotomayor's statements on the 3rd party doctrine in the US v. Jones case about GPS tracking. If you don't recall, the 3rd party doctrine basically says that you don't have privacy rights in information that you've left in the control of a third party. That's obviously quite problematic in an age of cloud computing, where all your data is probably in the hands of third parties. The government has been relying on this fact to access all sorts of data with little oversight for quite some time. It's good to see Kozinski hint at the idea that the 3rd party doctrine just isn't reasonable any more in the information era.

There were plenty of other tidbits, but basically it was an interesting discussion of privacy and anonymity, with a strong focus in how the government is collecting way too much information on us all these days. There was also some brief talk of how much information companies are collecting too -- including his apparent uncomfortableness with things like Google Maps' Street View and Satellite View (he joked about how you can see him sunbathing nude if you can find his house). But, for the most part, he seemed to think that this was an area where the government was doing a better job keeping companies somewhat in check.

Oh yeah, and one other amusing tidbit: in talking about how easy it is to track us all due to our mobile phones, he asked how many people had smartphones (or, more specifically, "phones with email on them") and noted that when he talks to lawyers, they all do. He noted that lawyers always had their email near them to respond to clients quickly, because otherwise you get fired, but this cool tool "given to you by work" just shackles you while also denting your privacy. And then he claimed that when work gives him a smartphone, he gets it without a sim and then sells the device on eBay. Maybe he should try selling it on Silk Road next time...

Permalink | Comments | Email This Story

DailyDirt: Big Data On Kid Development

Michael Ho — Thu, 28 Feb 2013 17:00:00 PST

Long term studies of children are expensive, but the data collected may be invaluable -- especially if wide-ranging policies are based on them. The amount of data Facebook is collecting on kids may be massive, but it'll probably only be data mined for marketing purposes. Here are just a few examples of some kid research that could influence the way people raise their kids.

Measuring the difference between heart rates when a baby inhales and exhales could lead to predictive indicators for whether or not that kid will overcome poverty. For "sensitive" babies, where the difference is large, a stable environment may have a greater effect on the child's future well-being. [url]
Preschool seems to be important for disadvantaged kids, but the benefits for middle class kids are not as clear. So flip a coin, tiger-moms, but some economists might have a good argument for government funded access to preschool for at-risk toddlers. [url]
The 7 Up documentary began in 1964 and followed seven year old kids from different economic backgrounds as they grew up. The latest installment, 56 Up checks in on these people now as they face retirement (or not) -- and presents a fascinating snapshot of society. [url]

If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) Techdirt post.

Permalink | Comments | Email This Story

State Audit Finds More Than Half Of Minnesota's 11,000 Law Enforcement Users Misused Driver Data

Tim Cushing — Thu, 28 Feb 2013 01:09:36 PST

It's been proven before by various government agencies and it still holds true: if you give someone more access than oversight when it comes to collected personal information, you can't be surprised when this tool gets abused.

The latest abuse of a government-harvested database was uncovered by state auditors in Minnesota. The report details extensive misuse of driver's license records by law enforcement agencies.

The review by the state's legislative auditor — highly anticipated by legislators and privacy advocates — said officers need better training in allowed uses of the protected data, and local and state agencies should do more to monitor use. Beyond 88 incidents of misuse documented in state records last year, auditors found even more suspicious activity buried in audit trails.

More than half of the 11,000 law enforcement users of the Driver and Vehicle Services (DVS) website in that time frame queried themselves or people with the same last name, for example, or disproportionately searched for people of one sex.

This study's findings will likely result in some additions to legislation proposed earlier this year, which seeks to add penalties and transparency to data breaches by government employees, requiring local agencies to post full investigation reports online should any breach occur. The legislation itself was written in response to a severe data breach traced back to a single government employee.

The legislation came on the heels of news that a former employee at the Department of Natural Resources had viewed thousands of drivers license records — almost exclusively of women — without a permissible use. That employee, John Hunt, is now facing criminal charges, and his actions have spurred five federal lawsuits against the state.

This employee might have been caught more quickly, but Hunt likely knew the limitations of the DPS auditing system and stayed below the radar, despite making nearly 19,000 queries to the database over the course of five years.

The report also recommended that because audits by the DPS largely detect heavy users, rather than suspicious use, local agencies should conduct more proactive monitoring. They suggested the department beef up its abilities to assist local agencies.

[Public safety commissioner Mona] Dohman said in an interview that the queries were so spread out that he did not emerge in their monthly review of the top 50 users.

In addition to the larger breaches, there were also cases where failure to deactivate accounts resulted in additional misuse of the DVS system.

During the 18 months ending June 30, 2012, 13 users conducted queries using access privileges associated with law enforcement agencies that no longer existed. Over the same time period, three former employees of state law enforcement agencies, as well as four former employees of local law enforcement agencies, accessed the DVS Web site using usernames and passwords that should have been disabled.

The current process for disabling accounts is almost farcical in its slowness. The report points out that the DVS allows accounts to remain dormant for 120 days before inactivating them. While this is a huge improvement over the 500 days it used to allow, it's still plenty of time for anyone looking to query a database they should no longer have access to.

Compounding the existing misuse issues is the fact that law enforcement agencies have exempted themselves from many of the policies affecting authorized civilian users. To begin with, sworn officers are not required to attend training or refresher courses on proper use of the DVS system, including policies regarding general security and appropriate data use. Officers are also exempted from the same user agreement that greets civilians at login and are otherwise not held accountable by any agreement when utilizing the DVS database.

DPS (Dept. of Public Safety) has not implemented other access management practices for all users. For example, DPS does not require a user agreement for sworn officers with access to the DVS Web site. Civilian law enforcement employees must sign a user agreement justifying their need for driver's license information, including their specific needs for access to driver's license photographs. DVS staff review the agreements before granting access. BCA (Bureau of Criminal Apprehension) has a signed intra-agency agreement with DVS. Agencies with employees who access BCA systems sign an agreement taking responsibility for access by their staff, among other things. Thus, it is only sworn officers who use the DVS Web site for whom DPS does not require an agreement, signed by the user or his or her employer, taking responsibility for appropriate access.

The findings of this study will certainly raise questions about this law enforcement double-standard. The proposed legislation and its attendant penalties and openness is, unsurprisingly, being fought by the law enforcement community.

House author Rep. Mary Liz Holberg, R-Lakeville, said she has already met resistance from some law enforcement entities.

"If you have bad actors in your bunch, then why shouldn't the public know about it?" Holberg said. "It seems like nobody wants any sunshine around this issue. And I think it would do a lot to rebuild the public trust if there was more public awareness of misuse and consequences."

It's pretty hard to rebuild public trust when you don't trust the public. Or, at least, don't trust them enough to be honest with them. The law enforcement fraternity has never been one for openness and consistency. As the study notes, misuse of the DVS system is handled differently by every law enforcement agency, if it's even punished at all. The lack of a codified "best practices" or even a basic "user agreement" that holds the individual officer responsible for his actions has led to widespread misuse. Minnesota's legislators are on the right track and this audit offers some very sound suggestions, but the feeling that those who enforce the law should be exempted from these same laws is somewhat endemic in law enforcement, meaning this has the potential to get worse before it gets any better. If they aren't careful, this legislation could reach passage with very few "teeth" intact, if it gets there at all.

Permalink | Comments | Email This Story

White House Orders Federal Agencies To Require More Open Access To Not Just Research, But Data

Mike Masnick — Fri, 22 Feb 2013 13:06:28 PST

Just a couple days ago, we were talking about the newly proposed FASTR bill, which would require federal agencies with greater than $100 million in research funds to require the end results of that research to be published in open ways within six months of publication. As we've noted, for years, the NIH (National Institute for Health) has had a plan that says all the research it funds needs to be published via open access after 12 months.

However, before anything happened with FASTR, it appears that the White House has stepped in to more or less push a fairly similar agenda via a White House policy memo. This comes in response to one of those We The People... petitions (which we had suggested you support). Dr. John Holdren, the Director of the White House Office of Science and Technology Policy, explained this rather big change in policy pretty clearly:

I have issued a memorandum today (.pdf) to Federal agencies that directs those with more than $100 million in research and development expenditures to develop plans to make the results of federally-funded research publically available free of charge within 12 months after original publication. As you pointed out, the public access policy adopted by the National Institutes of Health has been a great success. And while this new policy call does not insist that every agency copy the NIH approach exactly, it does ensure that similar policies will appear across government.

The interesting part is that, while this plan has the same 12 month delay as the NIH plan, rather than the 6 months in FASTR, it looks like the White House is really pushing agencies to go further. They're not just asking for the papers to be open access, but the more useful data as well:

In addition to addressing the issue of public access to scientific publications, the memorandum requires that agencies start to address the need to improve upon the management and sharing of scientific data produced with Federal funding. Strengthening these policies will promote entrepreneurship and jobs growth in addition to driving scientific progress. Access to pre-existing data sets can accelerate growth by allowing companies to focus resources and efforts on understanding and fully exploiting discoveries instead of repeating basic, pre-competitive work already documented elsewhere. For example, open weather data underpins the forecasting industry and provides great public benefits, and making human genome sequences publically available has spawned many biomedical innovations—not to mention many companies generating billions of dollars in revenues and the jobs that go with them. Going forward, wider availability of scientific data will create innovative economic markets for services related to data curation, preservation, analysis, and visualization, among others.

That part is big news, though most people will only focus on the open access to publications part. As he states, open access to data really helps power all sorts of interesting companies and research. In the past, publications have often tried to claim copyright over the data produced by experiments as well (even though you technically can't claim copyright over pure data). I've heard of researchers who have had to redo their own experiments because they had signed away the "rights" to their own data from previous experiments. That's not just wasteful, it's insane. Requiring open access to data is a massive step in the right direction. Kudos to the White House for doing this.

Next up: can we get them to realize the same thing should apply to patents on federally funded research as well, and that such things should not be allowed? Or is that just asking too much?

Permalink | Comments | Email This Story

What The Tesla / NY Times Fight Teaches Us About The Media

Mike Masnick — Fri, 15 Feb 2013 14:54:07 PST

For media watchers, the very public argument this week between Tesla and the NY Times has been quite fascinating. In case you happened to not be obsessively following each back and forth (what, you have lives?!?), it all began with a NY Times' less than enthusiastic review of the experience of trying to drive a Tesla S (the company's flagship electric car sedan) between a pair of Tesla's new "superchargers." You can read the full review yourself, but the short version is that it did not get the mileage expected, and at one point a flatbed truck needed to come pick up the totally dead car. I will admit that I'm impressed by the Tesla car in general, and most of the reviews have made it out to be about as close to a perfect car as you can imagine (which is pretty impressive considering that it's the first year of the car's existence and it's the first "mass" produced Tesla vehicle). But this review was less than thrilled, since the whole point was to test out the ability to drive between these "superchargers."

Upon publication, Tesla's famous CEO, Elon Musk, began tweeting up a storm about how the article was "fake" and that he had the vehicle logs to prove it. The author of the review, John Broder, responded to many of the tweeted charges, arguing that Musk was misrepresenting things -- leading many watchers to suggest that Musk was making a big mistake in attacking the NY Times.

Then, Musk published a blog post with a graphical representation of the log data they had, in which he argues that Broder lied and even purposely tried to run the car out of juice in order to write a negative story. Musk claims that after their dispute with Top Gear, they now keep logs on any media test drives (though it's unclear if they tell reporters that before giving them the cars). And, suddenly, a lot of people flipped sides, arguing that the data won and clearly the NY Times and Broder had some answering to do. After all, there were charts like this one: Except... then some people started to look more closely at the data and realize that perhaps Broder's story wasn't so crazy and Musk made a number of assumptions that aren't necessarily backed up by the data. For example, Musk insists that Broder claimed he turned down the climate control to low to conserve energy at 182 miles, and points to the fact that at 182 miles, Broder actually increased the temperature over 72 degrees. However, as Rebecca Greenfield points out, in her piece (linked above), it really looks like Musk may have simply assumed incorrectly that the point where this happened was 182 miles, and at about 250 miles it's quite clear that Broder does turn the climate control way down and keep it that way for a while (Greenfield added the purple box below). Then Broder chimed back in as well explaining away most of the accusations, including the charge by Musk that Broder drove the car around trying to run it out of energy:

When he first reached our Milford, Connecticut Supercharger, having driven the car hard and after taking an unplanned detour through downtown Manhattan to give his brother a ride, the display said "0 miles remaining." Instead of plugging in the car, he drove in circles for over half a mile in a tiny, 100-space parking lot. When the Model S valiantly refused to die, he eventually plugged it in.

Except, Broder notes, the "unplanned detour through downtown Manhattan" was not "unplanned" and had been communicated clearly to Tesla beforehand, did not actually go into "downtown" Manhattan, was partially recommended by Tesla employees who thought that the "regenerative braking" might help increase the range and only added two total miles to the trip length. Furthermore, as for the charge of driving around in circles in a parking lot?

Mr. Straubel said Tesla did not store data on exact locations where their cars were driven because of privacy concerns, although Tesla seemed to know that I had driven six-tenths of a mile “in a tiny 100-space parking lot.” While Mr. Musk has accused me of doing this to drain the battery, I was in fact driving around the Milford service plaza on Interstate 95, in the dark, trying to find the unlighted and poorly marked Tesla Supercharger.

Ouch.

In the end this is a fascinating story on many different levels. Dan Frommer makes an excellent point that "everyone's a media company now," noting that it's possible for companies to speak out on their own behalf if they disagree with a story. That used to be a lot harder. He compares that to the Quirky / OXO story we recently covered as well.

But, of course, if you're going to rebut charges made in a newspaper review, the information had better hold up, and it's not clear that it does here. Even worse, it really seems like Musk is making a much bigger deal of this than ever needed to be made. Sure, the initial review wasn't great, but it really didn't strike me as that bad. It basically said that if you try to drive it too far, or if you're unable to charge it enough, you might run out of juice. You know what? Same thing is true of a gas-powered car as well. But Musk has called much more attention to the story in a manner that doesn't necessarily lead to Tesla coming out on top. Carl Malmud's summary seems instructive:

Musk was offended that a reporter didn't operate the hardware properly. Blame the manual, tech support, PR, but not the user.

Musk is obviously quite passionate about the companies he runs and their products. And that's something that's actually quite appealing. Having followed his work for a while, you know that he really is striving to build "insanely great" products. So I can absolutely understand how his first emotional reaction is to lash out at someone who wrote a less than kind review (I've been there myself too many times). But, in the end, it seems like there would have been much better ways to handle this. I'm still a huge fan of the Tesla, and still dream of one day actually getting one, but I'd say that Musk's response probably made me more skeptical of the company than Broder's original article ever did.

When "everyone is the media," amazing and powerful things can happen. And, certainly, the ability to correct the record against questionable stories is something that really changes the game. But, at the same time, everyone is now a fact checker, and that makes for an interesting dynamic for both traditional media companies and those who wade in to respond to them.

Permalink | Comments | Email This Story

New Book About Data Mining To Find Love Online Has Author Admit To Possible CFAA Violations

Mike Masnick — Thu, 14 Feb 2013 08:51:53 PST

There's a new book out called Data, A Love Story by Amy Webb, in which she talks about how she gamed online dating sites. If you look at an account from the Washington Post, an excerpt at the Huffington Post and another excerpt at Slate, Webb discusses openly how she set up a ton of fake profiles on the online dating site JDate. Here's an example:

After figuring out just who she's seeking, Webb rejoins JDate, the Jewish dating site, as a man — creating 10 profiles for men she would want to date, with stock images and character sketches so elaborate you'd think she were outlining a novel. For example, we learn from the spreadsheet she makes for LawMan2346 that he and his younger brother, Mark, "didn't get along great as kids, but they're best friends now. Mark is the total opposite of him — plays sports, drinks beer. Typical man's man kind of guy."

But she's not Catfishing, she's doing opposition research. For a month, she corresponds with 96 female JDaters through these fake profiles, never meeting these women but interacting just enough to collect data (more spreadsheets!) on how they present themselves. Then, she can mimic her competitors and hopefully snag a better catch.

Interesting approach, I guess. Having met my wife through more traditional means at a time when online dating was in its infancy, I can only imagine the difficulty in successfully using those tools today. So, the appeal of "opposition research" and fake accounts for testing certainly must seem appealing. At the very least, it probably makes good fodder for a book... as it obviously did in this case.

But here's the problem. As we've been discussing, under the Computer Fraud and Abuse Act (CFAA), it's possible that she committed multiple felonies, and could face jail time. Now, let's be clear: no one has charged her with this and it's doubtful that anyone will. But in an age where we're finally starting to realize that perhaps we need to change and fix the CFAA, it's helpful to point out examples of how the law could easily be twisted.

Let's start with JDate's terms and conditions of service. There are a few clauses I want to call out. The first is in the "Registration and Subscription" section, in which it notes:

You agree to provide accurate, current and complete information about Yourself as prompted by Our registration form ("Registration Data"), and to maintain and update Your information to keep it accurate, current and complete."

In the "Proprietary Rights" section, it notes:

You represent and warrant to Us that the information posted in Your profile, including Your photograph, is posted by You and that You are the exclusive author of Your profile and the exclusive owner of Your photographs. You assign to Us, with full title guarantee, all copyright in Your profile, Your photographs posted, and any additional information sent to Us at any time in connection with Your use of the Service.

In the section "Your use of the service" it notes:

You will not post on the Service, or transmit to other members or to Us or Our employees, any defamatory, inaccurate, abusive, obscene, profane, offensive, sexually oriented, threatening, harassing, racially offensive, or illegal material, or any material that infringes or violates another party's rights

And also the following:

You will not harass or impersonate any person or entity. You will not use any manual or automatic device or process to retrieve, index, data mine, or, in any way reproduce or circumvent the navigational structure or presentation of the Service or its contents.

Now, you could make a case that in setting up ten completely fake profiles, including stock images, and then data mining the results of the women who communicated with those profiles, that she violated at least some, and possibly all of the clauses called out above.

Courts are not entirely in agreement on this, but certainly some courts have said that violating the terms of service of a website can potentially violate the CFAA (there are other factors that matter too). Even if we just look at the clauses of the CFAA that were used against Aaron Swartz, you could see how some (though not all) might apply to Webb as well. There's (a)(2)(c): intentionally accessing a computer without authorization or exceeding authorized access and thereby obtains information from any protected computer. There's (a)(4): knowingly and with intent to defraud, accessed a protected computer without authorization or by exceeding authorized access, and by means of such conduct furthers the intended fraud and obtains anything of value (as long as the thing of value is more than $5,000). Obviously, much of this is open to interpretation, but would you put it past a federal prosecutor arguing that Webb "knowingly and with intent to defraud" by "exceeding authorized access" obtained information and then obtained something of value more than $5,000? As the book reveals, Webb used these methods to meet her eventual husband. Is a husband something of value worth more than $5,000? Yes, perhaps it's a stretch, but... is it so much of a stretch that you couldn't see someone making the argument?

If you wanted to take it to even more ridiculous and extreme levels, you could argue that her "opposition research" may have enabled her to find a husband faster, thereby "cheating" JDate out of possible profits from keeping her as a paying customer for longer. Again, a long shot, but not a completely implausible reading.

And, again, if we can make the case that the value of the information she obtained by data mining these fake profiles exceeded $5,000 in value, then she has possibly set herself up for felony charges -- with maximum imprisonment of five years.

Would a court ever go that far? Almost certainly not. But given the lack of prosecutorial discretion we've seen in other cases, including many CFAA cases, is that something that really should be left to the prosecutors' and judges' discretion? Hopefully not.

Of course, no reasonable person thinks that Webb should be charged with anything for her activities, and it's not going to happen. But shouldn't we take a seriously look at fixing the law that makes it so that it's even possible she could have faced such charges?

Permalink | Comments | Email This Story

DailyDirt: Storing Data On DNA

Michael Ho — Mon, 4 Feb 2013 17:00:00 PST

There are lots of ways to store information nowadays -- from cloud services to nano-lithography to synthesizing custom strands of DNA. Some methods are cheaper or more convenient than others, but if physical space is really a premium, then encoding a gazillion bits of data on a few grams of DNA seems like the way to go. Here are just a few projects working on using DNA as an archiving medium.

A recent project encoded all of Shakespeare's sonnets (along with some other text and images) on DNA, putting 5.2 million bits of information (including some error correction) into some tangible goo. It costs roughly $12,400 to encode a megabyte of data and $220 to read it, and since sequencing DNA will never become obsolete (and is getting cheaper to do), DNA storage could be an excellent archiving technology. [url]
A milligram of DNA *could* contain all the text every book in the Library of Congress -- and all digital data that humans have ever created could be stored on a handful of DNA. We'd just have to make sure never to forget how to translate the digitized data from DNA back into a comprehensible form. [url]
Researchers at the J. Craig Venter Institute (JCVI) created the first self-replicating synthetic bacterial cell in 2010 -- and included a genetic watermark in the bacteria's DNA. The DNA watermark encoded some extra data such as the names of 46 researchers who worked on the project, as well as a URL and some famous quotations. [url]

If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) Techdirt post.

Permalink | Comments | Email This Story

Google Explains How It Handles Government Requests For Data; Why Don't More Companies Do This?

Mike Masnick — Mon, 28 Jan 2013 16:04:04 PST

Just recently, we pointed to Google latest Transparency Report, which showed a massive increase in requests for info on users from government agencies. However, it also showed that a much lower percentage of such requests were being honored, raising some questions about how Google handled such requests. Well, wonder no more (or, at least, wonder a little less) as Google has now explained the process by which it handles such requests, going into a fair bit of detail (you have to click through) in terms of the legal requirements and how Google handles different types of requests, and what data Google may be compelled to reveal. However, in an accompanying blog post, Google makes clear that it often pushes back:

When government agencies ask for our users’ personal information—like what you provide when you sign up for a Google Account, or the contents of an email—our team does several things:

We scrutinize the request carefully to make sure it satisfies the law and our policies. For us to consider complying, it generally must be made in writing, signed by an authorized official of the requesting agency and issued under an appropriate law.

We evaluate the scope of the request. If it’s overly broad, we may refuse to provide the information or seek to narrow the request. We do this frequently.

We notify users about legal demands when appropriate so that they can contact the entity requesting it or consult a lawyer. Sometimes we can’t, either because we’re legally prohibited (in which case we sometimes seek to lift gag orders or unseal search warrants) or we don’t have their verified contact information.

We require that government agencies conducting criminal investigations use a search warrant to compel us to provide a user’s search query information and private content stored in a Google Account—such as Gmail messages, documents, photos and YouTube videos. We believe a warrant is required by the Fourth Amendment to the U.S. Constitution, which prohibits unreasonable search and seizure and overrides conflicting provisions in ECPA.

This is definitely good to see -- and lots of other companies should do the same thing. However, it still remains an issue that governments can, and do, get lots of information with limited oversight -- even when companies push back.

Speaking of which, Twitter also came out with its latest transparency report, which highlights the information requests it gets as well. Both companies are really leading the way on transparency here, but it's a shame that these stories are even newsworthy, rather than the way most large companies act.

Permalink | Comments | Email This Story

DailyDirt: The Ever-Growing Growth Of Data...

Michael Ho — Thu, 24 Jan 2013 17:00:00 PST

There are a lot of reasons to be optimistic about the future. Some folks will always predict doom and gloom, but we say, "The Sky Is Rising!" (loud and proud -- and again with sequel The Sky Is Rising 2). The advent of digital information has created an enormous wealth of data, and the amount of this digital awesomeness seems to be growing all the time. Here are just a few more examples of the amazing abundance of media that surrounds us.

The Internet Archive has updated its Wayback Machine, indexing 5 petabytes of internet goodness, covering the web from 1996 to December 2012. That data is from over 240,000,000,000 URLs, and this virtual backup of the web doesn't even touch sites that have a login or a robot.txt file that blocks the Wayback Machine. [url]
Sandvine's global internet phenomena report contains a prediction that US internet traffic may rise to over 700,000 exabytes per year by 2019. And if Netflix continues to do well (accounting for about double the amount of traffic as YouTube and crushing Amazon Video, Hulu and HBO Go), a lot of that traffic will be people watching streaming movies and TV shows (legitimately, too, not just using BitTorrent). [url]
Every minute of the day, more and more data is generated. For example, 571 new websites per minute, 100K+ tweets per minute... gazillions of infographics and bazillions of random factoids. [url]
Since the beginning of time until 2003, humans generated about 5 billion gigabytes of data... and now we generate that much every 2 days. And that rate is accelerating (but humans are not exclusively generating all that data). [url]

If you'd like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) Techdirt post.

Permalink | Comments | Email This Story

Announcing: Our New Sky Is Rising Report!

Mike Masnick — Thu, 24 Jan 2013 06:00:00 PST

A year ago, we announced our Sky is Rising report, sponsored by CCIA, looking at the state of the global entertainment industry over the last decade -- and seeing that, despite the doom and gloom that we heard from some legacy players, the story actually showed tremendous opportunity. There was massive growth in content being produced, growth in revenue (though often distributed differently than through traditional gatekeepers) and tremendous new opportunities for content creators. It also found that, contrary to the claims that people just wanted content for free, people were spending more on entertainment. All in all, the data showed a much more positive picture than some have been spinning. That said, it did also highlight many of the challenges that content creators faced, with two key ones being important: the massive growth in content meant much more competition for consumers' dollars, while the changing technology and services landscape meant that the specific road map was a lot less clear.

This year, we're back with our second edition of the report, The Sky is Rising Two, once again, kindly sponsored by CCIA. This time around, we focused much more specifically on a few key countries in Europe: Germany, France, the UK, Italy, Russia and Spain. After the first report, we received some very reasonable questions about whether or not the global data reflected the situation across every country, or if one or two places (such as the US) might dominate and distort the picture in other countries. We chose to look at six of the larger economies in Europe individually, to see what we might find, and that's the focus of the second report. Once again, we've got a nice infographic that summarizes many of the findings: What we found this time was that, as expected, there are some big differences within different countries -- with some doing better in certain areas, and some struggling in other areas. But, on the whole, the general picture was the same. The various industries were growing. The amount of content being created was growing. The number of new services enabling new business models and new ways to distribute content were growing massively. It's an exciting time, and the key point is, once again, that there is no "conflict" between tech and content. Rather the two work hand in hand together quite well, with new services enabling all sorts of growth opportunities for artists.

Of course, the same caveats that we saw last year still apply. There is more competition, and the path to success may be less clear. So for individual creators, it may seem like a much greater struggle -- though their overall slate of opportunities continues to increase. Similarly, with new services and new opportunities, there remain challenges for the legacy players (especially if they had gatekeeper roles) to remain relevant. Please check out the full report below:

The Sky Is Rising 2 (PDF)The Sky Is Rising 2 (Text)

Also, since the report covered countries where English was not the primary language, we also had the report translated into German, Spanish, French, Italian and Russian. You can find each of those versions at the bottom of the Sky is Rising 2 page.

Permalink | Comments | Email This Story