We've talked quite a bit about how the federal government has been pretty aggressively shattering any remnants of the 4th amendment, and while there are some parts of the 1st amendment that are still respected, our government doesn't always seem so keen on that one either. Apparently, they've decided to kill two birds with one stone recently, in obtaining a broad collection of phone records concerning Associated Press journalists, which is almost certainly in violation of the law. The AP only just found out about this on Friday, despite the data already being obtained, and covering more than 20 separate phone lines (including work, home and mobile phones) for multiple AP journalists -- and a period covering approximately two months in early 2012. The AP has sent a quite reasonably angry letter to Attorney General Holder about this collection.

There can be no possible justification for such an overbroad collection of the telephone communications of The Associated Press and its reporters. These records potentially reveal communications with confidential sources across all of the newsgathering activities undertaken by the AP during a two-month period, provide a road map to AP’s newsgathering operations, and disclose information about AP’s activities and operations that the government has no conceivable right to know.

That the Department undertook this unprecedented step without providing any notice to the AP, and without taking any steps to narrow the scope of its subpoenas to matters actually relevant to an ongoing investigation, is particularly troubling.

The sheer volume of records obtained, most of which can have no plausible connection to any ongoing investigation, indicates, at a minimum, that this effort did not comply with 28 C.F.R. §50.10 and should therefore never have been undertaken in the first place. The regulations require that, in all cases and without exception, a subpoena for a reporter’s telephone toll records must be “as narrowly drawn as possible.’’ This plainly did not happen

The AP also (again, quite reasonably) notes that this appears to be a "serious interference with AP's constitutional rights to gather and report the news" and demand that the government destroy all copies of the data it received.

This really is an incredibly broad move by the government. Especially when it comes to reporters, the government has generally respected the right for reporters to keep their sources private, even if this administration has been known to threaten reporters if they won't reveal sources. In case you're wondering the law here is pretty clear about the limitations on getting this kind of info.

There should be reasonable ground to believe that a crime has been committed and that the information sought is essential to the successful investigation of that crime. The subpoena should be as narrowly drawn as possible; it should be directed at relevant information regarding a limited subject matter and should cover a reasonably limited time period. In addition, prior to seeking the Attorney General's authorization, the government should have pursued all reasonable alternative investigation steps as required by paragraph (b) of this section.

I'm sure that Eric Holder will try to tapdance around this one as well, but the claims here are very serious. On the positive side, perhaps this will finally help the press wake up to the continued expansion of the federal government's surveillance operations and their general disdain for the constitution if it helps them go after whoever they want. The press likes to go nuts when some startup accidentally leaks some data or tracks what people are doing online, but routinely ignores how the government seems to feel entitled to any bit of private data about anyone, often without a warrant. Perhaps having the press have their records taken will wake some of them up to the fact that it impacts them as well (perhaps even more than others).

Glyn Moody points us to an interesting analysis, by David Gerard, of yet another problem with the NY Times' moronic paywall: that it means the NY Times will be cited a lot less on Wikipedia, thereby handing over the "first draft of history" to other publications like The Guardian and the BBC.

But the even more interesting point comes after that:

Every journalist I've spoken to since 2006 uses Wikipedia as their handy universal backgrounder. Funnily enough, there's a distinct lack of donations to the Wikimedia Foundation from newspapers and media organisations. How much did the New York Times donate in the fundraiser?

We do this stuff for everyone to use and reuse. Journalists taking full advantage of this is absolutely fine. But claiming we should then pay the papers for the privilege is just a little odious.

Marcus Carab, who works in a newsroom, made a similar point in response to that article a few weeks ago when the NYTime's Bill Keller claimed that the Huffington Post was a "pirate site," in that newspaper reporters rely on tons of other sources that never get any credit, let alone payment (excuse Marcus' Canadianisms, he can't help it):

Forget the fact that pavement-pounding reporting is a form of aggregation from the public - newspapers actively aggregate from tonnes of published sources too. Every newsroom has a table covered in copies of every other newspaper in town - in case you missed something, or they got an angle you didn't, or you think one of the stories can be taken further. In addition to reporting, all journalists do research: they look up other articles on the topic, find past magazine interviews and pull data from published reports. Many science articles in newspapers are just summaries of journal articles.

And that's just what went on and still goes on in the traditional media ecosystem, amongst the old players. Newspapers actively aggregate from blogs too. Every journalist in entertainment or technology starts his morning looking for leads on blogs, and the first thing any reporter does when they get an assignment on a topic they aren't familiar with is look it up on Wikipedia.

So this raises a really good point about the silly claims from the NY Times and others about how they need to get paid, since they believe that they're the "originators" of the news. I do wonder how many of the people at the NY Times did contribute to Wikipedia? I would bet many of the folks who insist that their own work needs to be paid for by users, have in turn never once contributed to Wikipedia.

Information comes in all sorts of forms from all sorts of places. The NY Times is good at what it does. No one is denying that. But it's delusional in its thinking that it somehow is the piece of the puzzle that is worth the most here.

Famed investigative reporter Bob Woodward apparently doesn't spend much time "investigating" the state of the internet and online news before making statements. His latest is that he's not thrilled with this whole internet thing, saying that Eric Schmidt's tombstone should say "I killed newspapers." He followed this statement up with this bit of pure cluelessness:

"There's going to be something we're going to miss in journalism that will be very regrettable. I hope the young people who have developed Facebook and Google will say, 'We need to fix the information system and we need to get information to people that's well-researched and investigated.' "

There's something beautiful in a statement that disproves itself, and Woodward's statement above fits into that category perfectly. After all, he made the statement above, and it's pretty damn clear that he made it without investigating or researching what's actually going on. The fact is that Woodward seems to have conflated two totally separate issues: the fact that services like Google and Facebook exist... and the fact that not all reporting is well-researched or investigated. The problem is that those two things are not particularly related, and most certainly not causal. There are, for example, plenty of reports in newspapers that are neither well-researched nor investigated. At the same time, there are plenty of things found online that are both well-researched and investigated. Of course, there's plenty that isn't... but that doesn't harm the stuff that is. I never understand why people automatically think that all content online gets equal attention, and that any "bad" content somehow takes something away from good content.

But rather than researching and investigating what's really causing problems for newspapers (hint: massive debt-load and an astounding failure to adapt to the times), Woodward simply does what he claims to hate, and makes uninformed and ignorant statements, blaming "search engines" for the "screwed up information system." He also seems to think that while "people" exist offline, what happens online no longer involves people. The following paragraph, for example, makes no sense:

"Mark Felt, who was Deep Throat, didn't have a Facebook account. He wouldn't have had one. The news of Watergate came from human beings who were reluctant to talk. And the information was not on the Internet. You talk to college students and they say, 'Instead of two years before Nixon resigned, it would have happened in a week.' And I say, 'Why?' And they say, 'Because, people would have gone to the Internet and found it.' But I say, 'It wasn't there. Even if there was an Internet, the information would not be available.'"

Even if that latter exchange really happened (and I have my doubts), the whole thing seems to be based on the idea that a reporter today can't cultivate sources online. While I'm not an investigative reporter by any stretch of the imagination, I can say pretty clear that the internet has been a massive boon in building up a variety of sources of information that simply wouldn't be possible in the past. Just because it happens online doesn't mean it doesn't involve communicating with people. It's just that the process becomes more efficient. He's right that Mark Felt wouldn't have put info online... but would it have been possible for reporters to cultivate a source like Feld online? Absolutely.

Back in January, when I saw Daniel Ellsberg speak, one of the things he noted was how much more aggressive the Obama administration appeared to be in going after leakers than any previous administration. Ellsberg's theory -- which he admitted was based on just his intuition -- was that President Obama is actually quite embarrassed by some of the things he's doing and is, thus, more aggressive in trying to stop leaks, knowing that his actions are damaging his reputation. I don't know if that's true, but there is growing evidence of the level of questionable activities from the Obama administration even in going after leakers. Ellsberg noted at the time that the Obama administration has brought more indictments for leaking than all other presidents combined before him.

The latest is the report that came out late last week that the government, in going after leakers, got access to reporter James Risen's phone records, bank details and credit card statements. As the report notes, this is pretty extreme:

Although there have been other public controversies over subpoenas -- real and threatened -- to reporters in recent years, there have been few, if any, cases in which it has been documented that federal prosecutors obtained the bank records and credit reports of journalists.

It's not entirely clear if all of these activities took place under the Obama administration or previous administrations, but multiple people quoted in the article say this kind of activity has been much more common in the Obama Justice Department. For a President who has positioned himself as being a big supporter of press freedoms, this looks really hypocritical. Spying on reporters is bad. As the report notes, Risen was subpoenaed directly twice, but both times a judge reasonably quashed the subpoenas. So, for the administration to basically go around all that and get records from others is pretty bad.

If you haven't yet, set aside some time to read the NY Times' executive editor Bill Keller's account of the paper's association with Julian Assange. It gives some interesting (and not too surprising) background details about the relationship, Assange himself, and Keller's views on the overall impact of Wikileaks. Assange, not surprisingly, does not come off too well in the account, reinforcing the reports of his being someone who has an incredible ego and a conspiratorial bent. By this point, such claims are pretty much old news, but from an anecdotal level, the story is still entertaining.

Keller's final suggestion that Wikileaks' impact has been "overblown" is belied somewhat by the fact that Keller just spilled so much ink on the background of the paper's association (and subsequent blacklisting) with the organization. There are other points that could be nitpicked, but it's not surprising that Keller seeks to position the NY Times in the best possible light, and perhaps minimize the contributions of Wikileaks itself. But, just the fact that the NYT's is now considering its own Wikileaks, shows that the idea certainly has changed the way many people think about this aspect of reporting.

I'd argue -- as some others have -- that the impact of Wikileaks has actually been both over- and under-estimated. And part of the problem is that so many people are quick to conflate the idea of such an organization with the single organization itself (or worse, with a single person in the form of an easily dislikable Julian Assange). But it's a mistake to think that just because the particular organization itself is flawed, that its existence and what it's accomplished so far won't have profound effects on secrecy in organizations (government and corporate), the practice of journalism itself (which has suddenly gotten a hint of what's possible), and the idea of distributed or "stateless" organizations as enablers of information flow. It's that last point that I think many are ignoring, and that will later prove to be a mistake.

It's been funny watching critics of Wikileaks assume that if Julian Assange was somehow takendown, or that Wikileaks itself was somehow taken offline, that this concept of intermediaries to help publish leaked documents would somehow go away. In the past few months we've seen tons of new such operations spring up, and I'm sure some will start to find success (while others will fade away). However, what may be most interesting is that the mainstream press is finally waking up to the fact that they probably should have been doing this all along. The NY Times is apparently planning to create its own version of Wikileaks in the form of an "E-Z Pass lane for leakers." This idea of systems to help people leak info isn't going away. And, I'm curious if those who think that Wikileaks is criminally liable for "inducing" leaks, also think that the NY Times is similarly liable?

In the past couple of years, I've tried to be a lot more diligent about putting up primary documents when discussing them in relevant posts. Many of you have certainly noticed the DocStoc or Scribd embeds that I frequently use. This was after talking it over with a few people, who pointed out that giving readers the actual source material was a lot more useful (and after a few snarky "dissenters" suggested that I was "hiding" the real details). I've actually found that it's worked out quite well, with the comments around the actual documents often quite enlightening and informative. These days, I actually get annoyed when I read reports about a lawsuit or some other type of document and the document itself is not available. It makes it that much harder to actually build a real discussion around the topic.

So why is it that so many major news sources don't post source documents?

Felix Salmon has a long and interesting post about his discussion with the NY Times on this particular topic. Two really interesting points come out of this, neither of which is good:

1. Many (though, certainly not all) old school journalists come from an era in which they want to hoard the information, and dribble it out, because that's how journalism used to be. The power was in those who held the information, and the journalist's role was to just give you what s/he felt you needed, rather than giving you the full download to analyze yourself.
2. The NY Times and others then uses copyright law as a crutch to explain why they don't provide source material. They hide behind copyright law, claiming that it opens up too much liability to post certain documents that may be covered by copyright.

The first of these looks bads for those journalists, and speaks to how they haven't realized how the world has changed, and how their role has changed. The second shows how much copyright law can actively stifle real journalism, by limiting what a reporter can do in providing useful information to the public.

The one big thing I learned from talking to Samson is that when NYT journalists talk about copyright constraints preventing them from putting documents online, they're not particularly upset about that. In fact, they might secretly be quite happy that there's no question of posting the document they spent so much effort obtaining. Journalists are human, after all, and can be quite jealous and competitive: they don't want to simply give the story, on a plate, to their competitors, and will happily sit on documents rather than publishing them if they're given half a chance to do so. Samson said he couldn't think of a single instance where a journalist was begging him to be able to publish something and he said no, for copyright reasons.

After all, it's easy for the NYT to post copyrighted documents if it's so inclined -- it just needs to send them to any one of dozens of organizations who will happily put them online, and then link or embed the document into the story. Or the journalist can just ask their source to go ahead and post the document online, in some anonymous place where it can be linked to or embedded. But that never seems to happen. And even when there's no copyright at all, as in the case of the Hank Paulson ethics waiver, the NYT went on the record as saying that the reporters "would probably be uncomfortable simply handing over documents" even to one of their colleagues, let alone to the world in general. After all, said Tim O'Brien, an editor there, "they had spent a lot of time and energy to find, analyze, and report on" that document.

This is a rather antiquated view of the information economy these days. It's a view built on the idea that hoarding information is better than sharing it. In our own experience, that seems to create less valuable results, and for a publication like the NY Times, it seems like a huge waste. No one is saying that giving away the source material takes away from the actual reporting or commentary that happens on top of that. In fact, most people will still find that to be the most valuable part. But sharing the actual source material is an important part of the package.

The other chilling part of Salmon's conversation with Samson was one of his rationale's for hiding behind copyright:

"We want our readers to respect intellectual property," says Samson. "Intellectual property is arguably the biggest asset of this company. We value others' IP rights, and we want their IP rights to be respected."

No, your IP is not your biggest asset. The readers and the community you've built up is -- and if you treat them badly by hoarding information, they might start to go elsewhere. Hiding behind copyright law to provide less valuable reporting is a cheap cop out, that doesn't befit a news organization of the NY Times' stature.

Rupert Murdoch and his minions at News Corp. have been going around banging the drum that Google and others are "stealing" from News Corp. newspapers by linking to their stories and sending them traffic. But at the same time, they seem to have no problem totally taking credit for stories that they source from elsewhere. Late last year, the Times (of London), which is a News Corp. paper was caught publishing someone's blog post without their permission at all. And then there's the News Corp.-owned NY Post, which last year had a reporter admit that it was the paper's "policy" not to credit bloggers as the sources for stories. After that story came out, the NY Post insisted that wasn't true, but it appears the paper has been caught doing it again.

Andrew Fine alerted us to the news that suggests the NY Post used one of his posts as inspiration for a story. Fine had written about the rather disconcerting sign in a Chuck E. Cheese in Harlem. That blog post got some attention on various other blogs... and then just a couple of days later, the NY Post had an article about the very same sign (apparently, it took two reporters to write that article), with nary a mention of Fine's original blog post (or even any of the other blog posts that promoted Fine's original story).

Now, to be clear, while I do think it's good manners to cite where you sourced a story, it's certainly not required (legally or otherwise) by any means. But where it gets hypocritical is for this to come from an organization that claims that other sites merely linking to its articles are somehow "stealing." But when the NY Post comes in and blatantly borrows an idea from someone else, and does so without credit, that's perfectly fine? It seems like Murdoch and News Corp. have quite the double standard going.

While here in the US, we're still fighting over a federal shield law that will let journalists protect their sources, Michael Scott points us to the news that the European Court of Human Rights recently ruled that the media can protect anonymous sources.

The Court reiterates that freedom of expression constitutes one of the essential foundations of a democratic society and that, in that context, the safeguards guaranteed to the press are particularly important. Furthermore, protection of journalistic sources is one of the basic conditions for press freedom. Without such protection, sources may be deterred from assisting the press in informing the public on matters of public interest. As a result, the vital "public watchdog" role of the press may be undermined and the ability of the press to provide accurate and reliable reporting may be adversely affected. Having regard to the importance of the protection of journalistic sources for press freedom in a democratic society and the potentially chilling effect that an order for disclosure of a source has on the exercise of that freedom, such a measure cannot be compatible with Article 10 unless it is justified by an overriding requirement in the public interest.

This is definitely a good thing, and a bit surprising, since Europe has, in the past, often been a bit less protective of the right to speak anonymously.

There have been a number of cases recently that have tested whether various laws that protect journalists from having to give up their sources also apply to people publishing content online in forums, email groups or blogs. The latest, sent in by someone Anonymous, is taking place in New Jersey, where a woman who revealed a security breach in the software of a company called Too Much Media is being sued for slander in revealing the breach. There are numerous issues with the lawsuit, including the oddity that they're suing for slander for online comments, since slander is for spoken words, whereas libel is normally applied to the written word. It's also odd that they're suing considering the fact that they don't deny the security breach existed, but dispute the claim that customer info (including credit card details) were exposed, because they claim the security breach was brief and no info was compromised. That seems like a pretty weak defense.

However, the real battle seems to be over the attempt to determine how the woman, Shellee Hale, found out about the breach in the first place. She's refusing to give that up, claiming that she has a right to protect her sources, just like any journalist. And while Hale writes multiple different blogs, and has written for many mainstream publications (including the Wall Street Journal and Business Week), Too Much Media claims that she doesn't deserve protections afforded to journalists because she wasn't working for any real publication and is just a blogger. The article quotes someone who says that if the court sides with Hale:

"then everyone is a journalist and the privilege becomes meaningless."

I don't see how that's actually true. In fact, I'd argue the other way. It's not that it becomes meaningless, but that it becomes very, very meaningful -- especially in an era where we're looking for new ways to prop up investigative journalism. If everyone's a journalist, and everyone has a reasonable expectation that their sources are shielded, then we're much more likely to continue to root out corruption. If this protection is somehow reserved for some "special" credentialed people, then it becomes that much harder to expose corruption.

Unfortunately, it appears that the judge in the case is almost entirely computer and internet illiterate, needing to ask for explanations for a variety of things during the court proceedings. He seemed entirely confused by the very concept of people blogging for personal interest:

"Why would a guy put all this stuff on a blog? Does he have nothing better to do?" Locasio asked. "Does he get paid?"

The judge, who apparently is about to retire in a couple months, also didn't understand the difference between blogs, message boards and forums, and was apparently unfamiliar with instant messaging. It's difficult to see why someone entirely unfamiliar with the technology should be able to judge a case like this, where understanding what's happening online is crucial to understanding what the case is really about.