Weighing The Benefits And Costs Of DRM: Type I & Type II Errors
from the letting-in-too-much-or-not-enough dept
In statistics, there is the concept of “Type I errors” and “Type II errors,” with Type I errors representing “false positives” (reject, when should be allowed) and Type II errors representing “false negatives” (allowed, when should be rejected). After reading Simon Phipps’ recent excellent analysis of why DRM is “toxic to culture,” it occurred to me that one of the main areas of disagreement concerning DRM is over disagreements over the types of “errors” that DRM creates. Phipps compares subways in France and Germany — with the French subways involving a barrier which only rises if you insert a ticket. The idea here is, obviously, to prevent people from getting on the train without paying. In other words, they want to stop those who should not be in the set from riding the train. They’re trying to minimize Type II errors (get on the train when they shouldn’t be on the train). However, there are “costs.” The barriers cost money and need to be maintained. The technology may break at times, requiring repairs and blocking legitimate ticketholders from getting on their train (a Type I error). Law enforcement is needed to monitor the barriers to watch for “gate jumpers.”
In Germany, the system is much more open — there are no barriers, and no one may ever check your ticket. However, every so often tickets do get checked (somewhat randomly), at which point you would get fined for not having the proper ticket. This minimizes a different type of error — where someone who has paid and has a legitimate ticket has trouble getting through a gate. In other words, it’s minimizing Type I errors (blocking someone from getting on the train when they should be on the train). It also lowers many of those other costs (or takes them away entirely). Of course, the “cost” to such a system is that, obviously, some number will game the system and ride without paying (a Type II error).
I think one of the problems that people have in discussing DRM is that they only look at one type of error, and never bother to compare the two. As a result of that, those who support strong DRM tend to focus only on the “error” of letting people get a “free ride,” and ignore all of the collateral damage, as Phipps explains. Yet, when you compare the two, it’s difficult to see how one can argue that the “free ride” problem is worse than the problem of collateral damage from limiting legitimate uses. And that is why so many people have such problems with DRM. It’s not that we want a “free ride.” It’s that we worry about the costs associated with all of those collateral damage points.
Comments on “Weighing The Benefits And Costs Of DRM: Type I & Type II Errors”
That’s an interesting point. Something for DRM makers to consider.
Re: Re:
The DRM makers probably don’t give a crap about Type I errors, they most likely only care about convincing others that Type II errors are the only problem.
Those who use DRM in their stuff should care, not only for the Type I errors, but for the added cost of the DRM. Is the cost of the DRM and the cost of the Type I errors worth preventing Type II errors?
That’s also not going into the debate of the true cost of Type II errors.
Re: Re: Re:
u missed one thing, they probably see the occasional type 1 errors as a gain as some people might pay again
Just wondering.
To what extent do “type I” errors apply to security theater?
Actually, all about money
The sad thing is the issue has nothing to do with the customer at all, or their experience–it’s about maximizing (in the minds of the pro-DRM crowd) the amount of payments that get made. Keep in mind that blocking a paid user is considered no loss to people who support DRM, since they received your money already. Free rides irks them to no end since they don’t feel they were paid what they “are worth”.
It’s a primary disconnect. Customer service is, in the words of the borg, “irrelevant”.
Re: Actually, all about money
Sony rootkit sold me on DRM.
Don’t Remit Money.
Re: Actually, all about money
Good point, though I’d go further and invoke the old saying: fool me once…
Thinking of the Sony rootkit on music CDs, Starforce and Sony’s Securom on video games, those DRMs lost paying customers who’d been burned.
Re: Re: Actually, all about money
I haven’t bought one single Sony product since. I don’t feel like I’m missing anything, either… Thanks Sony! 🙂
Re: Re: Re: Actually, all about money
Yeah, I don’t buy Sony either, from software to hardware, because of that behavior. Nor any products that use Securom, or anything with value destroying DRM – which devalues in another way: mental transaction and/or having to research so much before I buy. I cannot trust producers in this respect anymore and it’s their fault, not mine as a potential customer.
Re: Actually, all about money
it’s about maximizing (in the minds of the pro-DRM crowd) the amount of payments that get made. Keep in mind that blocking a paid user is considered no loss to people who support DRM, since they received your money already. Free rides irks them to no end since they don’t feel they were paid what they “are worth”.
all true, but on the first day of marketing they tell you that it’s cheaper and easier to keep a customer than it is to get a new one.
the problem with DRM is that that blocking legitimate customers creates incentive to not return. in extreme cases it’s an incentive to pirate.
if you spend a dollar on something that makes your paying customers consider pirating, you haven’t just wasted that dollar, you have invested it in your undoing. it’s like you donated it to a competitor.
going beyond the carrot vs. the stick and into game theory, why would a customer come back and pay if they have been burned previously? especially given the fact that DRM doesn’t actually prevent piracy, and may actually encourage it?
personally, i think a better approach would be to focus on the paying customer instead of the pirate. it’s a safe allocation of resources, since those people have already paid you, and if you treat them well, they are likely to pay you again.
When did we stopped trusting others?
I know that in the USA trust, honor and things like that are mostly lost.
The Yakuza has a honor system that is based on their word, and that is good as a written contract. When criminals have more values than business people I get confused.
Re: Re:
wait did u imply business people airnt criminals
Type II doesn't exist for DRM
Consumers of DRM products are only affected by Type I errors, Type II doesn’t affect them.
Pirated copies aren’t affected by either type of error and as such have a higher probability to work as advertised.
Costs of type II errors
Type II errors have interesting costs. From a pure microeconomics perspective, if marginal costs are zero and there are no capacity constraints then the cost of a type II error are zero. Making digital copies of an existing file would be an example of this. The subway would be an example where there is a capacity constraint associated with a type II error because a person without a ticket might prevent someone with a ticket from boarding.
In practical terms there would be a cost to type II errors if buyers change from buyers to freeloaders because of lack of enforcement. rational buyers will compare the cost of the fine, the ratio of the original cost versus the fine times the percentage chance of being caught. Buyers might also figure in the value of feeling honest versus the cost of a guilty conscience.
In practice the chance of being caught is a critical factor. In a subway there are choke points where everyone going by that point must have a ticket and it is easy to identify who should have a ticket. The problem with the Internet is that there really are no checkpoints and there is no easy way to know what a person is doing. At one time Napster was a checkpoint but the RIAA shut that down. Recently they assumed torrent freak and similar sites were checkpoints and tried to shut them down. That isn’t a winning strategy because traffic just moves elsewhere.
Strong enforcement can keep the the cost of becoming a freeloader high, but the cost of enforcement can be enormous. That is why the RIAA and MPAA want the government and ISPs to do that for them.
DRM has upfront costs plus opportunity costs of lost customers. In addition, people find their ways around it so DRM just doesn’t work in the long term.
That leaves the ratio of price to fines. If the chance of getting caught is low them fines must be enormous. Ask Jamie Thomson about that. At some point the fines get so high the courts won’t or can’t enforce them.
There is one other way to address the price to fine ratio. Drop the price to the marginal cost of production. Economics always wins in the end.
Re: Costs of type II errors
> There is one other way to address the price to fine ratio. Drop the price to the marginal cost of production. Economics always wins in the end.
It is not only the price. It is also the convenience. A well-categorized and easy to use website is much better than having to sift through hundreds of badly-labeled pages trying to find one link which works.
Re: Costs of type II errors
The subway is only capacity limited if in fact they hit capacity at some point. Otherwise, the difference between the current situation and infinite subway availability is hard to see.
Put another way, if they removed the ticket booths altogether and made the subway free, would they run out of capacity? If the answer is no, then the availability doesn’t play into the discussion.
Being that both are publicly funded systems, there is little profit motivation. Depending on the way that the governments and the people look at things, systems are set to expectations. While it is nice to try to draw a parallel between subway ticket systems and DRM, the driving forces between the system choices have little to parallel.
The issue with the German system is that if enough people get the urge to become freeloaders, there is no way for the authorities to check all of them (without creating some sort of barrier system, which defeats the purpose of an open door / trust based system). As long as their system is not used to capacity, there is no real reason to deal with the issue, until they hit a tipping point.
If each month more and more people don’t buy monthly passes, knowing that the fines for getting caught are only 3 times a month pass, and that their chances of being checked are very small, they will work it out. If they are only asked for a ticket proof once every 6 months, it works out cheaper just not to pay.
in the end, it has absolute nothing to do with DRM, nor is it a particularly good parallel.
Re: Re: Costs of type II errors
There definitely is a capacity constraint on subways. The total capacity of the subway system is one cap, of course. But even if you don’t get to that cap, there is a cost of running additional cars. If you only ran one car on each train, then you would hit capacity pretty quickly. There is a pretty significant cost in running a subway car, so the marginal cost is considerably greater than zero. If you didn’t charge for subway rides then ridership would shoot up and you would not have a way to fund the cost of the additional riders.
Re: Re: Re: Costs of type II errors
As I said, there are capacity limits. But those capacity limits are so high, that it is doubtful that they would ever be hit. Further, as both are public systems, the only limits that exist could easily be overwritten by raiding the public purse (more trains, more tracks, whatever).
Most public transit systems are run with massive overcapacity. The cost of carrying one more person on the systems is effectively nothing, because they don’t need to add tracks, cars, or personnel. Until you have absorbed all the excess capacity in the system, there are no marginal cost to add free riders.
Marginal costs would only be incurred if they had to increase capacity. Otherwise, capacity is already a sunk cost.
Re: Re: Re:2 Costs of type II errors
My experience from a trip to Europe is that at certain times of the day, the subways definitely run at and above maximum capacity.
Further, as both are public systems, the only limits that exist could easily be overwritten by raiding the public purse (more trains, more tracks, whatever).
What do you mean, both are public systems? DRM isn’t public. Distribution of digital content isn’t public. What are you referring to?
And it doesn’t work to dismiss the limits of the subway system, because that’s exactly the difference between a train and for example an MP3 file. The latter cannot have limits, by its nature. Need more? Make more, for free, as much as you want. The problem with the train that you can solve with huge investments of money is a problem that cannot occur for digital goods. Thus, the difference in impact of Type II errors.
Another cost of DRM
There is another cost of DRM: it is completely incompatible with free software. A basic premise of free software is that you are free to modify your software as much as you want. A basic premise of DRM is that it tries to prevent you modifying your software in ways which can be used to bypass the DRM.
For example, if I change my operating system kernel, how can the DRM know if the change was something like a performance optimization or an attempt to bypass the DRM? If it only accepts unmodified kernels, I am not free to modify my operating system kernel; if it also accepts modified kernels, I am able to change the code to bypass the DRM. The same applies if it accepts limited modifications; I am not free to modify what it does not like, even if my changes would not be used to bypass the DRM.
Cost benefit analysis
For any given system, pushing down Type II errors (false negatives) by definition increases Type I errors (false positives). For DRM, this means failing to catch pirates (I) and inconveniencing legitimate customers (II).
However, I disagree with classifying the costs of the system as a Type II error, because these are not relevant to the statistical model. Instead, Type II errors fall into a bigger category that is “Cost to consumer”. In this context, you can set the stringency of your DRM (which affects relative ratios of Type I and II errors), and you can raise the specs of your DRM. All of these have a cost, but if we break it down, Type II errors and raising the specs cost more.
This is because Type II errors irritate your customers (never a good idea), and raising specs has costs that must be paid by the company, or passed to the customer. Type I errors do carry a cost in lost sales, but they also carry a gain in free advertising. Usually these gains far outstrip any lost sales. So if you add it all up, the road the entertainment industries have taken is completely illogical.
But we already knew that, right?
Type III
Then there’s the type-3 error. When I was in Paris over a decade ago, there was a scare over bombs being planted and security was tight. Some arab-looking guy beside us jumped the turnstile, only to run into a Metro cop followed by 6 army guys with machine guns. No escaping the fine there…
The problem is that ignoring type-II simply invites type-III wholesale bypass of the system.The number of free riders is proportional to the risk of being caught and the resultant fine. A ludicrous unreal fine that probably has a one-in-a-million chance of happening to you might as well be no deterrent.
Type I
Well, my story was particularly irritating to me. Having taken many many VHS home movies when my kids were very young, I wanted to transfer them to DVD to avoid the degredation issues one reads about, as well as keeping fairly current so I always have something to play them on. So I have bought (now, several) those dual deck machines with a VCR on one side and a DVD recorder on the other. Well, darned if the things didn’t accuse me of trying to copy ‘copy protected, copyright protected’ media when I was simply trying to duplicate media *I* had produced.
Once law-abiding, this experience has forced me to find, let’s just say, other ways to ‘get around’ these artificial restrictions being applied to my own self-produced media. Doing so, I have apparently trampled on the DMCA. Go figure.