Does Being More Vocal In Video Game Violence Debate Mean You Have The Better Argument?

from the quantity-vs.-quality? dept

A few folks sent over the news of some really bizarre research done by Brad Bushman and Craig Anderson on the question of whether or not violent video games harm teens. First of all, the research is already somewhat suspect, in that Anderson has a long history of claims about how violent video games must harm children based on questionable data. This new report is based on such questionable and loop methodology, you almost wonder why they even bothered.

What they did was take the amici briefs from the Supreme Court case concerning California’s anti-violent video game law, and run some numbers on who wrote the briefs and how many “published” studies they had. And that was how they determined which one was more credible. I’m not kidding. Quantity over quality:

The researchers analyzed the credentials of the 115 people who signed the Gruel brief, who believe video violence is harmful, and the 82 signers of the Millett brief, who believe video violence is not harmful. (The briefs are named after the lead attorneys for each side.)

The data for the study came from the PsycINFO database, which provides more than 3 million references to the psychological literature from the 1800s to the present, including peer-reviewed journal articles, book chapters or essays, and books.

For each of the signers of the two briefs, the researchers calculated how many articles and books they published on issues relating to violence and aggression in general and on media violence specifically.

The results showed that 60 percent of the Gruel brief signers (who believe video game violence is harmful) have published at least one scientific study on aggression or violence in general, compared to only 17 percent of the Millett brief signers.

Moreover, when the researchers looked specifically at the subject of media violence, 37 percent of Gruel brief signers have published at least one study in that area, compared to just 13 percent of the Millett brief signers.

And they claim that this is “a very objective approach.” It’s also a profoundly meaningless approach. In case you didn’t follow it, there were a ton of amici briefs filed by various parties in this case. This study picked just two of the briefs. The first one (pdf) filed by California State Senator Leland Yee (whom, I believe, may have written the legislation in question), the California Chapter of the American Academy of Pediatrics and the California Psychological Association. That brief supports California’s position in the case. The second one (pdf) is a brief from “social scientists, medical scientists and media effects scholars,” which is the one that supports the other side, saying that the law isn’t constitutional. You can read the two briefs that I linked to above, and you can judge the relative merits of both.

But that’s not what Bushman and Anderson did. They simply took the signers of each brief and measured how many of them have published studies on this specific question. Of course, that’s a meaningless and arbitrary number, especially when presented in percentages. Based on this methodology, it would mean that if only one person signed the amicus brief, but had published research, then that one would clearly be the most credible, since 100% of the signers would have published. Obviously, that makes no sense.

Now, Bushman and Anderson — clearly expecting the quantity over quality issue to make for easy mockery of such a ridiculous study — also added a second element to try to show “quality” as well:

In a further analysis, Bushman and Anderson examined where the signers of both briefs have published their research. The best academic journals have the highest standards and the most rigorous peer review, so only the best research should be published there, Bushman said.

The researchers used a well-established formula, called the impact factor, to determine the top-tier journals, and then calculated how many signers had published in these journals.

Results showed that signers of the Gruel brief had published over 48 times more studies in top-tier journals than did those who signed the Millett brief.

But, again, this attempt at showing “quality” is really a “quantity” study in disguise. It’s not looking at the actual credibility of any of the studies, but trying to create an aggregate (but meaningless) number. And, again, the entire basis of this result is a meaningless dataset. I’m really wondering who would possibly read this and think that the results are credible.

Oh yeah, and one final point. Guess what two academics signed that first brief? You guessed it: Craig Anderson and Brad Bushman. Talk about researcher objectivity huh? They create a bogus methodology to try to “prove” that the brief they signed is more credible than someone else’s brief? Honestly, when they present methodology like this, it serves mostly to raise questions about their methodology on any other study as well. They’ve made it clear that they’re not researching the truth. They’re starting with an established position and trying to figure out ways to present evidence to support that. That’s not science.

Filed Under: , , , , ,

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “Does Being More Vocal In Video Game Violence Debate Mean You Have The Better Argument?”

Subscribe: RSS Leave a comment
John Doe says:

Unfortunately that is what passes for science these days...

“They’re starting with an established position and trying to figure out ways to present evidence to support that. That’s not science.”

This is what passes for science in many circles today, not the least of which is the whole global warming lie.

John Doe says:

Re: Re: Unfortunately that is what passes for science these days...

Primates don’t grasp much of anything except bananas. You do realize that CO2 according to the GW freaks, only accounts for a fraction of a percentage of actual warming? So we are spending billions to tackle what, 1/2%? Solar activity and atmospheric water vapor accounts for the overwhelming majoring of warming and we have no control over solar activity and near zero understanding of why the atmosphere isn’t holding more water vapor than it currently does.

Besides, I am sure you superior human beings will drag us inferior primates along with your superior grasp of the ways of the world. So we inferior primates may be more evolved than the superior human beings since we can sit back and let you do all the work an yet we reap the same benefits.

Anonymous Coward says:

[…]They’re not researching the truth. They’re starting with an established position and trying to figure out ways to present evidence to support that. That’s not science.

Sounds like typical modern “science” to me. It’s a shame there aren’t more people genuinely trying to understand the world, instead of people like these hired by some cause or another to fabricate proof.

Christopher (profile) says:

Re: Apples and oranges

More like apples and watermelons! They aren’t even the same size.

For goodness sakes, I have played numerous exceedingly bloody games, as have my children and relatives children from younger than when I started…. we are goddamned near PACIFISTS!

That’s right….. we turn the other cheek so damned often (except when using words to snipe, we are masters at that) that we have been PHYSICALLY INJURED because we didn’t want to fight back when physical force was used against us in real life.

Mr. Smarta** says:

Nothing new... again

Arguments like this spawned back in the 70s and 80s when people were absolutely sure that playing Dungeons and Dragons led to worshipping the devil, performing satanic rituals involving human sacrifice, and butchering and murdering people.

Does that mean all the people who did an all-night D&D hack fest are the very individuals sitting on death row??? Wow! What a concept!! I think I’ve just figured this all out. Amazing…

Michael Price (profile) says:

Re: Re: Nothing new... again

The weird thing is if they really believed we could do the stuff our characters could do why would they want to fuck witnh us? I mean seriously considering what my characters can do why would a suburban housewife not just go, “Ok do what you want just don’t summon a basilisk in my living room.”? Of course that wouldn’t be necessary, they would just realise we’re not the hobbyists they’re looking for.

trrll (profile) says:

So one’s propensity to support the proposition that video game induced violence is a serious problem is correlated with one’s number of publications in the area–which probably also correlates with the amount of one’s funding predicated on the proposition that violence in video games is a serious problem justifying continued investment in such research. That doesn’t prove that they are wrong, but it sure doesn’t prove that they are right.

The real problem, of course, is that video game sales have increased, particularly to the young male demographic that is statistically most associated with violent behavior, and as video games have become more realistically violent, the incidence of violent crime has steadily decreased. That does not, of course, disprove the notion that there is a “pro-violence” influence of video games. What it does prove is that any such effect (if it exists at all) must be so small as to be swamped by other social and demographic factors influencing violent behavior.

Jason (profile) says:

Answers in Genesis Anyone?

This sounds like the intelligent design folks who decided there was a creator and then went and found enough evidence presented the right way to make their point (without ever proving their original assumption). It’s a near perfect system for folks who want to believe this already and aren’t willing to actually ask a critical question or even question what the authority has told them to beleive.

See the trick is to present the right “evidence” ingore anything to the contrary and use big science like words to sound like you are smart. Like lambs to theslaughter.

Anonymous Coward says:

Besides such wretched practices going on, and I don’t know if this stuff is BECAUSE it’s a high profile, or if it is just so common that we hear about it because it’s high profile. I can’t wait to see what the newest violence generator is (I really hope it goes to musical tv shows/movies). As more people who were kids that grew up on these violent video games turn into nice productive adults, this little myth will wither away. If I followed the statistics (the most abused area of math, poor thing) from fear mongering crap like this, I should be a mass murderer by now. Let’s see, violent video games? Check, my father bought me video games because they were on the news for being so violent because they looked cool (carmegeddon anyone?). He knew me and knew I understood the difference between a game and reality, I’m saddened to doubt how many parents could honestly say that. Anyways, Anti-social? Yeppers, it got me two degrees electrical engineering and computer science. Oh, and I’ve done several martial arts for over 17 years, that has to be more violent than the pretty little lights right? I should be foaming at the mouth by now with rage like a bad zombie movie.

Anonymous Coward says:

Re: Response to: Anonymous Coward on Apr 25th, 2011 @ 12:33pm

Aggravated assault and battery offences have are at a much higher rate than pre tv/video games. Most people aren’t affected by it, but a large enough percentage is that we live in a much more violent society. I worked as a paramedic and can tell you first hand that the rate of murder would be astronomical if it weren’t for modern medicine saving so many people. You did martial arts which specifically teaches you discipline. Go on YouTube and have a look at all the morons out there happily slapping each other around – you’ll notice they aren’t martial arts practitioners and they have no discipline. I call them ‘urban ninjas’ – a sarcastic name of course.

sam says:

Re: Re: Response to: Anonymous Coward on Apr 25th, 2011 @ 12:33pm

Do you have any evidence or did you just make all that up? And no anecdotal “I’m a paramedic” isn’t evidence, and neither is “oh look some youtube videos”. – violent crime has clearly been trending down since the release of the original Playstation.

And you really think that TV/video games are significant driver of violent crime rates, as opposed to say the war on drugs?

Anonymous Coward says:

Re: Re: Re: Response to: Anonymous Coward on Apr 25th, 2011 @ 12:33pm

Sam, I do have evidence – but lets first look at yours. Your report is a combination of two reports – one which is actual crime as reported by official police statistics – the other is a set of interviews with people who give anecdotal evidence so they can make an estimate for probable victimisation of crime. I’m don’t mind estimates – just not ones based on anecdotes.

Looking at the two separate graphs – actual crime has steadily increased but estimated crime has dramatically decreased. Combining the results is just total bullshit – a combination of an anecdotal estimate and actual figures is meaningless. Anyway – the graph clearly shows actual real life crime has gone up. It helps if you read the data first šŸ˜‰

Now – my anecdote was about the ability of medicine to decrease the murder rate by saving people that 40 years ago would have died. Now this isn’t just an anecdote – it is a fact. Medicine is much more advanced than it was 40 years ago. Quick example – back then a subdural bleed from a punch in the head meant your death – now they’ll save you. See for a comprehensive study supporting this (3 out of 4 murders now prevented by medical intervention).

Onto your request for my “evidence”. Firstly I’m not obliged to do this – this is a comment section of an article. These are all from Dave Grossman’s “On Combat”

Look at: FBI Uniform Crime Report – 1957 to 2000 shows a 5x increase in per capita crime rates.

Consider Australia –

The murder rate from the beginning of the 20th Century until about 1950 halved. People killing less, maybe, more likely medicine saved them. Of course the trend should be continually downward but it isn’t. It doubles again to in the next 50 years. Now combine this your new knowledge about medical advances and you can see that it would be 4 times this number without medical advances – or an 8 times increase in murder rate. The only thing stopping this is more advanced medicine.

Now my request. Please show how medicine is less advanced and saves less people than it did 50 years ago. If you can show that I’ll eat my words.

Yet another anonymous coward says:

Re: Re: Re:2 Response to: Anonymous Coward on Apr 25th, 2011 @ 12:33pm

But if you consider Australia, immigration has significantly changed the population demographic since the 50s. You could also argue that immigration has caused an increase in the murder rate. Which is just as specious an argument as the virtual violence begets real violence camp, because it uses one number to justify another based on no scientific process.

Anonymous Coward says:

Re: Re: Re:3 Response to: Anonymous Coward on Apr 25th, 2011 @ 12:33pm

Actually I happen to live in Australia (hence the Australian data). You’re actually on to something. As a paramedic I saw a significantly higher percentage of immigrants with mental illness versus the indigenous (as in anyone born in Aus) population. They had often come from war torn countries and brought with them extremely violent practices in terms of fighting. No more step outside and have a quick punch up, now it’s step outside and often – get knifed. Either way – murder rate has gone up, not down, and has been capped at a lower relative rate than it should be through advances in medicine.

Seriously though, suggesting that virtual violence begets real violence is not a specious argument. Exposure to anything at a high enough level both makes you intimate with it and numbs you to it.

Case in point – police, paramedics, firefighters, doctors and nurses all have a high amount of trauma and death around them. The regular population (or people new to the job) don’t cope particularly well with it. But with enough exposure you’ll be eating your lunch next to a stretcher dripping with blood from the last massive trauma patient and cracking jokes about deceased people – while they’re still warm (totally random examples of course…)

So, expose yourself to violence and expect that you’ll get pretty intimate with it and numb to the reality of it (until you get the shit kicked out of you because you can’t fight – like most urban ninjas). Keep in mind this is a totally accepted general principal – so why oh why are people trying to suspend it with this particular topic?

I’m happy to admit I’ve been an avid FPS gamer since 1990. I’ve played just about every good FPS game since then, although it’s getting hard to keep up with the amount that are released these days so I’m much more picky choosy now. Do I think it’s affected me? You betcha. Twenty years of killing mutant zombies has given me an itchy trigger finger should I ever see one. Same as most people really.

We’ll see if people find photorealistic virtual rape games acceptable when they hit the internet. I’m sure the users of it will be adamant that their virtual sickness won’t extend into real life. What if it’s the rape and/or killing of a virtual likeness of your own husband/wife/kids some freak is beating off to? I’m sure you won’t mind since it’s virtual (you as in anyone reading this – not the previous poster specifically). Don’t worry, 99.9% of the time their obsession won’t extend into real life. Good odds hey? Don’t worry, this is a specious argument and all šŸ˜‰

Anonymous Coward says:

Not all opinions are equal. Not all science is equal.

I think the Bushman/Anderson analysis is less goofy than it is being spun here.

The problem is that many people, including parents, politicians, and tech blog readers, have opinions on the ways in which media does (or does not) influence attitudes and aggressive behavior. But, because a parent or lawmaker has an opinion, doesn’t mean that he or she has a good understanding of the research literature on the topic, a literature that has both strengths and weaknesses. Some people have opinions that are supported by evidence. Some people have opinions that are supported by principle, coupled unrepresentative anecdotes.

What the Bushman/Anderson analysis demonstrates is that the people supporting one particular conclusion on this issue have more credibility: they’ve done the research themselves in many cases. As such, they are better positioned than you or I to understand how it informs the basic issues.

Sure, if could very well be the case that these authors reached a conclusion long ago and selectively publish studies that support their case. But that is merely an assumption on Mike’s part based on a distrust of the “science industry.” I don’t see any support for the assumption.

Beth W. (user link) says:


Thanks so much for writing this. It’s a subject I tackled recently on my own blog: It’s absolutely ridiculous that these guys tried pass this “study” off as “objective” or even convincing. Anderson is a pro at doing these studies-of-studies thing, too. I wish more folks in the press would dig deeper into these kinds of studies as you have, so we can show the public they have cause for suspicion.

Gene Cavanaugh (profile) says:

Does knowing more about a subject qualify you to give an opinion?

You blew it big time on this one, Michael!

When I first started in engineering, there was an enormous debate about education versus the so-called “school of hard knocks”, with the implication that experience was superior to an education. To a lesser extent, I saw this in marketing and other fields as well.

Now we are saying people who have studied the problem carefully (as shown by the papers they have contributed) know less than “knee-jerk” reactionists! That is so naive!!!!

Add Your Comment

Your email address will not be published.

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop Ā»

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...