DailyDirt: Can Computers Grade Written Essays?

from the urls-we-dig-up dept

Technology aimed at education could really benefit an incredible number of students by making classes and learning (potentially) a more pleasant and efficient experience. Computers can’t replace a really good human teacher, but they can make it easier for good human teachers to reach a vast audience of students. Massively open online courses (MOOCs) promise to change how education works, but there are some technological tools that might be missing. It’s pretty straightforward to test students on math problems in an automated way, but grading essays is a much more daunting problem. There have been some calls for automated grading software from various organizations (like the Hewlett Foundation). But at the same time, the National Council of Teachers of English argues that computers simply can’t grade essays. Here are just a few more links on this debate over the use of algorithms over English professors (or grad students).

If you’d like to read more awesome and interesting stuff, check out this unrelated (but not entirely random!) Techdirt post via StumbleUpon.

Filed Under: , , , , , , , , , , , ,
Companies: harvard, hewlett foundation, kaggle, mit

Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “DailyDirt: Can Computers Grade Written Essays?”

Subscribe: RSS Leave a comment
Anonymous Coward says:

Re: Re:

I have always thought that the way to restore sanity regarding zero tolerance is to threaten to do that.

Nothing is as influential as a bureaucrat trying to preserve their own job. If it goes through without a return to sanity well you punished the morons behind it by making them unemployed and you aren’t worse off. Not to mention the tax revenue saved. It is a heads you lose tails I win situation.

Anonymous Anonymous Coward says:

Re: Depends

Along those lines, isn’t the current system just a set of ‘facts’ (loosely describing the current standardized testing system) and that the way to game the system is to know what is on the final test?

Reminds me of a segment regarding the author of In Search of Excellence. He was a new professor on his first day, and he handed out the final exam to all of the students. A more senior professor was passing by and noticed this atrocity. He cornered his junior and remonstrated him for giving out the final exam. The new to the campus professor retorted, (to the best of my memory) “Not only am I giving them the final exam on the first day, I intend to spend the rest of the semester giving them the answers.”

As an experienced employer, with at times thousands of employees, my philosophy on hiring is: I don’t care what you know, I care what you can do! Education provides knowledge. Knowledge, TRAINING, AND EXPERIENCE provide skill. Skill is what one sells on the marketplace. Either to a new employer, or a customer.

Unfortunately, other employers are not so enlightened. They care what your grades are and what is on your Facebook page, and if those keywords weren’t on the resume, we wouldn’t even be talk’n.

For me, teach the student the basics (read’n, rite’n, and rithmitic) then teach them to think, make decisions, take stances, be critical without animosity, resolve issues, expand creativity (I wish art and music were required, but creativity can come elsewhere, even in math), and to ‘do it, try it, fix it.

I care not what the grades were, I care what they can do, and if they might grow with us.

Anonymous Coward says:

Re: Re: Depends

The problem really is that there is a hierarchy of knowledge (use information-gathering mechanism of choice for finding out about Bloom’s Taxonomy and related). At the lowest levels, you have things characterised by words like ‘recall’ and ‘recite’. At higher levels, you progress through things like ‘understand’ and ‘apply’ through to ‘analyse’ up to ‘create’.

Evaluating the lower levels ranges from trivially easy to simple. As you look for higher-level skills, you begin to require expert judgment. I’ll assert that expert judgment is almost impossible to implement as an algorithm.

I’m doing a MOOC atm. It is sitting squarely in the lower cognitive domains. It’ll replace a bad teacher easily. It won’t replace the best a university can do.

Anonymous Coward says:

Re: Depends

Yes. Les Perelman, a director of writing at MIT, did an analysis of e-Rater algorithms last year. He came up with a completely nonsensical and factually incorrect essay that got a top score, and a well-argued and well-written essay that scored lower.

Here’s the NYT article about it:

And here’s his top-scoring essay:

If nothing else, it’s worth skimming through the essay. Among other things, it includes a line from Howl and claims that the union of teaching assistants is more powerful than the Freemasons (although less powerful than the Jedi Knights).

Textual analysis is hard. It can potentially test for writing style, but it can’t test for comprehension or content.

Anonymous Coward says:

Need to worry about people gaming the models

Here’s a quick summary of the most insightful comments I’ve seen about this issue:

Yes, computers can apparently grade essays written for humans as well as underpaid and overworked graders do (which is badly). But once you start having computers actually grade the students, the students will start writing their essays to fool the computer graders (likely with help of computer programs)…

timmaguire42 (profile) says:

MS Word has an auto-summary feature that is disturbingly good, but just because software can create a summary doesn’t mean it can recognize a good one.

The day they write software that can recognize and appreciate a pun is the day they can dream about writing software that can recognize and appreciate a sound argument.

This sounds more like the kind of dumbing down we get from standardized testing. And I suppose if they really do want to reduce our children to mindless drones, then this is another step in the right direction

123movies says:

MS Word has a disturbingly good auto-summary feature, but just because software can build a summary doesn’t mean it can recognize a good one.

The day they’re writing software that recognizes and appreciates a joke is the day they can dream of writing software that can recognize and appreciate a sound statement.

Watch online movies 2020 on 123Movies new site.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Follow Techdirt

Techdirt Daily Newsletter

Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...