Are Automated Essay Graders Sucking The Life Out Of Writing?

from the who-needs-humans? dept

A slightly disturbing story about how English essays are increasingly being graded by machines. The folks behind the program claim that the computer's results have a very high percentage correlation to scores given by human graders. Of course, the machine is scoring the essays more on technical points concerning sentence structure and the use of certain key words. The same company that makes the software for grading exams is also offering a "training" application that students can use to get feedback on their own papers. As someone in the article points out, while it's nice to have a system to help teach students proper writing techniques, there is also the fear that such systems will beat out any creativity from students by focusing exclusively on the technical (rather than the creative) aspects of writing.

10 Comments | Leave a Comment..


If you liked this post, you may also be interested in...
 

Reader Comments (rss)

(Flattened / Threaded)

  1.  

    worksaving technology

    identicon
    larry kniepkamp, Sep 4th, 2003 @ 5:51am

    Your comments sound very much like the ones made when engineers stopped using slide rules and log tables because they had hand held calculators. Engineers are still creative, they are just more efficient.

    reply to this | link to this | view in thread ]

  2.  

    No Subject Given

    identicon
    bastard sam, Sep 4th, 2003 @ 6:39am

    I think that's an unfair assumption mike. If it correlates to human graders like they say, then creative writing or not should make no difference. Might be interesting to follow this one.

    reply to this | link to this | view in thread ]

  3.  

    Creativite Writing ...

    identicon
    AMetamorphosis, Sep 4th, 2003 @ 8:11am

    If a creative paper is graded by a machine, how can the " creativity " included in the paper be properly assessed ?
    Machines are great for use in grading assignments that have a predetermined outcome. ie: Multiple choice, True & False, or specific words or numerical values.
    This topic greatly intrigues me and I wonder what grade a Shakespearean piece or a poem would garner.

    reply to this | link to this | view in thread ]

  4.  

    Quality verses Stupidity

    identicon
    AMetamorphosis, Sep 4th, 2003 @ 8:26am

    From Article: " E-rater cannot read or judge an essay's quality but uses statistical analysis to determine which linguistic features are characteristic of each human scoring level, typically on a scale from 1 to 6. "
    Clearly, these programs do NOT judge or have the ability to even read the essay's QUALITY.
    I think it would be quite easy to come up with a formulation that would pick enough necessary words, phrases & other items the programs are looking for in order to achieve a high grade ... even though the quality may be abhorrent.

    reply to this | link to this | view in thread ]

  5.  

    Teach the Test

    identicon
    Beck, Sep 4th, 2003 @ 8:40am

    If these become common then it's only a matter of time until private companies develop courses that teach students how to make the machine happy. Instead of teaching writing they will be teaching how to beat the machine.

    I hope these machines have better grammar checking than the green underlines in Microsft Word.

    reply to this | link to this | view in thread ]

  6.  

    Look at the upside

    identicon
    digibum, Sep 4th, 2003 @ 8:41am

    Hey at least our students will be able to get high paying jobs as search engine optimization consultants.
    I think this trend will die out when someone figures out a way to write a total jibberish essay that is somehow graded very highly.

    reply to this | link to this | view in thread ]

  7.  

    Essay, Schmessay

    identicon
    Dan Z., Sep 4th, 2003 @ 9:04am

    Given that automated scoring can't judge quality, why use essay testing in the first place? If you're only looking for certain vocabulary words in certain places, why not just offer a Mad Lib style essay template where students fill in the blanks? At least then students and graders would agree 100% of the time on what belonged where, computers wouldn't be penalizing originality, and they wouldn't be giving reinforcement to poorly written essays and poorly thought out arguments.

    reply to this | link to this | view in thread ]

  8.  

    my college exit exams

    identicon
    mark, Sep 4th, 2003 @ 9:23am

    In order to graduate, I had to take some sort of basic competency exam and thought, man is this stupid. How could anyone get passed through 18 years of school and still not have a grasp of basic grammer and vocabulary?

    Anyway, when it came to the essay segments, I wrote my answers like a 4th grader. Short, simple sentances. See the cat. The cat is blue. I didn't attempt anything sophisticated like using a semicolon. Whoever graded the work must have thought I was a moron, but I didn't care, I thought the test itself was a waste of my time. Might as well use a machine.

    reply to this | link to this | view in thread ]

  9.  

    Anyone actually read the article?

    identicon
    LittleW0lf, Sep 4th, 2003 @ 1:59pm

    If they would have (ok, a couple folks who replied had read it,) they would have realized that nobody is proposing using the system alone to grade. The article specifically mentions that the grade is based both on the computer and the human grader's interpretation of the work, so it isn't likely to be cheated any time soon.

    I think it is a great idea...I write, and often I find it very difficult to find someone who will critique my work. Having a computer do it for me would be extremely helpful, so long as it does a much better job than the normal spell & grammer checkers do. And I can see where this could be a big advantage for the teacher, who can let the software grade the technical aspects of the work while they admire the literary aspects. Teachers who grade the technical aspects of a work often loose the literary aspects, since they spend most of their time grading the technical aspects.

    Sounds like a great idea to me.

    reply to this | link to this | view in thread ]

  10.  

    Re: Anyone actually read the article?

    identicon
    Anonymous Coward, Sep 4th, 2003 @ 2:40pm

    I like your use of moderation and a blending of two for a harmonious outcome ...

    reply to this | link to this | view in thread ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here
Get Techdirt’s Daily Email
Save me a cookie
  • Note: A CRLF will be replaced by a break tag (<br>), all other allowable HTML will remain intact
  • Allowed HTML Tags: <b> <i> <a> <em> <br> <strong> <blockquote> <hr> <tt>


A word from our Sponsors...
Follow Techdirt
Flattr rss rss
From the Techdirt Archive...
A word from our Sponsors...

Close

Email This