Automated Moderation Means Distance Learning Students Are Being Called Cheaters Because Reasons

(Mis)Uses of Technology

from the let's-make-an-uncontrollably-bad-situation-worse dept

The response to the COVID-19 pandemic in America was, at best, inadequate. Many voters who felt subservient to a failed businessman chose to view the pandemic as a conspiracy meant to unseat Donald Trump. Millions died. Many more millions continue to suffer.

Schools reacted by offering a variety of learning options to students, including the opportunity to avoid breathing the same air President Trump believed to be innocuous, despite all evidence to the contrary.

But schools felt they needed to punish students who had opted out of superspreader events by remaining at home and trimming down their COVID exposure possibilities to immediate family members. Assuming every distance learning student was a cheater, schools signed contracts with private companies to vastly increase the amount of in-home surveillance they could perform under the heading of “education integrity.”

Donald Trump no longer fails the country on a daily basis. He has been replaced by a Democratic moderate who will likely fail the country on a weekly basis. Joe Biden is a more capable statesman and continues to monitor the COVID situation in the US, but there’s little he can do to undo the status quo.

Students still need to follow through on classes. And contracts signed during the height of the pandemic still remain operative. For whatever reason, school officials still seem to believe students pursuing studies at home are only in it for the cheating opportunities. Rather than dial things back, officials have let the COVID status quo remain in place. This is good for students who’d rather not be exposed to COVID. But it remains bad for students who’d rather remain home than spend time in crowded classrooms, rubbing elbows with the infected.

Kashmir Hill says the status quo is going to hurt students. Her report for the New York Times shows how surveillance software, embedded in distance learning options, continues to punish students who’ve chosen (either by their own choice or the choices of their guardians) to remain home and out of harm’s way.

A Florida teenager taking a biology class at a community college got an upsetting note this year. A start-up called Honorlock had flagged her as acting suspiciously during an exam in February. She was, she said in an email to The New York Times, a Black woman who had been “wrongfully accused of academic dishonesty by an algorithm.”

What happened, however, was more complicated than a simple algorithmic mistake. It involved several humans, academic bureaucracy and an automated facial detection tool from Amazon called Rekognition. Despite extensive data collection, including a recording of the girl, 17, and her screen while she took the test, the accusation of cheating was ultimately a human judgment call: Did looking away from the screen mean she was cheating?

This is symptomatic of any effort made to engage in moderation at scale. It remains impossible, whether it’s Facebook dealing with billions of users or educational spyware meant to keep distance learning students from getting a leg up on their in-school competitors by doing a bit of Googling.

So, when things seem questionable, it’s always the end users that suffer. In this case, human moderators likely backstopped algorithmic decisions. But when moderators are employed to stop distance learning cheating by third-party contractors, there’s no better way to show you’re paying attention than punishing students for things that would otherwise be considered normal behavior by kids during standardized tests. In a physical classroom, a student looking in another direction (other than straight ahead) would be viewed as harmless. Under the scrutiny of algorithmic tools that cannot actually perceive efforts to cheat, it looks like cheating.

Honorlock is cheap. That’s a boon for educators who often have very limited budgets. But you get what you pay for, even if you’re using other people’s money to buy it. The name suggest results the company is seemingly unable to deliver. But bang-for-buck pricing says it’s worth the roll of the (public) dice.

Honorlock, based in Boca Raton, Fla., was founded by a couple of business school graduates who were frustrated by classmates they believed were gaming tests. The start-up administered nine million exams in 2021, charging about $5 per test or $10 per student to cover all the tests in the course. Honorlock has raised $40 million from investors, the vast majority of it since the pandemic began.

The company will likely continue to secure funding. It offers an unbeatable price point with little downside. While it may eventually be sued for punishing students for non-cheating movements declared to be cheating by its AI, the suing students will need to demonstrate financial loss and other harmful outcomes of the system’s faulty AI before securing wins. This is unlikely to happen because standardized test scores mean little to nothing out in the real world and students branded cheaters by schools will always have to the opportunity to move to other schools. Courts don’t really seem to care how burdensome moving schools is on parents and their students. As long as the option remains, schools using third-party spyware are likely to escape lawsuits over their intrusive distance learning software.

Hill’s article lists other victims of third party spyware deployed by schools. But as long as students have options (no matter how unrealistic), courts will likely side with the purveyors of spyware and the public entities that have spent public money to prevent cheating via algorithms and always-on surveillance of public school students.

That doesn’t mean the problem isn’t real. In-home spying, engaged in by schools under the theory it will prevent cheating, is highly problematic.

When the student met with the dean and Dr. Orridge by video, she said, she told them that she looks down to think, and that she fiddles with her hands to jog her memory. They were not swayed. The student was found “responsible” for “noncompliance with directions,” resulting in a zero on the exam and a warning on her record.

“Who stares at a test the entire time they’re taking a test? That’s ridiculous. That’s not how humans work,” said Cooper Quintin, a technologist at the Electronic Frontier Foundation, a digital rights organization. “Normal behaviors are punished by this software.”

The question is: will courts care? The COVID pandemic is an anomaly. And when things go weird, the government is given considerable latitude to maintain normalcy. That means encroachment on rights will be ignored, if not rewarded, until everything settles down. Students will be victims, even if courts won’t recognize this fact at this point in time. By the time everything actually returns to a pre-2020 “normal,” all of this intrusion is likely to be forgotten. But make no mistake: the COVID pandemic did almost as much for surveillance as the 9/11 attacks. The difference is most of the involved parties were private contractors. Every crisis is an opportunity, and the government will always make the most of departures from the mean.

Filed Under: covid, distance learning, proctors, remote testing, surveillance
Companies: honorlock