The best solution to this problem is usually to stop trying to evaluate based on single events with limited context, and instead look for patterns of behavior. Malicious users rarely do things once and then vanish, and non-malicious users rarely use threatening language over and over again, even in contextually appropriate ways.
More evidence increases the odds that context and level of intent will be accurately assessed - not just clearing innocent users but also making it possible to pick out threats that are genuinely credible and imminent. The space is admittedly tricky and perfect certainty is unrealistic, but things can be improved by focusing the priority where it should be with threats of violence - dangerous and abusive users, not the display of bad content.
Unfortunately there are a lot of internal pressures that push moderation teams away from this approach. The fear of missing obvious individual examples, the boost to reportable numbers from rating 100 comments instead of 10 accounts, desire for smaller discrete tasks to automate or send to vendors for scalability - all of these things play against user focused strategies in this space.
Techdirt has not posted any stories submitted by James Gresham.
The best solution to this problem is usually to stop trying to evaluate based on single events with limited context, and instead look for patterns of behavior. Malicious users rarely do things once and then vanish, and non-malicious users rarely use threatening language over and over again, even in contextually appropriate ways. More evidence increases the odds that context and level of intent will be accurately assessed - not just clearing innocent users but also making it possible to pick out threats that are genuinely credible and imminent. The space is admittedly tricky and perfect certainty is unrealistic, but things can be improved by focusing the priority where it should be with threats of violence - dangerous and abusive users, not the display of bad content. Unfortunately there are a lot of internal pressures that push moderation teams away from this approach. The fear of missing obvious individual examples, the boost to reportable numbers from rating 100 comments instead of 10 accounts, desire for smaller discrete tasks to automate or send to vendors for scalability - all of these things play against user focused strategies in this space.