Putting Pinners First: How Pinterest Is Building Partnerships For Compassionate Content Moderation

from the rethinking-how-moderation-works dept

Last week, Santa Clara University hosted a gathering of tech platform companies to discuss how they actually handle content moderation questions. Many of the participants in the event have written essays about the questions that were discussed at the event. Last week we published five of those essays and this week we're continuing to publish more of them, including this one.

The way platforms develop content moderation rules can seem mysterious or arbitrary. At first glance, the result of this seemingly inscrutable process is varying guidelines across different platforms, with only a vague hint of an industry standard -- what might be banned on one platform seems to be allowed on another. While each platform may have nuances in the way they create meaningful content moderation rules, these teams generally seek to align with the platform's/company's purpose, and use policies and guidelines to support an overarching mission. Different platforms delivering unique value propositions to users' accounts for variations in content moderation approaches.

At Pinterest, our purpose is clear: we help people discover and do what they love by showing them ideas that are relevant, interesting, and personal. For people to feel confident and encouraged to explore new possibilities, or try new things on Pinterest, it's important that the Pinterest platform continues to prioritize an environment of safety and security. To accomplish that, a team of content policy professionals, skilled in collaborating across different technical and non-technical functions at the company, decide where we draw the lines on what we consider acceptable boundaries for content and behavior. Drawing upon the feedback of Pinterest users, and staying up to date on prevailing discourse about online content moderation, this team of dedicated content generalists brings diverse perspectives to bear upon the guidelines and processes that keep divisive, disturbing, or unsafe content off Pinterest.

We know how impactful Pinterest can be in helping people make decisions in their daily life, like what to eat or what to wear, because we hear directly from the Pinterest community. We've also heard how people use Pinterest to find resources to process illness or trauma they may have experienced. Sometimes, the content that people share during these difficult moments can be polarizing or triggering to others, and we have to strike the right balance of letting people rely on Pinterest as a tool for navigating these difficult issues, and living up to our goal of removing divisive, disturbing, or unsafe content. As a team, we have to consider the broad range of use cases for content on Pinterest. For example, important historical yet graphic images of war can be collected in the context of learning about world events, or to glorify violence. Our team takes different contextual signals into account during the review process in order to make meaningful content moderation choices that ensure a positive experience for our community. If we wish to have the impact we hope to have in people's lives, we must also take responsibility for their entire experience.

To be responsible for the online environment that our community experiences, and to be aware of how that experience connects in a concrete way to their life offline, means we cultivate the humility to realize our team's  limitations. We can't claim to be experts in fields like grief counseling, eating disorder treatment, or suicide prevention -- areas that many groups and individuals have dedicated their careers to supporting -- so it's crucial that we partner with experts for the guidance, specialized skills, and knowledge that will enable us to better serve our community with respect, sensitivity, and compassion.

A couple years ago, we began reexamining our approach to one particularly difficult issue - eating disorders - to understand the way our image-heavy platform might contribute to perpetuating unhealthy stereotypes about the ideal body. We had already developed strict rules about content promoting self-harm, but wanted to ensure we were being thoughtful about content offering "thinspiration" or unhealthy diets from all over the internet. To help us navigate this complicated issue, we sought out the expertise of the National Eating Disorder Association (NEDA) to audit our approach, and understand all of the ways we might engage with people using the platform in this way.

Prior to reaching out to NEDA, we put together a list of search queries and descriptive keyword terms that we believed strongly signaled a worrying interest in self-harm behaviors. We limit the search results we show when people seek out content using these queries, and also use these terms as a guide for Pinterest's operational teams to decide if any given piece of self-harm-related content should be removed or hidden from public areas of the service. The subject matter experts at NEDA generously agreed to review our list to see if our bar for problematic terms was consistent with their expert knowledge, and they provided us with the feedback we needed to ensure we were aligned. We were relieved to hear that our list was fairly comprehensive, and that our struggle with grey area queries and terms was not unique. Since beginning that partnership with NEDA, they have developed a rich Pinterest profile to inspire people by sharing stories of recovery, content about body positivity, and tips for self-care and illness management. By maintaining a dialogue with NEDA, the Pinterest team has continued to consider and operationalize innovative features to facilitate possible early intervention on the platform. For example, we provide people seeking eating disorder content with an advisory that also links to specialized resources on NEDA's website, and supported their campaign for National Eating Disorder Awareness Week. Through another partnership and technical integration with Koko, a third party service that provides platforms with automated and peer-to-peer chat support for people in crisis, we're also able to provide people who may be engaging in self-harm behaviors with direct, in-the-moment crisis prevention.

Maintaining a safe and secure environment in which people can feel confident to try new things requires a multifaceted approach and multifaceted perspectives. Our team is well-equipped to grapple with broad online safety and content moderation issues, but we have to recognize when we might lack in-house expertise in more complex areas that require additional knowledge and sensitivity. We have much more work to do, but these types of partnerships help us adapt and grow as we continue to support people using Pinterest to discover and do the things they love.

Adelin Cai runs the Policy Team at Pinterest


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • This comment has been flagged by the community. Click here to show it
    identicon
    Anonymous Coward, 5 Feb 2018 @ 1:38pm

    I hate Pinterest, I avoid their links and immediately leave their sight the moment I accidentally click a link that leads me there.

    Should call those asshole the facebook of adversiting but the advertising industry already bet the fuck out of everyone for the title of everdoosh!

    reply to this | link to this | view in chronology ]

    • icon
      Mononymous Tim (profile), 5 Feb 2018 @ 2:39pm

      Re:

      Yeah, I've been adding -pinterest to the end of my Internet searches lately to keep them from clogging up the results, because every single Pinterest hit I've clicked on has been a useless dead end.

      A marketing scheme that makes people hate your service is a big fail.

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 5 Feb 2018 @ 3:37pm

        Re: Re:

        I do the same to avoid pinterest simply because they require a login even if I only want to view images that have popped up during a google image search.

        Good on them for curating content for people with eating disorders or who are suicidal or whatever. That's a plus I guess.

        reply to this | link to this | view in chronology ]

        • identicon
          Lawrence D’Oliveiro, 6 Feb 2018 @ 10:26am

          Re: they require a login even if I only want to view images that have popped up during a google image search

          I wonder how they can tell you’re Google? If your browser can look like a search engine to them, they should allow you to see the image without a login, shouldn’t they?

          reply to this | link to this | view in chronology ]

      • This comment has been flagged by the community. Click here to show it
        icon
        DocGerbil100 (profile), 5 Feb 2018 @ 3:59pm

        Re: Re:

        ^This. FUCKING EXACTLY THIS.

        My utter hatred for Pinterest knows few limits. I find it astonishing that they actually have the gall to try and pretend they care about self-harming behaviours, when they're easily one of the worst companies for driving any number of internet users into a state of psychotic rage.

        For the benefit of any and all Pinterest employees or shareholders reading this message: please die. Seriously, please, just die. If you could arrange to do so while shitting out your bloody, cancer-ridden entrails in front of your terrified, screaming children, as an object lesson to others: that would be helpful, but it's not essential, by any means.

        Removing yourself, along with as much of your hateful, malevolent, endlessly-spamming corporate nightmare from the world as you can take with you: that will be more than sufficient.

        reply to this | link to this | view in chronology ]

        • identicon
          Anonymous Coward, 5 Feb 2018 @ 4:34pm

          Re: Re: Re:

          For the benefit of any and all Pinterest employees or shareholders reading this message: please die. Seriously, please, just die. If you could arrange to do so while shitting out your bloody, cancer-ridden entrails in front of your terrified, screaming children, as an object lesson to others: that would be helpful, but it's not essential, by any means.

          WHY is this comment not at least hidden? The person specializes in bizarre vulgarity, and explicitly states " My utter hatred for Pinterest knows few limits."

          Nothing but HATE.

          Oh, right. Fanboy. Their comments are never even hidden.

          reply to this | link to this | view in chronology ]

          • This comment has been flagged by the community. Click here to show it
            identicon
            Anonymous Coward, 5 Feb 2018 @ 4:37pm

            Re: Re: Re: Re: Thanks 'dirters, for saving me time here.

            I was going to point up Techdirt's contradictions, but you've made that unnecessary!

            Read again slowly, and NOTHING BUT OFF-TOPIC HATE.

            reply to this | link to this | view in chronology ]

          • icon
            An Onymous Coward (profile), 5 Feb 2018 @ 4:43pm

            Re: Re: Re: Re:

            Give it time. Comments aren't hidden automatically based on their content. The community votes them down and that's never instant.

            If the community wants to hide the message they will.

            Your message, and mine for that matter, are entirely off topic and deserve to be hidden. The previous vitriol is at least on topic if off color.

            reply to this | link to this | view in chronology ]

            • This comment has been flagged by the community. Click here to show it
              identicon
              Anonymous Coward, 5 Feb 2018 @ 5:21pm

              Re: Re: Re: Re: Re:

              Give it time. Comments aren't hidden automatically based on their content. The community votes them down and that's never instant.

              THAT IS A LIE unless YOU know all about Techdirt's system. And you do not, unless YOU are an administrator and now astro-turfing. Fanboys here ALWAYS talk authoritatively but Techdirt itself NEVER responds. Let alone warns in advance.

              I know from first-hand experience that comments are (to be accurate for today, were) hidden within five minutes, and on nearly dead topics.

              If the community wants to hide the message they will.

              ANOTHER LIE if you don't have real knowledge. You are just making stuff up to protect the site. -- There is surely an administrator involved. State how many clicks, then. And again, how do YOU know how it works when I don't, having asked for years?

              Your message, and mine for that matter, are entirely off topic and deserve to be hidden. The previous vitriol is at least on topic if off color.

              No, they aren't / don't. There's no written guideline here and I'm asking a relevant board question. Not for the first time, but at least the hundredth, after getting MANY of my fine by common law comments hidden today.

              reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 6 Feb 2018 @ 12:07am

    It might be the gentlest and most compassionate...

    ... and yet it's the number one annoyance to anybody doing image searches today.

    I don't think there exists a business that made me want to interact with their product less than pinterest.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 7 Feb 2018 @ 9:02am

    Pinterest is just image spam for google searches. I can't wait for them to go away so I don't have to add -pinterest.

    reply to this | link to this | view in chronology ]

  • icon
    DocGerbil100 (profile), 7 Feb 2018 @ 3:31pm

    So, here we are, a few days later and the only kind comment about Pinterest on this page still consists exactly of the words:

    "Good on them for curating content for people with eating disorders or who are suicidal or whatever. That's a plus I guess."

    ... and that was by an AC who also says he or she blocks Pinterest from image search results.

    Pinterest people, please know and understand: outside of your own established community, no-one cares about your moderation efforts. Almost everyone who has encountered your links hates you passionately.

    How you've even managed to build a community is something of a mystery, given the utter contempt you routinely show for everyone else in your marketing methods. I can only assume your users are all either permanently stoned off their tits or masochists who enjoy being abused.

    I was sure my earlier and absurdly-hostile comment would get hidden, especially with another poster encouraging it - but at the time of writing, it seems the Techdirt community dislikes Liars far more than they dislike abusive language. While my comment is undoubtedly offensive, it has the virtue of being entirely honest in its shameless, naked hatred for all things Pinterest.

    It's a hatred that's well deserved. You are a parasitic troll of a company, damaging users efforts to find what we want on the internet, damaging other companies' efforts to provide useful image search services - and now damaging the truth in this debate, with your ridiculous attempt to pretend you care about anyone other than yourselves.

    The best moderation effort anyone, anywhere could possibly make would be to excise you vile, corporate spongers from all of our search results, once and for all.

    reply to this | link to this | view in chronology ]

  • identicon
    aguila, 12 Feb 2018 @ 8:00am

    what's that, white guys?

    you viciously oppose space that doesn't cater to your own needs?
    well, that's unique! take this to the presses! apocalypse now!

    pinterest - keep on keepin' on

    reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here
Get Techdirt’s Daily Email
Use markdown for basic formatting. HTML is no longer supported.
  Save me a cookie
Follow Techdirt
Techdirt Gear
Shop Now: I Invented Email
Advertisement
Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Advertisement
Report this ad  |  Hide Techdirt ads
Recent Stories
Advertisement
Report this ad  |  Hide Techdirt ads

Close

Email This

This feature is only available to registered users. Register or sign in to use it.