Just As Everyone's Starting To Worry About 'Deepfake' Porn Videos, SESTA Will Make The Problem Worse

from the what-is-congress-thinking? dept

Over the last few months, if you haven't been hiding under a tech news rock, you've probably heard at least something about growing concerns about so-called "Deepfakes" which are digitally altered videos, usually of famous people edited into porn videos. Last month, Reddit officially had to ban its deepfakes subreddit. And, you can't throw a stone without finding some mainstream media freaking out about the threat of deepfakes. And, yes, politicians are getting into the game, warning that this is going to be used to create fake scandals or influence elections.

But, at the same time, many of the same politicians suddenly concerned about deepfakes are still pushing forward with SESTA. However, as Charles Duan notes, if SESTA becomes law, it will make it much more difficult for platforms to block or filter deepfakes:

Under it, websites that have “knowledge” that some posted material relates to illegal sex-trafficking can be deemed legally responsible for that material. What it means for a website to have “knowledge” remains an open question, especially if the site uses automatic or artificial intelligence systems to review user posts. Therefore, this language opens the door to a potentially wide range of lawsuits and prosecution.

The worst case scenario is that, to avoid having “knowledge” of sex trafficking, Internet services will stop content-moderation entirely. This scenario, which some experts call the “moderator’s dilemma,” would most likely affect smaller websites—including message boards and forums that serve special interests—that can’t afford the advanced filtering systems or armies of content editors that the big sites use. These smaller sites have already faced difficult problems with content moderation, and would be even less likely to spend resources on cleaning up after their users if doing so might lead to a lawsuit.

Duan points out that it goes beyond just the moderator's dilemma aspect of this. Even if sites do decide to moderate, at this point Congress is making it clear that whatever moral panic of the day that excites it may lead to new laws demanding action. But if they're desperately chasing the last problem, they have even less time to deal with the new one.

One of the good things about CDA 230, is that it actually allows platforms to experiment and try out different ways of moderating content. If they fail (as they often do!), they hear about it from their users, or the press, or from politicians. In short, they're allowed to experiment, but have incentives to try to find the right balance. But if Congress is enacting carve-outs that make any failure to properly filter a crime, then it becomes almost impossible, and the incentive is just to avoid doing anything at all. That's not at all healthy.


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • identicon
    Anonymous Coward, 14 Mar 2018 @ 1:52pm

    I feel like this gonna bite legislators in the ass later on because they're gonna claim sites aren't doing enough to solve the problem and all it's gonna take is people saying they can't for fear of being prosecuted and sued and point out they voted for this to happen and well, some are gonna be really embarrassed.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 14 Mar 2018 @ 2:10pm

      Re:

      yea, right....

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 14 Mar 2018 @ 2:11pm

      Re:

      Politicians avoid continued embarrassment by enacting new laws, and mandatory moderation would remove their embarrassment.

      reply to this | link to this | view in chronology ]

    • identicon
      Agammamon, 14 Mar 2018 @ 11:52pm

      Re:

      Nope. It will simply mean that *more* laws will be needed, *more* political control.

      While this isn't 'intended' and there's no conspiracy here, everything the government does leads to the government finding justification to do more. The ratchet tightens and rarely loosens.

      reply to this | link to this | view in chronology ]

  • icon
    Uriel-238 (profile), 14 Mar 2018 @ 1:53pm

    This was a sci-fi idea I had in 2005.

    At the time I was thinking one could take mo-caps of a variety of basic porn actions, couple them with body shapes and skin maps of famous figures extrapolated from a sufficient database of photos and video footage and then make plausible porn of anyone boffing anyone else.

    Originally, the idea was that as soon as we figured out how to render Humphrey Bogart and Marilyn Monroe cheaply enough to resume making movies featuring them as leads, we would.

    Then someone would figure we could go ahead and make Bogart/Monroe porn. Soon to follow: Customized I had sex with Marilyn porn where the client was rendered into the scene doing Ms. Monroe. Then it was a matter of what other parings might be interesting or useful.

    It would cause a few scandals early on, but then would start serving to cover for scandals, since it would raise doubt regarding any legitimate sex tape. Anyone could say it wasn't me.

    Meanwhile, now we have early deepfakes. The process is slightly different, we're essentially right on schedule.

    Maybe it'll mean we'll get over ourselves and this notion that good people don't have sex or only have wholesome licensed child-producing sex.

    Naaah.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 14 Mar 2018 @ 1:55pm

      Re: This was a sci-fi idea I had in 2005.

      If only.

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 14 Mar 2018 @ 7:52pm

      Re: This was a sci-fi idea I had in 2005.

      also, child porn of child actors

      reply to this | link to this | view in chronology ]

    • identicon
      Agammamon, 14 Mar 2018 @ 11:54pm

      Re: This was a sci-fi idea I had in 2005.

      Sorry mate, that idea shows up in stories from 2005 (Accelerando for one) and earlier.

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 15 Mar 2018 @ 2:13pm

        Re: Re: This was a sci-fi idea I had in 2005.

        So what? Ideas are easy. Combine Forrest Gump, the advancement of special effects technology, and knowledge of what type of pictures people like to look at online, and you've got a prediction.

        reply to this | link to this | view in chronology ]

      • icon
        Uriel-238 (profile), 15 Mar 2018 @ 7:48pm

        Don't make me totally geek out on you.

        Star Trek TNG made it clear that with the holodeck and zero programming skill one could get any group of people to play out pretty much any scene ever. Babylon 5 even noted how this could be used for a number of political purposes. But both TV series were reflections of our culture, and were limited by what writers were willing to consider and what the producers were willing to show. Gene Roddenberry wanted Trek to show a sexually liberal future, but his team around him wasn't ready to follow through.

        (You might remember the controversy over the biracial kiss in Plato's Stepchildren. Evidently, Zoey's and Hoban's happy interracial marriage is one of the factors that turned the networks against Firefly In 2002.)

        In 2005 I was looking at the technology that was known and looking at the natural end results. (Any medium, once determined to work becomes quickly turned to porn if it can.) But despite the mid-twentieth century golden age of social sci fi (where we speculated that sex may not always look like it did in the 1960s) our exploration of technology that allows for unlimited porn with anyone has not yet really been fully explored, even in the age of Rule 34.

        We know it can be done, but we haven't yet fully played with what happens next.

        reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 14 Mar 2018 @ 2:47pm

    SESTA - brought to you by our "friends" at Google and Facebook.

    reply to this | link to this | view in chronology ]

    • icon
      That One Guy (profile), 14 Mar 2018 @ 3:00pm

      Re:

      If that is the best criticism you can come up with, you are clearly not paying attention and focused on entirely the wrong things.

      Seriously, there are bigger issues at play than two companies being short-sighted idiots, and focusing on them is a classic case of ignoring the forest for the trees(even if they are rather big ones).

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 14 Mar 2018 @ 6:11pm

        Re: Re:

        It's the same sort of troll who campaigned for SOPA because Google and Facebook were against it, but you won't be seeing him admit that it's not actually about Google or Facebook, aside from the gleeful digs he thinks is scoring him RIAA points.

        reply to this | link to this | view in chronology ]

  • icon
    That One Guy (profile), 14 Mar 2018 @ 3:04pm

    "... The what now?"

    "Honestly, I have no idea what you're talking about, we don't engage in any moderation of our platform, as that's far too risky should we miss something. If you have a problem with it, well, take it up with the legislators who made moderation an all or nothing proposition. Since we lack the resources to go with the former, we were forced to go with the latter or go under entirely."

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 14 Mar 2018 @ 3:33pm

    Deepfakes are the Trifecta

    Defamation, Vulgarity, Freedom of Expression

    I'm not sure how all of this is going to work out but I am uncomfortably excited to see all this debated by lawmakers and in lawsuits.

    reply to this | link to this | view in chronology ]

    • icon
      Uriel-238 (profile), 15 Mar 2018 @ 7:54pm

      No artistic value

      There's also the matter that comes when someone like Neil Gaiman or Allan Moore figures out a deepfake scene that has artistic value. At that point, deepfakes as a general medium process can no longer be regarded as obscene.

      reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 14 Mar 2018 @ 4:28pm

    I've actually viewed a deepfake (for...uh..."research" purposes, of course).

    It was so fucking creepy I had to find other material to fin- er, compare it against.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 14 Mar 2018 @ 5:44pm

    What is the likelihood of SESTA being challenged in the courts?

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 14 Mar 2018 @ 6:16pm

      Re:

      High. The odds of any challenge succeeding are lower. I think it's in the category of stupid but constitutional.

      reply to this | link to this | view in chronology ]

      • icon
        Mike Masnick (profile), 14 Mar 2018 @ 11:22pm

        Re: Re:

        Yes, it will almost certainly be challenged. I would say that there are decent arguments for why a constitutional challenge could win, but it's not a slam dunk by any means.

        reply to this | link to this | view in chronology ]

        • icon
          Hephaestus (profile), 15 Mar 2018 @ 10:47am

          Re: Re: Re:

          To name a few reasons parody, political satire, and the first amendment it would be successfully challenged.

          reply to this | link to this | view in chronology ]

          • icon
            Hephaestus (profile), 15 Mar 2018 @ 10:50am

            Re: Re: Re: Re:

            Look away if you do not wish to have an image in your head that will never go away.

            For a video of a politician like Hillary doing a Katherine the great video ...

            reply to this | link to this | view in chronology ]

  • identicon
    Agammamon, 14 Mar 2018 @ 11:50pm

    "Reddit officially had to ban its deepfakes subreddit."

    No, no it did not - Reddit *chose* to ban it. Agree or disagree with that decision, it was still something Reddit chose to do.

    reply to this | link to this | view in chronology ]

    • icon
      The Wanderer (profile), 15 Mar 2018 @ 6:07am

      Re:

      I read that sentence as being equivalent to "things reached a point such that Reddit felt that it had no choice but to officially ban" et cetera.

      I agree that it would have been better if phrased differently, though.

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 15 Mar 2018 @ 11:20pm

        Re: Re:

        Them jumping the gun on deepfakes is downright bizarre and inconsistent in the first place. Porn isn't banned. Photoshops aren't banned. Photoshopping of celebrities into porn isn't banned. But deepfakes are? That sounds like the classic mocked 'but on a computer' moral panic - for something already on a computer no less!

        Making matters worse this one hundred percent ethical sexual content gets banned? I mean this isn't /r/jailbait or/r/creepshots. Nobody tried to present anything is real - they are explicitly fake and have some minor uncanny valley to it - especially with comedic mismatches. Lets face it - the deepfakes right now appear to be headswaps and it shows.

        reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here
Get Techdirt’s Daily Email
Use markdown for basic formatting. HTML is no longer supported.
  Save me a cookie
Follow Techdirt
Techdirt Gear
Shop Now: Copying Is Not Theft
Advertisement
Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Advertisement
Report this ad  |  Hide Techdirt ads
Recent Stories
Advertisement
Report this ad  |  Hide Techdirt ads

Close

Email This

This feature is only available to registered users. Register or sign in to use it.