Facebook Ups Surveillance Of Users To Keep Tabs On People Who Don't Like Facebook

from the threat-level:-oversharing dept

Tech companies are becoming far more than useful repositories of third party records. They're becoming far more active in terms of surveillance, pivoting from platform providers to private sector Big Brothers, weaponizing their data collection capabilities to keep tabs on customers and users.

Facebook has decided to start scanning its platform for threats. Not threats against the many nations it serves or threats targeting other users, but rather threats against Facebook itself.

One of the tools Facebook uses to monitor threats is a "be on lookout" or "BOLO" list, which is updated approximately once a week. The list was created in 2008, an early employee in Facebook's physical security group told CNBC. It now contains hundreds of people, according to four former Facebook security employees who have left the company since 2016.

Facebook notifies its security professionals anytime a new person is added to the BOLO list, sending out a report that includes information about the person, such as their name, photo, their general location and a short description of why they were added.

[...]

Users who publicly threaten the company, its offices or employees — including posting threatening comments in response to posts from executives like CEO Mark Zuckerberg and COO Sheryl Sandberg — are often added to the list. These users are typically described as making "improper communication" or "threatening communication," according to former employees.

It's not that Facebook shouldn't be on the lookout for credible threats. It's that it's turned its platform into a surveillance tool for its in-house knockoff law enforcement agency. It's not clear whether the company is turning over its internal BOLO list to actual law enforcement, but if it is, that raises even more concerns. Certainly the company should be concerned about legitimate threats. But the company is flagging people simply for expressing their displeasure with Facebook in general.

While some users end up on the list after repeated appearances on company property or long email threats, others might find themselves on the BOLO list for saying something as simple as "F--- you, Mark," "F--- Facebook" or "I'm gonna go kick your a--," according to a former employee who worked with the executive protection team.

This undercuts Facebook's official statements about "rigorous reviews" of detected threats. So does the claim made by former employees that fired employees are automatically added to the BOLO list, despite nearly 100% of fired employees from all vocations posing no threat to their former employers.

And it goes further than simply flagging people (and, apparently, displaying their photos on monitors in the threat detection center). Facebook also tracks listed individuals using their smartphones, thanks to permissions granted to the Facebook app. The app comes pre-installed on most smartphones and most users are unaware how much data Facebook is gathering even when the app isn't in use.

Presumably, if some "F--- you, Mark" person gets too close to the Facebook campus, actual law enforcement is alerted. This sort of situation can only lead to positive outcomes. A person mildly displeased with Facebook's endless fuckery will be greeted by armed officers under the impression a credible threat has been made against the company. Good times.

More good times await. Facebook is also promising to "help" the suicidal by sending the cops after them.

Since 2011, Facebook has allowed users to flag potential suicidal content; reports prompted emails from Facebook urging the poster to call the National Suicide Prevention Lifeline. But starting in 2017, Facebook introduced bots to search out and report potential suicidal content. The bots report suspected cries for help to human moderators, who may then "work with first responders, such as police departments to send help," says CNN.

That's right: Facebook might call the cops on you because a bot thought you seemed sad. Facebook executives think that if a user exhibits signs of depression, it's up to Facebook—not the user's friends, family, or community—to intervene.

Rather than trying to track down friends or family, Facebook is turning this over to "first responders." In most cases, the first responder on the scene is going to be the local PD. Given how often police officers have helped talk people out of suicide by killing them, this effort by Facebook is going to result in more dead suicidal people than simply doing nothing.

I'm not saying Facebook should do nothing about threats against the company or to aid people with suicidal thoughts. But these efforts aren't going to make anything better and they're a misuse of Facebook's vast data collections and moderation efforts. There's an abuse of trust happening here and Facebook's efforts are so scattershot and half-assed they're going to cause a lot of collateral damage.

Filed Under: privacy, surveillance, threats
Companies: facebook


Reader Comments

The First Word

Subscribe: RSS

View by: Time | Thread


  • icon
    Anonymous Anonymous Coward (profile), 21 Feb 2019 @ 1:32pm

    Add me to the list, here is my opinion of Facebook, in part

    I haven't used Facebook. I don't use Facebook. I won't use Facebook. I haven't, won't, and don't do business with any business who's only access is Facebook. I block Facebook in my HOSTS file (whatever good that does) and in Umatrix. I think Facebook is pointless, scummy, dangerous, and sociopathic. I don't like Facebook, I don't like their attitude, I don't like their position or their ability to have the impacts they have. I don't like they way they spin serious accusations against them as being less nefarious than I think they actually are.

    If Facebook HQ and all of it's servers everywhere in the world were destroyed, irrecoverably, my opinion would be that the world would be better off, and that in the long run the employees of Facebook would be better off as well, as they would likely find employment in an actually useful environment. It would be nice is Zuckerberg 's bank account was accidentally drained at the same time.

    If I haven't yet fully expressed my opinion that the world could not only exist without Facebook, but that it would be a better world for its lack of existence it is due only to a lack of negative adjectives and adverbs at my current disposal. I could get out a thesaurus and go to town, but frankly, Facebook just isn't worth it. Better to just ignore it and pity those who have succumbed to the promise never delivered.

    Other than pity which I express only here, I have nothing against any user of Facebook.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 21 Feb 2019 @ 1:52pm

      Re: Add me to the list, here is my opinion of Facebook, in part

      AOL was 100x worse in the mid-1990s, when they had control over not just a website, but your internet connection itself.

      Curse words were like saying "Beetlejuice!" as a GUIDE would enter your chat just to keep things clean. Only a wage lawsuit put an end to that, btw.

      The market fixes this problem much more quickly than the government, but most people are their own tabloid anyway.

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 22 Feb 2019 @ 7:16am

      Re: Add me to the list, here is my opinion of Facebook, in part

      So glad I am not the only one who has never used fb or likes buttons. I have a friend who will never step foot into a walmart. But he loves fb. I go to walmart every couple of weeks. We are good friends anyway!

      reply to this | link to this | view in chronology ]

    • identicon
      Dave P., 22 Feb 2019 @ 10:18am

      Re: Add me to the list, here is my opinion of Facebook, in part

      Just one word........Agree!!!!!!!!

      reply to this | link to this | view in chronology ]

    • icon
      btr1701 (profile), 22 Feb 2019 @ 11:44am

      Re: Add me to the list, here is my opinion of Facebook, in part

      I haven't used Facebook. I don't use Facebook. I won't use Facebook. I haven't, won't, and don't do business with any business who's only access is Facebook. I block Facebook in my HOSTS file (whatever good that does) and in Umatrix. I think Facebook is pointless, scummy, dangerous, and sociopathic. I don't like Facebook, I don't like their attitude, I don't like their position or their ability to have the impacts they have. I don't like they way they spin serious accusations against them as being less nefarious than I think they actually are.

      But will you use Facebook with green eggs and ham?

      reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 21 Feb 2019 @ 1:51pm

    CELEBRITY: "I was threatened on Twitter!"

    FANS: "So you admit to reading every tweet which mentions you!"

    reply to this | link to this | view in chronology ]

  • icon
    That Anonymous Coward (profile), 21 Feb 2019 @ 2:00pm

    We don't allow people to fly based on fewer facts, where is the problem?

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 21 Feb 2019 @ 2:16pm

    Rigor

    This undercuts Facebook's official statements about "rigorous reviews" of detected threats.

    Not really. It may be that you're interpreting "rigorous" differently than them. "Rigor" basically means "inflexible" or "harsh"—and a hard rule to list people saying "Fuck Mark" as dangerous certainly qualifies. Hell, maybe they have 98 other employees verify the person really said it.

    Did you think Facebook meant "reasonable"? They didn't say that, and like interpreting NSA press releases, we're well past the point where we can assume good faith.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 21 Feb 2019 @ 2:26pm

      Re: Rigor

      "Rigor" basically means…

      Please recall that this is Techdirt, and some proportion of the “technology” folks do have a fair amount of semi-advanced mathematical education in their backgrounds.

      INot that engineers can or should ever be confused with ”real mathmaticians” — we know those real mathematicians — they sometimes invite us to their parties — probably just so someone's around to change their lightbulbs for 'em. 'Cept for SWEs, they can't deal with h/w either.

      But the point is that we're familiar with “rigor” as a general concept, and have absorbed some rather strong connotative notions about rigor.

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 21 Feb 2019 @ 3:46pm

        Re: Re: Rigor

        But the point is that we're familiar with “rigor” as a general concept, and have absorbed some rather strong connotative notions about rigor.

        Do you think Facebook doesn't know the difference between how people are likely to interpret their statements, and what can be shown to have been technically not a lie?

        reply to this | link to this | view in chronology ]

  • icon
    Bamboo Harvester (profile), 21 Feb 2019 @ 2:18pm

    I can't believe...

    ...this is a surprise to anyone.

    Talk bad about your boss or the company you work for IN THE OFFICE, and you're going to have problems. What makes the "online world" any different than, in the world of facebook, IRL?

    reply to this | link to this | view in chronology ]

    • icon
      Matthew Cline (profile), 21 Feb 2019 @ 2:27pm

      Re: I can't believe...

      Talk bad about your boss or the company you work for IN THE OFFICE, and you're going to have problems.

      Problems which shouldn't extend to being labelled a potentially dangerous person.

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 21 Feb 2019 @ 3:19pm

      Re: I can't believe...Zuckerberg

      ... does Zuckerberg really know what's going on in the daily operations of his company -- or is he just a figurehead ??

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 21 Feb 2019 @ 6:03pm

      Re: I can't believe...

      Talk bad about your boss or the company you work for IN THE OFFICE, and you're going to have problems.

      Do American workers not have the right to communicate to improve their working conditions, without fear of retaliation?

      reply to this | link to this | view in chronology ]

  • identicon
    Personanongrata, 21 Feb 2019 @ 2:39pm

    Will Facebook Face-Plant?

    Facebook Ups Surveillance Of Users To Keep Tabs On People Who Don't Like Facebook

    As the hour hands of time slowly flow on like molasses on a cold winter day it would appear as if Facebook is going to Face-plant.

    Like the internet behemoths (ie CompuServe, MySpace, Yahoo, AOL, etc) that preceded it's rise Facebook may find it is now on a seemingly unstoppable downward trajectory toward intertube irrelevance.

    https://www.forbes.com/sites/ninaangelovska/2019/01/07/facebook-loosing-users-to-pinterest-youtube- and-twitter-market-share-by-region/#48bd464d7746

    https://techcrunch.com/2018/07/26/facebook-official ly-loses-123-billion-in-value/

    https://www.urbandictionary.com/define.php?term=intertubes

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 21 Feb 2019 @ 2:41pm

    It now contains hundreds of people, according to four former Facebook security employees who have left the company since 2016.

    Well, I know who some of those people are: I'm pretty sure the list includes four former Facebook security employees and Mike Masnick. It possibly includes me, despite my blackholing of FB assets wherever possible.

    Not because of threatening behavior mind you, but because of "improper communication".

    reply to this | link to this | view in chronology ]

  • identicon
    Glenn, 21 Feb 2019 @ 3:07pm

    Is it alright for me to suggest that facebook seems to itself be committing suicide now? (from the death-by-idiocy dept.)

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 21 Feb 2019 @ 4:16pm

      Re:

      Bury enough bodies in your wake and you spend all your time trying to prevent them from climbing back out for the world to see.

      reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 21 Feb 2019 @ 4:41pm

    More good times await. Facebook is also promising to "help" the suicidal by sending the cops after them.

    Wasn't Masnick screeching a while back about how this was a great thing and how GDPR was bad because it might prevent Facebook from "helping" people in this way?

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 21 Feb 2019 @ 5:58pm

      Re:

      Sending the cops to a suicidal person is generally not helpful. Especially since that's not really their area of expertise or purpose.

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 21 Feb 2019 @ 9:18pm

      Re:

      As usual, more lying.

      You can read the article you're talking about here: https://www.techdirt.com/articles/20180128/21505439105/unintended-consequences-eus-new-interne t-privacy-rules-facebook-wont-use-ai-to-catch-suicidal-users.shtml

      There is no "screeching" present.

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 21 Feb 2019 @ 11:38pm

      Re: Why must you turn this forum into a house of lies?

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 22 Feb 2019 @ 7:37am

        Re: Re: Why must you turn this forum into a house of lies?

        I know what having comments anonymously flagged at Techdirt feels like. It feels like.. death. I think that qualifies me to make a few points about seeing some corporations going out of their way to fuck with the public with virtually no fears of reprisal from governments.

        reply to this | link to this | view in chronology ]

        • identicon
          Anonymous Coward, 22 Feb 2019 @ 8:03am

          Re: Re: Re: Why must you turn this forum into a house of lies?

          I know what having comments anonymously flagged at Techdirt feels like. It feels like.. death.

          Then I suggest finding some new hobbies and finding your self-worth apart from whether or not people like your comments. If that's all it takes to mess up your day then you need help. Fast. Because life is going to throw a whole lot worse at you than some anonymous internet citizens disagreeing with your opinion and hiding your comments.

          reply to this | link to this | view in chronology ]

          • identicon
            Anonymous Coward, 22 Feb 2019 @ 9:15am

            Re: Re: Re: Re: Why must you turn this forum into a house of lie

            Why you trying to hurt me sis?

            reply to this | link to this | view in chronology ]

            • identicon
              Rog S., 22 Feb 2019 @ 11:26am

              Re: Re: Re: Re: Re: Why must you turn this forum into a house of

              There are actually military contractors who have “kill books ” who harass, and stalk people to suicide, and worse.

              Google "Dan Love and Kill Book "

              These shitbag progressive psycopaths are so far outside of due process, they make John Ashcroft look sane, and liberal.

              reply to this | link to this | view in chronology ]

              • identicon
                Anonymous Coward, 25 Feb 2019 @ 8:50am

                Re: Re: Re: Re: Re: Re: Why must you turn this forum into a hous

                And this psychological psychobabble has what to do with anything? Including the comment you replied to?

                reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 22 Feb 2019 @ 1:29am

    Scattershot is right! I'm still a 1st amendment kind of person. They will probably not end with just their users, but the entire internet! So, obligatory once again.. fuck facebook.

    reply to this | link to this | view in chronology ]

  • identicon
    Rog S., 22 Feb 2019 @ 8:37am

    I told you so.....

    Corporate gang stalking DOES involve the police, at many levels, including slander, and actual, chronic surveillance.

    The BOLO originates in many places ranging from corporations, to local LEOs, to DHS private contractors, and is even given to Paul Blart the Mall Cop by police, etc.

    The Association of Threat Assessment Professionals (this eras forensic science / junk science fraud ) calls these collusions the "colliding parallel investigation "

    https://c.ymcdn.com/sites/www.atapworldwide.org/resource/resmgr/2018-ATAP-Preliminary-TMC-Pr. pdf

    ATAP is the primary crisis PR mouthpiece that weaponizes psychological psychobabble, and paints liostick on high heeled pigs so that these agencies can continue slurping at the trough.

    The linked. pdf even describes it in an attempt to legitimize the "colluding parallel investigation " which is colloquially known as gang stalking.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 22 Feb 2019 @ 8:53am

      Re: I told you so.....

      psychological psychobabble

      You mean like every single one of your comments?

      reply to this | link to this | view in chronology ]

      • identicon
        Rog S., 22 Feb 2019 @ 11:19am

        Re: Re: I told you so.....

        I cant even imagine what kind of person you are.

        But basically, you are human garbage. I wouldnt shit on you if flies ruled the world.

        But you do look like a king right now.

        Surely your cowardly, anonymous, cowardly minions will swarm to you.

        And, your defense contractor friends.

        reply to this | link to this | view in chronology ]

        • identicon
          Anonymous Coward, 25 Feb 2019 @ 8:49am

          Re: Re: Re: I told you so.....

          That's the best you've got? Just more psychological psychobabble from a Rog S-tered troll.

          reply to this | link to this | view in chronology ]

  • identicon
    Rog S., 22 Feb 2019 @ 11:20am

    your types are losing the war against due process right now.

    Troll harder, piggy.

    reply to this | link to this | view in chronology ]

  • icon
    btr1701 (profile), 22 Feb 2019 @ 11:41am

    That's right: Facebook might call the cops on you because a bot thought you seemed sad. Facebook executives think that if a user exhibits signs of depression, it's up to Facebook—not the user's friends, family, or community—to intervene.

    This happened to me with Twitter. Not the cops thing, but I got a message from Twitter one day about a month ago saying that a recent post I'd made raised concern that I might be in danger of self-harm and it provided me a link to suicide prevention resources.

    I scoured my posting history and for the life of me I couldn't find anything I'd posted that remotely suggested any kind of self-harm, not even a joke response of some kind. And of course Twitter provides no ability for the user to respond and ask what the hell they're talking about.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 22 Feb 2019 @ 3:46pm

      Re:

      for the life of me

      I've notified the authorities. Help is on the way. Make sure there is nothing in your hands and keep them where they can be seen at all times.

      reply to this | link to this | view in chronology ]

    • identicon
      Rog S., 22 Feb 2019 @ 9:57pm

      Re: Fusion Center Twitter Firehose

      I got a lot of those after the 2016 election, saying they were cautioning me that I had reposted Fancy Bears circus routine, or some other Russian troll farm stuff.

      But then again, purported FBI whistle blower Sibel Edmonds, and half of Israeli Squad 3200 are my “followers ” aka cyberstalkers.

      And that, verified with empirical evidence.

      It seems the icky, bad Russian spies, hackers, and ideologues inside and outside of Twitter are somehow more icky and bad than British / Israeli/US spies, hackers, and ideologues inside and outside of Twitter.

      reply to this | link to this | view in chronology ]

  • identicon
    Max, 22 Feb 2019 @ 2:42pm

    I'm finding less and less incentive to read comments I can't downvote (some of them x1000 - well, one can dream...) on a site where absolute, utter lunacy seems to be the norm in the comments. But hey, that's just me...

    reply to this | link to this | view in chronology ]

    • identicon
      Rog S., 22 Feb 2019 @ 9:50pm

      Poor Max

      Take your medication, snowflake, and follow the other posters advice to “keep your hands where the cops and paranedics can see them ”when we get you 5150d....for your own good, of course, and so that the mean words stop hurting you so much.

      Surely the comments section is out to get you, just you, personally.

      reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Close

Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Follow Techdirt
Techdirt Gear
Shop Now: Copying Is Not Theft
Advertisement
Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Advertisement
Report this ad  |  Hide Techdirt ads
Recent Stories
Advertisement
Report this ad  |  Hide Techdirt ads

Close

Email This

This feature is only available to registered users. Register or sign in to use it.