If Someone Is Prattling On About Spam On Twitter And How That Helps Elon Musk, Send Them This Article

from the setting-the-record-straight dept

We’ve explained the details of Twitter’s case against Elon Musk and the legal issues at play multiple times, but for reasons I don’t fully understand, the general narrative on Twitter seems to be that Elon is likely to win the case, and it’s because “Twitter lied about spam.” This is not even close to accurate, so I thought I’d lay out the details here in a format that can be sent to people if they start spouting off about how “Twitter lied about spam!” This post, by the way, was inspired by this tweet, which Elon promoted (he responded to the same guy who re-upped the post a few hours later).

If you can’t see the image, it’s a screenshot of a tweet by some dude named Pranay Pathole, who seems to exist only to tweet nonsense that fluffs up Musk’s projects, and it says:

Why is it that everyone is talking about @elonmusk waiving his right to due diligence & he now has to face the consequences or buy Twitter, but no one is talking about how Twitter omitted or didn’t publicly disclose the actual number of bots on the platform? Seems dubious

So, let’s break apart this tweet, and some other similar claims, and show why they’re all misleading at best, or just outright lies at worst.

Did Twitter lie about spam on the platform?

There is little to no evidence to support this claim. The argument people are making is that Twitter “lied” about there being only 5% bots on the platform. Here’s another version of the same kind of argument from another account that seems to only tweet pro-Musk content, showing what this guy claims (incorrectly) is “visualizing the change in Twitter fake/spam accounts over the past 30 quarters (7.5 years)” and showing a flat line at 5%:

Visualizing the change in @Twitter fake/spam accounts over the past 30 quarters (7.5 years) @elonmusk pic.twitter.com/rr1padXzGd

— ALEX (@ajtourville) July 16, 2022

Elon also promoted this tweet, and one has to assume that at least one of Elon’s lawyers has explained to him why this is not what he seems to be pretending it is, which leads me to believe that Elon doesn’t care and enjoys misleading his rather vocal fanbase.

So, let’s get this out of the way: Twitter does not report “the number of spam accounts on the platform.” The 5% number is not, and has never been, the number of spam accounts on the platform. What Twitter reports, and what the 5% actually is, is a review of how much spam is left after Twitter removes tons of spam accounts through a variety of automated and other means. As Twitter’s CEO explained, they have processes in place that remove over half a million potential spam accounts every day, and often lock others until they can prove they’re real (e.g. by providing a phone number for verification).

Then, AFTER the company has already removed all that spam, it regularly (basically every day) conducts a sample, reviewed by humans, to see how much spam is left on the platform. And while Musk insists this process is ineffective, he’s wrong. As we’ve explained, using basic statistical design practices, if you want a 95% confidence interval with Twitter’s population size, sampling 100 random accounts per day gets you 9000 per quarter, which gives you a margin of error of 1%.

Next, what Twitter actually reports in its filing never comes even remotely close to saying “we have 5% spam on the platform.” Here’s what the company actually says in its SEC filings:

While these numbers are based on whatwe believe to be reasonable estimates for the applicable period of measurement, there are inherent challenges in measuring usage and engagement across ourlarge number of total accounts around the world. Furthermore, our metrics may be impacted by our information quality efforts, which are our overall efforts toreduce malicious activity on the service, inclusive of spam, malicious automation, and fake accounts. For example, there are a number of false or spam accountsin existence on our platform. We have performed an internal review of a sample of accounts and estimate that the average of false or spam accounts during thefourth quarter of 2021 represented fewer than 5% of our mDAU during the quarter. The false or spam accounts for a period represents the average of false orspam accounts in the samples during each monthly analysis period during the quarter. In making this determination, we applied significant judgment, so ourestimation of false or spam accounts may not accurately represent the actual number of such accounts, and the actual number of false or spam accounts couldbe higher than we have estimated. We are continually seeking to improve our ability to estimate the total number of spam accounts and eliminate them from thecalculation of our mDAU, and have made improvements in our spam detection capabilities that have resulted in the suspension of a large number of spam,malicious automation, and fake accounts. We intend to continue to make such improvements. After we determine an account is spam, malicious automation, orfake, we stop counting it in our mDAU, or other related metrics. We also treat multiple accounts held by a single person or organization as multiple mDAUbecause we permit people and organizations to have more than one account. Additionally, some accounts used by organizations are used by many people withinthe organization. As such, the calculations of our mDAU may not accurately reflect the actual number of people or organizations using our platform.

There’s a lot to unpack here. However, first off, it’s all full of very clear caveats. It is quite clear that they are not, in any way, saying that it is definitive that 5% of the accounts are spam. The company admits that the number could be higher, while also detailing how it is calculating this number.

Also, it notes that mDAU — monetizable daily active users — is only a subset of overall users and (importantly) once the company identifies spam accounts (again, around half a million per day), it immediately removes them from mDAU. So let’s emphasize this again: the “less than 5%” number is AFTER the company has already removed hundreds of thousands of spam accounts per day from that calculation. It’s what remains after the company has used other methods to find and remove spam.

Then, the company is pretty clear that after it’s gone through that process, it samples randomly on an ongoing basis. And, as we showed above, this presents a very statistically robust setup. Given the 1% margin of error, what this almost certainly means is that each quarter, as Twitter manually reviews 9000 accounts, it finds that the amount of spam that it missed is likely anywhere between 2 and 4%, and that it varies over time (contrary to the tweet above). With a 1% margin of error, the company could feel confident just putting that 5% upper bound and noting that its review indicates less than 5% of mDAUs are spam, and it has fundamental statistics to back that up.

When put into context, what this all means is that:

• Contrary to claims you’ve heard, Twitter does not actually report how many spam accounts are on the platform.
• Instead, Twitter uses a variety of automated and other processes to remove a ton of spam every single day. Whenever a spam account is removed from the platform, it is also removed from the company’s mDAU calculation
• AFTER all that spam is removed, THEN Twitter (on a daily basis) has human experts randomly sample a bunch of accounts, in a statistically meaningful way, to see if any spam accounts remain. It consistently finds less than 5% of accounts that get through their earlier anti-spam processes are still spam accounts, and does so at a statistically significant rate, with a margin of error of 1%.
• It then reports that, based on this statistically rigorous process, less than 5% of the remaining accounts are found to be spam.
• It carefully caveats all of this in its SEC filing, and explains the many ways in which there could be problems with this process, and why no one should rely on it as an accurate measure of spam on the platform. That’s not why it’s there.

So, again, just to emphasize this: Twitter does not actually report the “amount of spam” on its platform, because that’s meaningless. Instead, it has extensive systems in place to try to delete spam, and then after it does that (averaging about half a million accounts per day), it goes further and runs a sample with human review every day to find out how much spam still slipped through the process. It then tracks that number.

Also, again, as soon as it identifies spam, it removes that from the mDAU count.

So, no, Twitter does not appear to have misled anyone about how much spam is on the platform.

The only argument you could make about the original tweet is that, for reasons the guy did not intend, he’s correct that Twitter “didn’t publicly disclose the actual number of bots on the platform” because that number is meaningless. Twitter does publicly disclose a reasonable measure of how many fake accounts sneak through all of their other systems designed to catch and stop fake accounts from being monetizable.

Does any of this matter for the lawsuit?

Again the answer is no, not really, for multiple reasons, many of which look really bad for Musk. First off, even before Musk purchased Twitter, both in public and in private (as revealed in the lawsuit), Musk made it clear that he, personally, believed that Twitter had a lot more spam than the company publicly revealed. Indeed, he insisted that it was his number one priority to rid the platform of that spam. So, in general, it’s sort of laughable for him to then try to get out of the deal by claiming there was more spam, when his initial explanation for why he was buying it was to get rid of the spam.

Next, a part of the made up narrative (not actually in the lawsuit) is that Musk “relied on” Twitter’s SEC statements regarding how much spam was on the platform. That seems unbelievable given the paragraph above and his public and private comments. But, even if that were true, that would mean that Musk would have to admit having read Twitter’s SEC filings… which explicitly say not to rely on those numbers as an accurate accounting of spam on the platform. So, that already looks bad for Musk, since he either read the details which say “don’t rely on this” or he didn’t… and relied on the numbers he didn’t actually read as a basis to agree to spend $44 billion. And that seems even more dubious in light of the next part:

If the amount of spam being much higher than he believed was actually an issue of concern to Musk, then there were multiple steps he could have taken prior to the deal being signed. First, Musk had every opportunity to ask Twitter if he could conduct due diligence on the amount of spam on the platform. If he were actually concerned about the percentage of spam, that would be not just the most logical step to take, it would be a necessary condition on the deal. There was no indication of an alternative buyer. Musk had all the time in the world to conduct the necessary due diligence.

Second, purchase agreements often have representations and warranties on issues like this — effectively having the purchase agreement say something to the effect of “Twitter represents that the amount of spam counted in our mDAU is less than 5%” and then there would be a warranty or indemnity if that turned out to be untrue, enabling Musk to force Twitter to make him whole for misrepresenting the number.

Musk chose not to do this.

In other words, there were multiple ways that Musk could have easily baked into the purchase agreement an effective guarantee on Twitter’s counting of fake accounts in the mDAU calculations (again, not the total spam on the platform). But he explicitly waived the right to examine that issue in more detail and to his own satisfaction before the deal, and also chose not to include a basic rep and warranty in the final deal, which is something anyone actually concerned about this number would absolutely do.

Given those two (conceivably very bad) choices by Musk, he is left with no “out” on the contract over the amount of spam in the mDAU calculation (which, again, as we showed above, has not been shown in any way to be inaccurate). Even if it were inaccurate, Musk left on the cutting room floor his opportunities to contractually deal with higher-than-believed spam.

So, nothing in the lawsuit is actually about how much spam is on the platform.

But Twitter amended their filings right after the deal! Isn’t that material? Doesn’t that change things?

This is part of the argument that Musk makes in his response to the lawsuit, and I’ve seen his fans harp on it as well. But even in the lawsuit they don’t make that much about it, because there’s not much to make. You can see the details in the filing, on page two. It notes that the company had revised how it was calculating mDAO after discovering an error in the earlier calculation, and they explained what the error was:

In March of 2019, we launched a feature that allowed people to link multiple separate accounts together in order to conveniently switch between accounts. An error was made at that time, such that actions taken via the primary account resulted in all linked accounts being counted as mDAU. This resulted in an overstatement of mDAU from Q1’19 through Q4’21.

So, basically, the mDAU calculations were just slightly off because of one type of account, one that is rarely used — where an account holder links together multiple accounts (for something like a corporate account, for example) and that counting error slightly inflated the mDAUs. Was it material? Uh, not really. In the last quarter accounted for, Q4 of 2021, the global mDAU was 214.7 million instead of 216.6 million. So, the corrected method decreased mDAU by… wait for it… less than 1%.

This was no big secret plot to overstate spam accounts. It’s not even about spam accounts. It was about a calculation error of one kind of account, that had a miniscule impact on overall mDAU.

The restatement was not misleading, was not a material misstatement, and does not actually have a meaningful impact on Twitter’s ability to monetize. It also has fuck all to do with spam accounts.

So what is the lawsuit actually about?

It seems clear that at some point, Musk decided he wanted out of the deal. His lawyers scoured the agreement, and must have slapped their heads at Musk’s initial failures to ask for due diligence, let alone the lack of reps and warranties on this issue. The only thing they picked up on was a clause that said that Twitter had to provide Musk with information necessary to close the deal, if Twitter felt it was reasonable and safe to provide that info to Musk.

Here’s the language in the original merger agreement. The key part is Section 6.4, and just to be clear “Parent” is Elon Musk and “the Company” is Twitter.

Section 6.4 Access to Information; Confidentiality. Upon reasonable notice, the Company shall (and shall cause each of its Subsidiaries to) afford to the representatives, officers, directors, employees, agents, attorneys, accountants and financial advisors (“Representatives”) of Parent reasonable access (at Parent’s sole cost and expense), in a manner not disruptive in any material respect to the operations of the business of the Company and its Subsidiaries, during normal business hours and upon reasonable written notice throughout the period commencing on the date of this Agreement until the earlier of the Effective Time and the termination of this Agreement pursuant to Article VIII, to the properties, books and records of the Company and its Subsidiaries and, during such period, shall (and shall cause each of its Subsidiaries to) furnish promptly to such Representatives all information concerning the business, properties and personnel of the Company and its Subsidiaries as may reasonably be requested in writing, in each case, for any reasonable business purpose related to the consummation of the transactions contemplated by this Agreement; provided, however, that nothing herein shall require the Company or any of its Subsidiaries to disclose any information to Parent or Acquisition Sub if such disclosure would, in the reasonable judgment of the Company, (i) cause significant competitive harm to the Company or its Subsidiaries if the transactions contemplated by this Agreement are not consummated, (ii) violate applicable Law or the provisions of any agreement to which the Company or any of its Subsidiaries is a party, or (iii) jeopardize any attorney-client or other legal privilege. No investigation or access permitted pursuant to this Section 6.4 shall affect or be deemed to modify any representation or warranty made by the Company hereunder. Each of Parent and Acquisition Sub agrees that it will not, and will cause its Representatives not to, use any information obtained pursuant to this Section 6.4 (or otherwise pursuant to this Agreement) for any competitive or other purpose unrelated to the consummation of the transactions contemplated by this Agreement. Parent will use its reasonable best efforts to minimize any disruption to the respective business of the Company and its Subsidiaries that may result from requests for access under this Section 6.4 and, notwithstanding anything to the contrary herein, the Company may satisfy its obligations set forth above by electronic means if physical access is not reasonably feasible or would not be permitted under applicable Law as a result of COVID-19 or any COVID-19 Measures. Prior to any disclosure, the Company and Parent shall enter into a customary confidentiality agreement with respect to any information obtained pursuant to this Section 6.4 (or otherwise pursuant to this Agreement).

Again, note how carefully caveated this is. Musk is allowed to request information, but that information can only be requested “for any reasonable business purpose related to the consummation of the transactions” AND on top of that, the agreement makes clear that the Company, Twitter, gets to decide not to share that information if it believes it might harm Twitter.

That puts Musk in a pretty tough spot. He can ask for information, but it has to be for the purpose of consummating the transaction. And there is no evidence that Musk needs details on spam accounts to consummate the transaction. If he did, he should have put that into the purchase agreement. But he didn’t.

Also, the purchase agreement makes it clear that Twitter gets to decide if it feels Musk is going to use the information for malicious purposes — including anti-competitive purposes — and thus refuse to provide the info to Musk without violating the deal.

In Twitter’s complaint, the company notes that Musk stated both publicly and privately to Twitter execs and board members that he was also considering just starting up a competitor. So, demanding access to all sorts of proprietary info from Twitter could easily raise concerns that Musk was just fishing for useful information for a would-be competitor, rather than using this info to consummate the deal.

That said, Twitter still provided Musk with all sorts of information anyway, so it’s difficult to argue that it didn’t live up to this clause in the agreement. Musk claims that some information requested wasn’t delivered, but much of that information likely doesn’t exist. This is likely on purpose. If the reason for these information requests were not to consummate the deal, but rather to search for an excuse to bail on the deal, it would make sense that Musk would keep asking for more and more ridiculous and impossible-to-obtain info.

And that’s what he did. Musk’s information requests were increasingly ridiculous, including the “daily measures of mDAU for the past 8 quarters” with “access to the sample set used and calculations performed” by the human reviewers to determine if accounts (again, out of the random sample of all who made it through the more aggressive culling process) were spam. But it seems likely that Twitter is not keeping track of the daily detailed notes of every single one of these human reviews. Because that would be silly.

Either way, note that the supposed breach of this clause (which again, does not appear to have actually been breached) is what the attempted termination, and thus the lawsuit, is actually about. Musk has to show that he requested information that was necessary for completing the deal and that Twitter didn’t give it to him.

Twitter just has to show that (1) it gave him what he asked for and (2) anything it didn’t give him either didn’t exist or in Twitter’s estimation was either not necessary for consummating the deal or could be abused by Musk.

Note that “how much spam is on the platform” doesn’t even remotely play into any of this.

A note on terminology:

There’s a lot going on here, but I also did want to note that the terminology here is also somewhat confusing, because people seem to be using “spam,” “bots,” and “fake accounts” interchangeably, even though they’re not all the same. There are human spammers. There are (often useful!) non-spam bot accounts. There are satirical fake accounts. Lumping all three categories together can lead to some problems.

The real concern for Twitter (and Musk) should be about what percentage of spam bots (not just “bots”) are making it through Twitter’s big list of defenses. Otherwise the conversation gets weird fast.

Final thoughts:

I’ve seen plenty of very smart people getting confused by all of this and insisting that the number of spam accounts must be the crux of the issue here. So don’t feel bad if you were confused. Musk has not helped things by continuing to push this false narrative (I have no clue if he doesn’t understand it, or just doesn’t care and wants to push the false narrative because it works in the court of public opinion). I’m not even sure what giving him the benefit of the doubt looks like here. Either he doesn’t understand his own case (doubtful) or he is deliberately misleading his most loyal fans (disgusting).

But the key points here are that (1) there is not, in fact, evidence that Twitter is misleading in spam counts. The methodology used appears to be sound, but even if it’s not, Twitter more than adequately disclaims the possible issues with this number. (2) Musk had ample opportunity prior to the deal to raise this issue and do due diligence or put in a contractual term in the purchase agreement to account for potential differences. He did none of that. (3) Musk’s own statements completely erase the idea that he was somehow taken by surprise by the amount of spam on the platform. (4) The actual reasons for the termination, while the lawyers try to connect it to the spam issue, are not actually about the spam issue at all, but rather about what information Twitter is providing Musk after the agreement was signed. (5) Based on the purchase agreement it appears that Musk is asking for information way beyond the scope of what is required in the agreement and Twitter has adequate contractual protections for refusing to share some of it (again, something Musk could have negotiated on but didn’t). Even so, it appears that Twitter still went above and beyond and provided Musk with a lot more information that he was entitled to under the contract.

So, no, the amount of spam on the platform is really not at issue in this case, even if Musk and his fans will continue to pretend otherwise.

Please feel free to share this article widely whenever you see people spreading misinformation about the lawsuit and, in particular, the claims about Twitter spam.

Filed Under: contract, disclosures, due diligence, elon musk, information breach, purchase agreement, representations and warranties, spam, statistical significance
Companies: twitter