Kathryn Tewson's Techdirt Profile

Kathryn Tewson

About Kathryn Tewson

Posted on Techdirt - 27 January 2023 @ 04:55pm

AI Lawyer Has A Sad: Bans People From Testing Its Lawyering After Being Mocked

Well, a lot has happened since I first started looking into the “World’s First Robot Lawyer,” from DoNotPay. First, Joshua Browder, DoNotPay’s CEO, reached out to me via direct message (DM) and told me he would get me access to my documents by 2 PM the next day – Tuesday, January 24th – saying that the delay was caused by my account being locked for “inauthentic activity,” a term he did not explain or define. Then, Josh claimed he was going to pull out of the industry entirely, canceling his courtroom stunt and saying he would disable all the legal tools on DoNotPay.com. He said he was doing it because it was a distraction, but the fact that he cited exactly the same two documents that I was waiting to receive seemed like a hell of a coincidence. 

But plus ça change, plus c’est la même fucking chose, as the poet says. Here’s what hasn’t changed: I still don’t have my documents, and Josh still hasn’t answered the questions I asked him like he said I would.

In his direct message, Josh said he would be willing to answer any questions I asked in good faith. I took him at his word, and responded to the email he sent me announcing his pullout with the following four questions:

  1. Can you describe for me the process DoNotPay used to identify the relevant law for a demand letter? (Cf. “Based on your location, DoNotPay will generate a formal demand letter on your behalf with the most relevant state legislation regarding defamation,” from here: https://web.archive.org/web/20230127063724/https://donotpay.com/learn/cease-and-desist-order/)
  1. Were humans involved in the generation of any client documents described by anything under your “Legal Tools” section? I don’t mean the creation of the templates, etc., I mean in the production of a document based on client responses to prompts.
  1. Are the articles in the “Learn” section of your website written by ChatGPT or equivalent, or by humans?
  1. Who signed the subpoena for the officer in the traffic case that was referenced in your now-deleted tweet?

I asked all these questions in good faith, and for good reasons. Josh represents — over and over and over and over again — that DoNotPay features a robot lawyer with artificial intelligence, going so far as to say that it uses AI instead of “human knowledge.” The sole document I got was one that didn’t make any kind of promise of customization or that it would contain “the most” relevant legislation for anything; the ones that did require that kind of analysis were the ones that got hung up in multi-hour deadlines and never ultimately delivered. Given how much weight he puts on these claims, I think it’s fair to interrogate and test them.

The articles, or “blog posts” as Josh calls them, are a slightly different situation. There are a TON of them, and they are all published without dates or bylines. But many of those articles have significant errors, both legal and factual, and if someone relied upon them, they could be in big trouble. And while I didn’t actually expect him to answer the question about the subpoena, he opened the door by bragging about it in the first place, as far as I’m concerned. 

(Only two sentences in this screenshot above are completely accurate. I’m not going to represent to you which two they are, because I am not a lawyer.)

This email, to my great disappointment, went completely ignored.

It wasn’t because he was too busy taking down all his bots, either. On Thursday night, I logged back into the site to check, and discovered that all the prompts I had accessed before were still available, save the two that Josh mentioned specifically in his tweet — the defamation demand letter and the divorce settlement. But even those were still being advertised; every “blog post” on the site has a signup teaser in it advertising access to the site’s legal and other services with no sign that they’re inaccessible until after DoNotPay has your money. One particularly egregious blog post that advertised “free legal advice” to those looking for help navigating the immigration process to become American citizens finally pushed me over the edge, and so I pinged Josh to remind him that I was still waiting for my documents and an answer to my questions, let that sit for a bit, and then started another thread about all the ways he was being less than forthright with the truth. In the course of writing that thread, I discovered that I was suddenly banned from the site; not only could I not log in, but any attempts to do so gave me an error message that read merely “something went wrong.”

It took Josh less than an hour to get back to me via DM, informing me that my money had been completely refunded and complaining that I was lying about the “bots” still being up, although he later admitted that only 7 bots had come down in the previous 36 hours (out of an advertised 1,000):

When I told him I had tested them myself and even generated new documents and cases, he demanded “Was your usage authentic?” I responded “It certainly complied with every provision of the Terms of Service.” At this point, Josh disappeared from the conversation.

After this pause had stretched out for a while, something was kind of nagging at me. I went back and looked at that question and answer again and thought “what is it about the Terms of Service that suddenly had him needing to leave immediately?”

By sheer coincidence, someone — not me (at least I don’t think so) — had archived the DoNotPay Terms of Service just about exactly when I started tweeting my thread, so I know exactly what they said then.

It also meant that I could spot exactly what had changed between that time and this one, a mere two hours later: Josh added a clause to the TOS prohibiting users from testing the website prior to using it in a live dispute. 

If you can’t see that, right after I told him I was not violating his terms, he appeared to add this to his terms:

You represent that any dispute or request submitted is an authentic problem you are having. You are responsible for any damages to DoNotPay or others from fake, inauthentic or test disputes.

So now “test” disputes violate the terms?

This change was made after he banned me, without any warning. He claims he told me to “keep it real,” but he absolutely did not, and his claim that I “triggered his anti-spam” by making 10 or 15 new cases seems to run against his site’s promises that one can create an “unlimited number of documents.”  

He didn’t answer my questions outside of saying “no the letters aren’t being hand typed out and no we didn’t write them,” which… didn’t answer my questions in the slightest. And then he blocked me.

So there you go. Joshua Browder, CEO of DoNotPay.com, would rather block me, ban my account, retcon his terms of service to disallow any test usage at all, and claim to pull out of the “Legal Services” industry that his site is PLASTERED with branding for, rather than show me the two documents I generated and tried to buy.

I wonder what he doesn’t want me to see?

Posted on Techdirt - 24 January 2023 @ 03:29pm

The World’s First Robot Lawyer Isn’t A Lawyer, And I’m Not Sure It’s Even A Robot

Note: This post is an adaptation of what started initially as a Twitter thread.

I’ve been going pretty hard on DoNotPay and its founder/CEO Joshua Browder for the past couple of days, and I’ve had a lot of people defending the service, saying that it could be a real boon to those who can’t otherwise afford legal aid.

So, I thought maybe I should give it a fair shake — after all, I’m mostly arguing with what my idea of a “legal AI” is, right? So I signed up for an account at DoNotPay and took the service for a little whirl. There’s no option to test out the real-time AI, or at least not one that I could find. But the site does offer a dazzling array of services under the category “legal tools.” I used the site to undertake three different tasks: Defamation Demand Letter, Divorce Settlement Agreement, and Sue Anyone in Small Claims Court.

Those of you who know me will be unsurprised to learn that I started with “Defamation Demand Letter.” I was a little taken aback by the description “File a demand letter” — you don’t file a demand letter, you just send it — but decided to press on.

The site leads you through a fairly basic set of question and answer prompts, asking you to identify the potential defendant and the defamatory statements, state the basis for their falsity, explain how they damaged you, etc.

After I filled in the prompts, which required my full name, address, and phone number, I pressed “next,” eager to see the “expertly drafted demand letter” DoNotPay generated on my behalf with the “most relevant state legislation regarding defamation” based on my location.

Instead, I got a little progress bar, informing me that my defamation demand letter would be ready in an hour.

That . . . seems a little slow for something that is supposed to be able to respond to a judge and give instructions in real time. But whatever, let’s press on.

I went back to the front page of the site to select “Divorce Settlement Agreement,” observing in passing that DoNotPay describes itself in full color as “the home of the world’s first robot lawyer” and promises the ability to “sue anyone at the press of a button.”

Once again, I stepped through my various prompts, giving information like my address, my putative soon-to-be-ex’s address, our comparative incomes, the number of hypothetical children we had, etc. so that I could get “the fair terms [I] deserve.”

This time, after I was done, I got a little “task progress” bar that said that my divorce settlement would be ready in EIGHT hours. Y’all, eight hours seems like a really, really long time for an AI to need to generate a document. But, whatever, we’re giving it a fair shake.

Time to Sue Anyone in Small Claims Court! I typed “Sue Now” into the search bar and was greeted with the following prompt:

This gave me significant pause. First of all, it says “I’m owed $500” at the top, but I haven’t entered any specific information yet. Second, “generating court filings” and giving people “a script to read in court” is… I mean, that’s the practice of law. It just is. Third, though, just from a UX perspective, the prompt ends with a yes or no question (“Are you ready to proceed?”) but has a text box which will only accept a dollar amount. What am I supposed to enter here?

So I entered a dollar amount and pressed “Enter.”

The next page presented what I thought was an odd question. Why would I have received a demand letter? I’m the one suing, right?

(Later, from context clues, I managed to figure out that what they meant was if I had already generated a demand letter with DoNotPay. I guess you can only use this service to create filing documents if you have already used it for a demand letter. But I digress.)

Another UX note — I originally found this prompt from an article entitled “Sue Anyone for Assault in Small Claims Court,” but “Assault” isn’t one of the options listed here. So I figure, go with what you know.

Ooooookay. I… this is not how I would ever phrase this question. But I’m not a lawyer at all, never mind the World’s First Robot Lawyer. Let’s press on.

Fortunately, these are the only types of contracts where you could ever have a breach and be owed money damages, right? Right?

At this point in the process, I seem to suddenly have been switched from drafting a lawsuit to drafting another demand letter. Not sure when that happened — the service certainly didn’t notify me about it.

But in for a penny, in for a pound. As requested, I selected all the appropriate dates for my hypothetical breach of contract: contract formation on 9/6/2019, due date on 12/6/2019, most recent request for payment on 11/11/2022, final due date of 1/31/2023. As requested, I gave details of what services I had theoretically performed.

. . . and got this extremely puzzling prompt in reply. Photographic evidence? What? At no point did the service ask me to provide any documentary evidence or give any terms of the contract other than the date the payment was due.

And HEY PRESTO! This time, no timer, no progress bar, just an instant PDF or Word demand letter to download!

The PDF document had all my personal information in it and was somehow in a format that could not be redacted. I’ve never seen this behavior before — it persisted through all my tricks like saving it as an archivable PDF and everything.

The author of the PDF was listed as some guy named Platon Konstantinos Mazarakis.

Out of curiosity, I did the zip trick on the Word file so that I could look at the properties without opening it. Interestingly, it has some custom XML in it; I’m not much of an XML expert, so I asked software engineer Debbie Mia to take a look at it. The docprops/core.xml file has Mazarakis as the document author once again but also shows that this Word document, generated by the World’s First Robot Lawyer, was last edited by “Denise.”

In order to get the PDF into a redactable format, I had to export it to TIFF and then reconvert it. There’s a big loss in quality, unfortunately. But let’s take a look at this demand letter, generated for me when I asked to sue someone in small claims court!

There is literally nothing AI about this. This is a straight-up plug-and-chug document wizard, and it is not well done at all.

I will admit, I threw it a couple of curve balls, like placing my defendant in Canada. But it didn’t attempt to do any kind of jurisdictional analysis at all — I could have put him on the moon and it wouldn’t have mattered. There is no date on the first page of the letter.

There was no widow or orphan control. After my “signature” there’s a space for “Additional Remarks,” which it never prompted me for. There’s also a random orphaned date in the second page header.

Did the contract say I would be paid for OFFERING the following services, or for PERFORMING them?

In addition to there being zero jurisdictional analysis, there’s also zero thought to engage with the fact that depending on the choice of law in the contract (which the prompt also didn’t ask me about) this action may well already be time-barred.

Or with the fact that Small Claims Courts have an upper limit on the amount you can claim, and I’m pretty sure $17K exceeds it. And, yes, that fucking orphaned “Sincerely,” is still killing me.

Also also? The prompt never asked whether I was amenable to a payment plan. Just went ahead and hucked that in there on its own. (Also never asked me about the interest rate, I have no idea where it got that.)

Let me be clear: this is a terrible demand letter. Absolutely terrible. Useless or worse than useless — if an actual attorney saw this, she would instantly know that the sender was unsophisticated, unrepresented, and gullible as fuck.

For this service, DoNotPay charged me $36 ($18/month and it charges you for two months at a time). But hey, there’s an unlimited number of documents we can generate, right? Let’s go see how the others are doing!

Turns out… not so great, Bob. The minute the defamation letter hit its one-hour limit, it flipped over to a little clock icon and said it would need more time. Same for the divorce settlement’s eight-hour limit. That was two days ago.

No updates since then. No contact from the company, no way to generate a support ticket, no hint as to what the trouble is. Just a cheerful, brightly colored promise that it’s doing its best.

I have literally no way to know what the fuck is actually going on here, but I can think of two likely options. The first is that the whole tool is just fucking broken, and Joshua Browder is scamming people out of almost $20 a month for a service that simply does not work. The second, though — and I find this much more likely based on the one-hour and eight-hour timelines given — is that this isn’t AI at all; DoNotPay collects the information from the prompt and then hands it to a human to go find the relevant law and customize the doc.

That would explain why the defamation demand letter gave me a one-hour deadline while the breach of contract letter happened instantly. Remember, the defamation demand SPECIFICALLY promises the “most relevant state legislation” “based on your location.”

It doesn’t take an AI an hour to look up the relevant law based on a physical address. Westlaw can do it… well, I was going to say instantly, but anyone who has used Westlaw knows that’s a lie. But Westlaw can do it in thirty seconds. If there are human beings doing the Googling and making the decision as to what the “most relevant state legislation” is for a defamation action (and, let me tell you, there’s a sign of a pretty glaring error just in that phrase on its own) then any argument that this isn’t unauthorized practice of law (UPL) goes straight out the fucking window.

But hey, maybe those aren’t the only two options! Maybe Browder recognized my usage of the site — after all, I did sign up for it with my real name, which I also tweet under — and panic-blocked me because he was salty that I think his idea is dumb. Or maybe there’s some other perfectly innocent reason why this brand-new revolutionary technology that is getting its debut in a live courtroom setting in less than a month has been wedged for almost 48 hours with no updates or signs of life.

Regardless, though, I don’t think any of the lawyers I know have any cause to fear for their jobs any time soon.

P.S. Browder, who had blocked me a few days ago, unblocked and followed me today!

But apparently only for long enough to promise to listen to feedback, and then to block me again. So, you know, good talk etc.

Awww, and the robot blocked me too!

Update: Josh Browder unblocked me and messaged me again, explaining that my account was automatically flagged because my usage “didn’t seem like authentic activity” and informing me that he had refunded my subscription. I asked him if I could see the two other documents I had already generated or unlock my account so that I could generate similar documents; he said he would try to make them available by tomorrow afternoon (January 25th), explaining that “[t]he engineer who understands the blocking code is out until 12 or so.”

Josh also stressed that “the letters aren’t being typed out by hand and in general are all generated instantly 😂.” Although I did not claim, nor did I intend to suggest, that the documents on DoNotPay.com were typed by hand, I have included his statement here in the spirit of good faith and full disclosure.