Lawyer Ranking Site Realizes That It Might Need A Better Ranking System
from the for-example... dept
You may recall that recently there was a class action suit against a new website that claimed to “rank” lawyers based on a variety of factors. The rankings were quite controversial for pretty good reasons. It wasn’t at all clear how some were ranked, and seeing some Supreme Court Justices with low rankings raised plenty of questions. The company has now said it’s revamping its ranking system, saying it will drop the controversial numerical ranking on lawyers it hasn’t collected much info on (such as Supreme Court Justices?). Amusingly (and not very convincingly), the company claims that the changes have nothing to do with the lawsuit, along with some blather about how such sites need to stay “dynamic.” Of course, it’s still not clear what’s illegal about the site. It’s simply compiling publicly available data and then coming up with a rank — the same thing that Google does for websites (instead of lawyers). And, as we’ve seen from a variety of lawsuits, you can’t sue just because you don’t like your ranking, since that ranking is considered an opinion.
Comments on “Lawyer Ranking Site Realizes That It Might Need A Better Ranking System”
Why a ranking?
That’s part of what’s wrong in this country today. People trying to quantify every detail into an easy to look at number (like credit reports) in order to save themselves the effort of looking up info.
I’m all for collecting statistical data but the problems start when you try to put all those different numbers together into a single all telling score.
Re: Why a ranking?
Have you actually looked at a credit report? The ‘score’ is the only ‘easy-to-look-at number’ on the whole report. Everything else is compiled data about your credit history.
Re: Re: Why a ranking?
The ‘score’ is the only ‘easy-to-look-at number’ on the whole report
The score is exactly what I was talking about. I guess I should have said “score” instead of “report”
My Ranking
I ran myself through that site and came up with a ranking of 7.1, better than several of those Supreme Court justices that the article mentions.
And here’s the clincher– I’m not even an active member of the bar any more. I went inactive almost ten years ago when I went into federal law enforcement.
Seems like the site treats no data on a lawyer as a positive so if you’re like me and haven’t even been actively practicing, you get a high score.
Is the lawyer ranking site is just a VC game?
Look at the amount of money that bas been poured into this lawyer ranking website – for something so inherently controversial and risky. Of course they knew the risks – I mean, suing is what lawyers do for a living! Which make me think that this is just a VC game. If they can win the first couple of lawsuits, set precedents to prevent further lawsuits, then they might actually have a decent business model. If not, it is not too bad – they haven’t really spent a lot of money.
Add an Error Estimate
There’s nothing inherently wrong with giving a ranking based on limited information–statisticians do it all the time. But you do have to give an explicit indication of the amount of uncertainty in the estimate: less information means greater uncertainty. In a chart plot, these uncertainties are shown in the form of error bars.
If their rankings included such uncertainty estimates, that would give you a much better idea of how seriously to take them.