If you only pay for one legal research platform, here's what you're not seeing

I started in digital marketing in 1999, when the whole job was figuring out why one page showed up first and another didn’t. The lesson stuck. A ranking is not a fact about the world. It’s somebody’s decision about what you should see first.

Legal research has the same problem, and almost nobody talks about it.

In 2017, Susan Nevelow Mart ran 50 searches on six legal databases and compared the top ten results from each. Her finding was uncomfortable. The databases returned mostly different cases for the same question. Between 36% and 43% of each platform’s results were unique to that platform. She called it “every algorithm has a POV”.

That was ten years ago. Search got more powerful since then, and every vendor now sells an AI layer on top. So I wanted to know if the gap closed.

It didn’t close. By my count it got wider.

What I did

I took 49 of her 50 searches (I skipped one that mixes three jurisdictions) and ran each on Westlaw, Lexis, Google Scholar and CourtListener, with the same courts and the same top-ten cutoff she used. I did this in October 2026, with AI assistance running the searches and coding relevance against her codebook.

Finding the cases is the easy part. The hard part is deciding whether each one actually answers the legal question. In research studies, the people who do that are called coders. Mart’s coders read each case and rated how likely it was to be relevant, following written rules she’d set out in a codebook.

I’m not a lawyer, so I didn’t want my own judgment deciding what counts as relevant. Instead I gave an AI the same job. For each case it got the legal question, an excerpt of the opinion and her codebook. It was never told which platform returned the case or where it ranked, so it couldn’t play favorites.

To see if that was reasonable, I had it rate 314 cases her human coders had already rated in 2016. It made the same call as they did on 80% of them. It runs about nine points more generous, calling more cases relevant than they did. That means you shouldn’t compare my percentages to hers from 2016. Compare the platforms to each other.

This is a replication with a small sample. It’s not a ranking of products.

What came back

Westlaw and Lexis shared only 16% of the distinct cases in their top tens. In 2016 it was 24%. On 13 of the 49 searches they had no case in common at all. In 2016 that happened three times.

Relevance, by platform: Westlaw 84%, Lexis 74%, Google Scholar 65%, CourtListener 49%.

Westlaw’s lead over Lexis was about ten points. Her coders found a lead of about the same size in 2016 (68% to 57%), so if you were hoping this had settled by now, it hasn’t.

Then the number I keep thinking about. Across all four platforms there were 1,391 distinct cases. 81% of them came from only one platform. Of those, 554 were relevant.

So whichever platform you pay for, the others had relevant cases yours didn’t show you.

Is it just noise?

I wondered that too. I reran ten of the searches on Westlaw and Lexis four days later. 168 of 171 top-ten slots were unchanged. Westlaw returned the same ten cases in the same order on every search I repeated. Lexis was nearly as steady. So the differences between platforms are real and they aren’t random.

What a solo should do with this

If you’re a one-person shop, you probably pay for one platform and treat it as the law. That’s reasonable. You can’t afford two.

But it means your results are one algorithm’s point of view, and the cases it left out aren’t flagged anywhere. The fix is cheap. Run your key search a second time on a free source. Google Scholar and CourtListener cost nothing, and in this study they were the only place 324 of those 554 relevant cases turned up, more than Westlaw and Lexis combined found on their own (230). Then read the opinions yourself before you cite them. A ranking tells you where to start looking, not where to stop.

That’s also how I think about AI in a law practice generally. It can do a lot of the legwork. Everything it produces is a draft, and a lawyer checks it against the source.

What I don’t know

Some of the Lexis result sets were small, which can push overlap down, and I compared courts the way Mart did rather than every possible filter, so treat the exact percentages as close, not precise. I don’t know how much of the gap comes from the platforms’ search engines and how much from how each one defines its database. I don’t know whether the AI research tools both vendors now sell narrow the gap or widen it, because this study used plain search. And I don’t know what a lawyer who does this for a living would add to the relevance calls, though I’d like to find out.

I’m going to keep running pieces of this, including how the platforms’ own citation counts compare. If you want the next one, there’s a signup below.