How to Objectively Compare Two Ranked Lists in Python
A simplified explanation and implementation of Rank Biased Overlap
Imagine you and your friend have both watched all 8 Harry Potter movies.
But there’s a catch — you have watched each movie the day it was released, without missing a single premier.
Your friend, however, watched the 2nd movie first, then the 4th and 5th, and then binge-watched the rest when it was available on Netflix.
Theoretically, you and your friend are on an equal footing — both have watched all the movies of the series.
Is it really equal though?
You being a true fan, can never consider it equal.
How could someone not watch the movies of a book series, In. A. Sequence?!
You consider it a sacrilege!
The good news is — the math is on your side. You can boil this problem down into a ranked list comparison.
There are several methods to compare lists.
In this post, you will:
- Understand why we actually need a ranked list comparison
- See the requirements a comparison measure should satisfy
- Get an overview of a method called Rank Biased Overlap
- Understand the mathematical approach behind it
- Create a simple implementation in python
At the end of it, you will finally settle the disagreement with your friend once and for all:
You != Your Friend
Why do you need a ranked list comparison?
Apart from the above movie viewing order disagreement, we are surrounded by examples of comparing lists!
In fact, we all do such comparisons all the time.
- Which results are better for the same query on Google vs. Bing?
- How similar is the New York Times bestseller list to that from USA Today?
