Matching Users Directly to Other Users
To complement the present system could we match users to each other directly. The idea is this, instead of predicting the ratings directly, what you would do is use a best match function of your own ratings with those of another user – you are essentially trying to find your other self. The key advantage of this method is possibly finding hidden gems – a lot of the time my predicted films are ones I know about already.
One problem with ratings is that some people’s ratings are generally cardinally higher than others which could skew this somewhat, as could their standard deviation – the lowest mark I have ever given anything is 2 stars and have only given one film 5 stars.
As such you could calculate the median rating of each user then calculate how far away the actual rating is. Then to standardise this across the board by declaring everyone’s median as 5 then adding and subtracting accordingly (there is probably a more sophisticated way to do this using standard deviations/variances but it alludes me at present)
e.g. My rating of the Trial is 10. My median rating is 7. So it’s plus 3. With a standardised median of 5 it will give me a rating of 8.
Using this you could best match users. You could further divide it via tags so you could see that a particular user is overall my best match but rates horrors much more differently than I. So when looking for horror films you can best match just in that genre.
When seeing a user you could then see the actual (though adjusted) ratings of films of an individual and browse the ones he’s rated but you haven’t.
This is probably a lot of work (and my way might not be best) but I think it would be an excellent function.
Hi Physiocrat,
The algorithm that you describe is actually the grandfather of recommendation algorithms, known as “user-user collaborative filtering”. This algorithm has lots of nice properties, especially because it is such an intuitive and interesting approach to the problem of finding new content, but its performance characteristics make it unsuitable for systems like movielens.
Your interface ideas that build on the intuition are really interesting, and there are some fun projects in there (especially combining user-user with tag information).