Book Recommendation Engine using KNN: getting wrong results

Python
#1

Heya,

I’m struggling to get the expected test numbers on the Book Recommendation Machine Learning Challenge .

I deleted all ratings from users with less than 200 ratings and all ratings from books with less than 100 ratings. Then I pass the remaining ratings into a csr_matrix where I replace missing values with 0. When I pass this into my NearestNeighbour-fit, I’m getting other results though. It gives me recommendations, but except for 2 books, they are not the expected ones and the distances are off by magnitudes. Could it be that the dataset changed as well?

These are the recommendations and distances I get for the test:

[
  "Where the Heart Is (Oprah's Book Club (Paperback))", 
  [
    ['The Surgeon', 61.286213], 
    ['Unspeakable', 61.522354],
    ['The Perks of Being a Wallflower', 61.579216], 
    ['Gap Creek: The Story Of A Marriage', 61.676575], 
    ['The Weight of Water', 61.75759]
  ]
]

Down below is the link to my google colab, would be thankful for any feedback :slight_smile:

#2

Hello Spreeni,

This might not help much, I faced similar issues., but got different distances.
I used ‘cosine’ as a metric, instead of ‘euclidean’ or ‘minkowski’

neighbours = NearestNeighbors(metric = 'cosine',n_neighbors=5, algorithm='auto')

That give me distance between –1 and 1

[
   "Where the Heart Is (Oprah's Book Club (Paperback))", 
    [
        ["The Lovely Bones: A Novel", 0.7230184], 
        ["The Pilot's Wife : A Novel", 0.81926787], 
        ["The Joy Luck Club", 0.81986046], 
        ["The Notebook', 0.823043]
    ]
]

Although I’m not able to get the same books.

#3

I also haven’t been able to get the same books, but I don’t believe using .fillna(0) is proper since some ratings are actually zero.

I tried -99999 and -1 and it still doesn’t match.

I’d be interested in the correct approach if someone figures it out in the future…

#4

I am sorry to inform you that the tests in this task are faulty. You have to twist and turn your data to get the correct answer.

There is another thread which solves this issue. The link to the thread is - Book Recommendation KNN System

#5

i havent get it how could we do that? can you explain?