cheetm/lightfm_deep_dive

[Knostromo]

Description

Added LightFM deep dive with accompanying utils

Related Issues

#1097

Checklist:

• I have followed the contribution guidelines and code style for this project.
• I have added tests covering my contributions.
• I have updated the documentation accordingly.
• This PR is being made to staging and not master.

[review-notebook-app]

Check out this pull request on 

You'll be able to see Jupyter notebook diff and discuss changes. Powered by ReviewNB.

[msftclas]

All CLA requirements met.

[miguelgfierro]

@cheetm this looks really good, I added some comments. It would be great if you could connect with Nile, she has already a lot of experience on the repo

[miguelgfierro]

@niwilso I'm trying to add you as reviewer but for some reason I can't. Feel free to add comments and suggestions as well

[anargyri]

Hey @cheetm this PR is very nice. Thanks for the great work!
My main comment is that we follow the practice of using the metrics computations from this repo, instead of the ones from lightfm.evaluation. This allows for consistency and comparisons between different algorithms and frameworks.

[Knostromo]

Hey @cheetm this PR is very nice. Thanks for the great work!
My main comment is that we follow the practice of using the metrics computations from this repo, instead of the ones from lightfm.evaluation. This allows for consistency and comparisons between different algorithms and frameworks.

Hi @anargyri thanks for your comments.
Perhaps it is not clear from the notebook, all the evaluation metrics were computed using the repo's methods up to section 3. In section 4 the progress were tracked and measured using LightFM's method because they are significantly more efficient. Furthermore, Section 4 is purely to demonstrate how one can track the model fitting process so it shouldn't be critical.

You can use the output from cell 18 and/or cell 32 for comparison with other algorithms.

Hope thats okay.
Thanks

[miguelgfierro]

@cheetm about this:
In section 4 the progress were tracked and measured using LightFM's method because they are significantly more efficient.

• do you have some metrics of the difference?
• do you know where the speed up comes from and/or how to improve our current metrics?

FYI @yueguoguo

[miguelgfierro]

really good job

[Knostromo]

@cheetm about this:
In section 4 the progress were tracked and measured using LightFM's method because they are significantly more efficient.

• do you have some metrics of the difference?
• do you know where the speed up comes from and/or how to improve our current metrics?

FYI @yueguoguo

I have added section 3.6 to demonstrate the difference (albeit very naively), the difference is so great I don't think we need to resort to timeit.
Also note that if I were to use the repo's metrics, each epoch in section 4 would require ~13 minutes to compute which is why I've decided to use the LightFM's methods.

I haven't got time to properly investigate your second point, but I am guessing if you have already fitted the latent matrices, the scores can be retrieved much more efficiently all at once instead of individually.