Fix bugs in RBM notebooks #1581

pradnyeshjoshi · 2021-12-13T19:04:58Z

Description

RBM algorithm in its current implementation requires ratings to be integers. Added code to map float ratings to integers in RBM quick start and deep dive notebooks.
Minor latex change in RBM deep dive notebook so that the equations render properly.
Disabled TF warnings in RBM deep dive notebook.
Previously the batch size for precision calculation, which is a part of the training loop, was hardcoded and equal to the number of users (which caused OOM issue during model fitting for 10M and 20M movielens datasets). Changing this batch size resolves the OOM error during model fitting.

Related Issues

Checklist:

I have followed the contribution guidelines and code style for this project.
I have added tests covering my contributions.
I have updated the documentation accordingly.
This PR is being made to staging branch and not to main branch.

review-notebook-app · 2021-12-13T19:05:02Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

codecov-commenter · 2021-12-13T20:01:54Z

Codecov Report

Merging #1581 (593c2d6) into staging (dc3455d) will decrease coverage by 0.01%.
The diff coverage is n/a.

❗ Current head 593c2d6 differs from pull request most recent head af4fb2a. Consider uploading reports for the commit af4fb2a to get more accurate results

@@             Coverage Diff             @@
##           staging    #1581      +/-   ##
===========================================
- Coverage    61.89%   61.88%   -0.02%     
===========================================
  Files           84       84              
  Lines         8458     8458              
===========================================
- Hits          5235     5234       -1     
- Misses        3223     3224       +1

Flag	Coverage Δ
pr-gate	`61.88% <ø> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
recommenders/models/rbm/rbm.py	`73.18% <ø> (ø)`
recommenders/evaluation/spark_evaluation.py	`86.60% <0.00%> (-0.45%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca70a97...af4fb2a. Read the comment docs.

miguelgfierro · 2021-12-14T09:17:20Z

examples/00_quick_start/rbm_movielens.ipynb

@@ -60,7 +60,7 @@
      "System version: 3.7.11 (default, Jul 27 2021, 14:32:16) \n",
      "[GCC 7.5.0]\n",
      "Pandas version: 1.3.4\n",
-      "Tensorflow version: 2.6.1\n"
+      "Tensorflow version: 1.15.5\n"


hey @pradjoshi can you merge the latest stating to your branch, reinstall using TF 2 and rereun again?

hi @miguelgfierro I have reinstalled recommenders using TF2 and rerun the notebooks.

…l fitting

review-notebook-app · 2021-12-15T08:54:38Z

View / edit / reply to this conversation on ReviewNB

miguelgfierro commented on 2021-12-15T08:54:38Z
----------------------------------------------------------------

Same comment as before with the verbosity

pradnyeshjoshi commented on 2021-12-15T16:54:45Z
----------------------------------------------------------------

Andreas resolved this.

review-notebook-app · 2021-12-15T08:54:39Z

View / edit / reply to this conversation on ReviewNB

miguelgfierro commented on 2021-12-15T08:54:38Z
----------------------------------------------------------------

we need to revisit the change in metrics, maybe in a different PR. Is there an issue to track this?

pradnyeshjoshi commented on 2021-12-15T16:54:14Z
----------------------------------------------------------------

I have created a task for it. Should I create an issue on GitHub instead?

review-notebook-app · 2021-12-15T08:54:40Z

View / edit / reply to this conversation on ReviewNB

miguelgfierro commented on 2021-12-15T08:54:39Z
----------------------------------------------------------------

Line #41.    print("Pandas version: {}".format(pd.__version__))

maybe here it would be good to also add tf version

pradnyeshjoshi commented on 2021-12-15T16:58:41Z
----------------------------------------------------------------

Done.

review-notebook-app · 2021-12-15T08:54:40Z

View / edit / reply to this conversation on ReviewNB

miguelgfierro commented on 2021-12-15T08:54:40Z
----------------------------------------------------------------

This is super verbose. Ideally what we want is just the old output, with the metrics loop:

training epoch 0 rmse 0.961369 
training epoch 10 rmse 0.902065 
training epoch 20 rmse 0.864718 
training epoch 30 rmse 0.848415 
done training, Training time 10.9566452 
Train set accuracy 0.3561482 
Test set accuracy 0.3516242

It looks the command to show only the errors is not working:

tf.get_logger().setLevel('ERROR') # only show error messages

Maybe there is a new command in TF. If you can't find it, it is better to manually erase the cell

pradnyeshjoshi commented on 2021-12-15T16:52:10Z
----------------------------------------------------------------

Looks like Andreas resolved this already!

miguelgfierro

@pradnyeshjoshi to me this looks ok to go, after the minor comments on logs.

@anargyri do you want to include this in the release?

anargyri · 2021-12-15T08:59:25Z

@pradnyeshjoshi to me this looks ok to go, after the minor comments on logs.

@anargyri do you want to include this in the release?

I don't know, maybe if there is time. First we need to resolve the tests that fail in the release.

anargyri · 2021-12-15T10:07:13Z

View / edit / reply to this conversation on ReviewNB

miguelgfierro commented on 2021-12-15T08:54:38Z ----------------------------------------------------------------

Same comment as before with the verbosity

Yes, this is strange. Other algos like deeprec don't have this amount of output. Maybe you could check what the difference is from those.

anargyri · 2021-12-15T16:18:12Z

View / edit / reply to this conversation on ReviewNB
miguelgfierro commented on 2021-12-15T08:54:38Z ----------------------------------------------------------------
Same comment as before with the verbosity

Yes, this is strange. Other algos like deeprec don't have this amount of output. Maybe you could check what the difference is from those.

I think I fixed it according to tensorflow/tensorflow#18146

…s into pradjoshi/test_rbm

pradnyeshjoshi · 2021-12-15T16:52:11Z

Looks like Andreas resolved this already!

View entire conversation on ReviewNB

pradnyeshjoshi · 2021-12-15T16:54:15Z

I have created a task for it. Should I create an issue on GitHub instead?

View entire conversation on ReviewNB

pradnyeshjoshi · 2021-12-15T16:54:46Z

Andreas resolved this.

View entire conversation on ReviewNB

pradnyeshjoshi · 2021-12-15T16:58:41Z

Done.

View entire conversation on ReviewNB

anargyri · 2021-12-15T17:00:28Z

I have created a task for it. Should I create an issue on GitHub instead?

View entire conversation on ReviewNB

The task should be enough.

Ubuntu and others added 5 commits December 3, 2021 16:33

map float ratings in 1movielens dataset to integers from 1 to K

9aadd53

fixed issue with gen_affinity_matrix return type

73685fa

minor fixes in rbm notebooks

84616f3

rerun rbm deep dive notebook

24079a8

disable tf warnings

8f85daf

pradnyeshjoshi requested review from anargyri, gramhagen, loomlike, miguelgfierro and wutaomsft as code owners December 13, 2021 19:04

Merge branch 'staging' into pradjoshi/test_rbm

94d3a47

pradnyeshjoshi self-assigned this Dec 13, 2021

pradnyeshjoshi changed the title ~~Pradjoshi/test rbm~~ Fix bugs in RBM notebooks Dec 13, 2021

miguelgfierro reviewed Dec 14, 2021

View reviewed changes

pradnyeshjoshi added 2 commits December 14, 2021 19:41

reran rbm notebooks with TF2

303cd82

change batch size for precision calc to resolve OOM error during mode…

c6361bd

…l fitting

miguelgfierro approved these changes Dec 15, 2021

View reviewed changes

Merge branch 'staging' into pradjoshi/test_rbm

7218a3a

anargyri approved these changes Dec 15, 2021

View reviewed changes

anargyri and others added 2 commits December 15, 2021 16:17

Disable excessive log messages

593c2d6

Merge branch 'staging' into pradjoshi/test_rbm

18c5471

anargyri added 2 commits December 15, 2021 16:41

Rerun notebooks

7f6c079

Merge branch 'pradjoshi/test_rbm' of github.com:microsoft/recommender…

6cd52a4

…s into pradjoshi/test_rbm

Merge branch 'staging' into pradjoshi/test_rbm

af4fb2a

anargyri merged commit ba24adc into staging Dec 15, 2021

miguelgfierro deleted the pradjoshi/test_rbm branch December 15, 2021 20:12

pradnyeshjoshi mentioned this pull request Dec 16, 2021

Remove explicit mapping of ratings to integers from RBM notebooks #1585

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix bugs in RBM notebooks #1581

Fix bugs in RBM notebooks #1581

pradnyeshjoshi commented Dec 13, 2021 •

edited by anargyri

Loading

review-notebook-app bot commented Dec 13, 2021

codecov-commenter commented Dec 13, 2021 •

edited

Loading

miguelgfierro Dec 14, 2021

pradnyeshjoshi Dec 14, 2021

review-notebook-app bot commented Dec 15, 2021 •

edited

Loading

review-notebook-app bot commented Dec 15, 2021 •

edited

Loading

review-notebook-app bot commented Dec 15, 2021 •

edited

Loading

review-notebook-app bot commented Dec 15, 2021 •

edited

Loading

miguelgfierro left a comment

anargyri commented Dec 15, 2021

anargyri commented Dec 15, 2021

anargyri commented Dec 15, 2021

pradnyeshjoshi commented Dec 15, 2021

pradnyeshjoshi commented Dec 15, 2021

pradnyeshjoshi commented Dec 15, 2021

pradnyeshjoshi commented Dec 15, 2021

anargyri commented Dec 15, 2021

Fix bugs in RBM notebooks #1581

Fix bugs in RBM notebooks #1581

Conversation

pradnyeshjoshi commented Dec 13, 2021 • edited by anargyri Loading

Description

Related Issues

Checklist:

review-notebook-app bot commented Dec 13, 2021

codecov-commenter commented Dec 13, 2021 • edited Loading

Codecov Report

miguelgfierro Dec 14, 2021

Choose a reason for hiding this comment

pradnyeshjoshi Dec 14, 2021

Choose a reason for hiding this comment

review-notebook-app bot commented Dec 15, 2021 • edited Loading

review-notebook-app bot commented Dec 15, 2021 • edited Loading

review-notebook-app bot commented Dec 15, 2021 • edited Loading

review-notebook-app bot commented Dec 15, 2021 • edited Loading

miguelgfierro left a comment

Choose a reason for hiding this comment

anargyri commented Dec 15, 2021

anargyri commented Dec 15, 2021

anargyri commented Dec 15, 2021

pradnyeshjoshi commented Dec 15, 2021

pradnyeshjoshi commented Dec 15, 2021

pradnyeshjoshi commented Dec 15, 2021

pradnyeshjoshi commented Dec 15, 2021

anargyri commented Dec 15, 2021

pradnyeshjoshi commented Dec 13, 2021 •

edited by anargyri

Loading

codecov-commenter commented Dec 13, 2021 •

edited

Loading

review-notebook-app bot commented Dec 15, 2021 •

edited

Loading

review-notebook-app bot commented Dec 15, 2021 •

edited

Loading

review-notebook-app bot commented Dec 15, 2021 •

edited

Loading

review-notebook-app bot commented Dec 15, 2021 •

edited

Loading