Add tabular regression example #254

lazarust · 2022-12-15T00:57:00Z

No description provided.

adrinjalali · 2022-12-15T08:45:53Z

I'm okay with adding this, but it'd need to add a few things for the example to be more useful:

use skops format skops.io instead of pickle
Have a story behind the example. It should be something where we could also explain limitations of the model, and disparities if they exist, to highlight how the model card should be used and written.
Add a more detailed model card, with all sections filled and add a few different inspection and interpretation methods to better explain the model

Thanks for the work @lazarust

lazarust · 2022-12-16T00:22:04Z

Hi @adrinjalali, thanks for taking a look at my PR! I've addressed your first point and replaced using pickle with skops.io. I did have some questions about your other two issues though. I tweaked the model text and comments but was unsure what sections I was missing. I based this example off of examples/plot_text_classification.py. If you could let me know what else you want to be added I can go ahead and add it and update the text classification example too. Thanks!

adrinjalali

Thanks,

This should also add the generating file the same way as it's done here:

skops/examples/plot_hf_hub.py

Lines 85 to 87 in 495abe6

    
           if "__file__" in locals():  # __file__ not defined during docs built 
        
               # Add this script itself to the files to be uploaded for reproducibility 
        
               hub_utils.add_files(__file__, dst=local_repo)

adrinjalali · 2023-01-02T11:18:10Z

examples/plot_tabular_regression.py

+# Inference
+# =========
+# Let's see if the model works.
+prediction_data = [[100, 200, 300, 400, 500, 600, 700, 800, 900, 1000]]


this should use a few rows from X_test instead.

adrinjalali · 2023-01-02T11:18:56Z

examples/plot_tabular_regression.py

+# to push your models please refer to
+# :ref:`this example <sphx_glr_auto_examples_plot_hf_hub.py>`.
+
+model_card.save(Path(local_repo) / "README.md")


missing new line at the end of the file.

lazarust · 2023-01-04T01:03:54Z

@adrinjalali I've made those requested changes

merveenoyan

Overall looks good, it's only nits, thank you!!!

examples/plot_tabular_regression.py

merveenoyan · 2023-01-05T06:59:53Z

examples/plot_tabular_regression.py

+    " This model is pretty limited and should just be used as an example of how to user `skops` and Hugging Face Hub."
+)
+model_card_authors = "skops_user, lazarust"
+get_started_code = (


this is generated when use skops template (which is default), see here and here) should we keep it like this for this tutorial? @adrinjalali

no we should remove them now.

examples/plot_tabular_regression.py

lazarust · 2023-01-10T02:33:51Z

@merveenoyan I think I've addressed all of your comments.

merveenoyan

Looks good to me! Thanks a lot!

merveenoyan

Just noticed tests aren't passing :/

merveenoyan · 2023-01-16T10:41:08Z

examples/plot_tabular_regression.py

+model_card_authors = "skops_user, lazarust"
+citation_bibtex = "bibtex\n@inproceedings{...,year={2022}}"
+model_card.add(**{
+    "How to Get Started with the Model": get_started_code,


Suggested change

"How to Get Started with the Model": get_started_code,

You can remove it as discussed 🙂

lazarust · 2023-01-16T21:01:44Z

@merveenoyan Tests should be fixed now. I didn't realize that there was a check for formatting using Black.

merveenoyan · 2023-01-17T11:25:21Z

@lazarust yesterday I saw the workflow failing for docs tests, as in, your script was failing, that's why. All seems good now!

merveenoyan

LGTM!

examples/plot_tabular_regression.py

merveenoyan · 2023-01-18T10:57:40Z

@BenjaminBossan could you skim through it and see if it's ok to merge?

BenjaminBossan

Overall, this looks very good to me, thanks for the work. I have some minor comments, please take a look.

There is also a bigger issue: If I'm not mistaken, this example isn't linked anywhere in the docs. That means it would be really hard for readers to find it, they would have to manually change the URL to https://skops.readthedocs.io/en/stable/auto_examples/ to get there. Probably it would be best if we linked to the auto examples from some place, not necessarily to each auto example individually.

BenjaminBossan · 2023-01-18T11:14:10Z

docs/changes.rst

@@ -83,4 +83,4 @@ Contributors

 :user:`Adrin Jalali <adrinjalali>`, :user:`Merve Noyan <merveenoyan>`,
 :user:`Benjamin Bossan <BenjaminBossan>`, :user:`Ayyuce Demirbas
-<ayyucedemirbas>`, :user:`Prajjwal Mishra <p-mishra1>`
+<ayyucedemirbas>`, :user:`Prajjwal Mishra <p-mishra1>`, :user:`Thomas Lazarus <lazarust>`


Should there not also be an entry (in a new section "v0.5") about what was added? Otherwise, it's hard to see the changes at a glance.

BenjaminBossan · 2023-01-18T11:16:39Z

examples/plot_tabular_regression.py

+prediction_data = X_test[:5]
+prediction = model.predict(prediction_data)
+print(prediction)


Suggested change

prediction_data = X_test[:5]

prediction = model.predict(prediction_data)

print(prediction)

y_pred = model.predict(X_test[:5])

print(y_pred)

This is not a must, but I would prefer we use the canonical y_pred for the output of model.predict (further below, we also use y_pred). I know that in plot_text_classification.py, we also don't use y_pred, but I'd rather change it there as well, than keeping it here.

adrinjalali · 2023-01-19T09:35:16Z

We can have a page where we manually list all examples in well defined categories for better discoverability.

BenjaminBossan · 2023-01-19T09:45:59Z

We can have a page where we manually list all examples in well defined categories for better discoverability.

Isn't that https://skops.readthedocs.io/en/stable/auto_examples/? We just need to link to it somewhere.

adrinjalali · 2023-01-19T11:28:36Z

That file is auto generated and not really nice to read @BenjaminBossan , would be nicer for us to have a file which we maintain instead of linking to this autogenerated one.

lazarust · 2023-01-26T01:54:49Z

@BenjaminBossan I added the page to the documentation, but I don't super love the title and description of the examples. Let me know if you want me to change them!

BenjaminBossan · 2023-01-26T10:49:52Z

docs/examples.rst

@@ -0,0 +1,9 @@
+.. _examples:
+
+Examples of using skops


I think the "of using skops" part is a bit redundant inside of the skops documentation ;)

Suggested change

Examples of using skops

Examples of interactions with the Hugging Face Hub

BenjaminBossan · 2023-01-26T10:53:03Z

docs/examples.rst

+- Tabular Regression:
+    `Here <https://github.com/skops-dev/skops/blob/main/examples/plot_tabular_classification.py>`_ is an example of using skops to serialize a tabular regression model and create a model card and a Hugging Face Hub repository.
+- Text Classification:
+    `Here <https://github.com/skops-dev/skops/blob/main/examples/plot_text_classification.py>`_ is an example of using skops to serialize a text classification model and create a model card and a Hugging Face Hub repository.


I would expect all auto-examples to be listed here:

https://skops--254.org.readthedocs.build/en/254/auto_examples/

At the same time, the paragraph on the start page of the skops page becomes a bit redundant with this new page:

The following examples are good starting points: ...

(link)

Should we remove it in favor of just linking to this page? @adrinjalali @merveenoyan WDYT?

I think that paragraph can stay since we can only link to a few examples where people can start, while also linking to this page with all examples listed.

@adrinjalali I'm a little confused, are we wanting to link from the start page to this examples page?

Yes we do @lazarust

adrinjalali · 2023-01-26T12:24:43Z

docs/examples.rst

@@ -0,0 +1,9 @@
+.. _examples:
+
+Examples of using skops


Suggested change

Examples of using skops

Examples of interactions with the Hugging Face Hub

adrinjalali · 2023-01-26T12:25:00Z

docs/examples.rst

+- Tabular Regression:
+    `Here <https://github.com/skops-dev/skops/blob/main/examples/plot_tabular_classification.py>`_ is an example of using skops to serialize a tabular regression model and create a model card and a Hugging Face Hub repository.
+- Text Classification:
+    `Here <https://github.com/skops-dev/skops/blob/main/examples/plot_text_classification.py>`_ is an example of using skops to serialize a text classification model and create a model card and a Hugging Face Hub repository.


please keep the lines at max 79 chars

adrinjalali · 2023-01-26T12:25:54Z

docs/examples.rst

+- Tabular Regression:
+    `Here <https://github.com/skops-dev/skops/blob/main/examples/plot_tabular_classification.py>`_ is an example of using skops to serialize a tabular regression model and create a model card and a Hugging Face Hub repository.
+- Text Classification:
+    `Here <https://github.com/skops-dev/skops/blob/main/examples/plot_text_classification.py>`_ is an example of using skops to serialize a text classification model and create a model card and a Hugging Face Hub repository.


I think that paragraph can stay since we can only link to a few examples where people can start, while also linking to this page with all examples listed.

lazarust · 2023-01-27T03:34:19Z

@adrinjalali @BenjaminBossan I've addressed your comments other than linking from the start page (if we want to do that).

BenjaminBossan · 2023-01-27T11:18:15Z

docs/examples.rst

+    regression model and create a model card and a Hugging Face Hub repository.
+- Text Classification:
+    `Here <https://github.com/skops-dev/skops/blob/main/examples/plot_text_cl
+    assification.py>`_ is an example of using skops to serialize a text classi


You accidentally split the word "classification" into 2.

BenjaminBossan · 2023-01-27T11:19:07Z

docs/examples.rst

+    _classification.py>`_ is an example of using skops to serialize a tabular 
+    regression model and create a model card and a Hugging Face Hub repository.
+- Text Classification:
+    `Here <https://github.com/skops-dev/skops/blob/main/examples/plot_text_cl


All the links here are to the .py files on GH. However, it is better to point them to the rendered documentation. E.g. for text classification, that would be: https://skops.readthedocs.io/en/stable/auto_examples/plot_text_classification.html

lazarust · 2023-01-28T16:56:00Z

@BenjaminBossan I've made the updates you requested, but it looks like there's an error happening in plot_hf_hub.py when it uploads the model to the Hub. Would you be able to help me figure out why this is happening?

BenjaminBossan · 2023-01-30T10:19:28Z

@lazarust I have no idea, it seems unlikely that any change you made could cause that. I tried to trigger a rebuild of the docs in case the problem was just temporary, but didn't see an option to do that. Could you please push an empty commit to trigger it?

adrinjalali · 2023-01-30T12:08:43Z

I triggered a rebuild on the readthedocs side and it passes

adrinjalali · 2023-01-30T12:09:34Z

@lazarust please avoid force pushing to your branch. We squash and merge at the end, so it doesn't matter how many commits with what commit messages you have in between.

lazarust · 2023-01-31T02:02:23Z

@adrinjalali Should I update the branch via rebase or merge commit? I've also added the link to the index page.

Resolves skops-dev#248 This example showcases how to use custom templates with the skops model card class. The new document is not yet explicitly linked in the documentation yet. Ideally, this can be done once skops-dev#254 has been finalized.

lazarust · 2023-02-08T14:55:31Z

@BenjaminBossan This is ready for you to review again

BenjaminBossan

Just this typo, then we should be good :)

docs/examples.rst

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>

lazarust · 2023-02-08T15:52:42Z

@BenjaminBossan Fixed!

BenjaminBossan

Great, thanks a lot for working on this!

* [skip ci] Add example of using a custom template Resolves #248 This example showcases how to use custom templates with the skops model card class. The new document is not yet explicitly linked in the documentation yet. Ideally, this can be done once #254 has been finalized. * [skip ci] Add entry to changes.rst

lazarust mentioned this pull request Dec 15, 2022

Have example scripts for every task #117

Open

3 tasks

lazarust force-pushed the add-tabular-regression-example branch 2 times, most recently from 05aa485 to 3cde33b Compare December 15, 2022 01:09

lazarust force-pushed the add-tabular-regression-example branch 4 times, most recently from bd88712 to 8dc669e Compare December 20, 2022 15:04

adrinjalali reviewed Jan 2, 2023

View reviewed changes

lazarust force-pushed the add-tabular-regression-example branch from 8dc669e to 86af97e Compare January 4, 2023 00:13

lazarust requested a review from adrinjalali January 4, 2023 01:04

adrinjalali approved these changes Jan 4, 2023

View reviewed changes

adrinjalali requested a review from merveenoyan January 4, 2023 10:14

merveenoyan approved these changes Jan 5, 2023

View reviewed changes

lazarust force-pushed the add-tabular-regression-example branch from 3476e20 to 441c2a5 Compare January 10, 2023 02:27

lazarust force-pushed the add-tabular-regression-example branch from 876ada2 to 7c34274 Compare January 10, 2023 15:32

merveenoyan approved these changes Jan 16, 2023

View reviewed changes

merveenoyan reviewed Jan 16, 2023

View reviewed changes

lazarust force-pushed the add-tabular-regression-example branch from 7c34274 to 2fc3fd5 Compare January 16, 2023 20:56

merveenoyan approved these changes Jan 17, 2023

View reviewed changes

examples/plot_tabular_regression.py Show resolved Hide resolved

BenjaminBossan requested changes Jan 18, 2023

View reviewed changes

Adds Examples to documentation

1ab80e4

lazarust force-pushed the add-tabular-regression-example branch from 35dccf1 to 1ab80e4 Compare January 26, 2023 01:51

BenjaminBossan reviewed Jan 26, 2023

View reviewed changes

adrinjalali reviewed Jan 26, 2023

View reviewed changes

Adds all examples from auto example page

bff9f26

lazarust force-pushed the add-tabular-regression-example branch from d5b8b67 to bff9f26 Compare January 27, 2023 03:31

BenjaminBossan requested changes Jan 27, 2023

View reviewed changes

Updates to link to docs

2ab5941

lazarust force-pushed the add-tabular-regression-example branch from 190c2f3 to 2ab5941 Compare January 28, 2023 16:45

Adds link to more examples

50efed5

lazarust added 2 commits January 30, 2023 20:12

Fixes link

f390b49

Fixes link with custom text

01cf817

BenjaminBossan mentioned this pull request Feb 3, 2023

DOC Add example of using a custom template with model cards #291

Merged

Merge branch 'main' into add-tabular-regression-example

25dc0e6

BenjaminBossan reviewed Feb 8, 2023

View reviewed changes

docs/examples.rst Outdated Show resolved Hide resolved

Update docs/examples.rst

c3c853d

Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>

BenjaminBossan approved these changes Feb 8, 2023

View reviewed changes

BenjaminBossan merged commit b324b8d into skops-dev:main Feb 8, 2023

lazarust deleted the add-tabular-regression-example branch April 7, 2023 01:55

	if "__file__" in locals(): # __file__ not defined during docs built
	# Add this script itself to the files to be uploaded for reproducibility
	hub_utils.add_files(__file__, dst=local_repo)

	Examples of using skops
	Examples of interactions with the Hugging Face Hub

Add tabular regression example #254

Add tabular regression example #254

Conversation

lazarust commented Dec 15, 2022

adrinjalali commented Dec 15, 2022

lazarust commented Dec 16, 2022

adrinjalali left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lazarust commented Jan 4, 2023

merveenoyan left a comment

Choose a reason for hiding this comment

merveenoyan Jan 5, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lazarust commented Jan 10, 2023

merveenoyan left a comment

Choose a reason for hiding this comment

merveenoyan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lazarust commented Jan 16, 2023

merveenoyan commented Jan 17, 2023 • edited Loading

merveenoyan left a comment

Choose a reason for hiding this comment

merveenoyan commented Jan 18, 2023

BenjaminBossan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adrinjalali commented Jan 19, 2023

BenjaminBossan commented Jan 19, 2023 • edited Loading

adrinjalali commented Jan 19, 2023

lazarust commented Jan 26, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lazarust commented Jan 27, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lazarust commented Jan 28, 2023

BenjaminBossan commented Jan 30, 2023

adrinjalali commented Jan 30, 2023

adrinjalali commented Jan 30, 2023

lazarust commented Jan 31, 2023 • edited Loading

lazarust commented Feb 8, 2023

BenjaminBossan left a comment

Choose a reason for hiding this comment

lazarust commented Feb 8, 2023

BenjaminBossan left a comment

Choose a reason for hiding this comment

merveenoyan Jan 5, 2023 •

edited

Loading

merveenoyan commented Jan 17, 2023 •

edited

Loading

BenjaminBossan commented Jan 19, 2023 •

edited

Loading

lazarust commented Jan 31, 2023 •

edited

Loading