Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MNT: refactor model card to render sections lazily #310

Commits on Mar 2, 2023

  1. [WIP] Refactor Card to lazily format sections

    Currently, the way model cards work is that if a user adds new content,
    a Section object is created and added as a (sub)section. That object
    contains the *formatted* content of that section as str, which is then
    used when rendering.
    
    So if the user adds, e.g., a table, the table is eagerly formatted to a
    str and during rendering, the str is simply returned.
    
    With this refactor, when a user adds new content, a specific type of
    section is added instead, which does not eagerly format the string.
    
    For simple text content, this is just a PlainSection which holds the content
    as a str, so no big change. However, for more complex content like
    tables or figures, a special section type is added that does not contain
    the final content as rendered.
    
    When the model card is rendered, the section.format() method is called
    to render the actual content, i.e. the formatting is now called lazily.
    
    The main benefit of this refactor is that the sections in a model card
    are now "aware" of what type of section they are. This allows us to more
    easily modify them later. E.g. let's say we want to modify the path to a
    figure or a row in a table: Right now, this would require some quite
    complex regex to modify the already formatted str. With this refactor,
    we can just access the attributes of the section, e.g. section.path or
    section.table, and modify them.
    
    A minor benefit could also be performance, because we don't render until
    it's strictly needed, but that's probably very minor.
    
    From the user perspective, this refactor shouldn't change anything as
    long as they were sticking with the official API. When calling render()
    or save(), the output is the same as before. Only if the users were
    directly interacting with sections might there be a breakage.
    
    One minor visible change this PR brings as well is that the repr of the
    card changes slightly. Now it indicates what type of section we're
    dealing with, e.g. TableSection(...), except if it's a PlainSection,
    which, same as previously, just returns the repr of its content.
    
    This commit is WIP because I also added new "description" parameters to
    add_* methods that were missing it so far. Now, if a user adds, say, a
    plot, they can optionally add a description for the plot too. This
    functionality lacks unit tests as of now.
    BenjaminBossan committed Mar 2, 2023
    Configuration menu
    Copy the full SHA
    85503ed View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d2585a0 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0669b28 View commit details
    Browse the repository at this point in the history

Commits on Mar 3, 2023

  1. Add description param to permutation importances

    The add_permutation_importances method now also has a description
    argument.
    
    I also reworked the tests for permutation importances a bit:
    
    - added a new test for description
    - put all related tests inside a test class
    - moved the permutation importance computation into a fixture
    - moved to the new testing method of selecting section and checking its
      content exactly instead of checking a substring in model_card.render()
    BenjaminBossan committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    833aced View commit details
    Browse the repository at this point in the history
  2. Remove Section abstract class

    Now, Section is a concrete class (taking the role of PlainSection).
    BenjaminBossan committed Mar 3, 2023
    Configuration menu
    Copy the full SHA
    87547df View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    18c0855 View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2023

  1. Reviewer comment: how to init class

    Missing from last commit.
    BenjaminBossan committed Mar 6, 2023
    Configuration menu
    Copy the full SHA
    b66db87 View commit details
    Browse the repository at this point in the history