DOC Add secure persistence docs

README.rst

@@ -45,6 +45,10 @@ download or load the model.
   and how it should be used. The model card can then be stored as the
   ``README.md`` file on the Hugging Face Hub, with pre-populated metadata to
   help Hub understand the model.
+- ``skops.io``: Secure persistence of sklearn estimators and more, without using
+  ``pickle``. Visit `the docs
+  <https://skops.readthedocs.io/en/latest/persistence.html>`_ for more
+  information.
 
 Please refer to our `documentation <https://skops.readthedocs.io/en/latest/>`_
 on using the library as user, which includes user guides on the above topics as

docs/changes.rst

@@ -16,6 +16,10 @@ v0.3
 - Add ``private`` as an optional argument to :meth:`.hub_utils.push` to
   optionally set the visibility status of a repo when pushing to the hub.
   :pr:`130` by `Adrin Jalali`_.
+- First release of the skops secure persistence feature (:pr:`128`) by `Adrin
+  Jalali`_ and `Benjamin Bossan`_. Visit :ref:`persistence` for more
+  information. This feature is not production ready yet but we're happy to
+  receive feedback from users.
 
 v0.2
 ----

docs/index.rst

@@ -38,6 +38,7 @@ User Guide / API Reference
    installation
    hf_hub
    model_card
+   persistence
    modules/classes
    changes
 

docs/modules/classes.rst

@@ -8,11 +8,15 @@ This is the class and function reference of skops.
 
 :mod:`skops.hf_hub`: Hugging Face Hub Integration
 =================================================
-
 .. automodule:: skops.hub_utils
     :members:
 
 :mod:`skops.card`: Model Card Utilities
 =======================================
 .. automodule:: skops.card
     :members:
+
+:mod:`skops.io`: Secure persistence
+===================================
+.. automodule:: skops.io
+    :members:

docs/persistence.rst

@@ -0,0 +1,84 @@
+.. _persistence:
+
+Secure persistence with skops
+=============================
+
+.. warning::
+
+   This feature is very early in development, which means the API is
+   unstable and it is **not secure** at the moment. Therefore, use the same
+   caution as you would for ``pickle``: Don't load from sources that you
+   don't trust. In the future, more security will be added.
+
+Skops offers a way to save and load sklearn models without using :mod:`pickle`.
+The ``pickle`` module is not secure, but with skops, you can securely save and
+load sklearn models without using ``pickle``.
+
+``Pickle`` is the standard serialization format for sklearn and for Python in
+general. One of the main advantages of ``pickle`` is that it can be used for
+almost all Python code but this flexibility also makes it inherently insecure.
+This is because loading certain types of objects requires the ability to run
+arbitrary code, which can be misused for malicious purposes. For example, an
+attacker can use it to steal secrets from your machine or install a virus. As
+the Python docs say:
+
+.. warning::
+
+    The pickle module is not secure. Only unpickle data you trust. It is
+    possible to construct malicious pickle data which will execute arbitrary
+    code during unpickling. Never unpickle data that could have come from an
+    untrusted source, or that could have been tampered with.
+
+In contrast to ``pickle``, the :func:`skops.io.save` and :func:`skops.io.load`
+functions cannot be used to save arbitrary Python code, but they bypass
+``pickle`` and are thus more secure.
+
+Usage
+-----
+
+The code snippet below illustrates how to use :func:`skops.io.save` and
+:func:`skops.io.load`:
+
+.. code:: python
+
+    from sklearn.linear_model import LogisticRegression
+    from skops.io import load, save
+
+    clf = LogisticRegression(random_state=0, solver="liblinear")
+    clf.fit(X_train, y_train)
+    save(clf, "my-logistic-regression.skops")
+    # ...
+    loaded = load("my-logistic-regression.skops")
+    loaded.predict(X_test)
+
+At the moment, we support the vast majority of sklearn estimators. This includes
+complex use cases such as :class:`sklearn.pipeline.Pipeline`,
+:class:`sklearn.model_selection.GridSearchCV`, classes using Cython code, such
+as :class:`sklearn.tree.DecisionTreeClassifier`, and more. If you discover an sklearn
+estimator that does not work, please open an issue on the skops `GitHub page
+<https://github.com/skops-dev/skops/issues>`_ and let us know.
+
+In contrast to ``pickle``, skops cannot persist arbitrary Python code. This
+means if you have custom functions (say, a custom function to be used with
+:class:`sklearn.preprocessing.FunctionTransformer`), it will not work. However,
+most ``numpy`` and ``scipy`` functions should work. Therefore, you can actually
+save built-in functions like``numpy.sqrt``.
+
+Roadmap
+-------
+
+Currently, it is still possible to run insecure code when using skops
+persistence. For example, it's possible to load a save file that evaluates arbitrary
+code using :func:`eval`. However, we have concrete plans on how to mitigate
+this, so please stay updated.
+
+On top of trying to support persisting all relevant sklearn objects, we plan on
+making persistence extensible for other libraries. As a user, this means that if
+you trust a certain library, you will be able to tell skops to load code from
+that library. As a library author, there will be a clear path of what needs to
+be done to add secure persistence to your library, such that skops can save and
+load code from your library.
+
+To follow what features are currently planned, filter for the `"persistence"
+label <https://github.com/skops-dev/skops/labels/persistence>`_ in our GitHub
+issues.

skops/io/_persist.py

@@ -28,6 +28,30 @@
 
 
 def save(obj, file):
+    """Save an object using the skops persistence format.
+
+    Skops aims at providing a secure persistence feature that does not rely on
+    :mod:`pickle`, which is inherently insecure. For more information, please
+    visit the :ref:`persistence` documentation.
+
+    .. warning::
+
+       This feature is very early in development, which means the API is
+       unstable and it is **not secure** at the moment. Therefore, use the same
+       caution as you would for ``pickle``: Don't load from sources that you
+       don't trust. In the future, more security will be added.
+
+    Parameters
+    ----------
+    obj: object
+        The object to be saved. Usually a scikit-learn compatible model.
+
+    file: str
+        The file name. A zip archive will automatically created. As a matter of
+        convention, we recommend to use the ".skops" file extension, e.g.
+        ``save(model, "my-model.skops")``.
+
+    """
     with tempfile.TemporaryDirectory() as dst:
         with open(Path(dst) / "schema.json", "w") as f:
             state = get_state(obj, dst)
@@ -43,6 +67,30 @@ def save(obj, file):
 
 
 def load(file):
+    """Load an object saved with the skops persistence format.
+
+    Skops aims at providing a secure persistence feature that does not rely on
+    :mod:`pickle`, which is inherently insecure. For more information, please
+    visit the :ref:`persistence` documentation.
+
+    .. warning::
+
+       This feature is very early in development, which means the API is
+       unstable and it is **not secure** at the moment. Therefore, use the same
+       caution as you would for ``pickle``: Don't load from sources that you
+       don't trust. In the future, more security will be added.
+
+    Parameters
+    ----------
+    file: str
+        The file name of the object to be loaded.
+
+    Returns
+    -------
+    instance: object
+        The loaded object.
+
+    """
     with ZipFile(file, "r") as input_zip:
         schema = input_zip.read("schema.json")
         instance = get_instance(json.loads(schema), input_zip)