Skip to content

Commit

Permalink
Docs-Guides - Add Index Entries for Index (#1969)
Browse files Browse the repository at this point in the history
Co-authored-by: above3 <anthony_bove@apple.com>
  • Loading branch information
tonybove-apple and above3 authored Sep 12, 2023
1 parent 60d1398 commit d7a1479
Show file tree
Hide file tree
Showing 53 changed files with 528 additions and 20 deletions.
8 changes: 2 additions & 6 deletions docs-guides/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ Core ML Tools

This guide includes instructions and examples. For details about using the API classes and methods, see the `coremltools API Reference <https://apple.github.io/coremltools/index.html>`_.

:ref:`genindex` | :ref:`search`

--------------

.. toctree::
Expand Down Expand Up @@ -79,9 +81,3 @@ This guide includes instructions and examples. For details about using the API c
source/updatable-model-examples.rst


Index
-----

* :ref:`genindex`
* :ref:`search`

14 changes: 14 additions & 0 deletions docs-guides/source/classifiers.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
```{eval-rst}
.. index::
single: classifier; preview and produce
```

# Classifiers

A [classifier](https://deepai.org/machine-learning-glossary-and-terms/classifier) is a special kind of [Core ML](https://developer.apple.com/documentation/coreml) model that provides a class label and class name to a probability dictionary as outputs. This topic describes the steps to produce a classifier model using the [Unified Conversion API](unified-conversion-api) by using the [ClassifierConfig](https://apple.github.io/coremltools/source/coremltools.converters.mil.input_types.html#classifierconfig) class.
Expand Down Expand Up @@ -46,6 +51,11 @@ To produce a classifier model, follow these steps:
)
```

```{eval-rst}
.. index::
single: PIL
```

3. Use PIL to load and resize the image to the expected size:

```python
Expand All @@ -69,4 +79,8 @@ To produce a classifier model, follow these steps:
All Core ML models use the Core ML framework and its APIs. However, with image input models you can also use the [Vision Classifier Observation API](https://developer.apple.com/documentation/vision/vnclassificationobservation), which provides image analysis and additional convenience features for preprocessing images.
```

```{eval-rst}
.. index::
single: Vision Classifier Observation API
```

23 changes: 23 additions & 0 deletions docs-guides/source/comparing-ml-programs-and-neural-networks.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
```{eval-rst}
.. index::
single: ML program; compared to neural network
single: neural network; compared to ML program
```


# Comparing ML Programs and Neural Networks

As ML models evolve in sophistication and complexity, their [representations](#model-representations) are also evolving to describe how they work. _ML programs_ are models that are represented as operations in code. The ML program model type is the foundation for future Core ML improvements.
Expand All @@ -12,6 +19,15 @@ A few of the major the differences between a neural network and an ML program ar
| Limited control over precision | More granular control over precision |


```{eval-rst}
.. index::
single: typed execution
single: GPU runtime
single: Metal Performance Shaders Graph framework
single: compiling an ML program
single: ML program; compiling
```

## ML Program Benefits

Converting to an ML program provides the following advantages over converting to a neural network:
Expand Down Expand Up @@ -43,6 +59,10 @@ There are several ways to represent a deep learning model. At a foundational lev
Three ways to represent a deep learning model.
```

```{eval-rst}
.. index:: Core ML NeuralNetwork
```

To express a neural network, the mathematical descriptions are often abstracted into a _computational graph_, which is a more concise and scalable representation (as shown in the center of the previous figure). A computational graph is a directed graph in which computational layers connect to each other — the input feeds into the _source_ layers and undergoes a series of mathematical transformations to generate the outputs through the _sink_ layers.

At the center of the Core ML [NeuralNetwork](https://apple.github.io/coremltools/mlmodel/Format/NeuralNetwork.html) are the layers that form the nodes of the graph. There are many different types of layers, and each describes a particular mathematical operation. Each layer specifies its input and output names, which are the _edges_ of the graph. Connections are established by matching an output name of a layer to another layer’s input name.
Expand Down Expand Up @@ -91,6 +111,9 @@ While the new ML program model type supports most of the functionality supported

ML programs currently support both float 16 and float 32 typed weights and activations.

```{eval-rst}
.. index:: MIL, Model Intermediate Language
```

## ML Programs and MIL

Expand Down
10 changes: 10 additions & 0 deletions docs-guides/source/composite-operators.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
```{eval-rst}
.. index::
single: composite operators
```

# Composite Operators

As machine learning continually evolves, new operations are regularly added to source frameworks such as [TensorFlow](https://www.tensorflow.org/) and [PyTorch](https://pytorch.org/). While converting a model to [Core ML](https://developer.apple.com/documentation/coreml), you may encounter an unsupported operation.
Expand Down Expand Up @@ -42,6 +47,11 @@ You may need to first install [Transformers](https://huggingface.co/transformers
![Not-implemented error](images/not-implemented-error.png)


```{eval-rst}
.. index::
single: MIL operators
```

## Decompose into Existing MIL Operators

The TensorFlow [documentation on Einsum](https://www.tensorflow.org/api_docs/java/org/tensorflow/op/core/Einsum) refers to Einstein summation notation. You can use this notation to represent a variety of tensor operations such as `reduce_sum`, `transpose`, and `trace`, using a string. Einsum is usually a complicated operation, but with this example you don't need to know all the possible cases, just the particular notation that this model uses.
Expand Down
6 changes: 6 additions & 0 deletions docs-guides/source/convert-a-tensorflow-1-deepspeech-model.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
```{eval-rst}
.. index::
single: TensorFlow 1; convert DeepSpeech model
```


# Converting a TensorFlow 1 DeepSpeech Model

The following example explores the automatic handling of flexible shapes and other related capabilities of the Core ML Tools converter. It uses an [automatic speech recognition](https://en.wikipedia.org/wiki/Speech_recognition#End-to-end_automatic_speech_recognition) (ASR) task in which the input is a speech audio file and the output is the text transcription of it.
Expand Down
8 changes: 8 additions & 0 deletions docs-guides/source/convert-a-tensorflow-1-image-classifier.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
```{eval-rst}
.. index::
single: TensorFlow 1; convert image classifier
single: classifier; convert TensorFlow 1
single: classifier; image
```


# Converting a TensorFlow 1 Image Classifier

The following example converts the TensorFlow Inception V1 image classifier to a Core ML neural network classifier model that directly predicts the class label of the input image. It demonstrates the importance of setting the image preprocessing parameters correctly to get the right results.
Expand Down
23 changes: 23 additions & 0 deletions docs-guides/source/convert-a-torchvision-model-from-pytorch.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
```{eval-rst}
.. index::
single: PyTorch; convert torchvision model
single: torchvision model
```

# Converting a torchvision Model from PyTorch

The following example shows how to convert into Core ML a [MobileNetV2](https://pytorch.org/hub/pytorch_vision_mobilenet_v2/) model trained using PyTorch. MobileNet is a type of convolutional neural network designed for mobile and embedded vision applications.
Expand Down Expand Up @@ -69,6 +75,13 @@ class_labels = class_labels[1:] # remove the first class which is background
assert len(class_labels) == 1000
```

```{eval-rst}
.. index::
single: PyTorch; preprocess image input
single: preprocessing for images
```


## Preprocess the Image Input for torchvision Models

Image-based models typically require the input image to be preprocessed before using it with the converted model. For the details of how to preprocess image input for torchvision models, see [Preprocessing for Torch](image-inputs.md#preprocessing-for-torch).
Expand Down Expand Up @@ -140,6 +153,10 @@ Right-click the following image and save it as `daisy.jpg` in the same folder as
Right-click this image and save it as `daisy.jpg` in the same folder as your Python project.
```

```{eval-rst}
.. index::
single: protobuf spec
```

## Get the protobuf spec

Expand All @@ -154,6 +171,11 @@ for out in spec.description.output:
break
```

```{eval-rst}
.. index::
single: PyTorch; make prediction
```

## Make a Core ML Prediction

You can now make a prediction with the converted model, using the test image. To learn more about making predictions, see [Model Prediction](model-prediction). The code for `coreml_out_dict["classLabel"]` returns the top-level class label.
Expand Down Expand Up @@ -186,6 +208,7 @@ class name: vase, raw score value: 8.516773223876953
class name: ant, raw score value: 8.169312477111816
```


## Make a PyTorch Prediction and Compare

To test the accuracy of the converted model with respect to the traced (TorchScript) model, make a prediction with the test image using the original PyTorch model.
Expand Down
6 changes: 6 additions & 0 deletions docs-guides/source/convert-nlp-model.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Converting a Natural Language Processing Model

```
.. index::
single: PyTorch; combine tracing and scripting
single: PyTorch; convert natural language proocessing model
```

The following example demonstrates how you can combine [model tracing](model-tracing) and [model scripting](model-scripting) in order to properly convert a model that includes a data-dependent control flow, such as a loop or conditional.

```{warning}
Expand Down
7 changes: 7 additions & 0 deletions docs-guides/source/convert-pytorch-workflow.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
```{eval-rst}
.. index::
single: PyTorch; convert workflow
single: Torchscript
```


# PyTorch Conversion Workflow

```{admonition} Minimum Deployment Target
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
```{eval-rst}
.. index::
single: TensorFlow 2; convert BERT Transformer Models
```


# Converting TensorFlow 2 BERT Transformer Models

The following examples demonstrate converting TensorFlow 2 models to Core ML using Core ML Tools.
Expand Down
19 changes: 19 additions & 0 deletions docs-guides/source/convert-to-ml-program.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
```{eval-rst}
.. index::
pair: ML program; convert to
```


# Convert Models to ML Programs

This section describes the `ML program` model type. It is an evolution of the neural network model type that has been available since the first version of Core ML.
Expand All @@ -20,6 +26,12 @@ model = ct.convert(source_model)

The above example produces an `mlprogram` with an `iOS15`/`macOS12` deployment target (or newer). You can override this behavior by providing a `minimum_deployment_target` value, such as `minimum_deployment_target=target.iOS14` or older.

```{eval-rst}
.. index::
single: precision type
single: ML program; precision type
```

## Set the ML Program Precision

You can optionally set the precision type (float 16 or float 32) of the weights and the intermediate tensors in the ML program during conversion. The ML program type offers an additional `compute_precision` parameter as shown in the following example:
Expand All @@ -45,6 +57,13 @@ For details on ML program precision, see [Typed Execution](typed-execution).
For ML programs, Core ML Tools version 5.0b3 and newer produces a model with float 16 precision by default (previous beta versions produced float 32 by default). You can override the default precision by using the `compute_precision` parameter of [`coremltools.convert()`](https://apple.github.io/coremltools/source/coremltools.converters.convert.html#coremltools.converters._converters_entry.convert).
```

```{eval-rst}
.. index::
single: model package
single: ML program; save as model package
single: save a model package
```

## Save ML Programs as Model Packages

The ML program type uses the [Core ML model package](https://developer.apple.com/documentation/coreml/updating_a_model_file_to_a_model_package) container format that separates the model into components and offers more flexible metadata editing. Since an ML program decouples the weights from the program architecture, it cannot be saved as an `.mlmodel` file.
Expand Down
6 changes: 6 additions & 0 deletions docs-guides/source/convert-to-neural-network.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
```{eval-rst}
.. index::
pair: neural network; convert to
```


# Convert Models to Neural Networks

With versions of Core ML Tools older than 7.0b2, if you didn't specify the model type, or your `minimum_deployment_target` was a version older than iOS15, macOS12, watchOS8, or tvOS15, the model was converted by default to a neural network.
Expand Down
5 changes: 5 additions & 0 deletions docs-guides/source/converting-compressed-source-models.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
```{eval-rst}
.. index::
single: convert compressed models
```

# Converting Compressed Source Models

This section shows how you can indicate to the converter to use the sparse or palettized representations to store the weights. This is required when you bring in a source model whose weights are compressed but still represented in dense `float`format. This is the case for PyTorch models that are updated and fine-tuned using the [`ct.optimize.torch`](optimizetorch-api-overview) APIs.
Expand Down
5 changes: 5 additions & 0 deletions docs-guides/source/custom-operators.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
```{eval-rst}
.. index::
single: custom operators
```

# Custom Operators

While converting a model to [Core ML](https://developer.apple.com/documentation/coreml), you may encounter an unsupported operation that can't be represented by a [composite operator](composite-operators).
Expand Down
6 changes: 6 additions & 0 deletions docs-guides/source/data-dependent-pruning.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
```{eval-rst}
.. index::
single: pruning; training-time
single: MagnitudePruner
```

# Training-Time Pruning

The [Training-Time Pruning](https://apple.github.io/coremltools/source/coremltools.optimize.torch.pruning.html#training-time-pruning) API in `coremltools.optimize.torch` builds on top of the [BasePruningMethod](https://pytorch.org/docs/stable/generated/torch.nn.utils.prune.BasePruningMethod.html) API in PyTorch and extends it to:
Expand Down
8 changes: 8 additions & 0 deletions docs-guides/source/data-dependent-quantization.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
```{eval-rst}
.. index::
single: quantization; training-time
single: PyTorch; quantization APIs
single: quantization; PyTorch APIs
single: LinearQuantizer
```

# Training-Time Quantization

The [`LinearQuantizer`](https://apple.github.io/coremltools/source/coremltools.optimize.torch.quantization.html#coremltools.optimize.torch.quantization.LinearQuantizer) class implements training-time quantization, also known as quantization-aware training (QAT) as described in the paper [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://arxiv.org/pdf/1712.05877.pdf). [`LinearQuantizer`](https://apple.github.io/coremltools/source/coremltools.optimize.torch.quantization.html#coremltools.optimize.torch.quantization.LinearQuantizer) quantizes both weights and activations, whereas [Post-Training Quantization](data-free-quantization) quantizes only the weights.
Expand Down
6 changes: 6 additions & 0 deletions docs-guides/source/data-free-quantization.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
```{eval-rst}
.. index::
single: quantization; post-training
single: linear_quantize_weights, OpLinearQuantizerConfig
```

# Post-Training Quantization

You can linearly quantize the weights of your Core ML model by using the [`linear_quantize_weights`](https://apple.github.io/coremltools/source/coremltools.optimize.coreml.post_training_quantization.html#coremltools.optimize.coreml.linear_quantize_weights) method as follows:
Expand Down
17 changes: 17 additions & 0 deletions docs-guides/source/flexible-inputs.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
```{eval-rst}
.. index::
single: input shapes
single: flexible input shapes
single: shapes
```

# Flexible Input Shapes

While some neural network models accept only fixed-size input, such as an image with a resolution of 224 x 224 pixels, other models require flexible input shapes that are determined at runtime. Examples are language models that work on arbitrary input lengths, and style transfer models that work on multiple resolutions.
Expand All @@ -8,6 +15,11 @@ When converting a model to Core ML using Core ML Tools, you can specify a fixed
- [Set a bounded range for each dimension](#set-the-range-for-each-dimension) to define the minimum and maximum for more dynamic shapes. Using a bounded range provides more opportunity for the runtime compiler to make optimizations, which is harder with an unbounded range.
- [Enable unbounded ranges](#enable-unbounded-ranges), if necessary, for maximum flexibility.

```{eval-rst}
.. index::
single: enumerated shapes
```

## Select From Predetermined Shapes

To optimize performance, ensure model accuracy, or limit the range of inputs for other purposes, specify a set of predetermined shapes for input. To make Core ML models using this constraint, set [`EnumeratedShapes`](https://apple.github.io/coremltools/source/coremltools.converters.mil.input_types.html#enumeratedshapes) as shown in the following example.
Expand Down Expand Up @@ -83,6 +95,11 @@ Core ML preallocates the memory for the default shape, so the first prediction w
For a multi-input model, only one of the inputs can be marked with `EnumeratedShapes`; the rest must have fixed single shapes. If you require multiple inputs to be flexible, set the range for each dimension.
```

```{eval-rst}
.. index::
single: RangeDim
```

## Set the Range for Each Dimension

If you know that the input shape will be within a specific interval in each dimension, set the range for each dimension with [`RangeDim`](https://apple.github.io/coremltools/source/coremltools.converters.mil.input_types.html#rangedim) as shown in the following example.
Expand Down
6 changes: 6 additions & 0 deletions docs-guides/source/graph-passes-intro.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
```{eval-rst}
.. index::
single: graph passes
single: pass_pipeline
```

# Graph Passes

During conversion, Core ML Tools optimizes the model by applying graph transformations, called _graph passes_, which simplify and canonicalize the representation for a more efficient execution by the Core ML runtime.
Expand Down
Loading

0 comments on commit d7a1479

Please sign in to comment.