-
Notifications
You must be signed in to change notification settings - Fork 639
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
24 changed files
with
576 additions
and
62 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,5 @@ | ||
| name: Code Scanning | ||
| permissions: read-all | ||
|
|
||
| on: | ||
| workflow_dispatch: # run on request (no need for PR) | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,5 @@ | ||
| name: Pre-Merge Checks | ||
| permissions: read-all | ||
|
|
||
| on: | ||
| push: | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,6 @@ | ||
| name: Upload coverage | ||
| permissions: read-all | ||
|
|
||
| on: | ||
| workflow_run: | ||
| workflows: ["Pre-Merge Checks"] | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
9 changes: 9 additions & 0 deletions
9
docs/source/markdown/guides/reference/pipelines/base/generator.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # Generator | ||
|
|
||
| ```{eval-rst} | ||
| .. autoclass:: anomalib.pipelines.components.base.job.JobGenerator | ||
| :members: | ||
| :inherited-members: | ||
| :show-inheritance: | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # Job | ||
|
|
||
| ```{eval-rst} | ||
| .. autoclass:: anomalib.pipelines.components.base.job.Job | ||
| :members: | ||
| :inherited-members: | ||
| :show-inheritance: | ||
| ``` |
9 changes: 9 additions & 0 deletions
9
docs/source/markdown/guides/reference/pipelines/base/pipeline.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| # Pipeline Base Class | ||
|
|
||
| The `Pipeline` class is the base class for all pipelines. It provides the following methods: | ||
|
|
||
| ```{eval-rst} | ||
| .. automodule:: anomalib.pipelines.components.base.pipeline | ||
| :members: | ||
| :show-inheritance: | ||
| ``` |
10 changes: 10 additions & 0 deletions
10
docs/source/markdown/guides/reference/pipelines/benchmark/generator.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # Benchmark Job Generator | ||
|
|
||
| ```{eval-rst} | ||
| .. autoclass:: anomalib.pipelines.benchmark.generator.BenchmarkJobGenerator | ||
| :members: | ||
| :inherited-members: | ||
| :show-inheritance: | ||
| ``` |
91 changes: 91 additions & 0 deletions
91
docs/source/markdown/guides/reference/pipelines/benchmark/index.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| # Benchmarking Pipeline | ||
|
|
||
| The benchmarking pipeline allows you to run multiple models across combination of parameters and dataset categories to collect metrics. The benchmarking run is configured using a config file that specifies the grid-search parameters. A sample config file is shown below: | ||
|
|
||
| ```yaml | ||
| accelerator: | ||
| - cuda | ||
| - cpu | ||
| benchmark: | ||
| seed: 42 | ||
| model: | ||
| class_path: | ||
| grid_search: [Padim, Patchcore] | ||
| data: | ||
| class_path: MVTec | ||
| init_args: | ||
| category: | ||
| grid: | ||
| - bottle | ||
| - cable | ||
| - capsule | ||
| ``` | ||
|
|
||
| The `accelerator` parameter is specific to the pipeline and is used to configure the runners. When `cuda` is passed it adds a [parallel](../runners/parallel.md) runner with number of jobs equal to the number of cuda devices. The idea is that since job is independent, we can increase the throughput by distributing each on an individual accelerator. The `cpu` jobs are run [serially](../runners/serial.md). | ||
|
|
||
| ## Running the Benchmark Pipeline | ||
|
|
||
| There are two ways to run the benchmark pipeline; as a subcommand, or as a standalone entrypoint. | ||
|
|
||
| :::::{dropdown} CLI | ||
| :icon: code | ||
|
|
||
| ::::{tab-set} | ||
| :::{tab-item} Anomalib subcommand | ||
| :sync: label-1 | ||
|
|
||
| ```{literalinclude} ../../../../../snippets/pipelines/benchmark/cli_anomalib.txt | ||
| :language: bash | ||
| ``` | ||
|
|
||
| ::: | ||
|
|
||
| :::{tab-item} Standalone entrypoint | ||
| :sync: label-2 | ||
|
|
||
| ```{literalinclude} ../../../../../snippets/pipelines/benchmark/cli_tools.txt | ||
| :language: bash | ||
| ``` | ||
|
|
||
| ::: | ||
|
|
||
| ::::: | ||
|
|
||
| ## Benchmark Pipeline Class | ||
|
|
||
| ```{eval-rst} | ||
| .. autoclass:: anomalib.pipelines.benchmark.pipeline.Benchmark | ||
| :members: | ||
| :inherited-members: | ||
| :show-inheritance: | ||
| ``` | ||
|
|
||
| ::::{grid} | ||
| :margin: 1 1 0 0 | ||
| :gutter: 1 | ||
|
|
||
| :::{grid-item-card} Job | ||
| :link: ./job | ||
| :link-type: doc | ||
|
|
||
| Benchmark Job | ||
| ::: | ||
|
|
||
| :::{grid-item-card} Generator | ||
| :link: ./generator | ||
| :link-type: doc | ||
|
|
||
| Benchmark Job Generator | ||
| ::: | ||
|
|
||
| :::: | ||
|
|
||
| ```{toctree} | ||
| :caption: Benchmark | ||
| :hidden: | ||
| ./job | ||
| ./generator | ||
| ``` |
10 changes: 10 additions & 0 deletions
10
docs/source/markdown/guides/reference/pipelines/benchmark/job.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # Benchmark Job | ||
|
|
||
| ```{eval-rst} | ||
| .. autoclass:: anomalib.pipelines.benchmark.job.BenchmarkJob | ||
| :members: | ||
| :inherited-members: | ||
| :show-inheritance: | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| # Pipelines | ||
|
|
||
| ```{danger} | ||
| The pipelines feature is experimental and might be changed without backward compatibility. | ||
| ``` | ||
|
|
||
| ## Introduction | ||
|
|
||
| Tasks such as Benchmarking, Ensemble Tiling, and Hyper-parameter optimization requires running multiple models and chaining multiple stages together. The pipelines feature provides a way to define and run such tasks. Each part of the pipeline is designed to be independent and composable so that they can be reused across different pipelines. | ||
|
|
||
| ## Terminology | ||
|
|
||
| - **Pipeline**: Pipeline is the main entity that defines the sequence of [jobs](./base/job.md) to be executed. It is responsible for creating and running the jobs. The job itself is generated using a [job generator](./base/generator.md). And, these are chained using a [runner](./runners/index.md). | ||
|
|
||
| - **Runner**: A runner is responsible for scheduling and running the jobs. It also passes the output of the previous job, if available. It also calls the right hooks to gather and save the results from the jobs and passes the gathered results to the next runner. | ||
|
|
||
| - **Job Generator**: The job generator is responsible for generating jobs based on the configuration. It is used by the runner to create jobs. | ||
|
|
||
| - **Job**: A job is an atomic unit of work that can be run independently. It is responsible for running a single task. For example, training a model or computing metrics. The idea behind this is to ensure that it can be attached to any runner without making changes to the job itself. This is useful when you want to distribute the jobs to increase the throughput of your pipeline. | ||
|
|
||
| ```{admonition} Detailed Walkthrough | ||
| :class: tip | ||
| For more clarity on creating a custom pipeline, refer to the [How-To Guide](../../how_to/pipelines/index.md). | ||
| ``` | ||
|
|
||
| ## Base classes | ||
|
|
||
| ::::{grid} | ||
| :margin: 1 1 0 0 | ||
| :gutter: 1 | ||
|
|
||
| :::{grid-item-card} {octicon}`workflow` Pipeline | ||
| :link: ./base/pipeline | ||
| :link-type: doc | ||
|
|
||
| Base class for pipeline. | ||
| ::: | ||
|
|
||
| :::{grid-item-card} {octicon}`file` Job | ||
| :link: ./base/job | ||
| :link-type: doc | ||
|
|
||
| Base class for job. | ||
| ::: | ||
|
|
||
| :::{grid-item-card} {octicon}`iterations` Job Generator | ||
| :link: ./base/generator | ||
| :link-type: doc | ||
|
|
||
| Base class for job generator. | ||
| ::: | ||
|
|
||
| :::{grid-item-card} {octicon}`play` Runner | ||
| :link: ./runners/index | ||
| :link-type: doc | ||
|
|
||
| Base class for runner. | ||
| ::: | ||
|
|
||
| :::: | ||
|
|
||
| ## Available Pipelines | ||
|
|
||
| ::::{grid} | ||
| :margin: 1 1 0 0 | ||
| :gutter: 1 | ||
|
|
||
| :::{grid-item-card} {octicon}`number` Benchmarking | ||
| :link: ./benchmark/index | ||
| :link-type: doc | ||
|
|
||
| Compute metrics across models using a grid-search. | ||
| ::: | ||
|
|
||
| :::: | ||
|
|
||
| ```{toctree} | ||
| :caption: Pipelines | ||
| :hidden: | ||
| ./benchmark/index | ||
| ./runners/index | ||
| ``` |
Oops, something went wrong.