Update docs build process

[frazane]

Type of changes

• Bug fix
• New feature
• Documentation / docstrings
• Tests
• Other

Description

This PR brings changes to the doc build process, specifically to how the example notebooks are handled. In short, we stop using some mkdocs plugins and instead use jupytext and nbcovert directly, which is more robust and only adds an extra command to the building process. Here's a summary of the changes:

• all the examples are moved from docs/examples to examples under the top level of the directory, which gives them more visibility and separates them for the doc building process (i.e. from the corresponding markdown files)
• a new script to convert .py notebooks to markdown files is added (docs/scripts/gen_examples.py), which stores the generated markdown files under docs/_examples. Includes CLI parameters to optionally execute notebooks, to specify the number of notebooks to be processed in parallel and whether to only process notebooks that have been updated.

With these changes, the documentation can now be generated with:

poetry run python docs/scripts/gen_examples.py && poetry run mkdocs serve

[thomaspinder]

Once linters pass, I am happy to merge this.

[Thomas-Christie]

FYI I realised Ruff has been pinned to an old version for a while - I upgraded it and fixed the raised issues (some of which are present in this PR) in #461

[thomaspinder]

FYI I realised Ruff has been pinned to an old version for a while - I upgraded it and fixed the raised issues (some of which are present in this PR) in #461

Thanks @Thomas-Christie. I merged that PR, so you can probably just rebase from that @frazane