From 5f8db3ed0e8b062cb88fb0ce06f08240e749cfcf Mon Sep 17 00:00:00 2001 From: David Robertson Date: Wed, 30 Mar 2022 14:49:17 +0100 Subject: [PATCH] DOCS: recommend poetry where appropriate In particular the documentaion for contributors should only advocate poetry. I am not thrilled at the propsect of now having N+1 installation methods---but at the very least we ought to mention poetry here. --- README.rst | 30 +++++++++++----------- docs/code_style.md | 9 ++----- docs/development/contributing_guide.md | 35 +++++++++++--------------- docs/upgrade.md | 33 +++++++++++++----------- 4 files changed, 51 insertions(+), 56 deletions(-) diff --git a/README.rst b/README.rst index 595fb5ff62a6..0b30207afcd3 100644 --- a/README.rst +++ b/README.rst @@ -293,24 +293,26 @@ directory of your choice:: git clone https://github.com/matrix-org/synapse.git cd synapse -Synapse has a number of external dependencies, that are easiest -to install using pip and a virtualenv:: +Synapse has a number of external dependencies. We maintain a fixed development +environment using [poetry](). First, install poetry. We recommend - python3 -m venv ./env - source ./env/bin/activate - pip install -e ".[all,dev]" + pip install --user pipx + pipx install poetry -This will run a process of downloading and installing all the needed -dependencies into a virtual env. If any dependencies fail to install, -try installing the failing modules individually:: +but see the `poetry installation docs `_ +for more details. Then ask poetry to create a virtualenvironment from the project +and install Synapse's dependencies:: + + poetry install --extras "all test" - pip install -e "module-name" +This will run a process of downloading and installing all the needed +dependencies into a virtual env. We recommend using the demo which starts 3 federated instances running on ports `8080` - `8082` - ./demo/start.sh + poetry run ./demo/start.sh -(to stop, you can use `./demo/stop.sh`) +(to stop, you can use `poetry run ./demo/stop.sh`) See the [demo documentation](https://matrix-org.github.io/synapse/develop/development/demo.html) for more information. @@ -318,14 +320,14 @@ for more information. If you just want to start a single instance of the app and run it directly:: # Create the homeserver.yaml config once - python -m synapse.app.homeserver \ + poetry run python -m synapse.app.homeserver \ --server-name my.domain.name \ --config-path homeserver.yaml \ --generate-config \ --report-stats=[yes|no] # Start the app - python -m synapse.app.homeserver --config-path homeserver.yaml + poetry run python -m synapse.app.homeserver --config-path homeserver.yaml Running the unit tests @@ -334,7 +336,7 @@ Running the unit tests After getting up and running, you may wish to run Synapse's unit tests to check that everything is installed correctly:: - trial tests + poetry run trial tests This should end with a 'PASSED' result (note that exact numbers will differ):: diff --git a/docs/code_style.md b/docs/code_style.md index ebda6dcc85f4..10b5bb6a7b09 100644 --- a/docs/code_style.md +++ b/docs/code_style.md @@ -6,13 +6,8 @@ The Synapse codebase uses a number of code formatting tools in order to quickly and automatically check for formatting (and sometimes logical) errors in code. -The necessary tools are detailed below. - -First install them with: - -```sh -pip install -e ".[lint,mypy]" -``` +The necessary tools are detailed below. They should be installed by poetry as part +of Synapse's dev dependencies. See [the contributing guide](https://matrix-org.github.io/synapse/develop/development/contributing_guide.html#4-install-the-dependencies) for instructions. - **black** diff --git a/docs/development/contributing_guide.md b/docs/development/contributing_guide.md index 071202e1965f..639b3d8d145f 100644 --- a/docs/development/contributing_guide.md +++ b/docs/development/contributing_guide.md @@ -48,16 +48,16 @@ can find many good git tutorials on the web. # 4. Install the dependencies -Once you have installed Python 3 and added the source, please open a terminal and -setup a *virtualenv*, as follows: +Synapse uses the [poetry](https://python-poetry.org/) project to lock its dependencies +and development environment. Once you have installed Python 3 and added the +source, you should [install poetry](https://python-poetry.org/docs/#installation). +We recommend installing `poetry` using `pipx`; see their instructions for details. + +Next, open a terminal and install dependencies as follows: ```sh cd path/where/you/have/cloned/the/repository -python3 -m venv ./env -source ./env/bin/activate -pip install wheel -pip install -e ".[all,dev]" -pip install tox +poetry install --extras "all test" ``` This will install the developer dependencies for the project. @@ -117,11 +117,10 @@ The linters look at your code and do two things: - ensure that your code follows the coding style adopted by the project; - catch a number of errors in your code. -The linter takes no time at all to run as soon as you've [downloaded the dependencies into your python virtual environment](#4-install-the-dependencies). +The linter takes no time at all to run as soon as you've [downloaded the dependencies](#4-install-the-dependencies). ```sh -source ./env/bin/activate -./scripts-dev/lint.sh +poetry run ./scripts-dev/lint.sh ``` Note that this script *will modify your files* to fix styling errors. @@ -131,15 +130,13 @@ If you wish to restrict the linters to only the files changed since the last com (much faster!), you can instead run: ```sh -source ./env/bin/activate -./scripts-dev/lint.sh -d +poetry run ./scripts-dev/lint.sh -d ``` Or if you know exactly which files you wish to lint, you can instead run: ```sh -source ./env/bin/activate -./scripts-dev/lint.sh path/to/file1.py path/to/file2.py path/to/folder +poetry run ./scripts-dev/lint.sh path/to/file1.py path/to/file2.py path/to/folder ``` ## Run the unit tests (Twisted trial). @@ -148,16 +145,14 @@ The unit tests run parts of Synapse, including your changes, to see if anything was broken. They are slower than the linters but will typically catch more errors. ```sh -source ./env/bin/activate -trial tests +poetry run trial tests ``` If you wish to only run *some* unit tests, you may specify another module instead of `tests` - or a test class or a method: ```sh -source ./env/bin/activate -trial tests.rest.admin.test_room tests.handlers.test_admin.ExfiltrateData.test_invite +poetry run trial tests.rest.admin.test_room tests.handlers.test_admin.ExfiltrateData.test_invite ``` If your tests fail, you may wish to look at the logs (the default log level is `ERROR`): @@ -169,7 +164,7 @@ less _trial_temp/test.log To increase the log level for the tests, set `SYNAPSE_TEST_LOG_LEVEL`: ```sh -SYNAPSE_TEST_LOG_LEVEL=DEBUG trial tests +SYNAPSE_TEST_LOG_LEVEL=DEBUG poetry run trial tests ``` By default, tests will use an in-memory SQLite database for test data. For additional @@ -180,7 +175,7 @@ database state to be stored in a file named `test.db` under the trial process' working directory. Typically, this ends up being `_trial_temp/test.db`. For example: ```sh -SYNAPSE_TEST_PERSIST_SQLITE_DB=1 trial tests +SYNAPSE_TEST_PERSIST_SQLITE_DB=1 poetry run trial tests ``` The database file can then be inspected with: diff --git a/docs/upgrade.md b/docs/upgrade.md index 062e823333c7..8f52fdda7b17 100644 --- a/docs/upgrade.md +++ b/docs/upgrade.md @@ -19,32 +19,35 @@ this document. packages](setup/installation.md#prebuilt-packages), you will need to follow the normal process for upgrading those packages. +- If Synapse was installed using pip then upgrade to the latest + version by running: + + ```bash + pip install --upgrade matrix-synapse + ``` + - If Synapse was installed from source, then: - 1. Activate the virtualenv before upgrading. For example, if - Synapse is installed in a virtualenv in `~/synapse/env` then + 1. Obtain the latest version of the source code. Git users can run + `git pull` to do this. + + 2. If you're running Synapse in a virtualenv, make sure to activate it before + upgrading. For example, if Synapse is installed in a virtualenv in `~/synapse/env` then run: ```bash source ~/synapse/env/bin/activate + pip install --upgrade . ``` + Include any relevant extras with `-e`, e.g. `pip install --upgrade .[postgres,oidc]`. - 2. If Synapse was installed using pip then upgrade to the latest - version by running: - - ```bash - pip install --upgrade matrix-synapse - ``` - - If Synapse was installed using git then upgrade to the latest - version by running: - + 3. If you're using `poetry` to manage a Synapse installation, run: ```bash - git pull - pip install --upgrade . + poetry install ``` + Include any relevant extras with `-E`, e.g. `poetry install -E "postgres oidc"`. - 3. Restart Synapse: + 4. Restart Synapse: ```bash synctl restart