This repository has been archived by the owner on Apr 12, 2024. It is now read-only.
-
-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge tag 'v1.33.0' into babolivier/dinsic_1.41.0
Synapse 1.33.0 (2021-05-05) =========================== Features -------- - Build Debian packages for Ubuntu 21.04 (Hirsute Hippo). ([\#9909](matrix-org/synapse#9909)) Synapse 1.33.0rc2 (2021-04-29) ============================== Bugfixes -------- - Fix tight loop when handling presence replication when using workers. Introduced in v1.33.0rc1. ([\#9900](matrix-org/synapse#9900)) Synapse 1.33.0rc1 (2021-04-28) ============================== Features -------- - Update experimental support for [MSC3083](matrix-org/matrix-spec-proposals#3083): restricting room access via group membership. ([\#9800](matrix-org/synapse#9800), [\#9814](matrix-org/synapse#9814)) - Add experimental support for handling presence on a worker. ([\#9819](matrix-org/synapse#9819), [\#9820](matrix-org/synapse#9820), [\#9828](matrix-org/synapse#9828), [\#9850](matrix-org/synapse#9850)) - Return a new template when an user attempts to renew their account multiple times with the same token, stating that their account is set to expire. This replaces the invalid token template that would previously be shown in this case. This change concerns the optional account validity feature. ([\#9832](matrix-org/synapse#9832)) Bugfixes -------- - Fixes the OIDC SSO flow when using a `public_baseurl` value including a non-root URL path. ([\#9726](matrix-org/synapse#9726)) - Fix thumbnail generation for some sites with non-standard content types. Contributed by @rkfg. ([\#9788](matrix-org/synapse#9788)) - Add some sanity checks to identity server passed to 3PID bind/unbind endpoints. ([\#9802](matrix-org/synapse#9802)) - Limit the size of HTTP responses read over federation. ([\#9833](matrix-org/synapse#9833)) - Fix a bug which could cause Synapse to get stuck in a loop of resyncing device lists. ([\#9867](matrix-org/synapse#9867)) - Fix a long-standing bug where errors from federation did not propagate to the client. ([\#9868](matrix-org/synapse#9868)) Improved Documentation ---------------------- - Add a note to the docker docs mentioning that we mirror upstream's supported Docker platforms. ([\#9801](matrix-org/synapse#9801)) Internal Changes ---------------- - Add a dockerfile for running Synapse in worker-mode under Complement. ([\#9162](matrix-org/synapse#9162)) - Apply `pyupgrade` across the codebase. ([\#9786](matrix-org/synapse#9786)) - Move some replication processing out of `generic_worker`. ([\#9796](matrix-org/synapse#9796)) - Replace `HomeServer.get_config()` with inline references. ([\#9815](matrix-org/synapse#9815)) - Rename some handlers and config modules to not duplicate the top-level module. ([\#9816](matrix-org/synapse#9816)) - Fix a long-standing bug which caused `max_upload_size` to not be correctly enforced. ([\#9817](matrix-org/synapse#9817)) - Reduce CPU usage of the user directory by reusing existing calculated room membership. ([\#9821](matrix-org/synapse#9821)) - Small speed up for joining large remote rooms. ([\#9825](matrix-org/synapse#9825)) - Introduce flake8-bugbear to the test suite and fix some of its lint violations. ([\#9838](matrix-org/synapse#9838)) - Only store the raw data in the in-memory caches, rather than objects that include references to e.g. the data stores. ([\#9845](matrix-org/synapse#9845)) - Limit length of accepted email addresses. ([\#9855](matrix-org/synapse#9855)) - Remove redundant `synapse.types.Collection` type definition. ([\#9856](matrix-org/synapse#9856)) - Handle recently added rate limits correctly when using `--no-rate-limit` with the demo scripts. ([\#9858](matrix-org/synapse#9858)) - Disable invite rate-limiting by default when running the unit tests. ([\#9871](matrix-org/synapse#9871)) - Pass a reactor into `SynapseSite` to make testing easier. ([\#9874](matrix-org/synapse#9874)) - Make `DomainSpecificString` an `attrs` class. ([\#9875](matrix-org/synapse#9875)) - Add type hints to `synapse.api.auth` and `synapse.api.auth_blocking` modules. ([\#9876](matrix-org/synapse#9876)) - Remove redundant `_PushHTTPChannel` test class. ([\#9878](matrix-org/synapse#9878)) - Remove backwards-compatibility code for Python versions < 3.6. ([\#9879](matrix-org/synapse#9879)) - Small performance improvement around handling new local presence updates. ([\#9887](matrix-org/synapse#9887))
- Loading branch information
Showing
685 changed files
with
3,793 additions
and
2,498 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # Inherit from the official Synapse docker image | ||
| FROM matrixdotorg/synapse | ||
|
|
||
| # Install deps | ||
| RUN apt-get update | ||
| RUN apt-get install -y supervisor redis nginx | ||
|
|
||
| # Remove the default nginx sites | ||
| RUN rm /etc/nginx/sites-enabled/default | ||
|
|
||
| # Copy Synapse worker, nginx and supervisord configuration template files | ||
| COPY ./docker/conf-workers/* /conf/ | ||
|
|
||
| # Expose nginx listener port | ||
| EXPOSE 8080/tcp | ||
|
|
||
| # Volume for user-editable config files, logs etc. | ||
| VOLUME ["/data"] | ||
|
|
||
| # A script to read environment variables and create the necessary | ||
| # files to run the desired worker configuration. Will start supervisord. | ||
| COPY ./docker/configure_workers_and_start.py /configure_workers_and_start.py | ||
| ENTRYPOINT ["/configure_workers_and_start.py"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,140 @@ | ||
| # Running tests against a dockerised Synapse | ||
|
|
||
| It's possible to run integration tests against Synapse | ||
| using [Complement](https://github.com/matrix-org/complement). Complement is a Matrix Spec | ||
| compliance test suite for homeservers, and supports any homeserver docker image configured | ||
| to listen on ports 8008/8448. This document contains instructions for building Synapse | ||
| docker images that can be run inside Complement for testing purposes. | ||
|
|
||
| Note that running Synapse's unit tests from within the docker image is not supported. | ||
|
|
||
| ## Testing with SQLite and single-process Synapse | ||
|
|
||
| > Note that `scripts-dev/complement.sh` is a script that will automatically build | ||
| > and run an SQLite-based, single-process of Synapse against Complement. | ||
| The instructions below will set up Complement testing for a single-process, | ||
| SQLite-based Synapse deployment. | ||
|
|
||
| Start by building the base Synapse docker image. If you wish to run tests with the latest | ||
| release of Synapse, instead of your current checkout, you can skip this step. From the | ||
| root of the repository: | ||
|
|
||
| ```sh | ||
| docker build -t matrixdotorg/synapse -f docker/Dockerfile . | ||
| ``` | ||
|
|
||
| This will build an image with the tag `matrixdotorg/synapse`. | ||
|
|
||
| Next, build the Synapse image for Complement. You will need a local checkout | ||
| of Complement. Change to the root of your Complement checkout and run: | ||
|
|
||
| ```sh | ||
| docker build -t complement-synapse -f "dockerfiles/Synapse.Dockerfile" dockerfiles | ||
| ``` | ||
|
|
||
| This will build an image with the tag `complement-synapse`, which can be handed to | ||
| Complement for testing via the `COMPLEMENT_BASE_IMAGE` environment variable. Refer to | ||
| [Complement's documentation](https://github.com/matrix-org/complement/#running) for | ||
| how to run the tests, as well as the various available command line flags. | ||
|
|
||
| ## Testing with PostgreSQL and single or multi-process Synapse | ||
|
|
||
| The above docker image only supports running Synapse with SQLite and in a | ||
| single-process topology. The following instructions are used to build a Synapse image for | ||
| Complement that supports either single or multi-process topology with a PostgreSQL | ||
| database backend. | ||
|
|
||
| As with the single-process image, build the base Synapse docker image. If you wish to run | ||
| tests with the latest release of Synapse, instead of your current checkout, you can skip | ||
| this step. From the root of the repository: | ||
|
|
||
| ```sh | ||
| docker build -t matrixdotorg/synapse -f docker/Dockerfile . | ||
| ``` | ||
|
|
||
| This will build an image with the tag `matrixdotorg/synapse`. | ||
|
|
||
| Next, we build a new image with worker support based on `matrixdotorg/synapse:latest`. | ||
| Again, from the root of the repository: | ||
|
|
||
| ```sh | ||
| docker build -t matrixdotorg/synapse-workers -f docker/Dockerfile-workers . | ||
| ``` | ||
|
|
||
| This will build an image with the tag` matrixdotorg/synapse-workers`. | ||
|
|
||
| It's worth noting at this point that this image is fully functional, and | ||
| can be used for testing against locally. See instructions for using the container | ||
| under | ||
| [Running the Dockerfile-worker image standalone](#running-the-dockerfile-worker-image-standalone) | ||
| below. | ||
|
|
||
| Finally, build the Synapse image for Complement, which is based on | ||
| `matrixdotorg/synapse-workers`. You will need a local checkout of Complement. Change to | ||
| the root of your Complement checkout and run: | ||
|
|
||
| ```sh | ||
| docker build -t matrixdotorg/complement-synapse-workers -f dockerfiles/SynapseWorkers.Dockerfile dockerfiles | ||
| ``` | ||
|
|
||
| This will build an image with the tag `complement-synapse`, which can be handed to | ||
| Complement for testing via the `COMPLEMENT_BASE_IMAGE` environment variable. Refer to | ||
| [Complement's documentation](https://github.com/matrix-org/complement/#running) for | ||
| how to run the tests, as well as the various available command line flags. | ||
|
|
||
| ## Running the Dockerfile-worker image standalone | ||
|
|
||
| For manual testing of a multi-process Synapse instance in Docker, | ||
| [Dockerfile-workers](Dockerfile-workers) is a Dockerfile that will produce an image | ||
| bundling all necessary components together for a workerised homeserver instance. | ||
|
|
||
| This includes any desired Synapse worker processes, a nginx to route traffic accordingly, | ||
| a redis for worker communication and a supervisord instance to start up and monitor all | ||
| processes. You will need to provide your own postgres container to connect to, and TLS | ||
| is not handled by the container. | ||
|
|
||
| Once you've built the image using the above instructions, you can run it. Be sure | ||
| you've set up a volume according to the [usual Synapse docker instructions](README.md). | ||
| Then run something along the lines of: | ||
|
|
||
| ``` | ||
| docker run -d --name synapse \ | ||
| --mount type=volume,src=synapse-data,dst=/data \ | ||
| -p 8008:8008 \ | ||
| -e SYNAPSE_SERVER_NAME=my.matrix.host \ | ||
| -e SYNAPSE_REPORT_STATS=no \ | ||
| -e POSTGRES_HOST=postgres \ | ||
| -e POSTGRES_USER=postgres \ | ||
| -e POSTGRES_PASSWORD=somesecret \ | ||
| -e SYNAPSE_WORKER_TYPES=synchrotron,media_repository,user_dir \ | ||
| -e SYNAPSE_WORKERS_WRITE_LOGS_TO_DISK=1 \ | ||
| matrixdotorg/synapse-workers | ||
| ``` | ||
|
|
||
| ...substituting `POSTGRES*` variables for those that match a postgres host you have | ||
| available (usually a running postgres docker container). | ||
|
|
||
| The `SYNAPSE_WORKER_TYPES` environment variable is a comma-separated list of workers to | ||
| use when running the container. All possible worker names are defined by the keys of the | ||
| `WORKERS_CONFIG` variable in [this script](configure_workers_and_start.py), which the | ||
| Dockerfile makes use of to generate appropriate worker, nginx and supervisord config | ||
| files. | ||
|
|
||
| Sharding is supported for a subset of workers, in line with the | ||
| [worker documentation](../docs/workers.md). To run multiple instances of a given worker | ||
| type, simply specify the type multiple times in `SYNAPSE_WORKER_TYPES` | ||
| (e.g `SYNAPSE_WORKER_TYPES=event_creator,event_creator...`). | ||
|
|
||
| Otherwise, `SYNAPSE_WORKER_TYPES` can either be left empty or unset to spawn no workers | ||
| (leaving only the main process). The container is configured to use redis-based worker | ||
| mode. | ||
|
|
||
| Logs for workers and the main process are logged to stdout and can be viewed with | ||
| standard `docker logs` tooling. Worker logs contain their worker name | ||
| after the timestamp. | ||
|
|
||
| Setting `SYNAPSE_WORKERS_WRITE_LOGS_TO_DISK=1` will cause worker logs to be written to | ||
| `<data_dir>/logs/<worker_name>.log`. Logs are kept for 1 week and rotate every day at 00: | ||
| 00, according to the container's clock. Logging for the main process must still be | ||
| configured by modifying the homeserver's log config in your Synapse data volume. |
Oops, something went wrong.