Skip to content
This repository has been archived by the owner on Apr 12, 2024. It is now read-only.

Commit

Permalink
Merge tag 'v1.40.0' into babolivier/dinsic_1.41.0
Browse files Browse the repository at this point in the history
Synapse 1.40.0 (2021-08-10)
===========================

No significant changes.

Synapse 1.40.0rc3 (2021-08-09)
==============================

Features
--------

- Support [MSC3289: room version 8](matrix-org/matrix-spec-proposals#3289). ([\#10449](matrix-org/synapse#10449))

Bugfixes
--------

- Mark the experimental room version from [MSC2716](matrix-org/matrix-spec-proposals#2716) as unstable. ([\#10449](matrix-org/synapse#10449))

Improved Documentation
----------------------

- Fix broken links in `upgrade.md`. Contributed by @dklimpel. ([\#10543](matrix-org/synapse#10543))

Synapse 1.40.0rc2 (2021-08-04)
==============================

Bugfixes
--------

- Fix the `PeriodicallyFlushingMemoryHandler` inhibiting application shutdown because of its background thread. ([\#10517](matrix-org/synapse#10517))
- Fix a bug introduced in Synapse v1.40.0rc1 that could cause Synapse to respond with an error when clients would update read receipts. ([\#10531](matrix-org/synapse#10531))

Internal Changes
----------------

- Fix release script to open the correct URL for the release. ([\#10516](matrix-org/synapse#10516))

Synapse 1.40.0rc1 (2021-08-03)
==============================

Features
--------

- Add support for [MSC2033](matrix-org/matrix-spec-proposals#2033): `device_id` on `/account/whoami`. ([\#9918](matrix-org/synapse#9918))
- Update support for [MSC2716 - Incrementally importing history into existing rooms](matrix-org/matrix-spec-proposals#2716). ([\#10245](matrix-org/synapse#10245), [\#10432](matrix-org/synapse#10432), [\#10463](matrix-org/synapse#10463))
- Update support for [MSC3083](matrix-org/matrix-spec-proposals#3083) to consider changes in the MSC around which servers can issue join events. ([\#10254](matrix-org/synapse#10254), [\#10447](matrix-org/synapse#10447), [\#10489](matrix-org/synapse#10489))
- Initial support for [MSC3244](matrix-org/matrix-spec-proposals#3244), Room version capabilities over the /capabilities API. ([\#10283](matrix-org/synapse#10283))
- Add a buffered logging handler which periodically flushes itself. ([\#10407](matrix-org/synapse#10407), [\#10515](matrix-org/synapse#10515))
- Add support for https connections to a proxy server. Contributed by @Bubu and @dklimpel. ([\#10411](matrix-org/synapse#10411))
- Support for [MSC2285 (hidden read receipts)](matrix-org/matrix-spec-proposals#2285). Contributed by @SimonBrandner. ([\#10413](matrix-org/synapse#10413))
- Email notifications now state whether an invitation is to a room or a space. ([\#10426](matrix-org/synapse#10426))
- Allow setting transaction limit for database connections. ([\#10440](matrix-org/synapse#10440), [\#10511](matrix-org/synapse#10511))
- Add `creation_ts` to "list users" admin API. ([\#10448](matrix-org/synapse#10448))

Bugfixes
--------

- Improve character set detection in URL previews by supporting underscores (in addition to hyphens). Contributed by @srividyut. ([\#10410](matrix-org/synapse#10410))
- Fix events being incorrectly rejected over federation if they reference auth events that the server needed to fetch. ([\#10439](matrix-org/synapse#10439))
- Fix `synapse_federation_server_oldest_inbound_pdu_in_staging` Prometheus metric to not report a max age of 51 years when the queue is empty. ([\#10455](matrix-org/synapse#10455))
- Fix a bug which caused an explicit assignment of power-level 0 to a user to be misinterpreted in rare circumstances. ([\#10499](matrix-org/synapse#10499))

Improved Documentation
----------------------

- Fix hierarchy of providers on the OpenID page. ([\#10445](matrix-org/synapse#10445))
- Consolidate development documentation to `docs/development/`. ([\#10453](matrix-org/synapse#10453))
- Add some developer docs to explain room DAG concepts like `outliers`, `state_groups`, `depth`, etc. ([\#10464](matrix-org/synapse#10464))
- Document how to use Complement while developing a new Synapse feature. ([\#10483](matrix-org/synapse#10483))

Internal Changes
----------------

- Prune inbound federation queues for a room if they get too large. ([\#10390](matrix-org/synapse#10390))
- Add type hints to `synapse.federation.transport.client` module. ([\#10408](matrix-org/synapse#10408))
- Remove shebang line from module files. ([\#10415](matrix-org/synapse#10415))
- Drop backwards-compatibility code that was required to support Ubuntu Xenial. ([\#10429](matrix-org/synapse#10429))
- Use a docker image cache for the prerequisites for the debian package build. ([\#10431](matrix-org/synapse#10431))
- Improve servlet type hints. ([\#10437](matrix-org/synapse#10437), [\#10438](matrix-org/synapse#10438))
- Replace usage of `or_ignore` in `simple_insert` with `simple_upsert` usage, to stop spamming postgres logs with spurious ERROR messages. ([\#10442](matrix-org/synapse#10442))
- Update the `tests-done` Github Actions status. ([\#10444](matrix-org/synapse#10444), [\#10512](matrix-org/synapse#10512))
- Update type annotations to work with forthcoming Twisted 21.7.0 release. ([\#10446](matrix-org/synapse#10446), [\#10450](matrix-org/synapse#10450))
- Cancel redundant GHA workflows when a new commit is pushed. ([\#10451](matrix-org/synapse#10451))
- Mitigate media repo XSS attacks on IE11 via the non-standard X-Content-Security-Policy header. ([\#10468](matrix-org/synapse#10468))
- Additional type hints in the state handler. ([\#10482](matrix-org/synapse#10482))
- Update syntax used to run complement tests. ([\#10488](matrix-org/synapse#10488))
- Fix up type annotations to work with Twisted 21.7. ([\#10490](matrix-org/synapse#10490))
- Improve type annotations for `ObservableDeferred`. ([\#10491](matrix-org/synapse#10491))
- Extend release script to also tag and create GitHub releases. ([\#10496](matrix-org/synapse#10496))
- Fix a bug which caused production debian packages to be incorrectly marked as 'prerelease'. ([\#10500](matrix-org/synapse#10500))
  • Loading branch information
babolivier committed Sep 1, 2021
2 parents 262007f + 9f7c038 commit 82af1a2
Show file tree
Hide file tree
Showing 126 changed files with 4,094 additions and 948 deletions.
43 changes: 39 additions & 4 deletions .github/workflows/release-artifacts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ on:
# we do the full build on tags.
tags: ["v*"]

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

permissions:
contents: write

Expand Down Expand Up @@ -44,12 +48,43 @@ jobs:
distro: ${{ fromJson(needs.get-distros.outputs.distros) }}

steps:
- uses: actions/checkout@v2
- name: Checkout
uses: actions/checkout@v2
with:
path: src
- uses: actions/setup-python@v2
- run: ./src/scripts-dev/build_debian_packages "${{ matrix.distro }}"
- uses: actions/upload-artifact@v2

- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v1
with:
install: true

- name: Set up docker layer caching
uses: actions/cache@v2
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
- name: Set up python
uses: actions/setup-python@v2

- name: Build the packages
# see https://github.com/docker/build-push-action/issues/252
# for the cache magic here
run: |
./src/scripts-dev/build_debian_packages \
--docker-build-arg=--cache-from=type=local,src=/tmp/.buildx-cache \
--docker-build-arg=--cache-to=type=local,mode=max,dest=/tmp/.buildx-cache-new \
--docker-build-arg=--progress=plain \
--docker-build-arg=--load \
"${{ matrix.distro }}"
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
- name: Upload debs as artifacts
uses: actions/upload-artifact@v2
with:
name: debs
path: debs/*
Expand Down
28 changes: 26 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ on:
branches: ["develop", "release-*"]
pull_request:

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
lint:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -340,19 +344,39 @@ jobs:
working-directory: complement/dockerfiles

# Run Complement
- run: go test -v -tags synapse_blacklist,msc2403,msc2946,msc3083 ./tests
- run: go test -v -tags synapse_blacklist,msc2403,msc2946,msc3083 ./tests/...
env:
COMPLEMENT_BASE_IMAGE: complement-synapse:latest
working-directory: complement

# a job which marks all the other jobs as complete, thus allowing PRs to be merged.
tests-done:
if: ${{ always() }}
needs:
- lint
- lint-crlf
- lint-newsfile
- lint-sdist
- trial
- trial-olddeps
- sytest
- portdb
- complement
runs-on: ubuntu-latest
steps:
- run: "true"
- name: Set build result
env:
NEEDS_CONTEXT: ${{ toJSON(needs) }}
# the `jq` incantation dumps out a series of "<job> <result>" lines.
# we set it to an intermediate variable to avoid a pipe, which makes it
# hard to set $rc.
run: |
rc=0
results=$(jq -r 'to_entries[] | [.key,.value.result] | join(" ")' <<< $NEEDS_CONTEXT)
while read job result ; do
if [ "$result" != "success" ]; then
echo "::set-failed ::Job $job returned $result"
rc=1
fi
done <<< $results
exit $rc
101 changes: 101 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,104 @@
Synapse 1.40.0 (2021-08-10)
===========================

No significant changes.


Synapse 1.40.0rc3 (2021-08-09)
==============================

Features
--------

- Support [MSC3289: room version 8](https://github.com/matrix-org/matrix-doc/pull/3289). ([\#10449](https://github.com/matrix-org/synapse/issues/10449))


Bugfixes
--------

- Mark the experimental room version from [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) as unstable. ([\#10449](https://github.com/matrix-org/synapse/issues/10449))


Improved Documentation
----------------------

- Fix broken links in `upgrade.md`. Contributed by @dklimpel. ([\#10543](https://github.com/matrix-org/synapse/issues/10543))


Synapse 1.40.0rc2 (2021-08-04)
==============================

Bugfixes
--------

- Fix the `PeriodicallyFlushingMemoryHandler` inhibiting application shutdown because of its background thread. ([\#10517](https://github.com/matrix-org/synapse/issues/10517))
- Fix a bug introduced in Synapse v1.40.0rc1 that could cause Synapse to respond with an error when clients would update read receipts. ([\#10531](https://github.com/matrix-org/synapse/issues/10531))


Internal Changes
----------------

- Fix release script to open the correct URL for the release. ([\#10516](https://github.com/matrix-org/synapse/issues/10516))


Synapse 1.40.0rc1 (2021-08-03)
==============================

Features
--------

- Add support for [MSC2033](https://github.com/matrix-org/matrix-doc/pull/2033): `device_id` on `/account/whoami`. ([\#9918](https://github.com/matrix-org/synapse/issues/9918))
- Update support for [MSC2716 - Incrementally importing history into existing rooms](https://github.com/matrix-org/matrix-doc/pull/2716). ([\#10245](https://github.com/matrix-org/synapse/issues/10245), [\#10432](https://github.com/matrix-org/synapse/issues/10432), [\#10463](https://github.com/matrix-org/synapse/issues/10463))
- Update support for [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083) to consider changes in the MSC around which servers can issue join events. ([\#10254](https://github.com/matrix-org/synapse/issues/10254), [\#10447](https://github.com/matrix-org/synapse/issues/10447), [\#10489](https://github.com/matrix-org/synapse/issues/10489))
- Initial support for [MSC3244](https://github.com/matrix-org/matrix-doc/pull/3244), Room version capabilities over the /capabilities API. ([\#10283](https://github.com/matrix-org/synapse/issues/10283))
- Add a buffered logging handler which periodically flushes itself. ([\#10407](https://github.com/matrix-org/synapse/issues/10407), [\#10515](https://github.com/matrix-org/synapse/issues/10515))
- Add support for https connections to a proxy server. Contributed by @Bubu and @dklimpel. ([\#10411](https://github.com/matrix-org/synapse/issues/10411))
- Support for [MSC2285 (hidden read receipts)](https://github.com/matrix-org/matrix-doc/pull/2285). Contributed by @SimonBrandner. ([\#10413](https://github.com/matrix-org/synapse/issues/10413))
- Email notifications now state whether an invitation is to a room or a space. ([\#10426](https://github.com/matrix-org/synapse/issues/10426))
- Allow setting transaction limit for database connections. ([\#10440](https://github.com/matrix-org/synapse/issues/10440), [\#10511](https://github.com/matrix-org/synapse/issues/10511))
- Add `creation_ts` to "list users" admin API. ([\#10448](https://github.com/matrix-org/synapse/issues/10448))


Bugfixes
--------

- Improve character set detection in URL previews by supporting underscores (in addition to hyphens). Contributed by @srividyut. ([\#10410](https://github.com/matrix-org/synapse/issues/10410))
- Fix events being incorrectly rejected over federation if they reference auth events that the server needed to fetch. ([\#10439](https://github.com/matrix-org/synapse/issues/10439))
- Fix `synapse_federation_server_oldest_inbound_pdu_in_staging` Prometheus metric to not report a max age of 51 years when the queue is empty. ([\#10455](https://github.com/matrix-org/synapse/issues/10455))
- Fix a bug which caused an explicit assignment of power-level 0 to a user to be misinterpreted in rare circumstances. ([\#10499](https://github.com/matrix-org/synapse/issues/10499))


Improved Documentation
----------------------

- Fix hierarchy of providers on the OpenID page. ([\#10445](https://github.com/matrix-org/synapse/issues/10445))
- Consolidate development documentation to `docs/development/`. ([\#10453](https://github.com/matrix-org/synapse/issues/10453))
- Add some developer docs to explain room DAG concepts like `outliers`, `state_groups`, `depth`, etc. ([\#10464](https://github.com/matrix-org/synapse/issues/10464))
- Document how to use Complement while developing a new Synapse feature. ([\#10483](https://github.com/matrix-org/synapse/issues/10483))


Internal Changes
----------------

- Prune inbound federation queues for a room if they get too large. ([\#10390](https://github.com/matrix-org/synapse/issues/10390))
- Add type hints to `synapse.federation.transport.client` module. ([\#10408](https://github.com/matrix-org/synapse/issues/10408))
- Remove shebang line from module files. ([\#10415](https://github.com/matrix-org/synapse/issues/10415))
- Drop backwards-compatibility code that was required to support Ubuntu Xenial. ([\#10429](https://github.com/matrix-org/synapse/issues/10429))
- Use a docker image cache for the prerequisites for the debian package build. ([\#10431](https://github.com/matrix-org/synapse/issues/10431))
- Improve servlet type hints. ([\#10437](https://github.com/matrix-org/synapse/issues/10437), [\#10438](https://github.com/matrix-org/synapse/issues/10438))
- Replace usage of `or_ignore` in `simple_insert` with `simple_upsert` usage, to stop spamming postgres logs with spurious ERROR messages. ([\#10442](https://github.com/matrix-org/synapse/issues/10442))
- Update the `tests-done` Github Actions status. ([\#10444](https://github.com/matrix-org/synapse/issues/10444), [\#10512](https://github.com/matrix-org/synapse/issues/10512))
- Update type annotations to work with forthcoming Twisted 21.7.0 release. ([\#10446](https://github.com/matrix-org/synapse/issues/10446), [\#10450](https://github.com/matrix-org/synapse/issues/10450))
- Cancel redundant GHA workflows when a new commit is pushed. ([\#10451](https://github.com/matrix-org/synapse/issues/10451))
- Mitigate media repo XSS attacks on IE11 via the non-standard X-Content-Security-Policy header. ([\#10468](https://github.com/matrix-org/synapse/issues/10468))
- Additional type hints in the state handler. ([\#10482](https://github.com/matrix-org/synapse/issues/10482))
- Update syntax used to run complement tests. ([\#10488](https://github.com/matrix-org/synapse/issues/10488))
- Fix up type annotations to work with Twisted 21.7. ([\#10490](https://github.com/matrix-org/synapse/issues/10490))
- Improve type annotations for `ObservableDeferred`. ([\#10491](https://github.com/matrix-org/synapse/issues/10491))
- Extend release script to also tag and create GitHub releases. ([\#10496](https://github.com/matrix-org/synapse/issues/10496))
- Fix a bug which caused production debian packages to be incorrectly marked as 'prerelease'. ([\#10500](https://github.com/matrix-org/synapse/issues/10500))


Synapse 1.39.0 (2021-07-29)
===========================

Expand Down
43 changes: 40 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ source ./env/bin/activate
./scripts-dev/lint.sh path/to/file1.py path/to/file2.py path/to/folder
```

## Run the unit tests.
## Run the unit tests (Twisted trial).

The unit tests run parts of Synapse, including your changes, to see if anything
was broken. They are slower than the linters but will typically catch more errors.
Expand Down Expand Up @@ -186,7 +186,7 @@ SYNAPSE_TEST_LOG_LEVEL=DEBUG trial tests
```


## Run the integration tests.
## Run the integration tests ([Sytest](https://github.com/matrix-org/sytest)).

The integration tests are a more comprehensive suite of tests. They
run a full version of Synapse, including your changes, to check if
Expand All @@ -203,6 +203,43 @@ $ docker run --rm -it -v /path/where/you/have/cloned/the/repository\:/src:ro -v
This configuration should generally cover your needs. For more details about other configurations, see [documentation in the SyTest repo](https://github.com/matrix-org/sytest/blob/develop/docker/README.md).


## Run the integration tests ([Complement](https://github.com/matrix-org/complement)).

[Complement](https://github.com/matrix-org/complement) is a suite of black box tests that can be run on any homeserver implementation. It can also be thought of as end-to-end (e2e) tests.

It's often nice to develop on Synapse and write Complement tests at the same time.
Here is how to run your local Synapse checkout against your local Complement checkout.

(checkout [`complement`](https://github.com/matrix-org/complement) alongside your `synapse` checkout)
```sh
COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh
```

To run a specific test file, you can pass the test name at the end of the command. The name passed comes from the naming structure in your Complement tests. If you're unsure of the name, you can do a full run and copy it from the test output:

```sh
COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh TestBackfillingHistory
```

To run a specific test, you can specify the whole name structure:

```sh
COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh TestBackfillingHistory/parallel/Backfilled_historical_events_resolve_with_proper_state_in_correct_order
```


### Access database for homeserver after Complement test runs.

If you're curious what the database looks like after you run some tests, here are some steps to get you going in Synapse:

1. In your Complement test comment out `defer deployment.Destroy(t)` and replace with `defer time.Sleep(2 * time.Hour)` to keep the homeserver running after the tests complete
1. Start the Complement tests
1. Find the name of the container, `docker ps -f name=complement_` (this will filter for just the Compelement related Docker containers)
1. Access the container replacing the name with what you found in the previous step: `docker exec -it complement_1_hs_with_application_service.hs1_2 /bin/bash`
1. Install sqlite (database driver), `apt-get update && apt-get install -y sqlite3`
1. Then run `sqlite3` and open the database `.open /conf/homeserver.db` (this db path comes from the Synapse homeserver.yaml)


# 9. Submit your patch.

Once you're happy with your patch, it's time to prepare a Pull Request.
Expand Down Expand Up @@ -392,7 +429,7 @@ By now, you know the drill!
# Notes for maintainers on merging PRs etc

There are some notes for those with commit access to the project on how we
manage git [here](docs/dev/git.md).
manage git [here](docs/development/git.md).

# Conclusion

Expand Down
4 changes: 1 addition & 3 deletions debian/build_virtualenv
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,11 @@ esac
# Use --builtin-venv to use the better `venv` module from CPython 3.4+ rather
# than the 2/3 compatible `virtualenv`.

# Pin pip to 20.3.4 to fix breakage in 21.0 on py3.5 (xenial)

dh_virtualenv \
--install-suffix "matrix-synapse" \
--builtin-venv \
--python "$SNAKE" \
--upgrade-pip-to="20.3.4" \
--upgrade-pip \
--preinstall="lxml" \
--preinstall="mock" \
--extra-pip-arg="--no-cache-dir" \
Expand Down
28 changes: 28 additions & 0 deletions debian/changelog
Original file line number Diff line number Diff line change
@@ -1,3 +1,31 @@
matrix-synapse-py3 (1.40.0) stable; urgency=medium

* New synapse release 1.40.0.

-- Synapse Packaging team <packages@matrix.org> Tue, 10 Aug 2021 13:50:48 +0100

matrix-synapse-py3 (1.40.0~rc3) stable; urgency=medium

* New synapse release 1.40.0~rc3.

-- Synapse Packaging team <packages@matrix.org> Mon, 09 Aug 2021 13:41:08 +0100

matrix-synapse-py3 (1.40.0~rc2) stable; urgency=medium

* New synapse release 1.40.0~rc2.

-- Synapse Packaging team <packages@matrix.org> Wed, 04 Aug 2021 17:08:55 +0100

matrix-synapse-py3 (1.40.0~rc1) stable; urgency=medium

[ Richard van der Hoff ]
* Drop backwards-compatibility code that was required to support Ubuntu Xenial.

[ Synapse Packaging team ]
* New synapse release 1.40.0~rc1.

-- Synapse Packaging team <packages@matrix.org> Tue, 03 Aug 2021 11:31:49 +0100

matrix-synapse-py3 (1.39.0) stable; urgency=medium

* New synapse release 1.39.0.
Expand Down
2 changes: 1 addition & 1 deletion debian/compat
Original file line number Diff line number Diff line change
@@ -1 +1 @@
9
10
5 changes: 1 addition & 4 deletions debian/control
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,8 @@ Section: contrib/python
Priority: extra
Maintainer: Synapse Packaging team <packages@matrix.org>
# keep this list in sync with the build dependencies in docker/Dockerfile-dhvirtualenv.
# TODO: Remove the dependency on dh-systemd after dropping support for Ubuntu xenial
# On all other supported releases, it's merely a transitional package which
# does nothing but depends on debhelper (> 9.20160709)
Build-Depends:
debhelper (>= 9.20160709) | dh-systemd,
debhelper (>= 10),
dh-virtualenv (>= 1.1),
libsystemd-dev,
libpq-dev,
Expand Down
4 changes: 1 addition & 3 deletions debian/rules
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,5 @@ override_dh_shlibdeps:
override_dh_virtualenv:
./debian/build_virtualenv

# We are restricted to compat level 9 (because xenial), so have to
# enable the systemd bits manually.
%:
dh $@ --with python-virtualenv --with systemd
dh $@ --with python-virtualenv
Loading

0 comments on commit 82af1a2

Please sign in to comment.