Default values will be 1 room per minute, with a burst count of 10.
It's hard to imagine most users will be affected by this default rate,
but it's intentionally non-invasive in case of bots or other users that
need to create rooms at a large rate.
Server admins might want to down-tune this on their deployments.
---------
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
The main goal of this PR is to handle device list changes onto multiple
writers, off the main process, so that we can have logins happening
whilst Synapse is rolling-restarting.
This is quite an intrusive change, so I would advise to review this
commit by commit; I tried to keep the history as clean as possible.
There are a few things to consider:
- the `device_list_key` in stream tokens becomes a
`MultiWriterStreamToken`, which has a few implications in sync and on
the storage layer
- we had a split between `DeviceHandler` and `DeviceWorkerHandler` for
master vs. worker process. I've kept this split, but making it rather
writer vs. non-writer worker, using method overrides for doing
replication calls when needed
- there are a few operations that need to happen on a single worker at a
time. Instead of using cross-worker locks, for now I made them run on
the first writer on the list
---------
Co-authored-by: Eric Eastwood <erice@element.io>
This implements
https://github.com/matrix-org/matrix-spec-proposals/pull/4155, which
adds support for a new account data type that blocks an invite based on
some conditions in the event contents.
---------
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Be able to test `/login/sso/redirect` in Complement
Spawning from
https://github.com/element-hq/sbg/pull/421#discussion_r1854926218 where
we have a proxy that intercepts responses to
`/_matrix/client/v3/login/sso/redirect(/{idpId})` in order to upgrade
them to use OAuth 2.0 Pushed Authorization Requests (PAR). We have some
Complement tests in that codebase that go over this flow and these
changes are required [in order for the URL's to line
up](d648c8ce3f/synapse/rest/client/login.py (L652-L673)).
I have a use case where I'd like the Synapse image to start up a
postgres instance that I can use, but don't want to force Synapse to use
postgres as well.
This commit prevents postgres from being started when it has already
been explicitly enabled elsewhere.
* complement: enable dirty runs
* Add changelog
* Set a low connpool limit when running in Complement
Dirty runs can cause many containers to be running concurrently,
which seems to easily exhaust resources on the host. The increased
speedup from dirty runs also seems to use more db connections on
workers, which are misconfigured currently to have
`SUM(workers * cp_max) > max_connections`, causing
```
FATAL: sorry, too many clients already
```
which results in tests failing.
* Try p=2 concurrency to restrict slowness of servers which causes partial state join tests to flake
* Debug logging
* Only run flakey tests
* Only adjust connection pool limits in worker mode
* Move cp vars to somewhere where they get executed in CI
* Move cp values back to where they actually work
* Debug logging
* Try p=1 to see if this makes worker mode happier
* Remove debug logging
* Fully qualified docker image names for the main Dockerfile and Complement related.
* Fully qualified docker image names for Dockerfiles associated with building Debian release artifacts.
This one is harder and is separate from the other commit in case it wasn't correct or was unwanted. I decided to
do the expansion on the docker images in the Dockerfile itself, instead of the various source places that build
which distribution that is selected, as it would have been more invasive with the scripts breaking up the string
for tagging and such. This one is untested.
* Changelog
* Update docker/Dockerfile-workers
* Update docker/complement/Dockerfile
---------
Co-authored-by: reivilibre <olivier@librepush.net>
* Enable Complement tests for Faster Remote Room Joins on worker-mode
* (dangerous) Add an override to allow Complement to use FRRJ under workers
* Newsfile
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
* Fix race where we didn't send out replication notification
* MORE HACKS
* Fix get_un_partial_stated_rooms_token to take instance_name
* Fix bad merge
* Remove warning
* Correctly advance un_partial_stated_room_stream
* Fix merge
* Add another notify_replication
* Fixups
* Create a separate ReplicationNotifier
* Fix test
* Fix portdb
* Create a separate ReplicationNotifier
* Fix test
* Fix portdb
* Fix presence test
* Newsfile
* Apply suggestions from code review
* Update changelog.d/14752.misc
Co-authored-by: Erik Johnston <erik@matrix.org>
* lint
Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
Co-authored-by: Erik Johnston <erik@matrix.org>
Serving partial join responses is no longer experimental. They will only be served under the stable identifier if the the undocumented config flag experimental.msc3706_enabled is set to true.
Synapse continues to request a partial join only if the undocumented config flag experimental.faster_joins is set to true; this setting remains present and unaffected.
* Expose getting SYNAPSE_WORKER_TYPES from external, allowing override of workers requested.
* Add WORKER_TYPES variable option to complement.sh script that passes requested workers into start_for_complement.sh entrypoint.
* Update docs to reflect this new ability.
* Changelog
* Don't rely on soft wrapping to format long strings
Good idea dklimpel. Thanks for catching that.
Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
* Small nits just noticed in docs.
* Fixup new line in docs.
Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
Doing so in the base postgres image doesn't work with buildah because
changes in a declared VOLUME in the Dockerfile is supposed to be
discarded, cf https://docs.docker.com/engine/reference/builder/#volume
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
When building the docker images for complement testing, copy a preinstalled
complement over from a base image, rather than apt installing it. This avoids
network traffic and is much faster.
