25307 Commits

Author SHA1 Message Date
Eric Eastwood
41a2762e58
Be mindful of other logging context filters in 3rd-party code (#19068)
Be mindful that Synapse can be run alongside other code in the same
Python process. We shouldn't overwrite fields on given log record unless
we know it's relevant to Synapse.

(no clobber)


### Background

As part of Element's plan to support a light form of vhosting (virtual
host) (multiple instances of Synapse in the same Python process), we're
currently diving into the details and implications of running multiple
instances of Synapse in the same Python process.

"Per-tenant logging" tracked internally by
https://github.com/element-hq/synapse-small-hosts/issues/48
2025-10-31 10:12:05 -05:00
Erik Johnston
3ccc5184e0
Fix schema lint script to understand CREATE TABLE IF NOT EXISTS (#19020)
The schema lint tries to make sure we don't add or remove indices in
schema files (rather than as background updates), *unless* the table was
created in the same schema file.

The regex to pull out the `CREATE TABLE` SQL incorrectly didn't
recognise `IF NOT EXISTS`.

There is a test delta file that shows that we accept different types of
`CREATE TABLE` and `CREATE INDEX` statements, as well as an index
creation that doesn't have a matching create table (to show that we do
still catch it). The test delta should be removed before merge.
2025-10-31 13:16:47 +00:00
V02460
07e7980572
Fix Rust’s confusing lifetime lint (#19118)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2025-10-31 12:09:13 +00:00
V02460
3595ff921f
Pydantic v2 (#19071)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Andrew Morgan <andrew@amorgan.xyz>
2025-10-31 09:22:22 +00:00
Andrew Morgan
300c5558ab
Update check_dependencies to support markers (#19110) 2025-10-30 21:33:29 +00:00
Eric Eastwood
c0b9437ab6
Fix lost logcontext when using timeout_deferred(...) (#19090)
Fix lost logcontext when using `timeout_deferred(...)` and things
actually timeout.

Fix https://github.com/element-hq/synapse/issues/19087 (our HTTP client
times out requests using `timeout_deferred(...)`
Fix https://github.com/element-hq/synapse/issues/19066 (`/sync` uses
`notifier.wait_for_events()` which uses `timeout_deferred(...)` under
the hood)


### When/why did these lost logcontext warnings start happening?

```
synapse.logging.context - 107 - WARNING - sentinel - Expected logging context call_later but found POST-2453

synapse.logging.context - 107 - WARNING - sentinel - Expected logging context call_later was lost
```

In https://github.com/element-hq/synapse/pull/18828, we switched
`timeout_deferred(...)` from using `reactor.callLater(...)` to
[`clock.call_later(...)`](3b59ac3b69/synapse/util/clock.py (L224-L313))
under the hood. This meant it started dealing with logcontexts but our
`time_it_out()` callback didn't follow our [Synapse logcontext
rules](3b59ac3b69/docs/log_contexts.md).
2025-10-30 11:49:15 -05:00
Eric Eastwood
f0aae62f85
Cheaper logcontext debug logs (random_string_insecure_fast(...)) (#19094)
Follow-up to https://github.com/element-hq/synapse/pull/18966

During the weekly Backend team meeting, it was mentioned that
`random_string(...)` was taking a significant amount of CPU on
`matrix.org`. This makes sense as it relies on
[`secrets.choice(...)`](https://docs.python.org/3/library/secrets.html#secrets.choice),
a cryptographically secure function that is inherently computationally
expensive. And since https://github.com/element-hq/synapse/pull/18966,
we're calling `random_string(...)` as part of a bunch of logcontext
utilities.

Since we don't need cryptographically secure random strings for our
debug logs, this PR is introducing a new `random_string_insecure_fast(...)`
function that uses
[`random.choice(...)`](https://docs.python.org/3/library/random.html#random.choice)
which uses pseudo-random numbers that are "both fast and threadsafe".
2025-10-30 11:47:53 -05:00
Andrew Morgan
349599143e
Move reading of multipart response into try body (#19062) 2025-10-30 15:22:52 +00:00
Eric Eastwood
2c4057bf93
Prevent duplicate logging setup when running multiple Synapse instances (#19067)
Be mindful that it's possible to run Synapse multiple times in the same
Python process. So we only need to do some part of the logging setup
once.

- We only need to setup the global log record factory and context filter
once
 - We only need to redirect Twisted logging once


### Background

As part of Element's plan to support a light form of vhosting (virtual
host) (multiple instances of Synapse in the same Python process), we're
currently diving into the details and implications of running multiple
instances of Synapse in the same Python process.

"Per-tenant logging" tracked internally by
https://github.com/element-hq/synapse-small-hosts/issues/48
2025-10-30 10:21:56 -05:00
Andrew Morgan
f54ddbcace
Prevent duplicate GH releases being created during Synapse release process (#19096) 2025-10-30 12:40:53 +00:00
Andrew Morgan
728512918e
Exclude .lock file from /usr/local when building docker images (#19107) 2025-10-30 10:17:35 +00:00
Andrew Ferrazzutti
e0838c2567
Drop Python 3.9, bump tests/builds to Python 3.10 (#19099)
Python 3.9 EOL is on 2025-10-31
2025-10-29 12:15:00 -05:00
dependabot[bot]
32998d07d2
Bump sigstore/cosign-installer from 3.10.0 to 4.0.0 (#19075)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-29 15:39:07 +00:00
Eric Eastwood
6facf98a3a
Be mindful of other SIGHUP handlers in 3rd-party code (#19095)
Be mindful that Synapse can be run alongside other code in the same
Python process. We shouldn't clobber other `SIGHUP` handlers as only one
can be set at time.

(no clobber)

### Background

As part of Element's plan to support a light form of vhosting (virtual
host) (multiple instances of Synapse in the same Python process), we're
currently diving into the details and implications of running multiple
instances of Synapse in the same Python process.

"Per-tenant logging" tracked internally by
https://github.com/element-hq/synapse-small-hosts/issues/48

Relevant to logging as we use a `SIGHUP` to reload log config in
Synapse.
2025-10-29 10:28:05 -05:00
Eric Eastwood
0417296b9f
Remove logcontext problems caused by awaiting raw deferLater(...) (#19058)
This is a normal
problem where we `await` a deferred without wrapping it in
`make_deferred_yieldable(...)`. But I've opted to replace the usage of
`deferLater` with something more standard for the Synapse codebase.

Part of https://github.com/element-hq/synapse/issues/18905

It's unclear why we're only now seeing these failures happen with the
changes from https://github.com/element-hq/synapse/pull/19057

Example failures seen in
https://github.com/element-hq/synapse/actions/runs/18477454390/job/52645183606?pr=19057

```
builtins.AssertionError: Expected `looping_call` callback from the reactor to start with the sentinel logcontext but saw task-_resumable_task-0-IBzAmHUoepQfLnEA. In other words, another task shouldn't have leaked their logcontext to us.
```
2025-10-29 10:23:10 -05:00
Andrew Morgan
18232871d0 Merge branch 'master' into develop 2025-10-29 11:49:11 +00:00
Andrew Morgan
7897c8f6af
Add a docs page with common steps to review the release notes (#19109) 2025-10-29 11:32:33 +00:00
Andrew Morgan
facb81d97b Add Debian trixie notice to the top of the release notes v1.141.0 2025-10-29 11:03:56 +00:00
Andrew Morgan
1090c3ec81 Move MacOS wheels deprecation notice to the top 2025-10-29 11:02:46 +00:00
Andrew Morgan
5f9f3d72b8 1.141.0 2025-10-29 11:01:53 +00:00
Andrew Ferrazzutti
dc33ef90d3
Update docs on downstream Debian package (#19100) 2025-10-28 17:25:16 -05:00
Andrew Ferrazzutti
a07dd43ac4
Use Pillow's non-experimental getexif (#19098)
It has been available since Pillow 6, and Synapse is now pinned on
Pillow >=10.0.1.

Found this while looking at Debian-shipped dependencies, and figured
this may as well be updated.
2025-10-28 13:11:45 -05:00
Andrew Morgan
086b887f29 Merge branch 'release-v1.141' into develop 2025-10-28 12:44:07 +00:00
dependabot[bot]
e23e7ae48f
Bump actions/upload-artifact from 4 to 5 (#19106)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-28 11:56:29 +00:00
dependabot[bot]
1a78fc8a65
Bump pyyaml from 6.0.2 to 6.0.3 (#19105)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-28 11:55:12 +00:00
dependabot[bot]
66a42d4e54
Bump hiredis from 3.2.1 to 3.3.0 (#19103)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-28 11:54:00 +00:00
dependabot[bot]
cb0ed5ec76
Bump actions/download-artifact from 5.0.0 to 6.0.0 (#19102)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-28 11:53:36 +00:00
dependabot[bot]
0d20f762cb
Bump reqwest from 0.12.23 to 0.12.24 (#19077)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-28 11:49:26 +00:00
dependabot[bot]
5cfe873146
Bump tokio from 1.47.1 to 1.48.0 (#19076)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-28 11:48:44 +00:00
dependabot[bot]
77c6905805
Bump regex from 1.11.3 to 1.12.2 (#19074)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-28 11:47:23 +00:00
Andrew Morgan
7106f67470 Move MacOS wheel deprecation message to top of changelog v1.141.0rc2 2025-10-28 10:22:23 +00:00
Andrew Morgan
d8e6fcced7 1.141.0rc2 2025-10-28 10:20:39 +00:00
Andrew Morgan
db9a61c30f
Fix bcrypt errors preventing users from being able to log in (#19101) 2025-10-28 10:16:02 +00:00
dependabot[bot]
634f7cf18b
Bump types-psycopg2 from 2.9.21.20250915 to 2.9.21.20251012 (#19054)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-27 16:58:33 +00:00
dependabot[bot]
f6ef9c129a
Bump stefanzweifel/git-auto-commit-action from 6.0.1 to 7.0.0 (#19052)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-27 16:58:05 +00:00
Shay
f1695ac20e
Add an admin API to get the space hierarchy (#19021)
It is often useful when investigating a space to get information about
that space and it's children. This PR adds an Admin API to return
information about a space and it's children, regardless of room
membership. Will not fetch information over federation about remote
rooms that the server is not participating in.
2025-10-24 15:32:16 -05:00
Andrew Ferrazzutti
9d81bb703c
Always treat RETURNING as supported by SQL engines (#19047)
Can do this now that SQLite 3.35.0 added support for `RETURNING`.

> The RETURNING syntax has been supported by SQLite since version 3.35.0
(2021-03-12).
>
> *-- https://sqlite.org/lang_returning.html*

This also bumps the minimum supported SQLite version according to
Synapse's [deprecation
policy](https://element-hq.github.io/synapse/latest/deprecation_policy.html#platform-dependencies).

Fix https://github.com/element-hq/synapse/issues/17577
2025-10-24 13:21:49 -05:00
dependabot[bot]
40893be93c
Bump idna from 3.10 to 3.11 (#19053)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-24 13:24:16 +01:00
dependabot[bot]
1419b35a40
Bump ijson from 3.4.0 to 3.4.0.post0 (#19051)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-24 13:18:25 +01:00
dependabot[bot]
a2fa61d1b5
Bump msgpack from 1.1.1 to 1.1.2 (#19050)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-24 13:17:35 +01:00
Andrew Morgan
123eff1bc0
Update poetry dev dependencies name (#19081) 2025-10-24 11:19:40 +01:00
Andrew Morgan
a092d2053a
Fix deprecation warning in release script (#19080) 2025-10-24 11:19:04 +01:00
Andrew Morgan
45a042ae88
Remove cibuildwheel pp38* skip selector (#19085) 2025-10-24 10:39:29 +01:00
Andrew Morgan
72d0de9f30
Don't exit the release script if there are uncommitted changes (#19088) 2025-10-24 10:39:06 +01:00
Andrew Morgan
5556b491c1
Spruce up generated announcement text in the release script (#19089) 2025-10-24 10:19:44 +01:00
Bryce Servis
b835eb253c
Make optional networking and security settings for Redis more apparent in workers.md (#19073)
I couldn't really find any documentation regarding how to setup TLS
communication between Synapse and Redis, so I looked through the source
code and found it. I figured I should go ahead and document it here.
2025-10-23 10:10:10 -05:00
Andrew Ferrazzutti
fc244bb592
Use type hinting generics in standard collections (#19046)
aka PEP 585, added in Python 3.9

 - https://peps.python.org/pep-0585/
 - https://docs.astral.sh/ruff/rules/non-pep585-annotation/
2025-10-22 16:48:19 -05:00
Eric Eastwood
cba3a814c6
Fix lints on develop (#19092)
Snuck in with
ff242faad0
2025-10-22 10:39:04 -05:00
Andrew Morgan
3b59ac3b69 Merge branch 'release-v1.141' into develop 2025-10-21 16:48:09 +01:00
Andrew Morgan
ff242faad0 Don't exit the release script if there are uncommitted changes
Instead, all the user to fix them and retry.
2025-10-21 16:40:26 +01:00