Currently, Synapse workers ignore the X-Forwarded headers, which leads to internal Docker IP addresses randomly appearing in the users' device list.
This adds the `x_forwarded: true` option to the worker config, fixing the issue.
Source: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types
As an alternative, we should probably find a way to run one or a few
more generic workers (which will handle appservice and user_dir stuff) and
update `homeserver.yaml` so that it would point to the name of these workers using
`notify_appservices_from_worker` and `update_user_directory_from_worker` options.
For now, this solves the deprecation, so we can have a peace of mind
going forward.
We're force-setting these worker counts to 0, so that we can clean up
existing homeservers which use these worker types. In the future, these
options will either be removed or repurposed (so that they transparently
create more generic workers that handle user_dir/appservice loads).
Details here: https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_6.html#id36
Basically:
```yaml
- name: Prior to 2.13
debug:
msg: '[1] + {{ [2] }}'
- name: 2.13 and forward
debug:
msg: '{{ [1] + [2] }}'
```
Interestingly, we had been using the new/safe syntax in lofs of places.
We were using the broken one in many others though. Hopefully all
instances were fixed by this patch.
Related to https://gitlab.com/mx-puppet/discord/mx-puppet-discord/-/issues/117
Looks like the bridge is too quick to start and fails to initialize
itself by connecting to Synapse. It's mostly observed after a system
reboot, because Synapse (and everything else) is slower to start.
Once mx-puppet-discord fails to initialize itself, a "No relay found"
error will be observed any time you try to relay a Matrix message to
Discord. Relaying messages in the other direction (Discord to Matrix)
also fails.
With this workaround (longer delay on mx-puppet-discord startup), I
observe mx-puppet-discord working well, even after a full reboot.
Of course, a proper fix is preferable, instead of delaying by a magic
number of seconds.
These endpoints should not be proxied to a generic Synapse worker
without other preparation (setting up stream writers, sending traffic
to a specific stream writer, etc.).
Disabling them for now. In the future, we'd like to fix up our awk
script to disable them automatically.
This is a fix up for 058fedff91
This prevented us from keeping our workers reverse-proxying definitions
updated since Synapse v1.54.0.
The last `workers.md` file we could parse is at commit
02632b3504ad4512c5f5a4f859b3fe326b19c788.
Parsing regressed at commit c56bfb08bc071368db23f3b1c593724eb4f205f0,
because the introduction message for `synapse.app.generic_worker` said
"If":
> If a worker is set up to handle a..
.. which made the AWK script think that definitions below were
conditional (which they're not in this case).
This patch fixes up the regex for determining if a line is conditional
or not, so that it doesn't trip up. Hopefully, it doesn't miss something
important.
When setting `matrix_nginx_proxy_enabled: false` and enabling authentication on the metrics endpoint, the htpasswd file is hardcoded to the nginx-proxy container dir, this changes the hardcoded value to a variable so the path can be updated
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1775
Related to https://signald.org/articles/install/docker/#migrating-from-versions-before-0180
> Prior to 0.18.0 the signald container image used the root user, which is not recommended for security reasons. This was fixed in the 0.18.0 release which will start as root, fix permissions on the volume, then drop to the non-root user and start signald. Future images will start as the non-root user, so if you’re upgrading make sure to run 0.18.0 at least once.
> A special tag, 0.18.0-non-root, will be published. it starts as the non-root user and does not fix permissions on the volume.
* Add matrix-registration-bot
This adds an install and uninstall task plus helpers. The bot is disabled by default.
This commit does not include documentation, yet. In short, the bot can be enabled by adding
matrix_bot_matrix_registration_bot_enabled: true
matrix_bot_matrix_registration_bot_matrix_user_password: "verysecret"
matrix_bot_matrix_registration_bot_matrix_admin_token: "supersecret"
to the host_vars
* Change bot username to bot.matrix-registration-bot following convention
* Address smaller remarks, fix local docker build
* Switch to an env file
* Add environment variables extension for additional config
* Add documentation for the matrix-registration-bot
* Add screenshot on how to obtain admin access token
* Use bot as admin to only have one access token (bot and admin api)
* Use cleaner setting of matrix_synapse_registration_requires_token
* Use config file for cleaner more secure usage
* Delete unneeded env
* Rename vars to make usage clear
* Fix typos/wording and add notice about logging out
* Convert configuration to use |to_json
* Reorder role includes
Nothing should be after `matrix-common-after`.
`matrix-bot-matrix-registration-bot` can probably be anywhere, but it makes sense to put it next to the other `matrix-bot-*` roles.
* Minor group_vars/matrix_servers touchups
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
It is very confusing to debug why messages only go from Matrix to Slack
but not from Slack to Matrix. RTM should be enabled by default, as
that's the recommended way to make this work.
We no longer validate that there's an IP address defined.
Seems like Coturn can start without one as well, so there's no need to
require it.
If people populate `matrix_coturn_turn_external_ip_addresses` directly
to specify multiple addresses, they can leave
`matrix_coturn_turn_external_ip_address` empty.
We use the "select not equal to empty string" thing in the for loop
to avoid `matrix_coturn_turn_external_ip_address` leading to
`matrix_coturn_turn_external_ip_addresses: ['']` leading to
`external-ip=` in the Coturn configuration.
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1741
People often report and ask about these "failures".
More-so previously, when the `docker kill/rm` output was collected,
but it still happens now when people do `systemctl status
matrix-something` and notice that it says "FAILURE".
Suppressing to avoid further time being wasted on saying "this is
expected".
This switches the playbook from devture/rust-synapse-compress-state (a
container image which wraps the upstream-prebuilt amd64 binary of
rust-synapse-compress-state) to registry.gitlab.com/mb-saces/rust-synapse-compress-state
(https://gitlab.com/mb-saces/rust-synapse-compress-state), which builds
rust-synapse-compress-state from source and provides a multi-arch image
that currently works on amd64 and arm64.
Ideally, we'll stop using `:latest` and arm32 support will be made
available upstream as well at some point.
Discussed here: https://gitlab.com/mb-saces/rust-synapse-compress-state/-/issues/1
Changed the commit hash in matrix_synapse_ext_spam_checker_mjolnir_antispam_git_version to latest. Fixes a Synapse ImportError with mjolnir v1.4.1 leading to a Synapse crash-loop.
`roles/matrix-synapse/vars/workers.yml` has not been updated here,
because running `roles/matrix-synapse/files/workers-doc-to-yaml.awk`
seems to cause some trouble with the current `workers.md` file
available at https://github.com/matrix-org/synapse/raw/master/docs/workers.md,
namely lots of:
> FIXME: ADDITIONAL CONDITIONS REQUIRED: to be enabled manually
lines and commented out regex lines.
This is something that remains to be investigated/fixed.
For an unknown reason prometheus ignored the given "numeric" index and replaced it by 1. This made it not work properly, plus multiple workers of same types were not differentiable. With a "string" index, it works as intended.
The `to_nice_yaml` helper will by default wrap any string YAML values on
the first space after column 80. This can in worst case yield invalid
YAML syntax. More details in Ansible's documentation here:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#formatting-data-yaml-and-json
In short, you need to explicitly provide a custom width argument of a
high number of some kind to avoid the line wrapping.
matrix_synapse_federation_port_enabled can be disabled by users, for
example, when one wants to use the same port for client and federation
requests (docs/configuring-playbook-federation.md).
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1682
Previously, when matrix-postgres was disabled, we were setting
`matrix_mautrix_twitter_database_engine` to an invalid empty value.
Now, we always hardcode `matrix_mautrix_twitter_database_engine: postgres`,
but set/unset the database hostname and password values instead.
This allows disabling the automatic portal bridging, meaning bridges
must be manually setup via self-servicing, by setting:
matrix_appservice_discord_bridge_enableSelfServiceBridging: true
"Community" support
- has been removed from mautrix/facebook in v0.3.3:
31cac6fb5e
- has been removed from mautrix/signal in v0.2.2:
1f27a608a6
- will be removed in the next mautrix/instagram release:
e2ae1ca503
- will be removed in the next mautrix/twitter release:
3893075265
For now, we disable the new `com.devture.shared_secret_auth` login type
by default, because it causes problems with Element:
https://github.com/vector-im/element-web/issues/19605
This also becomes the first module to use the new Synapse module system
that got introduced in Synapse v1.46.0.
Despite these upgrades, things should remain functionally identical
as far as bridges, matrix-corporal or other consumers are concerned.
Fedora 35 is:
- `ansible_os_family = 'RedHat'`
- `ansible_distribution_major_version = '35'`
Our RedHat checks against v7/v8 are really for RHEL derivatives (CentOS, Rockylinux,
AlmaLinux), but the same checks (by coincidence) apply for Fedora 35.
The problem is that `'35' > '7'` (comparing these as strings) is
`false`.
This patch makes sure that we always cast
`ansible_distribution_major_version` to an integer.
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1610
This also removes the `matrix_synapse_version_arm64` variable we've
been dragging around for a long time.
Since https://github.com/matrix-org/synapse/pull/11810, a multiarch Synapse
container image (for AMD64 and ARM64) is released at the same time.
Not hardcoding 'CentOS' and using the OS family ('RedHat') instead,
we now behave better on Rockylinux and AlmaLinux, etc.
With that said, we may or may not fully support CentOS/Rockylinux/AlmaLinux v8 yet.
Certain things were improved in
https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/300.
v8 support is discussed here: https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/300
Certain things (firewalld?) may still be problematic. This patch does not try to address those.
If the remaining issues are confirmed to be fixed in the future, we can mark v8 as supported.
Reverts b1b4ba501f, 90c9801c56, a3c84f78ca, ..
I haven't really traced it (yet), but on some servers, I'm observing
`ansible-playbook ... --tags=start` completing very slowly, waiting
to stop services. I can't reproduce this on all Matrix servers I manage.
I suspect that either the systemd version is to blame or that some
specific service is not responding well to some `docker kill/rm` command.
`ExecStop` seems to work great in all cases and it's what we've been
using for a very long time, so I'm reverting to that.
1.0.2 is the first container image tag that is available as a multi-arch image
with support for linux/amd64, linux/arm64/v8 (arm64) and linux/arm/v7 (arm32),
so self-building is no longer necessary on all these platforms.
4.95-r0-1 was problematic, because `/etc/exim/exim.conf` in the
container had the wrong permissions (writable by the `exim` user).
Fixed in 697f3cff7e
which is built as 4.95-r0-2
4.95-r0-1 is the first container image tag that is available as a multi-arch image
with support for linux/amd64, linux/arm64/v8 (arm64) and linux/arm/v7 (arm32),
so self-building is no longer necessary on all these platforms.
Slavi Pantaleev
Aine
Julian-Samuel Gebühr
David Mehren
Aine
Vladimir Panteleev
3hhh
Ruben Hias
roughnecks
Erick Wibben
GoliathLabs
Vicx
Paul B
Toni Spets
Aaron Raimist
Daniel Sonck
Didier 'OdyX' Raboud
HarHarLinks
Brandon Kraft
Sekki21956
Andrea Tartaglia
Matthew Cengia
borisrunakov
Adriel Sand
Olivér Falvai
Lunar
Arthur Brugière
joecool1029
Christos Karamolegkos
Yan Minagawa
Hefty Zauk
heftyzauk
SaltireSoul
Yan Minagawa
PC-Admin
slikie
László Várady
Luke
RoiArthurB
pulmonarycosignerkindness
Catalan Lover
Peter
Thom Wiggers
Jim Myhrberg
Sweeny
Luis
Peter Schümann
Alejo Diaz
GoMatrixHosting
PC-Admin
Wm Salt Hale
Petteri Pucilowski
Wunderharke
Marko Weltzer
Marko Weltzer
Marko Weltzer