* Auto trust new signal identities
from signald doku: when a remote key changes, set trust level to TRUSTED_UNVERIFIED instead of UNTRUSTED
I find it much more convenient when new identities are automatically recognized as trusted, as the process to do that manually is cumbersome.
Should this the default behavior, or should i add an option to configure this behavior?
* Added option to trust new signal identities
* Using env file
* Renamed variable
* Corrected typo
* Use fully-qualified Ansible module name
* removed option trust_new_keys
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* if variable to bind an exporter container to a host port is set, have matrix-domain.conf (nginx) support this
* manipulate some variables to account for just port numbers or 0.0.0.0 IPs
* Make sure to use the right variable in the init.yml files
* Update roles/matrix-prometheus-node-exporter/tasks/init.yml
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* Update roles/matrix-prometheus-postgres-exporter/tasks/init.yml
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
* remove extraneous variables and whitespace
Co-authored-by: Luca Bilke <luca@gmail.com>
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
This is what upstream uses and also what
https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1977
used.
Initially, I wanted to make the prefix more unique, in case another
Kakaotalk bridge comes along, but.. it's probably on the new bridge to
come up with a unique puppet prefix, not on us now to override upstream
decisions.
Adds support for: https://src.miscworks.net/fair/matrix-appservice-kakaotalk
This is pretty similar to
https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1977
which just appeared, but has mostly been done independently.
I've taken some inspiration and did some fixups based on that PR.
Thanks to https://github.com/hnarjis for taking the time to contribute!
Notable differences between this branch compared to that PR:
- better naming and documentation around the "configuration" variables
- no unnecessary (5 sec.) intentional delay when starting `matrix-appservice-kakaotalk-node.service`
- stores configuration in `config/`, not in `data/`
- passes configuration as read-only and starts the bridge with (`--no-update`) to ensure no changes are made to it
- starts containers more securely - with `matrix:matrix` user:group (not `root`) and
reduced capabilities (`--cap-drop=ALL`)
- uses `tcp` for communication between the "node" and the appservice (simpler than sharing unix sockets)
- `registration.yaml` which is closer to the one generated by `matrix-appservice-kakaotalk` (no `de.sorunome.msc2409.push_ephemeral` stuff, etc.)
- `registration.yaml` which is more customizable (customizable bot username and prefix for puppets - see `matrix_appservice_kakaotalk_appservice_bot_username` and `matrix_appservice_kakaotalk_user_prefix`)
- less fragile and more extensible bridge permissions configuration via `matrix_appservice_kakaotalk_bridge_permissions`. Doing `{% if matrix_admin %}` in the bridge configuration sometimes causes syntax problems (I hit some myself) and is not ideal. Other bridges should be redone as well.
- configurable command prefix for the bridge, instead of hardcoding `!kt` (see `matrix_appservice_kakaotalk_command_prefix`)
- logging that is more consistent with the rest of the playbook (console / journald only, no logging to files), as well as configurable log level (via `matrix_appservice_kakaotalk_logging_level`)
- somewhat more detailed documentation (`docs/configuring-playbook-bridge-appservice-kakaotalk.md`)
- removed some dead code (data relocation tasks from `tasks/setup_install.yml`, as well as likely unnecessary SQLite -> Postgres migration)
Not doing {% if matrix_admin %} checks in the YAML also fixes some issues
with indentation being incorrect sometimes.
This should be backward compatible, except for mautrix-signal's case
where `matrix_mautrix_signal_bridge_permissions` previously existed
as a string, not a dictionary. `tasks/validate_config.yml` will catch
the problem an even provide a quick fix.
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1894
Because the configuration file is now mounted as readonly and maubot tries to update it on start,
we get this warning:
> Failed to create tempfile to write updated config to disk: [Errno 30] Read-only file system: '/config/tmpfa8vcb3y.yaml'
It doesn't seem to cause issues though.
Because the configuration is no longer overwritten on every bot start, each
next Ansible run should no longer overwrite it again and report a
"changed" task.
* Make interface hidden behind proxy by default
* Remove expose option and replace with http_bind_port
Reasoning: This is a similar binary trigger but allows to bin not on all interfaces
* Clarify maubot admin purpose
* Remove unnecessary edif
* Extend docs to prevent common misconceptions
* Make http_bind_port singular, do not allow multiple values
* Make optional again
Reasoning: setup_install.yml only runs on --tags=setup-all or on --tags=setup-bot-maubot.
If --tags=setup-nginx-proxy or similar commands are run, setup_install.yml will not run and the nginx configuration will be incomplete.
Reference: https://ansible-lint.readthedocs.io/en/latest/default_rules/#var-naming
We don't really fix these, but just suppress them,
because they're like that intentionally.
We try to name variables in a way that is consistent with the
configuration key they control. If the upstream component uses
camelCase, we also need to include camelCase in the variable name.
Reference: https://ansible-lint.readthedocs.io/en/latest/default_rules/#git-latest
Our variable naming is not necessarily consistent across roles.
I've tried to follow the naming conventions of each individual role.
All new variables are suffixed with `_version`, but the prefix may be
somewhat different.
This commit adds a 'matrix-ntfy' role that runs Ntfy server in Docker with
simple configuration, and plumbing to add the role to the playbook.
TODO: documentation, self-check, database persistence.
`localhost` may resolve to `::1` on some IPv6-enabled systems, which will
not work, because we only potentially expose container ports on
`127.0.0.1` when nginx is disabled (`matrix_nginx_proxy_enabled: false`),
not on `::1`.
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1914
This shall indicate that the public url of maubot is here configured the same as matrix_server_fqn_matrix but this must not be the case.
In the config I used the matrix fqnd directly as this part of the config is directly bound to the homeserver we want to connect to (but can not use the internal)
Currently, Synapse workers ignore the X-Forwarded headers, which leads to internal Docker IP addresses randomly appearing in the users' device list.
This adds the `x_forwarded: true` option to the worker config, fixing the issue.
Source: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types
As an alternative, we should probably find a way to run one or a few
more generic workers (which will handle appservice and user_dir stuff) and
update `homeserver.yaml` so that it would point to the name of these workers using
`notify_appservices_from_worker` and `update_user_directory_from_worker` options.
For now, this solves the deprecation, so we can have a peace of mind
going forward.
We're force-setting these worker counts to 0, so that we can clean up
existing homeservers which use these worker types. In the future, these
options will either be removed or repurposed (so that they transparently
create more generic workers that handle user_dir/appservice loads).
Details here: https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_6.html#id36
Basically:
```yaml
- name: Prior to 2.13
debug:
msg: '[1] + {{ [2] }}'
- name: 2.13 and forward
debug:
msg: '{{ [1] + [2] }}'
```
Interestingly, we had been using the new/safe syntax in lofs of places.
We were using the broken one in many others though. Hopefully all
instances were fixed by this patch.
Related to https://gitlab.com/mx-puppet/discord/mx-puppet-discord/-/issues/117
Looks like the bridge is too quick to start and fails to initialize
itself by connecting to Synapse. It's mostly observed after a system
reboot, because Synapse (and everything else) is slower to start.
Once mx-puppet-discord fails to initialize itself, a "No relay found"
error will be observed any time you try to relay a Matrix message to
Discord. Relaying messages in the other direction (Discord to Matrix)
also fails.
With this workaround (longer delay on mx-puppet-discord startup), I
observe mx-puppet-discord working well, even after a full reboot.
Of course, a proper fix is preferable, instead of delaying by a magic
number of seconds.
These endpoints should not be proxied to a generic Synapse worker
without other preparation (setting up stream writers, sending traffic
to a specific stream writer, etc.).
Disabling them for now. In the future, we'd like to fix up our awk
script to disable them automatically.
This is a fix up for 058fedff91
This prevented us from keeping our workers reverse-proxying definitions
updated since Synapse v1.54.0.
The last `workers.md` file we could parse is at commit
02632b3504ad4512c5f5a4f859b3fe326b19c788.
Parsing regressed at commit c56bfb08bc071368db23f3b1c593724eb4f205f0,
because the introduction message for `synapse.app.generic_worker` said
"If":
> If a worker is set up to handle a..
.. which made the AWK script think that definitions below were
conditional (which they're not in this case).
This patch fixes up the regex for determining if a line is conditional
or not, so that it doesn't trip up. Hopefully, it doesn't miss something
important.
When setting `matrix_nginx_proxy_enabled: false` and enabling authentication on the metrics endpoint, the htpasswd file is hardcoded to the nginx-proxy container dir, this changes the hardcoded value to a variable so the path can be updated
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1775
Related to https://signald.org/articles/install/docker/#migrating-from-versions-before-0180
> Prior to 0.18.0 the signald container image used the root user, which is not recommended for security reasons. This was fixed in the 0.18.0 release which will start as root, fix permissions on the volume, then drop to the non-root user and start signald. Future images will start as the non-root user, so if you’re upgrading make sure to run 0.18.0 at least once.
> A special tag, 0.18.0-non-root, will be published. it starts as the non-root user and does not fix permissions on the volume.
* Add matrix-registration-bot
This adds an install and uninstall task plus helpers. The bot is disabled by default.
This commit does not include documentation, yet. In short, the bot can be enabled by adding
matrix_bot_matrix_registration_bot_enabled: true
matrix_bot_matrix_registration_bot_matrix_user_password: "verysecret"
matrix_bot_matrix_registration_bot_matrix_admin_token: "supersecret"
to the host_vars
* Change bot username to bot.matrix-registration-bot following convention
* Address smaller remarks, fix local docker build
* Switch to an env file
* Add environment variables extension for additional config
* Add documentation for the matrix-registration-bot
* Add screenshot on how to obtain admin access token
* Use bot as admin to only have one access token (bot and admin api)
* Use cleaner setting of matrix_synapse_registration_requires_token
* Use config file for cleaner more secure usage
* Delete unneeded env
* Rename vars to make usage clear
* Fix typos/wording and add notice about logging out
* Convert configuration to use |to_json
* Reorder role includes
Nothing should be after `matrix-common-after`.
`matrix-bot-matrix-registration-bot` can probably be anywhere, but it makes sense to put it next to the other `matrix-bot-*` roles.
* Minor group_vars/matrix_servers touchups
Co-authored-by: Slavi Pantaleev <slavi@devture.com>
It is very confusing to debug why messages only go from Matrix to Slack
but not from Slack to Matrix. RTM should be enabled by default, as
that's the recommended way to make this work.
We no longer validate that there's an IP address defined.
Seems like Coturn can start without one as well, so there's no need to
require it.
If people populate `matrix_coturn_turn_external_ip_addresses` directly
to specify multiple addresses, they can leave
`matrix_coturn_turn_external_ip_address` empty.
We use the "select not equal to empty string" thing in the for loop
to avoid `matrix_coturn_turn_external_ip_address` leading to
`matrix_coturn_turn_external_ip_addresses: ['']` leading to
`external-ip=` in the Coturn configuration.
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/1741
People often report and ask about these "failures".
More-so previously, when the `docker kill/rm` output was collected,
but it still happens now when people do `systemctl status
matrix-something` and notice that it says "FAILURE".
Suppressing to avoid further time being wasted on saying "this is
expected".
This switches the playbook from devture/rust-synapse-compress-state (a
container image which wraps the upstream-prebuilt amd64 binary of
rust-synapse-compress-state) to registry.gitlab.com/mb-saces/rust-synapse-compress-state
(https://gitlab.com/mb-saces/rust-synapse-compress-state), which builds
rust-synapse-compress-state from source and provides a multi-arch image
that currently works on amd64 and arm64.
Ideally, we'll stop using `:latest` and arm32 support will be made
available upstream as well at some point.
Discussed here: https://gitlab.com/mb-saces/rust-synapse-compress-state/-/issues/1
Changed the commit hash in matrix_synapse_ext_spam_checker_mjolnir_antispam_git_version to latest. Fixes a Synapse ImportError with mjolnir v1.4.1 leading to a Synapse crash-loop.
`roles/matrix-synapse/vars/workers.yml` has not been updated here,
because running `roles/matrix-synapse/files/workers-doc-to-yaml.awk`
seems to cause some trouble with the current `workers.md` file
available at https://github.com/matrix-org/synapse/raw/master/docs/workers.md,
namely lots of:
> FIXME: ADDITIONAL CONDITIONS REQUIRED: to be enabled manually
lines and commented out regex lines.
This is something that remains to be investigated/fixed.
For an unknown reason prometheus ignored the given "numeric" index and replaced it by 1. This made it not work properly, plus multiple workers of same types were not differentiable. With a "string" index, it works as intended.
The `to_nice_yaml` helper will by default wrap any string YAML values on
the first space after column 80. This can in worst case yield invalid
YAML syntax. More details in Ansible's documentation here:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#formatting-data-yaml-and-json
In short, you need to explicitly provide a custom width argument of a
high number of some kind to avoid the line wrapping.
matrix_synapse_federation_port_enabled can be disabled by users, for
example, when one wants to use the same port for client and federation
requests (docs/configuring-playbook-federation.md).
Related to https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1682
Previously, when matrix-postgres was disabled, we were setting
`matrix_mautrix_twitter_database_engine` to an invalid empty value.
Now, we always hardcode `matrix_mautrix_twitter_database_engine: postgres`,
but set/unset the database hostname and password values instead.
This allows disabling the automatic portal bridging, meaning bridges
must be manually setup via self-servicing, by setting:
matrix_appservice_discord_bridge_enableSelfServiceBridging: true
"Community" support
- has been removed from mautrix/facebook in v0.3.3:
31cac6fb5e
- has been removed from mautrix/signal in v0.2.2:
1f27a608a6
- will be removed in the next mautrix/instagram release:
e2ae1ca503
- will be removed in the next mautrix/twitter release:
3893075265
For now, we disable the new `com.devture.shared_secret_auth` login type
by default, because it causes problems with Element:
https://github.com/vector-im/element-web/issues/19605
This also becomes the first module to use the new Synapse module system
that got introduced in Synapse v1.46.0.
Despite these upgrades, things should remain functionally identical
as far as bridges, matrix-corporal or other consumers are concerned.
Fedora 35 is:
- `ansible_os_family = 'RedHat'`
- `ansible_distribution_major_version = '35'`
Our RedHat checks against v7/v8 are really for RHEL derivatives (CentOS, Rockylinux,
AlmaLinux), but the same checks (by coincidence) apply for Fedora 35.
The problem is that `'35' > '7'` (comparing these as strings) is
`false`.
This patch makes sure that we always cast
`ansible_distribution_major_version` to an integer.
Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/1610
This also removes the `matrix_synapse_version_arm64` variable we've
been dragging around for a long time.
Since https://github.com/matrix-org/synapse/pull/11810, a multiarch Synapse
container image (for AMD64 and ARM64) is released at the same time.
Not hardcoding 'CentOS' and using the OS family ('RedHat') instead,
we now behave better on Rockylinux and AlmaLinux, etc.
With that said, we may or may not fully support CentOS/Rockylinux/AlmaLinux v8 yet.
Certain things were improved in
https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/300.
v8 support is discussed here: https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/300
Certain things (firewalld?) may still be problematic. This patch does not try to address those.
If the remaining issues are confirmed to be fixed in the future, we can mark v8 as supported.
Reverts b1b4ba501f, 90c9801c56, a3c84f78ca, ..
I haven't really traced it (yet), but on some servers, I'm observing
`ansible-playbook ... --tags=start` completing very slowly, waiting
to stop services. I can't reproduce this on all Matrix servers I manage.
I suspect that either the systemd version is to blame or that some
specific service is not responding well to some `docker kill/rm` command.
`ExecStop` seems to work great in all cases and it's what we've been
using for a very long time, so I'm reverting to that.
1.0.2 is the first container image tag that is available as a multi-arch image
with support for linux/amd64, linux/arm64/v8 (arm64) and linux/arm/v7 (arm32),
so self-building is no longer necessary on all these platforms.
4.95-r0-1 was problematic, because `/etc/exim/exim.conf` in the
container had the wrong permissions (writable by the `exim` user).
Fixed in 697f3cff7e
which is built as 4.95-r0-2
4.95-r0-1 is the first container image tag that is available as a multi-arch image
with support for linux/amd64, linux/arm64/v8 (arm64) and linux/arm/v7 (arm32),
so self-building is no longer necessary on all these platforms.
2.2.3 is the first container image tag that is available as a multi-arch image
with support for linux/amd64, linux/arm64/v8 (arm64) and linux/arm/v7 (arm32),
so self-building is no longer necessary on all these platforms.
Aine
Slavi Pantaleev
IUCCA
MdotAmaan
snailed
Aine
MdotAmaan
Maxdeso
Aaron Raimist
Simone
Julian-Samuel Gebühr
Cody Wyatt Neiman
teutates
Chirayu Desai
Julian Foad
ThellraAK
3hhh
Kabir Kwatra
ThellraAK
Stuart Mumford
David Mehren
Vladimir Panteleev
Ruben Hias
Erick Wibben
GoliathLabs
Vicx
Paul B
Toni Spets
Daniel Sonck
Didier 'OdyX' Raboud
HarHarLinks
Brandon Kraft
Sekki21956
Andrea Tartaglia
Matthew Cengia
borisrunakov
Adriel Sand
Olivér Falvai
Lunar
Arthur Brugière
joecool1029
Christos Karamolegkos
Yan Minagawa
Hefty Zauk
heftyzauk
SaltireSoul
Yan Minagawa
PC-Admin
slikie
László Várady
Luke
RoiArthurB
pulmonarycosignerkindness
Catalan Lover
Peter
Thom Wiggers
Jim Myhrberg
Sweeny
Luis
Peter Schümann
Alejo Diaz
GoMatrixHosting
PC-Admin
Wm Salt Hale
Petteri Pucilowski
Wunderharke
Marko Weltzer
Marko Weltzer
Marko Weltzer
AtomHare