Merge remote-tracking branch 'origin/master' into synapse-workers

CHANGELOG.md

@@ -1,3 +1,191 @@
+# 2021-01-22
+
+## (Breaking Change) Postgres changes that require manual intervention
+
+We've made a lot of changes to our Postgres setup and some manual action is required (described below). Sorry about the hassle.
+
+**TLDR**: people running an [external Postgres server](docs/configuring-playbook-external-postgres.md) don't need to change anything for now. Everyone else (the common/default case) is affected and manual intervention is required.
+
+### Why?
+
+- we had a default Postgres password (`matrix_postgres_connection_password: synapse-password`), which we think is **not ideal for security anymore**. We now ask you to generate/provide a strong password yourself. Postgres is normally not exposed outside the container network, making it relatively secure, but still:
+  - by tweaking the configuration, you may end up intentionally or unintentionally exposing your Postgres server to the local network (or even publicly), while still using the default default credentials (`synapse` + `synapse-password`)
+  - we can't be sure we trust all these services (bridges, etc). Some of them may try to talk to or attack `matrix-postgres` using the default credentials (`synapse` + `synapse-password`)
+  - you may have other containers running on the same Docker network, which may try to talk to or attack `matrix-postgres` using the default credentials (`synapse` + `synapse-password`)
+- our Postgres usage **was overly-focused on Synapse** (default username of `synapse` and default/main database of `homeserver`). Additional homeserver options are likely coming in the future ([Dendrite](https://matrix.org/docs/projects/server/dendrite), [Conduit](https://matrix.org/docs/projects/server/conduit), [The Construct](https://matrix.org/docs/projects/server/construct)), so being too focused on `matrix-synapse` is not great. From now on, Synapse is just another component of this playbook, which happens to have an *additional database* (called `synapse`) on the Postgres server.
+- we try to reorganize things a bit, to make the playbook even friendlier to people running an [external Postgres server](docs/configuring-playbook-external-postgres.md). Work on this will proceed in the future.
+
+So, this is some **effort to improve security** and to **prepare for a brighter future of having more homeserver options** than just Synapse.
+
+### What has really changed?
+
+- the default superuser Postgres username is now `matrix` (used to be `synapse`)
+- the default Postgres database is now `matrix` (used to be `homeserver`)
+- Synapse's database is now `synapse` (used to be `homeserver`). This is now just another "additional database" that the playbook manages for you
+- Synapse's user called `synapse` is just a regular user that can only use the `synapse` database (not a superuser anymore)
+
+### What do I do if I'm using the integrated Postgres server (default)?
+
+By default, the playbook runs an integrated Postgres server for you in a container (`matrix-postgres`). Unless you've explicitly configured an [external Postgres server](docs/configuring-playbook-external-postgres.md), these steps are meant for you.
+
+To migrate to the new setup, expect a few minutes of downtime, while you follow these steps:
+
+1. We believe the steps below are safe and you won't encounter any data loss, but consider [making a Postgres backup](docs/maintenance-postgres.md#backing-up-postgresql) anyway. If you've never backed up Postgres, now would be a good time to try it.
+
+2. Generate a strong password to be used for your superuser Postgres user (called `matrix`). You can use `pwgen -s 64 1` to generate it, or some other tool. The **maximum length** for a Postgres password is 100 bytes (characters). Don't go crazy!
+
+3. Update your playbook's `inventory/host_vars/matrix.DOMAIN/vars.yml` file, adding a line like this:
+```yaml
+matrix_postgres_connection_password: YOUR_POSTGRES_PASSWORD_HERE
+```
+
+.. where `YOUR_POSTGRES_PASSWORD_HERE` is to be replaced with the password you generated during step #2.
+
+4. Stop all services: `ansible-playbook -i inventory/hosts setup.yml --tags=stop`
+5. Log in to the server via SSH. The next commands will be performed there.
+6. Start the Postgres database server: `systemctl start matrix-postgres`
+7. Open a Postgres shell: `/usr/local/bin/matrix-postgres-cli`
+8. Execute the following query, while making sure to **change the password inside** (**don't forget the ending `;`**):
+
+```sql
+CREATE ROLE matrix LOGIN SUPERUSER PASSWORD 'YOUR_POSTGRES_PASSWORD_HERE';
+```
+
+.. where `YOUR_POSTGRES_PASSWORD_HERE` is to be replaced with the password you generated during step #2.
+
+9. Execute the following queries as you see them (no modifications necessary, so you can just **paste them all at once**):
+
+```sql
+CREATE DATABASE matrix OWNER matrix;
+
+ALTER DATABASE postgres OWNER TO matrix;
+ALTER DATABASE template0 OWNER TO matrix;
+ALTER DATABASE template1 OWNER TO matrix;
+
+\c matrix;
+
+ALTER DATABASE homeserver RENAME TO synapse;
+
+ALTER ROLE synapse NOSUPERUSER NOCREATEDB NOCREATEROLE;
+
+\quit
+```
+
+You may need to press *Enter* after pasting the lines above.
+
+10. Re-run the playbook normally: `ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start`
+
+### What do I do if I'm using an external Postgres server?
+
+If you've explicitly configured an [external Postgres server](docs/configuring-playbook-external-postgres.md), there are **no changes** that you need to do at this time.
+
+The fact that we've renamed Synapse's database from `homeserver` to `synapse` (in our defaults) should not affect you, as you're already explicitly defining `matrix_synapse_database_database` (if you've followed our guide, that is). If you're not explicitly defining this variable, you may wish to do so (`matrix_synapse_database_database: homeserver`), to avoid the new `synapse` default and keep things as they were.
+
+
+# 2021-01-20
+
+## (Breaking Change) The mautrix-facebook bridge now requires a Postgres database
+
+A new version of the [mautrix-facebook](https://github.com/tulir/mautrix-facebook) bridge has been released. It's a full rewrite of its backend and the bridge now requires Postgres. New versions of the bridge can no longer run on SQLite.
+
+**TLDR**: if you're NOT using an [external Postgres server](docs/configuring-playbook-external-postgres.md) and have NOT forcefully kept the bridge on SQLite during [The big move to all-on-Postgres (potentially dangerous)](#the-big-move-to-all-on-postgres-potentially-dangerous), you will be automatically upgraded without manual intervention. All you need to do is send a `login` message to the Facebook bridge bot again.
+
+Whether this change requires your intervention depends mostly on:
+- whether you're using an [external Postgres server](docs/configuring-playbook-external-postgres.md). If yes, then [you need to do something](#upgrade-path-for-people-running-an-external-postgres-server).
+- or whether you've force-changed the bridge's database engine to SQLite (`matrix_mautrix_facebook_database_engine: 'sqlite'` in your `vars.yml`) some time in the past (likely during [The big move to all-on-Postgres (potentially dangerous)](#the-big-move-to-all-on-postgres-potentially-dangerous)).
+
+As already mentioned above, you most likely don't need to do anything. If you rerun the playbook and don't get an error, you've been automatically upgraded. Just send a `login` message to the Facebook bridge bot again. Otherwise, read below for a solution.
+
+### Upgrade path for people NOT running an external Postgres server (default for the playbook)
+
+If you're **not running an external Postgres server**, then this bridge either already works on Postgres for you, or you've intentionally kept it back on SQLite with custom configuration (`matrix_mautrix_facebook_database_engine: 'sqlite'` in your `vars.yml`) .
+
+Simply remove that custom configuration from your `vars.yml` file (if it's there) and re-run the playbook. It should upgrade you automatically.
+You'll need to send a `login` message to the Facebook bridge bot again.
+
+Alternatively, [you can stay on SQLite for a little longer](#staying-on-sqlite-for-a-little-longer-temporary-solution).
+
+### Upgrade path for people running an external Postgres server
+
+For people using the internal Postgres server (the default for the playbook):
+- we automatically create an additional `matrix_mautrix_facebook` Postgres database and credentials to access it
+- we automatically adjust the bridge's `matrix_mautrix_facebook_database_*` variables to point the bridge to that Postgres database
+- we use [pgloader](https://pgloader.io/) to automatically import the existing SQLite data for the bridge into the `matrix_mautrix_facebook` Postgres database
+
+If you are using an [external Postgres server](docs/configuring-playbook-external-postgres.md), unfortunately we currently can't do any of that for you.
+
+You have 3 ways to proceed:
+
+- contribute to the playbook to make this possible (difficult)
+- or, do the migration "steps" manually:
+  - stop the bridge (`systemctl stop matrix-mautrix-facebook`)
+  - create a new `matrix_mautrix_facebook` Postgres database for it
+  - run [pgloader](https://pgloader.io/) manually (we import this bridge's data using default settings and it works well)
+  - define `matrix_mautrix_facebook_database_*` variables in your `vars.yml` file (credentials, etc.) - you can find their defaults in `roles/matrix-mautrix-facebook/defaults/main.yml`
+  - switch the bridge to Postgres (`matrix_mautrix_facebook_database_engine: 'postgres'` in your `vars.yml` file)
+  - re-run the playbook (`--tags=setup-all,start`) and ensure the bridge works (`systemctl status matrix-mautrix-facebook` and `journalctl -fu matrix-mautrix-facebook`)
+  - send a `login` message to the Facebook bridge bot again
+- or, [stay on SQLite for a little longer (temporary solution)](#staying-on-sqlite-for-a-little-longer-temporary-solution)
+
+### Staying on SQLite for a little longer (temporary solution)
+
+To keep using this bridge with SQLite for a little longer (**not recommended**), use the following configuration in your `vars.yml` file:
+
+```yaml
+# Force-change the database engine to SQLite.
+matrix_mautrix_facebook_database_engine: 'sqlite'
+
+# Force-downgrade to the last bridge version which supported SQLite.
+matrix_mautrix_facebook_docker_image: "{{ matrix_mautrix_facebook_docker_image_name_prefix }}tulir/mautrix-facebook:da1b4ec596e334325a1589e70829dea46e73064b"
+```
+
+If you do this, keep in mind that **you can't run this forever**. This SQLite-supporting bridge version is not getting any updates and will break sooner or later. The playbook will also drop support for SQLite at some point in the future.
+
+
+# 2021-01-17
+
+## matrix-corporal goes 2.0
+
+[matrix-corporal v2 has been released](https://github.com/devture/matrix-corporal/releases/tag/2.0.0) and the playbook also supports it now.
+
+No manual intervention is required in the common case.
+
+The new [matrix-corporal](https://github.com/devture/matrix-corporal) version is also the first one to support Interactive Authentication. If you wish to enable that (hint: you should), you'll need to set up the [REST auth password provider](docs/configuring-playbook-rest-auth.md). There's more information in [our matrix-corporal docs](docs/configuring-playbook-matrix-corporal.md).
+
+
+# 2021-01-14
+
+## Moving from cronjobs to systemd timers
+
+We no longer use cronjobs for Let's Encrypt SSL renewal and `matrix-nginx-proxy`/`matrix-coturn` reloading. Instead, we've switched to systemd timers.
+
+The largest benefit of this is that we no longer require you to install a cron daemon, thus simplifying our install procedure.
+
+The playbook will migrate you from cronjobs to systemd timers automatically. This is just a heads up.
+
+
+# 2021-01-08
+
+## (Breaking Change) New SSL configuration
+
+SSL configuration (protocols, ciphers) can now be more easily controlled thanks to us making use of configuration presets.
+
+We define a few presets (old, intermediate, modern), following the [Mozilla SSL Configuration Generator](https://ssl-config.mozilla.org/#server=nginx).
+
+A new variable `matrix_nginx_proxy_ssl_preset` controls which preset is used (defaults to `"intermediate"`).
+
+Compared to before, this changes nginx's `ssl_prefer_server_ciphers` to `off`  (used to default to `on`). It also add some more ciphers to the list, giving better performance on mobile devices, and removes some weak ciphers. More information in the [documentation](docs/configuring-playbook-nginx.md).
+
+To revert to the old behaviour, set the following variables:
+
+```yaml
+matrix_nginx_proxy_ssl_ciphers: "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"
+matrix_nginx_proxy_ssl_prefer_server_ciphers: "on"
+```
+
+Just like before, you can still use your own custom protocols by specifying them in `matrix_nginx_proxy_ssl_protocols`. Doing so overrides the values coming from the preset.
+
+
 # 2021-01-03
 
 ## Signal bridging support via mautrix-signal
@@ -48,7 +236,6 @@ If you went with the Postgres migration and it went badly for you (some bridge n
 - re-run the playbook (`ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start`)
 - [get in touch](README.md#support) with us
 
-
 # 2020-12-11
 
 ## synapse-janitor support removed

README.md

@@ -146,7 +146,7 @@ This playbook sets up your server using the following Docker images:
 
 - [devture/matrix-corporal](https://hub.docker.com/r/devture/matrix-corporal/) - [Matrix Corporal](https://github.com/devture/matrix-corporal): reconciliator and gateway for a managed Matrix server (optional)
 
-- [devture/zeratax-matrix-registration](https://hub.docker.com/r/devture/zeratax-matrix-registration/) - [matrix-registration](https://github.com/ZerataX/matrix-registration): a simple python application to have a token based matrix registration (optional)
+- [zeratax/matrix-registration](https://hub.docker.com/r/devture/zeratax-matrix-registration/) - [matrix-registration](https://github.com/ZerataX/matrix-registration): a simple python application to have a token based matrix registration (optional)
 
 - [nginx](https://hub.docker.com/_/nginx/) - the [nginx](http://nginx.org/) web server (optional)
 
@@ -170,7 +170,7 @@ This playbook sets up your server using the following Docker images:
 
 - [turt2live/matrix-appservice-webhooks](https://hub.docker.com/r/turt2live/matrix-appservice-webhooks) - the [Appservice Webhooks](https://github.com/turt2live/matrix-appservice-webhooks) bridge (optional)
 
-- [folivonet/matrix-sms-bridge](https://hub.docker.com/repository/docker/folivonet/matrix-sms-bridge) - the [matrix-sms-brdige](https://github.com/benkuly/matrix-sms-bridge) (optional)
+- [folivonet/matrix-sms-bridge](https://hub.docker.com/repository/docker/folivonet/matrix-sms-bridge) - the [matrix-sms-bridge](https://github.com/benkuly/matrix-sms-bridge) (optional)
 
 - [sorunome/mx-puppet-skype](https://hub.docker.com/r/sorunome/mx-puppet-skype) - the [mx-puppet-skype](https://github.com/Sorunome/mx-puppet-skype) bridge to [Skype](https://www.skype.com) (optional)
 

docs/README.md

@@ -1,6 +1,8 @@
 # Table of Contents
 
-- [Prerequisites](prerequisites.md)
+- [FAQ](faq.md) - lots of questions and answers. Jump to [Prerequisites](prerequisites.md) to avoid reading too much and to just start a guided installation.
+
+- [Prerequisites](prerequisites.md) - go here to a guided installation using this Ansible playbook
 
 - [Configuring your DNS server](configuring-dns.md)
 

docs/configuring-playbook-bridge-mautrix-telegram.md

@@ -12,7 +12,6 @@ matrix_mautrix_telegram_api_id: YOUR_TELEGRAM_APP_ID
 matrix_mautrix_telegram_api_hash: YOUR_TELEGRAM_API_HASH
 ```
 
-
 ## Set up Double Puppeting
 
 If you'd like to use [Double Puppeting](https://github.com/tulir/mautrix-telegram/wiki/Authentication#replacing-telegram-accounts-matrix-puppet-with-matrix-account) (hint: you most likely do), you have 2 ways of going about it.

docs/configuring-playbook-external-postgres.md

@@ -4,6 +4,7 @@ By default, this playbook would set up a PostgreSQL database server on your mach
 If that's alright, you can skip this.
 
 If you'd like to use an external PostgreSQL server that you manage, you can edit your configuration file  (`inventory/host_vars/matrix.<your-domain>/vars.yml`).
+
 It should be something like this:
 
 ```yaml

docs/configuring-playbook-matrix-corporal.md

@@ -11,7 +11,9 @@ The playbook can install and configure [matrix-corporal](https://github.com/devt
 In short, it's a sort of automation and firewalling service, which is helpful if you're instaling Matrix services in a controlled corporate environment.
 See that project's documentation to learn what it does and why it might be useful to you.
 
-If you decide that you'd like to let this playbook install it for you, you'd need to also [set up the Shared Secret Auth password provider module](configuring-playbook-shared-secret-auth.md).
+If you decide that you'd like to let this playbook install it for you, you'd need to also:
+- (required) [set up the Shared Secret Auth password provider module](configuring-playbook-shared-secret-auth.md)
+- (optional, but encouraged) [set up the REST authentication password provider module](configuring-playbook-rest-auth.md)
 
 
 ## Playbook configuration
@@ -24,6 +26,15 @@ You would then need some configuration like this:
 matrix_synapse_ext_password_provider_shared_secret_auth_enabled: true
 matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret: YOUR_SHARED_SECRET_GOES_HERE
 
+# When matrix-corporal is acting as the primary authentication provider,
+# you need to set up the REST authentication password provider module
+# to make Interactive User Authentication work.
+# This is necessary for certain user actions (like E2EE, device management, etc).
+#
+# See configuring-playbook-rest-auth.md
+matrix_synapse_ext_password_provider_rest_auth_enabled: true
+matrix_synapse_ext_password_provider_rest_auth_endpoint: "http://matrix-corporal:41080/_matrix/corporal"
+
 matrix_corporal_enabled: true
 
 matrix_corporal_policy_provider_config: |
@@ -40,9 +51,9 @@ matrix_corporal_policy_provider_config: |
 matrix_corporal_http_api_enabled: true
 matrix_corporal_http_api_auth_token: "AUTH_TOKEN_HERE"
 
-# If you need to change the reconciliator user's id from the default (matrix-corporal)..
+# If you need to change matrix-corporal's user id from the default (matrix-corporal).
 # In any case, you need to make sure this Matrix user is created on your server.
-matrix_corporal_reconciliation_user_id_local_part: "matrix-corporal"
+matrix_corporal_corporal_user_id_local_part: "matrix-corporal"
 
 # Because Corporal peridoically performs lots of user logins from the same IP,
 # you may need raise Synapse's ratelimits.

docs/configuring-playbook-matrix-registration.md

@@ -10,7 +10,7 @@ Use matrix-registration to **create unique registration links**, which people ca
 
 - **an API for creating registration tokens** (unique registration links). This API can be used via `curl` or via the playbook (see [Usage](#usage) below)
 
-- **a user registration page**, where people can use these registration tokens. By default, exposed at `https:///matrix.DOMAIN/matrix-registration`
+- **a user registration page**, where people can use these registration tokens. By default, exposed at `https://matrix.DOMAIN/matrix-registration`
 
 
 ## Installing
@@ -33,15 +33,18 @@ ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start
 
 ## Usage
 
-**matrix-registration** gets exposed at `https:///matrix.DOMAIN/matrix-registration`
+**matrix-registration** gets exposed at `https://matrix.DOMAIN/matrix-registration`
 
 It provides various [APIs](https://github.com/ZerataX/matrix-registration/wiki/api) - for creating registration tokens, listing tokens, disabling tokens, etc. To make use of all of its capabilities, consider using `curl`.
 
-We make the most common API (the one for creating unique registration tokens) easy to use via the playbook.
+We make the most common APIs easy to use via the playbook (see below).
 
-**To create a new user registration token (link)**, use this command:
 
-```
+### Creating registration tokens
+
+To **create a new user registration token (link)**, use this command:
+
+```bash
 ansible-playbook -i inventory/hosts setup.yml \
 --tags=generate-matrix-registration-token \
 --extra-vars="one_time=yes ex_date=2021-12-31"
@@ -51,3 +54,13 @@ The above command creates and returns a **one-time use** token, which **expires*
 Adjust the `one_time` and `ex_date` variables as you see fit.
 
 Share the unique registration link (generated by the command above) with users to let them register on your Matrix server.
+
+
+### Listing registration tokens
+
+To **list the existing user registration tokens**, use this command:
+
+```bash
+ansible-playbook -i inventory/hosts setup.yml \
+--tags=list-matrix-registration-tokens
+```

docs/configuring-playbook-nginx.md

@@ -24,6 +24,27 @@ matrix_nginx_proxy_proxy_matrix_nginx_status_allowed_addresses:
 - 1.1.1.1
 ```
 
+## Adjusting SSL in your server
+
+You can adjust how the SSL is served by the nginx server using the `matrix_nginx_proxy_ssl_preset` variable. We support a few presets, based on the Mozilla Server Side TLS
+Recommended configurations. These presets influence the TLS Protocol, the SSL Cipher Suites and the `ssl_prefer_server_ciphers` variable of nginx.
+Possible values are:
+
+- `"modern"` - For Modern clients that support TLS 1.3, with no need for backwards compatibility
+- `"intermediate"` (**default**) - Recommended configuration for a general-purpose server
+- `"old"` - Services accessed by very old clients or libraries, such as Internet Explorer 8 (Windows XP), Java 6, or OpenSSL 0.9.8
+
+**Be really carefull when setting it to `"modern"`**. This could break comunication with other Matrix servers, limiting your federation posibilities. The
+[Federarion tester](https://federationtester.matrix.org/) also won't work.
+
+Besides changing the preset (`matrix_nginx_proxy_ssl_preset`), you can also directly override these 3 variables:
+
+- `matrix_nginx_proxy_ssl_protocols`: for specifying the supported TLS protocols.
+- `matrix_nginx_proxy_ssl_prefer_server_ciphers`: for specifying if the server or the client choice when negotiating the cipher. It can set to `on` or `off`.
+- `matrix_nginx_proxy_ssl_ciphers`: for specifying the SSL Cipher suites used by nginx.
+
+For more information about these variables, check the `roles/matrix-nginx-proxy/defaults/main.yml` file.
+
 ## Synapse + OpenID Connect for Single-Sign-On
 
 If you want to use OpenID Connect as an SSO provider (as per the [Synapse OpenID docs](https://github.com/matrix-org/synapse/blob/develop/docs/openid.md)), you need to use the following configuration (in your `vars.yml` file) to instruct nginx to forward `/_synapse/oidc` to Synapse:

docs/configuring-playbook.md

@@ -10,7 +10,7 @@ You can then follow these steps inside the playbook directory:
 
 - create a directory to hold your configuration (`mkdir inventory/host_vars/matrix.<your-domain>`)
 
-- copy the sample configuration file (`cp examples/host-vars.yml inventory/host_vars/matrix.<your-domain>/vars.yml`)
+- copy the sample configuration file (`cp examples/vars.yml inventory/host_vars/matrix.<your-domain>/vars.yml`)
 
 - edit the configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`) to your liking. You may also take a look at the various `roles/ROLE_NAME_HERE/defaults/main.yml` files and see if there's something you'd like to copy over and override in your `vars.yml` configuration file.
 

docs/faq.md

@@ -0,0 +1,433 @@
+# Frequently Asked Questions
+
+This documentation page tries to answer various Frequently Asked Questions about all things [Matrix](https://matrix.org/), with a focus on this [Ansible](https://www.ansible.com/) playbook ([What is Ansible? How does it work?](#what-is-ansible-how-does-it-work)).
+
+This FAQ page does not intend to replace the [matrix.org FAQ](https://matrix.org/faq/) (please see that one too).
+
+We've only started this FAQ recently, so it's still somewhat empty.
+
+Also, we encourage you to not dig yourself into a hole by reading way too much. When you've heard enough, proceed to [Prerequisites](prerequisites.md) to get guided into installing Matrix.
+
+
+## Introductory
+
+## Where do I find more questions and answers about Matrix?
+
+This is a Frequently Asked Questions page focused on this [Ansible](https://www.ansible.com/) playbook ([What is Ansible? How does it work?](#what-is-ansible-how-does-it-work)) for deploying a [Matrix](https://matrix.org/) server.
+
+For a lot more generic questions and answers, see the [matrix.org FAQ](https://matrix.org/faq/).
+
+## What is Matrix? What is Element? What is Synapse? Why are you confusing me with so many terms?
+
+[Matrix](https://matrix.org/) is a new type of realtime communication (chat) network, the closest analogy to which is probably "email".
+
+You don't just use the "email" protocols (SMTP, POP3, IMAP) directly though. There's a some *server* somewhere which stores your data (`@gmail.com`, `@yahoo.com`, `@hotmail.com`, `@your-company.com`) and you access using these "email" protocol using use some *client* program (Outlook, Thunderbird, some website, etc).
+
+In the world of the Matrix chat protocol, there are various client programs. The first and currently most full-featured one is called [Element](https://element.io/) (used to be called Riot.im and Vector.im in the past). There are [many other clients](https://matrix.org/clients/). You can switch clients as much as you want until you find the one that is right for you on a given platform (you may use Element on your desktop, but Fluffychat on your phone, etc).
+
+Matrix is also like email due to the fact that are many servers around the world which can all talk to each other (you can send email from `@gmail.com` addresses to `@yahoo.com` and `@hotmail.com` addresses). It's the same with Matrix (`@bob:his-domain.com` can talk to `@alice:her-domain.org`).
+
+If someone else is hosting your Matrix server (you being `@user:matrix.org` or some other public server like this), all you need is a Matrix client program, like Element.
+
+If you'd like to host your own server (you being `@user:your-own-domain.com`), you'd need to set up a Matrix server program, like Synapse.
+
+In short:
+
+- Matrix is the protocol - a set of rules about how the chat network operates
+- Element is a client program you can use to participate on the Matrix chat network via some server (yours or someone else's). There are also [many other client programs](https://matrix.org/clients/).
+- Synapse is a server program you can use to host your very own Matrix server.
+
+This FAQ here mostly focuses on installing Matrix services using the Ansible automation tool. You can learn much more about Matrix in the [matrix.org FAQ](https://matrix.org/faq/).
+
+## People I wish to talk to are not on Matrix. Can I talk to them?
+
+You most likely can. Besides Matrix-native chats, Matrix also supports this concept of "bridging", which allows you to plug other networks into it.
+
+This Ansible playbook can help you install [tens of bridges for various networks](configuring-playbook.md#bridging-other-networks).
+
+Besides setting up your own bridges (preferable), you can also use some [public bridges hosted by others](https://publiclist.anchel.nl/#bridges).
+
+## How do I get started with Matrix?
+
+One of [Matrix](https://matrix.org/)'s distinguishing strengths (compared to other chat networks) is its decentralized nature. There's not just one entity (company, organization) controlling the servers. Rather there's thousands of servers operated by different people - one server being insecure, slow or disrespective toward its users does not affect the rest of the network. To participate in that decentralization in its fullest, consider hosting your own server or using some public server other than the largest/default one (`matrix.org`).
+
+There are 3 ways to get into Martix, depending on your technical ability and needs:
+
+- **using the existing default server** - the easiest way is to use an existing server. The largest public Matrix server is `matrix.org` and it's configured as a default server in clients such as [Element](https://element.io) and many others. Just use Element on the browser via that link (or download the Element app on a smartphone), create an account and start chatting.
+
+- **using some other server** - instead of using the largest public server (`matrix.org`), you can use another public one. Here's a [list of public Matrix servers](https://publiclist.anchel.nl/) to choose from. Again, you download [Element](https://element.io) or [some other client](https://matrix.org/clients/) of your choosing and adjust the homeserver URL during login.
+
+- **using your own server** - running your own server puts you in ultimate control of your data. It also lets you have your own user identifiers (e.g. `@bob:your-domain.com`). See [How do I set up my own Matrix server](#how-do-i-set-up-my-own-matrix-server).
+
+### How do I set up my own Matrix server?
+
+Normally, you'd first choose the [Matrix](https://matrix.org/) server software you'd like to run. At the time of this writing (January/2021), there's only one fully-featured server program, so there's only one reasonable choice. That's [Synapse](https://github.com/matrix-org/synapse).
+
+There are [many guides about installing Synapse](https://matrix.org/docs/guides/#installing-synapse). Using this Ansible playbook is just one way of doing it.
+
+Naturally, we're biased, so our usual recommendation is to go with this [Ansible](https://www.ansible.com/) playbook, instead of installing Synapse (and many many other things around it) manually.
+To get started with the playbook, start at the [Prerequisites](prerequisites.md) page.
+
+### What is Ansible? How does it work?
+
+[Ansible](https://www.ansible.com/) is an automation program. This "playbook" is a collection of tasks/scripts that will set up a [Matrix](https://matrix.org/) server for you, so you don't have to perform these tasks manually.
+
+We have written these automated tasks for you and all you need to do is execute them using the Ansible program.
+
+You can install Ansible and this playbook code repository on your own computer and tell it to install Matrix services at the server living at `matrix.DOMAIN`. We recommend installing Ansible on your own computer.
+
+Alternatively, you can download Ansible and the playbook itself directly on the `matrix.DOMAIN` server.
+
+To learn more, see our [dedicated Ansible documentation page](ansible.md).
+
+### Why use this playbook and not install Synapse and other things manually?
+
+There's various guides telling you how easy it is to install [Synapse](https://github.com/matrix-org/synapse).
+
+Reading this Ansible playbook's documentation, you may also be thinking:
+
+> I don't know what [Ansible](https://www.ansible.com/) is. I don't know what [Docker](https://www.docker.com/) is. This looks more complicated.
+
+.. so you may be leaning toward [installing Synapse manually](https://github.com/matrix-org/synapse/blob/master/INSTALL.md).
+
+The problem with a manual installation is:
+
+- Synapse is written in Python. If not packaged for your distribution, you'd need to install various Python modules, etc., and keep them updated.
+- Synapse requires a [Postgres](https://www.postgresql.org/) database (it can run on SQLite, but that's very much discouraged). So you'd need to install Postgres as well.
+- you may also need a reverse-proxy server in front of it (nginx, Apache), so you'd need to be familiar with that
+- SSL is required, so you'd need to obtain Let's Encrypt (or other free or non-free) certificates for one or more domain names. You'd need to be familiar with [certbot](https://certbot.eff.org/) (when using Let's Encrypt) or similar software.
+- for each additional component you'd like to add (client like [Element](https://element.io), bridge to some other chat network, Integration Manager (sitckers, other services), Identity Manager, etc.), you'll need to spend extra time installing and wiring it with the rest of the system in a way that works.
+- you'll likely get slower updates for all of these components, depending on your distro packaging or your own time and ability
+
+The playbook, on the other hand, installs a bunch of components for you by default, obtains SSL certificates for you, etc. If you'd like, you can enable various bridges and other services with very little effort. All the components are wired to work together.
+
+All services run in Docker containers (most being officially provided by each component's developers), so we're not at the mercy of distro packaging.
+
+### Why use this playbook and not just use the Docker image directly?
+
+Reasons are similar to the reasons for not installing manually.
+
+Besides Synapse, you'd need other things - a Postgres database, likely the [Element](https://element.io) client, etc., etc.
+
+Using the playbook, you get all these components in a way that works well together out of the box.
+
+
+## Server-related
+
+### What kind of server do I need to install Matrix using this Ansible playbook?
+
+We list our server requirements in [Prerequisites](prerequisites.md).
+
+### Why not run Matrix on Kubernetes?
+
+There's no reason not to run Matrix on [Kubernetes](https://kubernetes.io/).
+
+However, that's overly complicated for thousands of us who just want to run a single small (and sometimes not so small) Matrix server, either using "cloud" servers or even a [Raspberry Pi](https://www.raspberrypi.org/) at home.
+
+For us, a Kubernetes-based setup which requires a cluster of multiple computers and is more technically-involved is a no-go.
+
+There are others working on automating a Matrix-on-Kubernetes setup, such as this [Helm](https://helm.sh/) chart: https://github.com/dacruz21/matrix-chart.
+
+### Why don't you use Podman instead of Docker?
+
+We like the philosophy of a daemonless container runtime, but [Podman](https://podman.io) is just not ready for our use case yet.
+
+Learn more about our past experiences/attempts to give Podman a chance, by reading [this issue](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/520).
+
+In short, `alias podman=docker` is a lie (for us).
+
+### Why use Docker?
+
+[Docker](https://www.docker.com/) is one of our 2 hard dependencies (the other one being [systemd](https://systemd.io/)).
+
+It lets us run services in an isolated manner and independently of the (usually old) packages available for distributions.
+
+It also lets us have a unified setup which runs the same across various supported distros (see them on [Prerequisites](prerequisites.md)).
+
+### Is Docker a hard requirement?
+
+Yes. See [Why don't you use Podman instead of Docker?](#is-docker-a-hard-requirement) for why we're not using another container runtime.
+
+All of our services run in containers. It's how we achieve predictability and also how we support tens of different services across lots of distros.
+
+The only thing we need on the distro is systemd and Python (we install Docker ourselves, unless you ask us not to).
+
+### Why don't you use docker-compose?
+
+Instead of using [docker-compose](https://docs.docker.com/compose/), we prefer installing systemd services and scheduling those independently.
+
+There are people who have worked on turning this setup into a docker-compose-based one. See these experiments [here](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/64#issuecomment-603164625).
+
+### Can I run this on a distro without systemd?
+
+No. [systemd](https://systemd.io/) is one of our 2 hard dependencies (the other one being [Docker](https://www.docker.com/)).
+
+### Can I install this on a Raspberry Pi?
+
+Yes, you can. See our [Alternative Architectures](alternative-architectures.md) documentation page.
+
+Whether a Raspberry Pi has enough power to give you a good experience is another question. It depends on your use case.
+
+Also see: [What kind of server specs do I need?](#what-kind-of-server-specs-do-i-need).
+
+### What kind of server specs do I need?
+
+This largely depends on your use case. It's not so much the number of users that you plan to host, but rather the number of large rooms they will join.
+
+Federated rooms with lots of history and containing hundreds of other servers are very heavy CPU-wise and memory-wise.
+
+You can probably use a 1 CPU + 1GB memory server to host hundreds of local users just fine, but as soon as of them joins a federated room like `#matrix:matrix.org` (Matrix HQ) or some IRC-bridged room (say `##linux`), your server will get the need for a lot more power (at least 2GB RAM, etc).
+
+Running Matrix on a server with 1GB of memory is possible (especially if you disable some not-so-important services). See [How do I optimize this setup for a low-power server?](#how-do-i-optimize-this-setup-for-a-low-power-server).
+
+**We recommend starting with a server having at least 2GB of memory** and even then using it sparingly. If you know for sure you'll be joining various large rooms, etc., then going for 4GB of memory or more is a good idea.
+
+Besides the regular Matrix stuff, we also support things like video-conferencing using [Jitsi](configuring-playbook-jitsi.md) and other additional services which (when installed) may use up a lot of memory. Things do add up. Besides the Synapse Matrix server, Jitsi is especially notorious for consuming a lot of resources. If you plan on running Jitsi, we recommend a server with at least 2GB of memory (preferrably more). See our [Jitsi documentation page](configuring-playbook-jitsi.md) to learn how to optimize its memory/CPU usage.
+
+### Can I run this in an LXC container?
+
+If your distro runs within an [LXC container](https://linuxcontainers.org/), you may hit [this issue](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/703). It can be worked around, if absolutely necessary, but we suggest that you avoid running from within an LXC container.
+
+
+## Configuration
+
+### Why install my server at matrix.DOMAIN and not at the base DOMAIN?
+
+It's the same with email servers. Your email address is likely `name@company.com`, not `name@mail.company.com`, even though it's really `mail.company.com` that is really handling your data for `@company.com` email to work.
+
+Using a separate domain name is easier to manage (although it's a little hard to get right at first) and keeps your Matrix server isolated from your website (if you have one), from your email server (if you have one), etc.
+
+We allow `matrix.DOMAIN` to be the Matrix server handling Matrix stuff for `DOMAIN` by [Server Delegation](howto-server-delegation.md). During the installation procedure, we recommend that you set up server delegation using the [.well-known](configuring-well-known.md) method.
+
+If you'd really like to install Matrix services directly on the base domain, see [How do I install on matrix.DOMAIN without involving the base DOMAIN?](#how-do-i-install-on-matrixdomain-without-involving-the-base-domain).
+
+### I don't control anything on the base domain and can't set up delegation to matrix.DOMAIN. What do I do?
+
+If you're not in control of your base domain (or server handling it) at all, you can take a look at [How do I install on matrix.DOMAIN without involving the base DOMAIN?](#how-do-i-install-on-matrixdomain-without-involving-the-base-domain)
+
+### I can't set up HTTPS on the base domain. How will I get Matrix federating?
+
+If you really can't obtain an HTTPS certificate for your base domain, you can take a look at [How do I install on matrix.DOMAIN without involving the base DOMAIN?](#how-do-i-install-on-matrixdomain-without-involving-the-base-domain)
+
+### How do I install on matrix.DOMAIN without involving the base DOMAIN?
+
+This Ansible playbook guides you into installing a server for `DOMAIN` (user identifiers are like this: `@user:DOMAIN`), while the server is at `matrix.DOMAIN`.
+
+We allow `matrix.DOMAIN` to be the Matrix server handling Matrix stuff for `DOMAIN` by [Server Delegation](howto-server-delegation.md). During the installation procedure, we recommend that you set up server delegation using the [.well-known](configuring-well-known.md) method.
+
+If you're fine with uglier identifiers (`@user:matrix.DOMAIN`, which is the equivalent of having an email address like `bob@mail.company.com`, instead of just `bob@company.com`), you can do that as well using the following configuration in your `vars.yml` file:
+
+```yaml
+# This is what your identifiers are like (e.g. `@bob:matrix.YOUR_BASE_DOMAIN`).
+matrix_domain: "matrix.YOUR_BASE_DOMAIN"
+
+# This is where Matrix services
+matrix_server_fqn_matrix: "matrix.YOUR_BASE_DOMAIN"
+
+# This is where you access the Element web UI from (if enabled via `matrix_client_element_enabled: true`; enabled by default).
+# This and the Matrix FQN (see above) are expected to be on the same server.
+#
+# Feel free to use `element.matrix.YOUR_BASE_DOMAIN`, if you'd prefer that.
+matrix_server_fqn_element: "element.YOUR_BASE_DOMAIN"
+
+# This is where you access Dimension (if enabled via `matrix_dimension_enabled: true`; NOT enabled by default).
+#
+# Feel free to use `dimension.matrix.YOUR_BASE_DOMAIN`, if you'd prefer that.
+matrix_server_fqn_dimension: "dimension.YOUR_BASE_DOMAIN"
+
+# This is where you access Jitsi (if enabled via `matrix_jitsi_enabled: true`; NOT enabled by default).
+#
+# Feel free to use `jitsi.matrix.YOUR_BASE_DOMAIN`, if you'd prefer that.
+matrix_server_fqn_jitsi: "jitsi.YOUR_BASE_DOMAIN"
+```
+
+### I don't use the base domain for anything. How am I supposed to set up Server Delegation for Matrix services?
+
+If you don't use your base domain for anything, then it's hard for you to "serve files over HTTPS" on it -- something we ask you to do for the [.well-known](configuring-well-known.md) setup (needed for [Server Delegation](howto-server-delegation.md)).
+
+Luckily, the playbook can set up your Matrix server (at `matrix.DOMAIN`) to also handle traffic for the base domain (`DOMAIN`).
+
+See [Serving the base domain](configuring-playbook-base-domain-serving.md).
+
+### How do I optimize this setup for a low-power server?
+
+You can disable some not-so-important services to save on memory.
+
+```yaml
+# An identity server is not a must.
+matrix_ma1sd_enabled: false
+
+# Disabling this will prevent email-notifications and other such things from working.
+matrix_mailer_enabled: false
+
+# You can also disable this to save more RAM,
+# at the expense of audio/video calls being unreliable.
+matrix_coturn_enabled: true
+
+# This makes Synapse not keep track of who is online/offline.
+#
+# Keeping track of this and announcing such online-status in federated rooms with
+# hundreds of servers inside is insanely heavy (https://github.com/matrix-org/synapse/issues/3971).
+#
+# If your server does not federate with hundreds of others, enabling this doesn't hurt much.
+matrix_synapse_use_presence: false
+```
+
+You can also consider implementing a restriction on room complexity, in order to prevent users from joining very heavy rooms:
+
+```yaml
+matrix_synapse_configuration_extension_yaml: |
+  limit_remote_rooms:
+    enabled: true
+    complexity: 1.0 # this limits joining complex (~large) rooms, can be
+					# increased, but larger values can require more RAM
+```
+
+If you've installed [Jitsi](configuring-playbook-jitsi.md) (not installed by default), there are additional optimizations listed on its documentation page that you can perform.
+
+### I already have Docker on my server. Can you stop installing Docker via the playbook?
+
+Yes, we can stop installing Docker ourselves. Just use this in your `vars.yml` file:
+
+```yaml
+matrix_docker_installation_enabled: true
+```
+
+### I run another webserver on the same server where I wish to install Matrix. What now?
+
+By default, we install a webserver for you (nginx), but you can also use [your own webserver](configuring-playbook-own-webserver.md).
+
+### How is the effective configuration determined?
+
+Configuration variables are defined in multiple places in this playbook and are considered in this order:
+
+- there are defaults coming from each role's defaults file (`role/matrix*/defaults/main.yml`). These variable values aim to be good defaults for when the role is used standalone (outside of this collection of roles, also called playbook).
+
+- then, there are overrides in `group_vars/matrix_servers`, which aim to adjust these "standalone role defaults" to something which better fits the playbook in its entirety.
+
+- finally, there's your `inventory/host_vars/matrix.DOMAIN/vars.yml` file, which is the ultimate override
+
+### What configuration variables are available?
+
+You can discover the variables you can override in each role (`role/matrix*/defaults/main.yml`).
+
+As described in [How is the effective configuration determined?](#how-is-the-effective-configuration-determined), these role-defaults may be overriden by values defined in `group_vars/matrix_servers`.
+
+Refer to both of these for inspiration. Still, as mentioned in [Configuring the playbook](configuring-playbook.md), you're only ever supposed to edit your own `inventory/host_vars/matrix.DOMAIN/vars.yml` file and nothing else inside the playbook (unless you're meaning to contribute new features).
+
+### I'd like to adjust some configuration which doesn't have a corresponding variable. How do I do it?
+
+The playbook doesn't aim to expose all configuration settings for all services using variables.
+Doing so would amount is to hundreds of variables that we have to create and maintain.
+
+Instead, we only try to make some important basics configurable using dedicated variables you can see in each role.
+See [What configuration variables are available?](#what-configuration-variables-are-available).
+
+Besides that, each role (component) aims to provide a `matrix_SOME_COMPONENT_configuration_extension_yaml` (or `matrix_SOME_COMPONENT_configuration_extension_json`) variable, which can be used to override the configuration.
+
+Check each role's `role/matrix*/defaults/main.yml` for the corresponding variable and an example for how use it.
+
+
+## Installation
+
+### How do I run the installation?
+
+See [Installing](installing.md) to learn how to use Ansible to install Matrix services.
+
+Of course, don't just jump straight to Installing. Rather, start at [Prerequisites](prerequisites.md) and get guided from there (into [setting up DNS](configuring-dns.md), [configuring the playbook](configuring-playbook.md), etc).
+
+### I installed Synapse some other way. Can I migrate such a setup to the playbook?
+
+Yes, you can.
+
+You generally need to do a playbook installation (start at the [Prerequisites](prerequisites.md) page), followed by importing your existing data into it.
+
+This Ansible playbook guides you into installing a server for `DOMAIN` (user identifiers are like this: `@user:DOMAIN`), while the server is at `matrix.DOMAIN`. If your existing setup has a server name (`server_name` configuration setting in Synapse's `homeserver.yaml` file) other than the base `DOMAIN`, you may need to tweak some additional variables. This FAQ entry may be of use if you're dealing with a more complicated setup - [How do I install on matrix.DOMAIN without involving the base DOMAIN?](#how-do-i-install-on-matrixdomain-without-involving-the-base-domain)
+
+After configuring the playbook and installing and **before starting** services (done with `ansible-playbook ... --tags=start`) you'd import [your SQLite](importing-synapse-sqlite.md) (or [Postgres](importing-postgres.md)) database and also [import your media store](importing-synapse-media-store.md).
+
+### I've downloaded Ansible and the playbook on the server. It can't connect using SSH.
+
+If you're using the playbook directly on the server, then Ansible doesn't need to connect using SSH.
+
+It can perform a local connection instead. Just set `ansible_connection=local` at the end of the server line in `inventory/hosts` and re-run the playbook.
+
+If you're running Ansible from within a container (one of the possibilities we list on our [dedicated Ansible documentation page](ansible.md)), then using `ansible_connection=local` is not possible.
+
+
+## Troubleshooting
+
+### I get "Error response from daemon: configured logging driver does not support reading" when I do `docker logs matrix-synapse`.
+
+See [How can I see the logs?](#how-can-i-see-the-logs).
+
+### How can I see the logs?
+
+We utilize [systemd/journald](https://www.freedesktop.org/software/systemd/man/systemd-journald.service.html#Description) for logging.
+
+To see logs for Synapse, run `journalctl -fu matrix-synapse.service`. You may wish to see the [manual page for journalctl](https://www.commandlinux.com/man-page/man1/journalctl.1.html).
+
+Available service names can be seen by doing `ls /etc/systemd/system/matrix*.service` on the server.
+
+Some services also log to files in `/matrix/*/data/..`, but we're slowly moving away from that.
+
+We also disable Docker logging, so you can't use `docker logs matrix-*` either. We do this to prevent useless double (or even tripple) logging and to avoid having to rotate log files.
+
+We just simply delegate logging to journald and it takes care of persistenec and expiring old data.
+
+Also see: [How long do systemd/journald logs persist for?](#how-long-do-systemdjournald-logs-persist-for)
+
+### How long do systemd/journald logs persist for?
+
+On some distros, the journald logs are just in-memory and not persisted to disk.
+
+Consult (and feel free to adjust) your distro's journald logging configuration in `/etc/systemd/journald.conf`.
+
+To enable persistence and put some limits on how large the journal log files can become, adjust your configuration like this:
+
+```ini
+[Journal]
+RuntimeMaxUse=200M
+SystemMaxUse=1G
+RateLimitInterval=0
+RateLimitBurst=0
+Storage=persistent
+```
+
+
+## Maintenance
+
+### Do I need to do anything to keep my Matrix server updated?
+
+Yes. We don't update anything for you automatically.
+
+See our [documentation page about upgrading services](maintenance-upgrading-services.md).
+
+### How do I move my existing installation to another (VM) server?
+
+If you have an existing installation done using this Ansible playbook, you can easily migrate that to another server using [our dedicated server migration guide](maintenance-migrating.md).
+
+If your previous installation is done in some other way (not using this Ansible playbook), see [I installed Synapse some other way. Can I migrate such a setup to the playbook?](#i-installed-synapse-some-other-way-can-i-migrate-such-a-setup-to-the-playbook).
+
+### How do I back up the data on my server?
+
+We haven't document this properly yet, but the general advice is to:
+
+- back up Postgres by making a database dump. See [Backing up PostgreSQL](maintenance-postgres.md#backing-up-postgresql)
+
+- back up all `/matrix` files, except for `/matrix/postgres/data` (you already have a dump) and `/matrix/postgres/data-auto-upgrade-backup` (this directory may exist and contain your old data if you've [performed a major Postgres upgrade](maintenance-postgres.md#upgrading-postgresql)).
+
+You can later restore these roughly like this:
+
+- restore the `/matrix` directory and files on the new server manually
+- run the playbook again (see [Installing](installing.md)), but **don't** start services yet (**don't run** `... --tags=start`). This step will fix any file permission mismatches and will also set up additional software (Docker, etc.) and files on the server (systemd service, etc.).
+- perform a Postgres database import (see [Importing Postgres](importing-postgres.md)) to restore your database backup
+- start services (see [Starting the services](installing.md#starting-the-services))
+
+If your server's IP address has changed, you may need to [set up DNS](configuring-dns.md) again.
+
+### What is this `/matrix/postgres/data-auto-upgrade-backup` directory that is taking up so much space?
+
+When you [perform a major Postgres upgrade](maintenance-postgres.md#upgrading-postgresql), we save the the old data files in `/matrix/postgres/data-auto-upgrade-backup`, just so you could easily restore them should something have gone wrong.
+
+After verifying that everything still works after the Postgres upgrade, you can safely delete `/matrix/postgres/data-auto-upgrade-backup`

docs/importing-postgres.md

@@ -7,8 +7,8 @@ Run this if you'd like to import your database from a previous installation.
 ## Prerequisites
 
 For this to work, **the database name in Postgres must match** what this playbook uses.
-This playbook uses a Postgres database name of `homeserver` by default (controlled by the `matrix_postgres_db_name` variable).
-If your database name differs, be sure to change `matrix_postgres_db_name` to your desired name and to re-run the playbook before proceeding.
+This playbook uses a Postgres database name of `synapse` by default (controlled by the `matrix_synapse_database_database` variable).
+If your database name differs, be sure to change `matrix_synapse_database_database` to your desired name and to re-run the playbook before proceeding.
 
 The playbook supports importing Postgres dump files in **text** (e.g. `pg_dump > dump.sql`) or **gzipped** formats (e.g. `pg_dump | gzip -c > dump.sql.gz`).
 
@@ -21,10 +21,17 @@ Before doing the actual import, **you need to upload your Postgres dump file to
 
 To import, run this command (make sure to replace `<server-path-to-postgres-dump.sql>` with a file path on your server):
 
-	ansible-playbook -i inventory/hosts setup.yml --extra-vars='server_path_postgres_dump=<server-path-to-postgres-dump.sql>' --tags=import-postgres
+```sh
+ansible-playbook -i inventory/hosts setup.yml \
+--extra-vars='postgres_default_import_database=synapse server_path_postgres_dump=<server-path-to-postgres-dump.sql>' \
+--tags=import-postgres
+```
+
+We specify the `synapse` database as the default import database. If your dump is a single-database dump (`pg_dump`), then we need to tell it where to go to. If you're redefining `matrix_synapse_database_database` to something other than `synapse`, please adjust it here too. For database dumps spanning multiple databases (`pg_dumpall`), you can remove the `postgres_default_import_database` definition (but it doesn't hurt to keep it too).
 
 **Note**: `<server-path-to-postgres-dump.sql>` must be a file path to a Postgres dump file on the server (not on your local machine!).
 
+
 ## Troubleshooting
 
 A table ownership issue can occur if you are importing from a Synapse installation which was both:

docs/maintenance-postgres.md

@@ -19,6 +19,17 @@ You can use the `/usr/local/bin/matrix-postgres-cli` tool to get interactive ter
 
 If you are using an [external Postgres server](configuring-playbook-external-postgres.md), the above tool will not be available.
 
+By default, this tool puts you in the `matrix` database, which contains nothing.
+
+To see the available databases, run `\list` (or just `\l`).
+
+To change to another database (for example `synapse`), run `\connect synapse` (or just `\c synapse`).
+
+You can then proceed to write queries. Example: `SELECT COUNT(*) FROM users;`
+
+**Be careful**. Modifying the database directly (especially as services are running) is dangerous and may lead to irreversible database corruption.
+When in doubt, consider [making a backup](#backing-up-postgresql).
+
 
 ## Vacuuming PostgreSQL
 
@@ -53,6 +64,8 @@ pg_dumpall -h matrix-postgres \
 
 If you are using an [external Postgres server](configuring-playbook-external-postgres.md), the above command will not work, because the credentials file (`/matrix/postgres/env-postgres-psql`) is not available.
 
+If your server is on the ARM32 [architecture](alternative-architectures.md), you may need to remove the `-alpine` suffix from the image name in the command above.
+
 Restoring a backup made this way can be done by [importing it](importing-postgres.md).
 
 

docs/prerequisites.md

@@ -1,19 +1,23 @@
 # Prerequisites
 
-- An **x86** server running one of these operating systems:
+To install Matrix services using this Ansible playbook, you need:
+
+- (Recommended) An **x86** server running one of these operating systems:
   - **CentOS** (7 only for now; [8 is not yet supported](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/300))
-  - **Debian** (9/Stretch+)
-  - **Ubuntu** (16.04+, although [20.04 may be problematic](ansible.md#supported-ansible-versions))
+  - **Debian** (9/Stretch or newer)
+  - **Ubuntu** (16.04 or newer, although [20.04 may be problematic](ansible.md#supported-ansible-versions))
   - **Archlinux**
 
-This playbook doesn't support running on ARM (see [this issue](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/299)), however a minimal subset of the tools can be built on the host, which may result in a working configuration, even on a Raspberry pi (see [Alternative Architectures](alternative-architectures.md)). We only strive to support released stable versions of distributions, not betas or pre-releases. This playbook can take over your whole server or co-exist with other services that you have there.
+Generally, newer is better. We only strive to support released stable versions of distributions, not betas or pre-releases. This playbook can take over your whole server or co-exist with other services that you have there.
+
+This playbook somewhat supports running on non-`amd64` architectures like ARM. See [Alternative Architectures](alternative-architectures.md).
+
+If your distro runs within an [LXC container](https://linuxcontainers.org/), you may hit [this issue](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/703). It can be worked around, if absolutely necessary, but we suggest that you avoid running from within an LXC container.
 
 - `root` access to your server (or a user capable of elevating to `root` via `sudo`).
 
 - [Python](https://www.python.org/) being installed on the server. Most distributions install Python by default, but some don't (e.g. Ubuntu 18.04) and require manual installation (something like `apt-get install python3`). On some distros, Ansible may incorrectly [detect the Python version](https://docs.ansible.com/ansible/latest/reference_appendices/interpreter_discovery.html) (2 vs 3) and you may need to explicitly specify the interpreter path in `inventory/hosts` during installation (e.g. `ansible_python_interpreter=/usr/bin/python3`)
 
-- A `cron`-like tool installed on the server such as `cron` or `anacron` to automatically schedule the Let's Encrypt SSL certificates's renewal. *This can be ignored if you use your own SSL certificates.*
-
 - The [Ansible](http://ansible.com/) program being installed on your own computer. It's used to run this playbook and configures your server for you. Take a look at [our guide about Ansible](ansible.md) for more information, as well as [version requirements](ansible.md#supported-ansible-versions) and alternative ways to run Ansible.
 
 - Either the `dig` tool or `python-dns` installed on your own computer. Used later on, by the playbook's [services check](maintenance-checking-services.md) feature.

docs/self-building.md

@@ -18,8 +18,11 @@ List of roles where self-building the Docker image is currently possible:
 - `matrix-corporal`
 - `matrix-ma1sd`
 - `matrix-mailer`
+- `matrix-bridge-appservice-irc`
+- `matrix-bridge-appservice-slack`
 - `matrix-bridge-mautrix-facebook`
 - `matrix-bridge-mautrix-hangouts`
+- `matrix-bridge-mautrix-telegram`
 - `matrix-bridge-mx-puppet-skype`
 
 Adding self-building support to other roles is welcome. Feel free to contribute!

docs/uninstalling.md

@@ -23,15 +23,13 @@ If you prefer to uninstall manually, run these commands (most are meant to be ex
 
 - ensure all Matrix services are stopped: `ansible-playbook -i inventory/hosts setup.yml --tags=stop` (if you can't get Ansible working to run this command, you can run `systemctl stop 'matrix*'` manually on the server)
 
-- delete the Matrix-related systemd `.service` files (`rm -f /etc/systemd/system/matrix*.service`) and reload systemd (`systemctl daemon-reload`)
-
-- delete all Matrix-related cronjobs (`rm -f /etc/cron.d/matrix*`)
+- delete the Matrix-related systemd `.service` and `.timer` files (`rm -f /etc/systemd/system/matrix*.{service,timer}`) and reload systemd (`systemctl daemon-reload`)
 
 - delete some helper scripts (`rm -f /usr/local/bin/matrix*`)
 
 - delete some cached Docker images (`docker system prune -a`) or just delete them all (`docker rmi $(docker images -aq)`)
 
-- delete the Docker network: `docker network rm matrix` (might have been deleted already if you ran the `docker system prune` command)
+- delete the Docker networks: `docker network rm matrix matrix-coturn` (might have been deleted already if you ran the `docker system prune` command)
 
 - uninstall Docker itself, if necessary
 

examples/vars.yml

@@ -4,7 +4,7 @@
 # Note: this playbook does not touch the server referenced here.
 # Installation happens on another server ("matrix.<matrix-domain>").
 #
-# If you've deployed using the wrong domain, you'll have to run the Uninstalling step, 
+# If you've deployed using the wrong domain, you'll have to run the Uninstalling step,
 # because you can't change the Domain after deployment.
 #
 # Example value: example.com
@@ -18,12 +18,18 @@ matrix_domain: YOUR_BARE_DOMAIN_NAME_HERE
 # you won't be required to define this variable (see `docs/configuring-playbook-ssl-certificates.md`).
 #
 # Example value: someone@example.com
-matrix_ssl_lets_encrypt_support_email: YOUR_EMAIL_ADDRESS_HERE
+matrix_ssl_lets_encrypt_support_email: ''
 
 # A shared secret (between Coturn and Synapse) used for authentication.
 # You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`).
-matrix_coturn_turn_static_auth_secret: ""
+matrix_coturn_turn_static_auth_secret: ''
 
 # A secret used to protect access keys issued by the server.
 # You can put any string here, but generating a strong one is preferred (e.g. `pwgen -s 64 1`).
-matrix_synapse_macaroon_secret_key: ""
+matrix_synapse_macaroon_secret_key: ''
+
+# A Postgres password to use for the superuser Postgres user (called `matrix` by default).
+#
+# The playbook creates additional Postgres users and databases (one for each enabled service)
+# using this superuser account.
+matrix_postgres_connection_password: ''

group_vars/matrix_servers

@@ -110,6 +110,8 @@ matrix_appservice_webhooks_systemd_required_services_list: |
 # We don't enable bridges by default.
 matrix_appservice_slack_enabled: false
 
+matrix_appservice_slack_container_self_build: "{{ matrix_architecture != 'amd64' }}"
+
 # Normally, matrix-nginx-proxy is enabled and nginx can reach matrix-appservice-slack over the container network.
 # If matrix-nginx-proxy is not enabled, or you otherwise have a need for it, you can expose
 # matrix-appservice-slack's client-server port to the local host.
@@ -147,6 +149,8 @@ matrix_appservice_slack_database_password: "{{ matrix_synapse_macaroon_secret_ke
 # We don't enable bridges by default.
 matrix_appservice_irc_enabled: false
 
+matrix_appservice_irc_container_self_build: "{{ matrix_architecture != 'amd64' }}"
+
 # Normally, matrix-nginx-proxy is enabled and nginx can reach matrix-appservice-irc over the container network.
 # If matrix-nginx-proxy is not enabled, or you otherwise have a need for it, you can expose
 # matrix-appservice-irc's client-server port to the local host.
@@ -206,7 +210,8 @@ matrix_mautrix_facebook_login_shared_secret: "{{ matrix_synapse_ext_password_pro
 
 matrix_mautrix_facebook_bridge_presence: "{{ matrix_synapse_use_presence if matrix_synapse_enabled else true }}"
 
-# Postgres is the default, except if not using `matrix_postgres` (internal postgres)
+# We'd like to force-set people with external Postgres to SQLite, so the bridge role can complain
+# and point them to a migration path.
 matrix_mautrix_facebook_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}"
 matrix_mautrix_facebook_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mau.fb.db') | to_uuid }}"
 
@@ -305,6 +310,9 @@ matrix_mautrix_signal_database_password: "{{ matrix_synapse_macaroon_secret_key
 # We don't enable bridges by default.
 matrix_mautrix_telegram_enabled: false
 
+# Images are multi-arch (amd64 and arm64, but not arm32).
+matrix_mautrix_telegram_container_self_build: "{{ matrix_architecture not in ['arm64', 'amd64'] }}"
+
 matrix_mautrix_telegram_systemd_required_services_list: |
   {{
     ['docker.service']
@@ -669,6 +677,9 @@ matrix_corporal_matrix_homeserver_api_endpoint: "http://matrix-synapse:8008"
 
 matrix_corporal_matrix_auth_shared_secret: "{{ matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret }}"
 
+# This is only useful if there's REST auth provider to make use of it.
+matrix_corporal_http_gateway_internal_rest_auth_enabled: "{{ matrix_synapse_ext_password_provider_rest_auth_enabled }}"
+
 matrix_corporal_matrix_registration_shared_secret: "{{ matrix_synapse_registration_shared_secret }}"
 
 ######################################################################
@@ -1021,16 +1032,22 @@ matrix_ssl_pre_obtaining_required_service_name: "{{ 'matrix-dynamic-dns' if matr
 
 matrix_postgres_enabled: true
 
-matrix_postgres_connection_hostname: "matrix-postgres"
-matrix_postgres_connection_username: "synapse"
-# Please note that the max length of the password is 99 characters
-matrix_postgres_connection_password: "synapse-password"
-matrix_postgres_db_name: "homeserver"
+matrix_postgres_architecture: "{{ matrix_architecture }}"
+
+# We unset this if internal Postgres disabled, which will cascade to some other variables
+# and tell users they need to set it (either here or in those variables).
+matrix_postgres_connection_hostname: "{{ 'matrix-postgres' if matrix_postgres_enabled else '' }}"
 
 matrix_postgres_pgloader_container_image_self_build: "{{ matrix_architecture != 'amd64' }}"
 
 matrix_postgres_additional_databases: |
   {{
+    ([{
+      'name': matrix_synapse_database_database,
+      'username': matrix_synapse_database_user,
+      'password': matrix_synapse_database_password,
+    }] if (matrix_synapse_enabled and matrix_synapse_database_database != matrix_postgres_db_name and matrix_synapse_database_host == 'matrix-postgres') else [])
+    +
     ([{
       'name': matrix_ma1sd_database_name,
       'username': matrix_ma1sd_database_username,
@@ -1259,10 +1276,7 @@ matrix_synapse_container_metrics_api_host_bind_port: "{{ '127.0.0.1:9100' if (ma
 # For exposing the Synapse Manhole port (plain HTTP) to the local host.
 matrix_synapse_container_manhole_api_host_bind_port: "{{ '127.0.0.1:9000' if matrix_synapse_manhole_enabled else '' }}"
 
-matrix_synapse_database_host: "{{ matrix_postgres_connection_hostname }}"
-matrix_synapse_database_user: "{{ matrix_postgres_connection_username }}"
-matrix_synapse_database_password: "{{ matrix_postgres_connection_password }}"
-matrix_synapse_database_database: "{{ matrix_postgres_db_name }}"
+matrix_synapse_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'synapse.db') | to_uuid }}"
 
 # We do not enable TLS in Synapse by default.
 # TLS is handled by the matrix-nginx-proxy, which proxies the requests to Synapse.

roles/matrix-aux/defaults/main.yml

@@ -0,0 +1,72 @@
+---
+
+# matrix-aux is a role that manages auxiliary files and directories on your Matrix server.
+#
+# Certain components (like matrix-synapse, etc.) may sometimes require additional templates (email templates, privacy policies, etc.).
+# This role allows such files to be managed by the playbook.
+#
+# Note that files and directories created via this role are not automatically made available for containers to use.
+# If you use this role to put files in a directory that's already mounted into a container,
+# you can access the files without additional work.
+# Otherwise, you'd need to mount the file/directory to the container that needs it.
+# Roles usually provide a `matrix_*_additional_volumes` or `matrix_*_container_extra_arguments` variable
+# that you can use to mount an additional volume.
+
+# The default permission mode when creating directories using `matrix_aux_directory_definitions`
+matrix_aux_directory_default_mode: '0750'
+
+# Holds a list of directories to create on the server.
+#
+# By default, directories are:
+# - created with permissions as specified in `matrix_aux_directory_default_mode`
+# - owned by the `matrix_user_username` user and `matrix_user_groupname` group (usually `matrix:matrix`)
+#
+# Example:
+#
+# matrix_aux_directory_definitions:
+#   - dest: /matrix/aux
+#
+#   - dest: /matrix/another
+#     mode: '0700'
+#     owner: 'some-user'
+#     group: 'some-group'
+matrix_aux_directory_definitions: []
+
+# The default permission mode when creating directories using `matrix_aux_directory_definitions`
+matrix_aux_file_default_mode: '0640'
+
+# Holds a list of files to create on the server.
+#
+# By default, files are:
+# - created with permissions as specified in `matrix_aux_file_default_mode`
+# - owned by the `matrix_user_username` user and `matrix_user_groupname` group (usually `matrix:matrix`)
+#
+# You can define the file content inline (in your `vars.yml` file) or as an external file (see the example below).
+# Defining the content inline in `vars.yml` has the benefit of not splitting your configuration into multiple files,
+# but rather keeping everything inside `vars.yml` (which also gets backed up on the server in `/matrix/vars.yml`).
+#
+# Note: parent paths for files must exist.
+# If you've defined a file with a destination of `/matrix/some/path/file.txt`,
+# then you likely need to add `/matrix/some/path` to `matrix_aux_directory_definitions` as well.
+# You don't need to do this for directories that the playbook already creates for you.
+#
+# Example:
+#
+# matrix_aux_file_definitions:
+#   - dest: "{{ matrix_synapse_config_dir_path }}/something.html"
+#     content: |
+#       <!doctype html>
+#       <html><body>Something</body></html>
+#
+#   - dest: /matrix/aux/some-other-file.txt
+#     content: "Something"
+#     mode: '0600'
+#     owner: 'some-user'
+#     group: 'some-group'
+#
+#   - dest: /matrix/aux/yet-another-file.txt
+#     content: "{{ lookup('template', '/path/to/file.txt.j2') }}"
+#     mode: '0600'
+#     owner: 'some-user'
+#     group: 'some-group'
+matrix_aux_file_definitions: []

roles/matrix-aux/tasks/main.yml

@@ -0,0 +1,5 @@
+- import_tasks: "{{ role_path }}/tasks/setup.yml"
+  when: run_stop|bool
+  tags:
+    - setup-all
+    - setup-aux-files

roles/matrix-aux/tasks/setup.yml

@@ -0,0 +1,19 @@
+---
+
+- name: Ensure AUX directories are created
+  file:
+    dest: "{{ item.dest }}"
+    state: directory
+    owner: "{{ item.owner|default(matrix_user_username) }}"
+    group: "{{ item.group|default(matrix_user_groupname) }}"
+    mode: "{{ item.mode|default(matrix_aux_directory_default_mode) }}"
+  with_items: "{{ matrix_aux_directory_definitions }}"
+
+- name: Ensure AUX files are created
+  copy:
+    dest: "{{ item.dest }}"
+    content: "{{ item.content }}"
+    owner: "{{ item.owner|default(matrix_user_username) }}"
+    group: "{{ item.group|default(matrix_user_groupname) }}"
+    mode: "{{ item.mode|default(matrix_aux_file_default_mode) }}"
+  with_items: "{{ matrix_aux_file_definitions }}"

roles/matrix-base/defaults/main.yml

@@ -48,7 +48,16 @@ matrix_base_data_path_mode: "750"
 
 matrix_static_files_base_path: "{{ matrix_base_data_path }}/static-files"
 matrix_systemd_path: "/etc/systemd/system"
+
+# Specifies the path to use for the `HOME` environment variable for systemd unit files.
+# Docker 20.10 complains with `WARNING: Error loading config file: .dockercfg: $HOME is not defined`
+# if `$HOME` is not defined, so we define something to make it happy.
+matrix_systemd_unit_home_path: /root
+
+# This is now unused. We keep it so that cleanup tasks can use it.
+# To be removed in the future.
 matrix_cron_path: "/etc/cron.d"
+
 matrix_local_bin_path: "/usr/local/bin"
 
 matrix_host_command_docker: "/usr/bin/env docker"

roles/matrix-base/templates/usr-local-bin/matrix-remove-all.j2

@@ -20,8 +20,6 @@ else
 		rm -f {{ matrix_systemd_path }}/$s
 	done
 	systemctl daemon-reload
-	echo "Remove matrix cronjobs"
-	find /etc/cron.d/ -name "matrix-*" -delete
 	echo "Remove matrix scripts"
 	find {{ matrix_local_bin_path }}/ -name "matrix-*" -delete
 	echo "Remove unused Docker images and resources"

roles/matrix-bot-matrix-reminder-bot/tasks/init.yml

@@ -1,3 +1,3 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-bot-matrix-reminder-bot'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-bot-matrix-reminder-bot.service'] }}"
   when: matrix_bot_matrix_reminder_bot_enabled|bool

roles/matrix-bot-matrix-reminder-bot/templates/systemd/matrix-bot-matrix-reminder-bot.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-bot-matrix-reminder-bot
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-bot-matrix-reminder-bot
 

roles/matrix-bridge-appservice-discord/tasks/init.yml

@@ -7,7 +7,7 @@
   when: "matrix_appservice_discord_enabled and matrix_synapse_role_executed|default(False)"
 
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-appservice-discord'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-appservice-discord.service'] }}"
   when: matrix_appservice_discord_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-appservice-discord/templates/systemd/matrix-appservice-discord.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-appservice-discord
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-appservice-discord
 

roles/matrix-bridge-appservice-irc/defaults/main.yml

@@ -3,6 +3,10 @@
 
 matrix_appservice_irc_enabled: true
 
+matrix_appservice_irc_container_self_build: false
+matrix_appservice_irc_docker_repo: "https://github.com/matrix-org/matrix-appservice-irc.git"
+matrix_appservice_irc_docker_src_files_path: "{{ matrix_base_data_path }}/appservice-irc/docker-src"
+
 matrix_appservice_irc_docker_image: "docker.io/matrixdotorg/matrix-appservice-irc:release-0.17.1"
 matrix_appservice_irc_docker_image_force_pull: "{{ matrix_appservice_irc_docker_image.endswith(':latest') }}"
 

roles/matrix-bridge-appservice-irc/tasks/init.yml

@@ -7,7 +7,7 @@
   when: "matrix_appservice_irc_enabled|bool and matrix_synapse_role_executed|default(False)"
 
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-appservice-irc'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-appservice-irc.service'] }}"
   when: matrix_appservice_irc_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-appservice-irc/tasks/setup_install.yml

@@ -2,15 +2,17 @@
 
 - name: Ensure Appservice IRC paths exist
   file:
-    path: "{{ item }}"
+    path: "{{ item.path }}"
     state: directory
     mode: 0750
     owner: "{{ matrix_user_username }}"
     group: "{{ matrix_user_groupname }}"
   with_items:
-    - "{{ matrix_appservice_irc_base_path }}"
-    - "{{ matrix_appservice_irc_config_path }}"
-    - "{{ matrix_appservice_irc_data_path }}"
+    - { path: "{{ matrix_appservice_irc_base_path }}", when: true }
+    - { path: "{{ matrix_appservice_irc_config_path }}", when: true }
+    - { path: "{{ matrix_appservice_irc_data_path }}", when: true }
+    - { path: "{{ matrix_appservice_irc_docker_src_files_path }}", when: "{{ matrix_appservice_irc_container_self_build }}" }
+  when: item.when|bool
 
 - name: Check if an old passkey file already exists
   stat:
@@ -59,6 +61,26 @@
     source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
     force_source: "{{ matrix_appservice_irc_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
     force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_appservice_irc_docker_image_force_pull }}"
+  when: "matrix_appservice_irc_enabled|bool and not matrix_appservice_irc_container_self_build|bool"
+
+- name: Ensure matrix-appservice-irc repository is present when self-building
+  git:
+    repo: "{{ matrix_appservice_irc_docker_repo }}"
+    dest: "{{ matrix_appservice_irc_docker_src_files_path }}"
+    force: "yes"
+  register: matrix_appservice_irc_git_pull_results
+  when: "matrix_appservice_irc_enabled|bool and matrix_appservice_irc_container_self_build|bool"
+
+- name: Ensure matrix-appservice-irc Docker image is build
+  docker_image:
+    name: "{{ matrix_appservice_irc_docker_image }}"
+    source: build
+    force_source: yes
+    build:
+      dockerfile: Dockerfile
+      path: "{{ matrix_appservice_irc_docker_src_files_path }}"
+      pull: yes
+  when: "matrix_appservice_irc_enabled|bool and matrix_appservice_irc_container_self_build|bool and matrix_appservice_irc_git_pull_results.changed"
 
 - name: Ensure Matrix Appservice IRC config installed
   copy:

roles/matrix-bridge-appservice-irc/templates/systemd/matrix-appservice-irc.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-appservice-irc
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-appservice-irc
 

roles/matrix-bridge-appservice-slack/defaults/main.yml

@@ -3,6 +3,10 @@
 
 matrix_appservice_slack_enabled: true
 
+matrix_appservice_slack_container_self_build: false
+matrix_appservice_slack_docker_repo: "https://github.com/matrix-org/matrix-appservice-slack.git"
+matrix_appservice_slack_docker_src_files_path: "{{ matrix_base_data_path }}/appservice-slack/docker-src"
+
 matrix_appservice_slack_docker_image: "docker.io/matrixdotorg/matrix-appservice-slack:release-1.5.0"
 matrix_appservice_slack_docker_image_force_pull: "{{ matrix_appservice_slack_docker_image.endswith(':latest') }}"
 

roles/matrix-bridge-appservice-slack/tasks/init.yml

@@ -7,7 +7,7 @@
   when: "matrix_synapse_role_executed|default(False)"
 
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-appservice-slack'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-appservice-slack.service'] }}"
   when: matrix_appservice_slack_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-appservice-slack/tasks/setup_install.yml

@@ -8,9 +8,11 @@
     owner: "{{ matrix_user_username }}"
     group: "{{ matrix_user_groupname }}"
   with_items:
-    - "{{ matrix_appservice_slack_base_path }}"
-    - "{{ matrix_appservice_slack_config_path }}"
-    - "{{ matrix_appservice_slack_data_path }}"
+    - { path: "{{ matrix_appservice_slack_base_path }}", when: true }
+    - { path: "{{ matrix_appservice_slack_config_path }}", when: true }
+    - { path: "{{ matrix_appservice_slack_data_path }}", when: true }
+    - { path: "{{ matrix_appservice_slack_docker_src_files_path }}", when: "{{ matrix_appservice_slack_container_self_build }}" }
+  when: item.when|bool
 
 - set_fact:
     matrix_appservice_slack_requires_restart: false
@@ -35,6 +37,26 @@
     source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
     force_source: "{{ matrix_appservice_slack_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
     force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_appservice_slack_docker_image_force_pull }}"
+  when: "not matrix_appservice_slack_container_self_build|bool"
+
+- name: Ensure matrix-appservice-slack repository is present when self-building
+  git:
+    repo: "{{ matrix_appservice_slack_docker_repo }}"
+    dest: "{{ matrix_appservice_slack_docker_src_files_path }}"
+    force: "yes"
+  register: matrix_appservice_slack_git_pull_results
+  when: "matrix_appservice_slack_container_self_build|bool"
+
+- name: Ensure matrix-appservice-slack Docker image is built
+  docker_image:
+    name: "{{ matrix_appservice_slack_docker_image }}"
+    source: build
+    force_source: yes
+    build:
+      dockerfile: Dockerfile
+      path: "{{ matrix_appservice_slack_docker_src_files_path }}"
+      pull: yes
+  when: "matrix_appservice_slack_container_self_build|bool and matrix_appservice_slack_git_pull_results.changed"
 
 - name: Ensure Matrix Appservice Slack config installed
   copy:

roles/matrix-bridge-appservice-slack/templates/systemd/matrix-appservice-slack.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-appservice-slack
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-appservice-slack
 

roles/matrix-bridge-appservice-webhooks/tasks/init.yml

@@ -7,7 +7,7 @@
   when: "matrix_synapse_role_executed|default(False)"
 
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-appservice-webhooks'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-appservice-webhooks.service'] }}"
   when: matrix_appservice_webhooks_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-appservice-webhooks/templates/systemd/matrix-appservice-webhooks.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-appservice-webhooks
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-appservice-webhooks
 

roles/matrix-bridge-mautrix-facebook/defaults/main.yml

@@ -35,12 +35,15 @@ matrix_mautrix_facebook_homeserver_token: ''
 
 # Database-related configuration fields.
 #
-# To use SQLite, stick to these defaults.
+# To use SQLite:
+# - change the engine (`matrix_mautrix_facebook_database_engine: 'sqlite'`)
+# - change to the last bridge version that supported SQLite:
+#   `matrix_mautrix_facebook_docker_image: "{{ matrix_mautrix_facebook_docker_image_name_prefix }}tulir/mautrix-facebook:da1b4ec596e334325a1589e70829dea46e73064b"`
+# - plan your migration to Postgres, as this bridge does not support SQLite anymore (and neither will the playbook in the future).
 #
 # To use Postgres:
-# - change the engine (`matrix_mautrix_facebook_database_engine: 'postgres'`)
 # - adjust your database credentials via the `matrix_mautrix_facebook_postgres_*` variables
-matrix_mautrix_facebook_database_engine: 'sqlite'
+matrix_mautrix_facebook_database_engine: 'postgres'
 
 matrix_mautrix_facebook_sqlite_database_path_local: "{{ matrix_mautrix_facebook_data_path }}/mautrix-facebook.db"
 matrix_mautrix_facebook_sqlite_database_path_in_container: "/data/mautrix-facebook.db"

roles/matrix-bridge-mautrix-facebook/tasks/init.yml

@@ -1,5 +1,5 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-facebook'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-facebook.service'] }}"
   when: matrix_mautrix_facebook_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-mautrix-facebook/tasks/validate_config.yml

@@ -8,3 +8,24 @@
   with_items:
     - "matrix_mautrix_facebook_appservice_token"
     - "matrix_mautrix_facebook_homeserver_token"
+
+- block:
+    - name: Fail if on SQLite, unless on the last version supporting SQLite
+      fail:
+        msg: >-
+          You're trying to use the mautrix-facebook bridge with an SQLite database.
+          Going forward, this bridge only supports Postgres.
+          To learn more about this, see our changelog: https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/CHANGELOG.md#breaking-change-the-mautrix-facebook-bridge-now-requires-a-postgres-database
+      when: "not matrix_mautrix_facebook_docker_image.endswith(':da1b4ec596e334325a1589e70829dea46e73064b')"
+
+    - name: Inject warning if still on SQLite
+      set_fact:
+        matrix_playbook_runtime_results: |
+          {{
+            matrix_playbook_runtime_results|default([])
+            +
+            [
+              "NOTE: Your mautrix-facebook bridge setup is still on SQLite. Your bridge is not getting any updates and will likely stop working at some point. To learn more about this, see our changelog: https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/CHANGELOG.md#breaking-change-the-mautrix-facebook-bridge-now-requires-a-postgres-database"
+            ]
+          }}
+  when: "matrix_mautrix_facebook_database_engine == 'sqlite'"

roles/matrix-bridge-mautrix-facebook/templates/config.yaml.j2

@@ -8,6 +8,10 @@ homeserver:
     # Whether or not to verify the SSL certificate of the homeserver.
     # Only applies if address starts with https://
     verify_ssl: true
+    # Whether or not the homeserver supports asmux-specific endpoints,
+    # such as /_matrix/client/unstable/net.maunium.asmux/dms for atomically
+    # updating m.direct.
+    asmux: false
 
 # Application service host/registration related details
 # Changing these values requires regeneration of the registration.
@@ -22,11 +26,7 @@ appservice:
     # Usually 1 is enough, but on high-traffic bridges you might need to increase this to avoid 413s
     max_body_size: 1
 
-    # The full URI to the database. SQLite and Postgres are fully supported.
-    # Other DBMSes supported by SQLAlchemy may or may not work.
-    # Format examples:
-    #   SQLite:   sqlite:///filename.db
-    #   Postgres: postgres://username:password@hostname/dbname
+    # The full URI to the database. Only Postgres is currently supported.
     database: {{ matrix_mautrix_facebook_appservice_database|to_json }}
 
     # Public part of web server for out-of-Matrix interaction with the bridge.
@@ -38,6 +38,10 @@ appservice:
         # The base URL where the public-facing endpoints are available. The prefix is not added
         # implicitly.
         external: https://example.com/public
+        # Shared secret for integration managers such as mautrix-manager.
+        # If set to "generate", a random string will be generated on the next startup.
+        # If null, integration manager access to the API will not be possible.
+        shared_secret: generate
 
     # The unique ID of this appservice.
     id: facebook
@@ -46,12 +50,17 @@ appservice:
     # Display name and avatar for bot. Set to "remove" to remove display name/avatar, leave empty
     # to leave display name/avatar as-is.
     bot_displayname: Facebook bridge bot
-    bot_avatar: mxc://maunium.net/ddtNPZSKMNqaUzqrHuWvUADv
+    bot_avatar: mxc://maunium.net/ygtkteZsXnGJLJHRchUwYWak
 
     # Authentication tokens for AS <-> HS communication.
     as_token: "{{ matrix_mautrix_facebook_appservice_token }}"
     hs_token: "{{ matrix_mautrix_facebook_homeserver_token }}"
 
+# Prometheus telemetry config. Requires prometheus-client to be installed.
+metrics:
+    enabled: false
+    listen_port: 8000
+
 # Bridge config
 bridge:
     # Localpart template of MXIDs for Facebook users.
@@ -76,6 +85,7 @@ bridge:
     # "own_nickname" (user-specific!)
     displayname_preference:
     - name
+    - first_name
 
     # The prefix for commands. Only required in non-management rooms.
     command_prefix: "!fb"
@@ -120,6 +130,18 @@ bridge:
         # Default to encryption, force-enable encryption in all portals the bridge creates
         # This will cause the bridge bot to be in private chats for the encryption to work properly.
         default: false
+        # Options for automatic key sharing.
+        key_sharing:
+            # Enable key sharing? If enabled, key requests for rooms where users are in will be fulfilled.
+            # You must use a client that supports requesting keys from other users to use this feature.
+            allow: false
+            # Require the requesting device to have a valid cross-signing signature?
+            # This doesn't require that the bridge has verified the device, only that the user has verified it.
+            # Not yet implemented.
+            require_cross_signing: false
+            # Require devices to be verified by the bridge?
+            # Verification by the bridge is not yet implemented.
+            require_verification: true
     # Whether or not the bridge should send a read receipt from the bridge bot when a message has
     # been sent to Facebook.
     delivery_receipts: false
@@ -161,6 +183,10 @@ bridge:
     # Whether or not the bridge should try to "refresh" the connection if a normal reconnection
     # attempt fails.
     refresh_on_reconnection_fail: false
+    # Set this to true to tell the bridge to re-send m.bridge events to all rooms on the next run.
+    # This field will automatically be changed back to false after it,
+    # except if the config file is not writable.
+    resend_bridge_info: false
 
     # Permissions for using the bridge.
     # Permitted values:
@@ -192,9 +218,7 @@ logging:
     loggers:
         mau:
             level: DEBUG
-        fbchat:
-            level: DEBUG
-        hbmqtt:
+        paho:
             level: INFO
         aiohttp:
             level: INFO

roles/matrix-bridge-mautrix-facebook/templates/systemd/matrix-mautrix-facebook.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mautrix-facebook
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mautrix-facebook
 ExecStartPre={{ matrix_host_command_docker }} run --rm --name matrix-mautrix-facebook-db \

roles/matrix-bridge-mautrix-hangouts/tasks/init.yml

@@ -1,5 +1,5 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-hangouts'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-hangouts.service'] }}"
   when: matrix_mautrix_hangouts_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-mautrix-hangouts/templates/systemd/matrix-mautrix-hangouts.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mautrix-hangouts matrix-mautrix-hangouts-db
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mautrix-hangouts matrix-mautrix-hangouts-db
 ExecStartPre={{ matrix_host_command_docker }} run --rm --name matrix-mautrix-hangouts-db \

roles/matrix-bridge-mautrix-signal/tasks/init.yml

@@ -1,5 +1,5 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-signal', 'matrix-mautrix-signal-daemon'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-signal.service', 'matrix-mautrix-signal-daemon.service'] }}"
   when: matrix_mautrix_signal_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-mautrix-signal/tasks/setup_install.yml

@@ -35,6 +35,9 @@
     - "{{ matrix_mautrix_signal_base_path }}"
     - "{{ matrix_mautrix_signal_config_path }}"
     - "{{ matrix_mautrix_signal_daemon_path }}"
+    - "{{ matrix_mautrix_signal_daemon_path }}/avatars"
+    - "{{ matrix_mautrix_signal_daemon_path }}/attachments"
+    - "{{ matrix_mautrix_signal_daemon_path }}/data"
 
 - name: Ensure mautrix-signal config.yaml installed
   copy:

roles/matrix-bridge-mautrix-signal/templates/systemd/matrix-mautrix-signal-daemon.service.j2

@@ -13,6 +13,7 @@ Wants={{ service }}
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mautrix-signal-daemon
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mautrix-signal-daemon
@@ -20,9 +21,11 @@ ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mautrix-signal-daemon
 # Intentional delay, so that the homeserver (we likely depend on) can manage to start.
 ExecStartPre={{ matrix_host_command_sleep }} 5
 
+# We can't use `--read-only` for this bridge.
 ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-mautrix-signal-daemon \
 			--log-driver=none \
 			--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
+			--cap-drop=ALL \
 			--network={{ matrix_docker_network }} \
 			-v {{ matrix_mautrix_signal_daemon_path }}:/signald:z \
 			{{ matrix_mautrix_signal_daemon_docker_image }}

roles/matrix-bridge-mautrix-signal/templates/systemd/matrix-mautrix-signal.service.j2

@@ -13,6 +13,7 @@ Wants={{ service }}
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mautrix-signal
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mautrix-signal
 
@@ -22,16 +23,19 @@ ExecStartPre={{ matrix_host_command_sleep }} 5
 ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-mautrix-signal \
 			--log-driver=none \
 			--network={{ matrix_docker_network }} \
+			--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
+			--cap-drop=ALL \
+			--read-only \
 			{% if matrix_mautrix_signal_container_http_host_bind_port %}
 			-p {{ matrix_mautrix_signal_container_http_host_bind_port }}:29328 \
 			{% endif %}
 			-v {{ matrix_mautrix_signal_daemon_path }}:/signald:z \
-			-v {{ matrix_mautrix_signal_config_path }}:/data:z \
+			-v {{ matrix_mautrix_signal_config_path }}:/config:z \
 			{% for arg in matrix_mautrix_signal_container_extra_arguments %}
 			{{ arg }} \
 			{% endfor %}
 			{{ matrix_mautrix_signal_docker_image }} \
-			python3 -m mautrix_signal -c /data/config.yaml
+			python3 -m mautrix_signal -c /config/config.yaml
 
 ExecStop=-{{ matrix_host_command_docker }} kill matrix-mautrix-signal
 ExecStop=-{{ matrix_host_command_docker }} rm matrix-mautrix-signal

roles/matrix-bridge-mautrix-telegram/defaults/main.yml

@@ -3,6 +3,10 @@
 
 matrix_mautrix_telegram_enabled: true
 
+matrix_mautrix_telegram_container_self_build: false
+matrix_mautrix_telegram_docker_repo: "https://mau.dev/tulir/mautrix-telegram.git"
+matrix_mautrix_telegram_docker_src_files_path: "{{ matrix_base_data_path }}/mautrix-telegram/docker-src"
+
 # See: https://mau.dev/tulir/mautrix-telegram/container_registry
 matrix_mautrix_telegram_docker_image: "dock.mau.dev/tulir/mautrix-telegram:v0.9.0"
 matrix_mautrix_telegram_docker_image_force_pull: "{{ matrix_mautrix_telegram_docker_image.endswith(':latest') }}"

roles/matrix-bridge-mautrix-telegram/tasks/init.yml

@@ -1,5 +1,5 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-telegram'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-telegram.service'] }}"
   when: matrix_mautrix_telegram_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml

@@ -34,24 +34,46 @@
       when: "matrix_mautrix_telegram_sqlite_database_path_local_stat_result.stat.exists|bool"
   when: "matrix_mautrix_telegram_database_engine == 'postgres'"
 
-- name: Ensure Mautrix Telegram image is pulled
-  docker_image:
-    name: "{{ matrix_mautrix_telegram_docker_image }}"
-    source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
-    force_source: "{{ matrix_mautrix_telegram_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
-    force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mautrix_telegram_docker_image_force_pull }}"
-
 - name: Ensure Mautrix Telegram paths exist
   file:
-    path: "{{ item }}"
+    path: "{{ item.path }}"
     state: directory
     mode: 0750
     owner: "{{ matrix_user_username }}"
     group: "{{ matrix_user_groupname }}"
   with_items:
-    - "{{ matrix_mautrix_telegram_base_path }}"
-    - "{{ matrix_mautrix_telegram_config_path }}"
-    - "{{ matrix_mautrix_telegram_data_path }}"
+    - { path: "{{ matrix_mautrix_telegram_base_path }}", when: true }
+    - { path: "{{ matrix_mautrix_telegram_config_path }}", when: true }
+    - { path: "{{ matrix_mautrix_telegram_data_path }}", when: true }
+    - { path: "{{ matrix_mautrix_telegram_docker_src_files_path }}", when: "{{ matrix_mautrix_telegram_container_self_build }}" }
+  when: item.when|bool
+
+- name: Ensure Mautrix Telegram image is pulled
+  docker_image:
+    name: "{{ matrix_mautrix_telegram_docker_image }}"
+    source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
+    force_source: "{{ matrix_mautrix_telegram_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
+    force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mautrix_telegram_docker_image_force_pull }}"
+  when: "not matrix_mautrix_telegram_container_self_build|bool"
+
+- name: Ensure matrix-mautrix-telegram repository is present when self-building
+  git:
+    repo: "{{ matrix_mautrix_telegram_docker_repo }}"
+    dest: "{{ matrix_mautrix_telegram_docker_src_files_path }}"
+    force: "yes"
+  register: matrix_mautrix_telegram_git_pull_results
+  when: "matrix_mautrix_telegram_container_self_build|bool"
+
+- name: Ensure matrix-mautrix-telegram Docker image is build
+  docker_image:
+    name: "{{ matrix_mautrix_telegram_docker_image }}"
+    source: build
+    force_source: yes
+    build:
+      dockerfile: Dockerfile
+      path: "{{ matrix_mautrix_telegram_docker_src_files_path }}"
+      pull: yes
+  when: "matrix_mautrix_telegram_container_self_build|bool and matrix_mautrix_telegram_git_pull_results.changed"
 
 - name: Check if an old database file already exists
   stat:

roles/matrix-bridge-mautrix-telegram/templates/systemd/matrix-mautrix-telegram.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mautrix-telegram
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mautrix-telegram
 ExecStartPre={{ matrix_host_command_docker }} run --rm --name matrix-mautrix-telegram-db \

roles/matrix-bridge-mautrix-whatsapp/tasks/init.yml

@@ -1,5 +1,5 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-whatsapp'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-whatsapp.service'] }}"
   when: matrix_mautrix_whatsapp_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml

@@ -26,6 +26,7 @@
               engine_variable_name: 'matrix_mautrix_whatsapp_database_engine'
               engine_old: 'sqlite'
               systemd_services_to_stop: ['matrix-mautrix-whatsapp.service']
+              pgloader_options: ['--with "quote identifiers"']
 
         - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml"
 

roles/matrix-bridge-mautrix-whatsapp/templates/systemd/matrix-mautrix-whatsapp.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mautrix-whatsapp
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mautrix-whatsapp
 

roles/matrix-bridge-mx-puppet-discord/tasks/init.yml

@@ -1,5 +1,5 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mx-puppet-discord'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mx-puppet-discord.service'] }}"
   when: matrix_mx_puppet_discord_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-mx-puppet-discord/templates/config.yaml.j2

@@ -122,20 +122,4 @@ logging:
   lineDateFormat: MMM-D HH:mm:ss.SSS
   # Logging files
   # Log files are rotated daily by default
-  files:
-    # Log file path
-    - file: "/data/bridge.log"
-      # Log level for this file
-      # Allowed values starting with most verbose:
-      # silly, debug, verbose, info, warn, error
-      level: info
-      # Date and time formatting
-      datePattern: YYYY-MM-DD
-      # Maximum number of logs to keep.
-      # This can be a number of files or number of days.
-      # If using days, add 'd' as a suffix
-      maxFiles: 14d
-      # Maximum size of the file after which it will rotate. This can be a
-      # number of bytes, or units of kb, mb, and gb. If using the units, add
-      # 'k', 'm', or 'g' as the suffix
-      maxSize: 50m
+  files: []

roles/matrix-bridge-mx-puppet-discord/templates/systemd/matrix-mx-puppet-discord.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mx-puppet-discord
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mx-puppet-discord
 

roles/matrix-bridge-mx-puppet-instagram/tasks/init.yml

@@ -1,5 +1,5 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mx-puppet-instagram'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mx-puppet-instagram.service'] }}"
   when: matrix_mx_puppet_instagram_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-mx-puppet-instagram/templates/config.yaml.j2

@@ -66,20 +66,4 @@ logging:
   lineDateFormat: MMM-D HH:mm:ss.SSS
   # Logging files
   # Log files are rotated daily by default
-  files:
-    # Log file path
-    - file: "/data/bridge.log"
-      # Log level for this file
-      # Allowed values starting with most verbose:
-      # silly, debug, verbose, info, warn, error
-      level: info
-      # Date and time formatting
-      datePattern: YYYY-MM-DD
-      # Maximum number of logs to keep.
-      # This can be a number of files or number of days.
-      # If using days, add 'd' as a suffix
-      maxFiles: 14d
-      # Maximum size of the file after which it will rotate. This can be a
-      # number of bytes, or units of kb, mb, and gb. If using the units, add
-      # 'k', 'm', or 'g' as the suffix
-      maxSize: 50m
+  files: []

roles/matrix-bridge-mx-puppet-instagram/templates/systemd/matrix-mx-puppet-instagram.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mx-puppet-instagram
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mx-puppet-instagram
 

roles/matrix-bridge-mx-puppet-skype/tasks/init.yml

@@ -1,5 +1,5 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mx-puppet-skype'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mx-puppet-skype.service'] }}"
   when: matrix_mx_puppet_skype_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-mx-puppet-skype/templates/config.yaml.j2

@@ -42,30 +42,7 @@ logging:
   lineDateFormat: MMM-D HH:mm:ss.SSS
   # Logging files
   # Log files are rotated daily by default
-  files:
-    # Log file path
-    - file: "/data/bridge.log"
-      # Log level for this file
-      # Allowed values starting with most verbose:
-      # silly, debug, verbose, info, warn, error
-      level: info
-      # Date and time formatting
-      datePattern: YYYY-MM-DD
-      # Maximum number of logs to keep.
-      # This can be a number of files or number of days.
-      # If using days, add 'd' as a suffix
-      maxFiles: 14d
-      # Maximum size of the file after which it will rotate. This can be a
-      # number of bytes, or units of kb, mb, and gb. If using the units, add
-      # 'k', 'm', or 'g' as the suffix
-      maxSize: 50m
-      # Optionally enable/disable logging for certain modules
-      #disabled:
-      #  - PresenceHandler
-      #  - module: bot-sdk-MatrixLiteClient
-      #    regex: /_matrix/client/r0/presence/ # this regex needs to match to disable the log
-      #enabled:
-      #  - Store
+  files: []
 
 database:
 {% if matrix_mx_puppet_skype_database_engine == 'postgres' %}

roles/matrix-bridge-mx-puppet-skype/templates/systemd/matrix-mx-puppet-skype.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mx-puppet-skype
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mx-puppet-skype
 

roles/matrix-bridge-mx-puppet-slack/tasks/init.yml

@@ -1,5 +1,5 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mx-puppet-slack'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mx-puppet-slack.service'] }}"
   when: matrix_mx_puppet_slack_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-mx-puppet-slack/templates/config.yaml.j2

@@ -80,20 +80,4 @@ logging:
   lineDateFormat: MMM-D HH:mm:ss.SSS
   # Logging files
   # Log files are rotated daily by default
-  files:
-    # Log file path
-    - file: "/data/bridge.log"
-      # Log level for this file
-      # Allowed values starting with most verbose:
-      # silly, debug, verbose, info, warn, error
-      level: info
-      # Date and time formatting
-      datePattern: YYYY-MM-DD
-      # Maximum number of logs to keep.
-      # This can be a number of files or number of days.
-      # If using days, add 'd' as a suffix
-      maxFiles: 14d
-      # Maximum size of the file after which it will rotate. This can be a
-      # number of bytes, or units of kb, mb, and gb. If using the units, add
-      # 'k', 'm', or 'g' as the suffix
-      maxSize: 50m
+  files: []

roles/matrix-bridge-mx-puppet-slack/templates/systemd/matrix-mx-puppet-slack.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mx-puppet-slack
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mx-puppet-slack
 

roles/matrix-bridge-mx-puppet-steam/tasks/init.yml

@@ -1,5 +1,5 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mx-puppet-steam'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mx-puppet-steam.service'] }}"
   when: matrix_mx_puppet_steam_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-mx-puppet-steam/templates/config.yaml.j2

@@ -83,20 +83,4 @@ logging:
   lineDateFormat: MMM-D HH:mm:ss.SSS
   # Logging files
   # Log files are rotated daily by default
-  files:
-    # Log file path
-    - file: "/data/bridge.log"
-      # Log level for this file
-      # Allowed values starting with most verbose:
-      # silly, debug, verbose, info, warn, error
-      level: info
-      # Date and time formatting
-      datePattern: YYYY-MM-DD
-      # Maximum number of logs to keep.
-      # This can be a number of files or number of days.
-      # If using days, add 'd' as a suffix
-      maxFiles: 14d
-      # Maximum size of the file after which it will rotate. This can be a
-      # number of bytes, or units of kb, mb, and gb. If using the units, add
-      # 'k', 'm', or 'g' as the suffix
-      maxSize: 50m
+  files: []

roles/matrix-bridge-mx-puppet-steam/templates/systemd/matrix-mx-puppet-steam.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mx-puppet-steam
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mx-puppet-steam
 

roles/matrix-bridge-mx-puppet-twitter/tasks/init.yml

@@ -1,5 +1,5 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mx-puppet-twitter'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mx-puppet-twitter.service'] }}"
   when: matrix_mx_puppet_twitter_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-mx-puppet-twitter/templates/config.yaml.j2

@@ -76,20 +76,4 @@ logging:
   lineDateFormat: MMM-D HH:mm:ss.SSS
   # Logging files
   # Log files are rotated daily by default
-  files:
-    # Log file path
-    - file: "/data/bridge.log"
-      # Log level for this file
-      # Allowed values starting with most verbose:
-      # silly, debug, verbose, info, warn, error
-      level: info
-      # Date and time formatting
-      datePattern: YYYY-MM-DD
-      # Maximum number of logs to keep.
-      # This can be a number of files or number of days.
-      # If using days, add 'd' as a suffix
-      maxFiles: 14d
-      # Maximum size of the file after which it will rotate. This can be a
-      # number of bytes, or units of kb, mb, and gb. If using the units, add
-      # 'k', 'm', or 'g' as the suffix
-      maxSize: 50m
+  files: []

roles/matrix-bridge-mx-puppet-twitter/templates/systemd/matrix-mx-puppet-twitter.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mx-puppet-twitter
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mx-puppet-twitter
 

roles/matrix-bridge-sms/tasks/init.yml

@@ -7,7 +7,7 @@
   when: "matrix_sms_bridge_enabled and matrix_synapse_role_executed|default(False)"
 
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-sms-bridge'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-sms-bridge.service'] }}"
   when: matrix_sms_bridge_enabled|bool
 
 # If the matrix-synapse role is not used, these variables may not exist.

roles/matrix-bridge-sms/templates/systemd/matrix-sms-bridge.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-/usr/bin/docker kill matrix-sms-bridge
 ExecStartPre=-/usr/bin/docker rm matrix-sms-bridge
 

roles/matrix-client-element/defaults/main.yml

@@ -3,7 +3,7 @@ matrix_client_element_enabled: true
 matrix_client_element_container_image_self_build: false
 matrix_client_element_container_image_self_build_repo: "https://github.com/vector-im/riot-web.git"
 
-matrix_client_element_docker_image: "{{ matrix_client_element_docker_image_name_prefix }}vectorim/element-web:v1.7.16"
+matrix_client_element_docker_image: "{{ matrix_client_element_docker_image_name_prefix }}vectorim/element-web:v1.7.17"
 matrix_client_element_docker_image_name_prefix: "{{ 'localhost/' if matrix_client_element_container_image_self_build else 'docker.io/' }}"
 matrix_client_element_docker_image_force_pull: "{{ matrix_client_element_docker_image.endswith(':latest') }}"
 

roles/matrix-client-element/tasks/init.yml

@@ -1,5 +1,5 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-client-element'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-client-element.service'] }}"
   when: matrix_client_element_enabled|bool
 
 # ansible lower than 2.8, does not support docker_image build parameters

roles/matrix-client-element/templates/systemd/matrix-client-element.service.j2

@@ -9,6 +9,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-client-element
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-client-element
 

roles/matrix-common-after/tasks/start.yml

@@ -1,6 +1,6 @@
 ---
 
-- name: Deterimne whether we should make services autostart
+- name: Determine whether we should make services autostart
   set_fact:
     matrix_services_autostart_enabled_bool: "{{ true if matrix_services_autostart_enabled|default('') == '' else matrix_services_autostart_enabled|bool }}"
 
@@ -46,7 +46,7 @@
         Try running `systemctl status {{ item }}` and `journalctl -fu {{ item }}` on the server to investigate.
     with_items: "{{ matrix_systemd_services_list }}"
     when:
-      - "ansible_facts.services[item + '.service']|default(none) is none or ansible_facts.services[item + '.service'].state != 'running'"
+      - "item.endswith('.service') and (ansible_facts.services[item]|default(none) is none or ansible_facts.services[item].state != 'running')"
   when: " ansible_distribution != 'Archlinux'"
 
 - block:

roles/matrix-corporal/defaults/main.yml

@@ -24,7 +24,7 @@ matrix_corporal_systemd_required_services_list: ['docker.service']
 
 matrix_corporal_docker_image: "{{ matrix_corporal_docker_image_name_prefix }}devture/matrix-corporal:{{ matrix_corporal_docker_image_tag }}"
 matrix_corporal_docker_image_name_prefix: "{{ 'localhost/' if matrix_corporal_container_image_self_build else 'docker.io/' }}"
-matrix_corporal_docker_image_tag: "1.11.0"
+matrix_corporal_docker_image_tag: "2.1.0"
 matrix_corporal_docker_image_force_pull: "{{ matrix_corporal_docker_image.endswith(':latest') }}"
 
 matrix_corporal_base_path: "{{ matrix_base_data_path }}/corporal"
@@ -50,10 +50,16 @@ matrix_corporal_matrix_registration_shared_secret: ""
 matrix_corporal_matrix_timeout_milliseconds: 45000
 
 matrix_corporal_reconciliation_retry_interval_milliseconds: 30000
-matrix_corporal_reconciliation_user_id_local_part: "matrix-corporal"
+matrix_corporal_corporal_user_id_local_part: "matrix-corporal"
 
 matrix_corporal_http_gateway_timeout_milliseconds: 60000
 
+# If enabled, matrix-corporal exposes a `POST /_matrix/corporal/_matrix-internal/identity/v1/check_credentials` API
+# on the gateway (Client-Server API) server.
+# This API can then be used together with the REST Auth password provider by pointing it to matrix-corporal (e.g. `http://matrix-corporal:41080/_matrix/corporal`).
+# Doing so allows Interactive Authentication to work.
+matrix_corporal_http_gateway_internal_rest_auth_enabled: false
+
 matrix_corporal_http_api_enabled: false
 matrix_corporal_http_api_auth_token: ""
 matrix_corporal_http_api_timeout_milliseconds: 15000

roles/matrix-corporal/tasks/init.yml

@@ -1,3 +1,3 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-corporal'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-corporal.service'] }}"
   when: matrix_corporal_enabled|bool

roles/matrix-corporal/tasks/validate_config.yml

@@ -16,7 +16,6 @@
     msg: "The Matrix Corporal HTTP API is enabled (`matrix_corporal_http_api_enabled`), but no auth token has been set in `matrix_corporal_http_api_auth_token`"
   when: "matrix_corporal_http_api_enabled|bool and matrix_corporal_http_api_auth_token == ''"
 
-
 - name: (Deprecation) Catch and report renamed corporal variables
   fail:
     msg: >-
@@ -25,3 +24,4 @@
   when: "item.old in vars"
   with_items:
     - {'old': 'matrix_corporal_container_expose_ports', 'new': '<superseded by matrix_corporal_container_http_gateway_host_bind_port and matrix_corporal_container_http_api_host_bind_port>'}
+    - {'old': 'matrix_corporal_reconciliation_user_id_local_part', 'new': 'matrix_corporal_corporal_user_id_local_part'}

roles/matrix-corporal/templates/config.json.j2

@@ -7,14 +7,20 @@
 		"TimeoutMilliseconds": {{ matrix_corporal_matrix_timeout_milliseconds }}
 	},
 
+	"Corporal": {
+		"UserID": "@{{ matrix_corporal_corporal_user_id_local_part }}:{{ matrix_domain }}"
+	},
+
 	"Reconciliation": {
-		"UserId": "@{{ matrix_corporal_reconciliation_user_id_local_part }}:{{ matrix_domain }}",
 		"RetryIntervalMilliseconds": {{ matrix_corporal_reconciliation_retry_interval_milliseconds }}
 	},
 
 	"HttpGateway": {
 		"ListenAddress": "0.0.0.0:41080",
-		"TimeoutMilliseconds": {{ matrix_corporal_http_gateway_timeout_milliseconds }}
+		"TimeoutMilliseconds": {{ matrix_corporal_http_gateway_timeout_milliseconds }},
+		"InternalRESTAuth": {
+			"Enabled": {{ matrix_corporal_http_gateway_internal_rest_auth_enabled|to_json }}
+		}
 	},
 
 	"HttpApi": {

roles/matrix-corporal/templates/systemd/matrix-corporal.service.j2

@@ -9,6 +9,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-corporal
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-corporal
 

roles/matrix-coturn/defaults/main.yml

@@ -3,7 +3,7 @@ matrix_coturn_enabled: true
 matrix_coturn_container_image_self_build: false
 matrix_coturn_container_image_self_build_repo: "https://github.com/instrumentisto/coturn-docker-image.git"
 
-matrix_coturn_docker_image: "{{ matrix_coturn_docker_image_name_prefix }}instrumentisto/coturn:4.5.1.3"
+matrix_coturn_docker_image: "{{ matrix_coturn_docker_image_name_prefix }}instrumentisto/coturn:4.5.2"
 matrix_coturn_docker_image_name_prefix: "{{ 'localhost/' if matrix_coturn_container_image_self_build else 'docker.io/' }}"
 matrix_coturn_docker_image_force_pull: "{{ matrix_coturn_docker_image.endswith(':latest') }}"
 

roles/matrix-coturn/tasks/init.yml

@@ -1,7 +1,11 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-coturn'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-coturn.service'] }}"
   when: matrix_coturn_enabled|bool
 
+- set_fact:
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-coturn-reload.timer'] }}"
+  when: "matrix_coturn_enabled|bool and matrix_coturn_tls_enabled|bool"
+
 # ansible lower than 2.8, does not support docker_image build parameters
 # for self buildig it is explicitly needed, so we rather fail here
 - name: Fail if running on Ansible lower than 2.8 and trying self building

roles/matrix-coturn/tasks/main.yml

@@ -8,8 +8,14 @@
     - setup-all
     - setup-coturn
 
-- import_tasks: "{{ role_path }}/tasks/setup_coturn.yml"
-  when: run_setup|bool
+- import_tasks: "{{ role_path }}/tasks/setup_install.yml"
+  when: "run_setup|bool and matrix_coturn_enabled|bool"
+  tags:
+    - setup-all
+    - setup-coturn
+
+- import_tasks: "{{ role_path }}/tasks/setup_uninstall.yml"
+  when: "run_setup|bool and not matrix_coturn_enabled|bool"
   tags:
     - setup-all
     - setup-coturn

roles/matrix-coturn/tasks/setup_coturn.yml

@@ -1,137 +0,0 @@
----
-
-#
-# Tasks related to setting up Coturn
-#
-
-- name: Ensure Matrix Coturn path exists
-  file:
-    path: "{{ item.path }}"
-    state: directory
-    mode: 0750
-    owner: "{{ matrix_user_username }}"
-    group: "{{ matrix_user_groupname }}"
-  with_items:
-    - { path: "{{ matrix_coturn_docker_src_files_path }}", when: "{{ matrix_coturn_container_image_self_build }}"}
-  when: matrix_coturn_enabled|bool and item.when
-
-- name: Ensure Coturn image is pulled
-  docker_image:
-    name: "{{ matrix_coturn_docker_image }}"
-    source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
-    force_source: "{{ matrix_coturn_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
-    force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_coturn_docker_image_force_pull }}"
-  when: matrix_coturn_enabled|bool and not matrix_coturn_container_image_self_build
-
-- name: Ensure Coturn repository is present on self-build
-  git:
-    repo: "{{ matrix_coturn_container_image_self_build_repo }}"
-    dest: "{{ matrix_coturn_docker_src_files_path }}"
-    version: "{{ matrix_coturn_docker_image.split(':')[1] }}"
-    force: "yes"
-  register: matrix_coturn_git_pull_results
-  when: "matrix_coturn_enabled|bool and matrix_coturn_container_image_self_build"
-
-- name: Ensure Coturn Docker image is built
-  docker_image:
-    name: "{{ matrix_coturn_docker_image }}"
-    source: build
-    force_source: "{{ matrix_coturn_git_pull_results.changed }}"
-    build:
-      dockerfile: Dockerfile
-      path: "{{ matrix_coturn_docker_src_files_path }}"
-      pull: yes
-  when: "matrix_coturn_enabled|bool and matrix_coturn_container_image_self_build|bool"
-
-- name: Ensure Coturn configuration path exists
-  file:
-    path: "{{ matrix_coturn_base_path }}"
-    state: directory
-    mode: 0750
-    owner: "{{ matrix_user_username }}"
-    group: "{{ matrix_user_groupname }}"
-  when: matrix_coturn_enabled|bool
-
-- name: Ensure turnserver.conf installed
-  template:
-    src: "{{ role_path }}/templates/turnserver.conf.j2"
-    dest: "{{ matrix_coturn_config_path }}"
-    mode: 0644
-  when: matrix_coturn_enabled|bool
-
-- name: Ensure Coturn network is created in Docker
-  docker_network:
-    name: "{{ matrix_coturn_docker_network }}"
-    driver: bridge
-  when: matrix_coturn_enabled|bool
-
-- name: Ensure matrix-coturn.service installed
-  template:
-    src: "{{ role_path }}/templates/systemd/matrix-coturn.service.j2"
-    dest: "{{ matrix_systemd_path }}/matrix-coturn.service"
-    mode: 0644
-  register: matrix_coturn_systemd_service_result
-  when: matrix_coturn_enabled|bool
-
-- name: Ensure systemd reloaded after matrix-coturn.service installation
-  service:
-    daemon_reload: yes
-  when: "matrix_coturn_enabled|bool and matrix_coturn_systemd_service_result.changed"
-
-# This may be unnecessary when more long-lived certificates are used.
-# We optimize for the common use-case though (short-lived Let's Encrypt certificates).
-# Reloading doesn't hurt anyway, so there's no need to make this more flexible.
-- name: Ensure periodic reloading of matrix-coturn is configured for SSL renewal (matrix-coturn-reload)
-  template:
-    src: "{{ role_path }}/templates/cron.d/matrix-coturn-ssl-reload.j2"
-    dest: /etc/cron.d/matrix-coturn-ssl-reload
-    mode: 0644
-  when: "matrix_coturn_enabled|bool and matrix_coturn_tls_enabled|bool"
-
-
-#
-# Tasks related to getting rid of Coturn (if it was previously enabled)
-#
-
-- name: Ensure matrix-coturn-ssl-reload cronjob removed
-  file:
-    path: /etc/cron.d/matrix-coturn-ssl-reload
-    state: absent
-  when: "not matrix_coturn_enabled|bool or not matrix_coturn_tls_enabled|bool"
-
-- name: Check existence of matrix-coturn service
-  stat:
-    path: "{{ matrix_systemd_path }}/matrix-coturn.service"
-  register: matrix_coturn_service_stat
-  when: "not matrix_coturn_enabled|bool"
-
-- name: Ensure matrix-coturn is stopped
-  service:
-    name: matrix-coturn
-    state: stopped
-    daemon_reload: yes
-  register: stopping_result
-  when: "not matrix_coturn_enabled|bool and matrix_coturn_service_stat.stat.exists"
-
-- name: Ensure matrix-coturn.service doesn't exist
-  file:
-    path: "{{ matrix_systemd_path }}/matrix-coturn.service"
-    state: absent
-  when: "not matrix_coturn_enabled|bool and matrix_coturn_service_stat.stat.exists"
-
-- name: Ensure systemd reloaded after matrix-coturn.service removal
-  service:
-    daemon_reload: yes
-  when: "not matrix_coturn_enabled|bool and matrix_coturn_service_stat.stat.exists"
-
-- name: Ensure Matrix coturn paths don't exist
-  file:
-    path: "{{ matrix_coturn_base_path }}"
-    state: absent
-  when: "not matrix_coturn_enabled|bool"
-
-- name: Ensure coturn Docker image doesn't exist
-  docker_image:
-    name: "{{ matrix_coturn_docker_image }}"
-    state: absent
-  when: "not matrix_coturn_enabled|bool"

roles/matrix-coturn/tasks/setup_install.yml

@@ -0,0 +1,104 @@
+---
+
+# This is a cleanup/migration task. It can be removed some time in the future.
+- name: (Migration) Remove deprecated cronjob
+  file:
+    path: "{{ matrix_cron_path }}/matrix-coturn-ssl-reload"
+    state: absent
+
+- name: Ensure Matrix Coturn path exists
+  file:
+    path: "{{ item.path }}"
+    state: directory
+    mode: 0750
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+  with_items:
+    - { path: "{{ matrix_coturn_docker_src_files_path }}", when: "{{ matrix_coturn_container_image_self_build }}"}
+  when: "item.when|bool"
+
+- name: Ensure Coturn image is pulled
+  docker_image:
+    name: "{{ matrix_coturn_docker_image }}"
+    source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
+    force_source: "{{ matrix_coturn_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
+    force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_coturn_docker_image_force_pull }}"
+  when: "not matrix_coturn_container_image_self_build|bool"
+
+- block:
+    - name: Ensure Coturn repository is present on self-build
+      git:
+        repo: "{{ matrix_coturn_container_image_self_build_repo }}"
+        dest: "{{ matrix_coturn_docker_src_files_path }}"
+        version: "{{ matrix_coturn_docker_image.split(':')[1] }}"
+        force: "yes"
+      register: matrix_coturn_git_pull_results
+
+    - name: Ensure Coturn Docker image is built
+      docker_image:
+        name: "{{ matrix_coturn_docker_image }}"
+        source: build
+        force_source: "{{ matrix_coturn_git_pull_results.changed }}"
+        build:
+          dockerfile: Dockerfile
+          path: "{{ matrix_coturn_docker_src_files_path }}"
+          pull: yes
+  when: "matrix_coturn_container_image_self_build|bool"
+
+- name: Ensure Coturn configuration path exists
+  file:
+    path: "{{ matrix_coturn_base_path }}"
+    state: directory
+    mode: 0750
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+
+- name: Ensure turnserver.conf installed
+  template:
+    src: "{{ role_path }}/templates/turnserver.conf.j2"
+    dest: "{{ matrix_coturn_config_path }}"
+    mode: 0644
+    owner: "{{ matrix_user_username }}"
+    group: "{{ matrix_user_groupname }}"
+
+- name: Ensure Coturn network is created in Docker
+  docker_network:
+    name: "{{ matrix_coturn_docker_network }}"
+    driver: bridge
+
+- name: Ensure matrix-coturn.service installed
+  template:
+    src: "{{ role_path }}/templates/systemd/matrix-coturn.service.j2"
+    dest: "{{ matrix_systemd_path }}/matrix-coturn.service"
+    mode: 0644
+  register: matrix_coturn_systemd_service_change_results
+
+# This may be unnecessary when more long-lived certificates are used.
+# We optimize for the common use-case though (short-lived Let's Encrypt certificates).
+# Reloading doesn't hurt anyway, so there's no need to make this more flexible.
+- name: Ensure reloading systemd units installed, if necessary
+  template:
+    src: "{{ role_path }}/templates/systemd/{{ item }}.j2"
+    dest: "{{ matrix_systemd_path }}/{{ item }}"
+    mode: 0644
+  register: "matrix_coturn_systemd_service_change_results"
+  when: "matrix_coturn_tls_enabled|bool"
+  with_items:
+    - matrix-coturn-reload.service
+    - matrix-coturn-reload.timer
+
+# A similar task exists in `setup_uninstall.yml`
+- name: Ensure reloading systemd units uninstalled, if unnecessary
+  file:
+    path: "{{ item }}"
+    state: absent
+  register: "matrix_coturn_systemd_service_change_results"
+  when: "not matrix_coturn_tls_enabled|bool"
+  with_items:
+    - matrix-coturn-reload.service
+    - matrix-coturn-reload.timer
+
+- name: Ensure systemd reloaded if systemd units changed
+  service:
+    daemon_reload: yes
+  when: "matrix_coturn_systemd_service_change_results.changed"

roles/matrix-coturn/tasks/setup_uninstall.yml

@@ -0,0 +1,45 @@
+---
+
+- name: Check existence of matrix-coturn service
+  stat:
+    path: "{{ matrix_systemd_path }}/matrix-coturn.service"
+  register: matrix_coturn_service_stat
+  when: "not matrix_coturn_enabled|bool"
+
+- name: Ensure matrix-coturn is stopped
+  service:
+    name: matrix-coturn
+    state: stopped
+    daemon_reload: yes
+  when: "matrix_coturn_service_stat.stat.exists|bool"
+
+- name: Ensure matrix-coturn-reload.timer is stopped
+  service:
+    name: matrix-coturn
+    state: stopped
+    daemon_reload: yes
+  failed_when: false
+  when: "matrix_coturn_service_stat.stat.exists|bool"
+
+- name: Ensure systemd units don't exist
+  file:
+    path: "{{ matrix_systemd_path }}/{{ item }}"
+    state: absent
+  register: matrix_coturn_systemd_unit_uninstallation_result
+  with_items:
+    - matrix-coturn.service
+    - matrix-coturn-reload.service
+    - matrix-coturn-reload.timer
+
+- name: Ensure systemd reloaded after unit removal
+  service:
+    daemon_reload: yes
+  when: "matrix_coturn_systemd_unit_uninstallation_result.changed|bool"
+
+- name: Ensure Matrix coturn paths don't exist
+  file:
+    path: "{{ matrix_coturn_base_path }}"
+    state: absent
+
+# Intentionally not removing the Docker image when uninstalling.
+# We can't be sure it had been pulled by us in the first place.

roles/matrix-coturn/templates/cron.d/matrix-coturn-ssl-reload.j2

@@ -1 +0,0 @@
-20 4 */5 * * root {{ matrix_host_command_systemctl }} reload matrix-coturn.service

roles/matrix-coturn/templates/systemd/matrix-coturn-reload.service.j2

@@ -0,0 +1,6 @@
+[Unit]
+Description=Reloads matrix-coturn so that new SSL certificates can kick in
+
+[Service]
+Type=oneshot
+ExecStart={{ matrix_host_command_systemctl }} reload matrix-coturn.service

roles/matrix-coturn/templates/systemd/matrix-coturn-reload.timer.j2

@@ -0,0 +1,10 @@
+[Unit]
+Description=Reloads matrix-coturn periodically so that new SSL certificates can kick in
+
+[Timer]
+Unit=matrix-coturn-reload.service
+OnCalendar=Sunday *-*-* 13:00:00
+RandomizedDelaySec=3h
+
+[Install]
+WantedBy=timers.target

roles/matrix-coturn/templates/systemd/matrix-coturn.service.j2

@@ -9,6 +9,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-coturn
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-coturn
 

roles/matrix-dimension/tasks/init.yml

@@ -1,3 +1,3 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-dimension'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-dimension.service'] }}"
   when: matrix_dimension_enabled|bool

roles/matrix-dimension/templates/systemd/matrix-dimension.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-dimension
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-dimension
 

roles/matrix-dynamic-dns/tasks/init.yml

@@ -1,3 +1,3 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-dynamic-dns'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-dynamic-dns.service'] }}"
   when: "matrix_dynamic_dns_enabled|bool"

roles/matrix-dynamic-dns/tasks/uninstall.yml

@@ -22,3 +22,6 @@
   service:
     daemon_reload: yes
   when: "matrix_dynamic_dns_service_stat.stat.exists"
+
+# Intentionally not removing the Docker image when uninstalling.
+# We can't be sure it had been pulled by us in the first place.

roles/matrix-dynamic-dns/templates/systemd/matrix-dynamic-dns.service.j2

@@ -12,6 +12,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-dynamic-dns
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-dynamic-dns
 ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-dynamic-dns \

roles/matrix-email2matrix/tasks/init.yml

@@ -1,3 +1,3 @@
 - set_fact:
-    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-email2matrix'] }}"
+    matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-email2matrix.service'] }}"
   when: matrix_email2matrix_enabled|bool

roles/matrix-email2matrix/templates/systemd/matrix-email2matrix.service.j2

@@ -7,6 +7,7 @@ DefaultDependencies=no
 
 [Service]
 Type=simple
+Environment="HOME={{ matrix_systemd_unit_home_path }}"
 ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-email2matrix
 ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-email2matrix