diff --git a/CHANGELOG.md b/CHANGELOG.md index 27afb64e..1aa8c9b5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,85 @@ +# 2021-01-03 + +## Signal bridging support via mautrix-signal + +Thanks to [laszabine](https://github.com/laszabine)'s efforts, the playbook now supports bridging to [Signal](https://www.signal.org/) via the [mautrix-signal](https://github.com/tulir/mautrix-signal) bridge. See our [Setting up Mautrix Signal bridging](docs/configuring-playbook-bridge-mautrix-signal.md) documentation page for getting started. + +If you had installed the mautrix-signal bridge while its Pull Request was still work-in-progress, you can migrate your data to the new and final setup by referring to [this comment](https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/686#issuecomment-753510789). + + +# 2020-12-23 + +## The big move to all-on-Postgres (potentially dangerous) + +**TLDR**: all your bridges (and other services) will likely be auto-migrated from SQLite/nedb to Postgres, hopefully without trouble. You can opt-out (see how below), if too worried about breakage. + +Until now, we've only used Postgres as a database for Synapse. All other services (bridges, bots, etc.) were kept simple and used a file-based database (SQLite or nedb). + +Since [this huge pull request](https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/740), **all of our services now use Postgres by default**. Thanks to [Johanna Dorothea Reichmann](https://github.com/jdreichmann) for starting the work on it and for providing great input! + +Moving all services to Postgres brings a few **benefits** to us: + +- **improved performance** +- **improved compatibility**. Most bridges are deprecating SQLite/nedb support or offer less features when not on Postgres. +- **easier backups**. It's still some effort to take a proper backup (Postgres dump + various files, keys), but a Postgres dump now takes you much further. +- we're now **more prepared to introduce other services** that need a Postgres database - [Dendrite](https://github.com/matrix-org/dendrite), the [mautrix-signal](https://github.com/tulir/mautrix-signal) bridge (existing [pull request](https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/686)), etc. + +### Key takeway + +- existing installations that use an [external Postgres](https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/docs/configuring-playbook-external-postgres.md) server should be unaffected (they remain on SQLite/nedb for all services, except Synapse) + +- for existing installations which use our integrated Postgres database server (`matrix-postgres`, which is the default), **we automatically migrate data** from SQLite/nedb to Postgres and **archive the database files** (`something.db` -> `something.db.backup`), so you can restore them if you need to go back (see how below). + +### Opting-out of the Postgres migration + +This is a **very large and somewhat untested change** (potentially dangerous), so **if you're not feeling confident/experimental, opt-out** of it for now. Still, it's the new default and what we (and various bridges) will focus on going forward, so don't stick to old ways for too long. + +You can remain on SQLite/nedb (at least for now) by adding a variable like this to your `vars.yml` file for each service you use: `matrix_COMPONENT_database_engine: sqlite` (e.g. `matrix_mautrix_facebook_database_engine: sqlite`). + +Some services (like `appservice-irc` and `appservice-slack`) don't use SQLite, so use `nedb`, instead of `sqlite` for them. + +### Going back to SQLite/nedb if things went wrong + +If you went with the Postgres migration and it went badly for you (some bridge not working as expected or not working at all), do this: + +- stop all services (`ansible-playbook -i inventory/hosts setup.yml --tags=stop`) +- SSH into the server and rename the old database files (`something.db.backup` -> `something.db`). Example: `mv /matrix/mautrix-facebook/data/mautrix-facebook.db.backup /matrix/mautrix-facebook/data/mautrix-facebook.db` +- switch the affected service back to SQLite (e.g. `matrix_mautrix_facebook_database_engine: sqlite`). Some services (like `appservice-irc` and `appservice-slack`) don't use SQLite, so use `nedb`, instead of `sqlite` for them. +- re-run the playbook (`ansible-playbook -i inventory/hosts setup.yml --tags=setup-all,start`) +- [get in touch](README.md#support) with us + + +# 2020-12-11 + +## synapse-janitor support removed + +We've removed support for the unmaintained [synapse-janitor](https://github.com/xwiki-labs/synapse_scripts) script. There's been past reports of it corrupting the Synapse database. Since there hasn't been any new development on it and it doesn't seem too useful nowadays, there's no point in including it in the playbook. + +If you need to clean up or compact your database, consider using the Synapse Admin APIs directly. See our [Synapse maintenance](docs/maintenance-synapse.md) and [Postgres maintenance](docs/maintenance-postgres.md) documentation pages for more details. + + +## Docker 20.10 is here + +(No need to do anything special in relation to this. Just something to keep in mind) + +Docker 20.10 got released recently and your server will likely get it the next time you update. + +This is the first major Docker update in a long time and it packs a lot of changes. +Some of them introduced some breakage for us initially (see [here](https://github.com/spantaleev/matrix-docker-ansible-deploy/commit/d08b27784f222effcbce2abf924bf07bbe0893be) and [here](https://github.com/spantaleev/matrix-docker-ansible-deploy/commit/7593d969e316cc0144bce378a5be58c76c2c37ee)), but it should be all good now. + + +# 2020-12-08 + +## openid APIs exposed by default on the federation port when federation disabled + +We've changed some defaults. People running with our default configuration (federation enabled), are not affected at all. + +If you are running an unfederated server (`matrix_synapse_federation_enabled: false`), this may be of interest to you. + +When federation is disabled, but ma1sd or Dimension are enabled, we'll now expose the `openid` APIs on the federation port. +These APIs are necessary for some ma1sd features to work. If you'd like to prevent this, you can: `matrix_synapse_federation_port_openid_resource_required: false`. + + # 2020-11-27 ## Recent Jitsi updates may require configuration changes diff --git a/README.md b/README.md index ac2b7ca9..5a9278f3 100644 --- a/README.md +++ b/README.md @@ -46,6 +46,8 @@ Using this playbook, you can get the following services configured on your serve - (optional) the [mautrix-hangouts](https://github.com/tulir/mautrix-hangouts) bridge for bridging your Matrix server to [Google Hangouts](https://en.wikipedia.org/wiki/Google_Hangouts) +- (optional) the [mautrix-signal](https://github.com/tulir/mautrix-signal) bridge for bridging your Matrix server to [Signal](https://www.signal.org/) + - (optional) the [matrix-appservice-irc](https://github.com/matrix-org/matrix-appservice-irc) bridge for bridging your Matrix server to [IRC](https://wikipedia.org/wiki/Internet_Relay_Chat) - (optional) the [matrix-appservice-discord](https://github.com/Half-Shot/matrix-appservice-discord) bridge for bridging your Matrix server to [Discord](https://discordapp.com/) @@ -150,13 +152,15 @@ This playbook sets up your server using the following Docker images: - [certbot/certbot](https://hub.docker.com/r/certbot/certbot/) - the [certbot](https://certbot.eff.org/) tool for obtaining SSL certificates from [Let's Encrypt](https://letsencrypt.org/) (optional) -- [tulir/mautrix-telegram](https://hub.docker.com/r/tulir/mautrix-telegram/) - the [mautrix-telegram](https://github.com/tulir/mautrix-telegram) bridge to [Telegram](https://telegram.org/) (optional) +- [tulir/mautrix-telegram](https://mau.dev/tulir/mautrix-telegram/container_registry) - the [mautrix-telegram](https://github.com/tulir/mautrix-telegram) bridge to [Telegram](https://telegram.org/) (optional) + +- [tulir/mautrix-whatsapp](https://mau.dev/tulir/mautrix-whatsapp/container_registry) - the [mautrix-whatsapp](https://github.com/tulir/mautrix-whatsapp) bridge to [Whatsapp](https://www.whatsapp.com/) (optional) -- [tulir/mautrix-whatsapp](https://hub.docker.com/r/tulir/mautrix-whatsapp/) - the [mautrix-whatsapp](https://github.com/tulir/mautrix-whatsapp) bridge to [Whatsapp](https://www.whatsapp.com/) (optional) +- [tulir/mautrix-facebook](https://mau.dev/tulir/mautrix-facebook/container_registry) - the [mautrix-facebook](https://github.com/tulir/mautrix-facebook) bridge to [Facebook](https://facebook.com/) (optional) -- [tulir/mautrix-facebook](https://hub.docker.com/r/tulir/mautrix-facebook/) - the [mautrix-facebook](https://github.com/tulir/mautrix-facebook) bridge to [Facebook](https://facebook.com/) (optional) +- [tulir/mautrix-hangouts](https://mau.dev/tulir/mautrix-hangouts/container_registry) - the [mautrix-hangouts](https://github.com/tulir/mautrix-hangouts) bridge to [Google Hangouts](https://en.wikipedia.org/wiki/Google_Hangouts) (optional) -- [tulir/mautrix-hangouts](https://hub.docker.com/r/tulir/mautrix-hangouts/) - the [mautrix-hangouts](https://github.com/tulir/mautrix-hangouts) bridge to [Google Hangouts](https://en.wikipedia.org/wiki/Google_Hangouts) (optional) +- [tulir/mautrix-signal](https://mau.dev/tulir/mautrix-signal/container_registry) - the [mautrix-signal](https://github.com/tulir/mautrix-signal) bridge to [Signal](https://www.signal.org/) (optional) - [matrixdotorg/matrix-appservice-irc](https://hub.docker.com/r/matrixdotorg/matrix-appservice-irc) - the [matrix-appservice-irc](https://github.com/matrix-org/matrix-appservice-irc) bridge to [IRC](https://wikipedia.org/wiki/Internet_Relay_Chat) (optional) diff --git a/docs/README.md b/docs/README.md index 8d22477e..8c95eff3 100644 --- a/docs/README.md +++ b/docs/README.md @@ -10,13 +10,13 @@ - [Installing](installing.md) -- **Importing data from another Synapse server installation** +- **Importing data from another server installation** - - [Importing an existing SQLite database (from another installation)](importing-sqlite.md) (optional) + - [Importing an existing SQLite database (from another Synapse installation)](importing-synapse-sqlite.md) (optional) - [Importing an existing Postgres database (from another installation)](importing-postgres.md) (optional) - - [Importing `media_store` data files from an existing installation](importing-media-store.md) (optional) + - [Importing `media_store` data files from an existing Synapse installation](importing-synapse-media-store.md) (optional) - [Registering users](registering-users.md) diff --git a/docs/ansible.md b/docs/ansible.md index 776147bd..ff513bbf 100644 --- a/docs/ansible.md +++ b/docs/ansible.md @@ -9,9 +9,9 @@ If your local computer cannot run Ansible, you can also run Ansible on some serv ## Supported Ansible versions -Ansible 2.7.0 or newer is required. +Ansible 2.7.1 or newer is required ([last discussion about Ansible versions](https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/743)). -Ubuntu (at least 20.04) ships with a buggy version (see this [bug](https://bugs.launchpad.net/ubuntu/+source/ansible/+bug/1880359)), which can't be used in combination with a host running new systemd (more detaisl in [#517](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/517), [#669]([669](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/669))). If this problem affects you, you can: avoid running Ubuntu 20.04 on your host; run Ansible from another machine targeting your host; or try to upgrade to a newer Ansible version (see below). +Note: Ubuntu 20.04 ships with Ansible 2.9.6 which is a buggy version (see this [bug](https://bugs.launchpad.net/ubuntu/+source/ansible/+bug/1880359)), which can't be used in combination with a host running new systemd (more details in [#517](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/517), [#669](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/669)). If this problem affects you, you can: avoid running Ubuntu 20.04 on your host; run Ansible from another machine targeting your host; or try to upgrade to a newer Ansible version (see below). ## Checking your Ansible version @@ -51,7 +51,7 @@ docker run -it --rm \ -v `pwd`:/work \ -v $HOME/.ssh/id_rsa:/root/.ssh/id_rsa:ro \ --entrypoint=/bin/sh \ -devture/ansible:2.9.13-r0 +docker.io/devture/ansible:2.9.14-r0 ``` The above command tries to mount an SSH key (`$HOME/.ssh/id_rsa`) into the container (at `/root/.ssh/id_rsa`). diff --git a/docs/configuring-playbook-bridge-appservice-discord.md b/docs/configuring-playbook-bridge-appservice-discord.md index 13a5174b..f3efc555 100644 --- a/docs/configuring-playbook-bridge-appservice-discord.md +++ b/docs/configuring-playbook-bridge-appservice-discord.md @@ -22,10 +22,10 @@ matrix_appservice_discord_client_id: "YOUR DISCORD APP CLIENT ID" matrix_appservice_discord_bot_token: "YOUR DISCORD APP BOT TOKEN" ``` -4. If you've already installed Matrix services using the playbook before, you'll need to re-run it (`--tags=setup-all,start`). If not, proceed with [configuring other playbook services](configuring-playbook.md) and then with [Installing](installing.md). Get back to this guide once ready. -5. Retrieve Discord invite link from the `{{ matrix_appservice_discord_config_path }}/invite_link` file on the server (this defaults to `/matrix/appservice-discord/config/invite_link`). You need to peek at the file on the server via SSH, etc., because it's not available via HTTP(S). -6. Invite the Bot to Discord servers you wish to bridge. Administrator permission is recommended. -7. Room addresses follow this syntax: `#_discord_guildid_channelid`. You can easily find the guild and channel ids by logging into Discord in a browser and opening the desired channel. The URL will have this format: `discordapp.com/channels/guild_id/channel_id`. Once you have figured out the appropriate room addrss, you can join by doing `/join #_discord_guildid_channelid` in your Matrix client. +5. If you've already installed Matrix services using the playbook before, you'll need to re-run it (`--tags=setup-all,start`). If not, proceed with [configuring other playbook services](configuring-playbook.md) and then with [Installing](installing.md). Get back to this guide once ready. +6. Retrieve Discord invite link from the `{{ matrix_appservice_discord_config_path }}/invite_link` file on the server (this defaults to `/matrix/appservice-discord/config/invite_link`). You need to peek at the file on the server via SSH, etc., because it's not available via HTTP(S). +7. Invite the Bot to Discord servers you wish to bridge. Administrator permission is recommended. +8. Room addresses follow this syntax: `#_discord_guildid_channelid`. You can easily find the guild and channel ids by logging into Discord in a browser and opening the desired channel. The URL will have this format: `discordapp.com/channels/guild_id/channel_id`. Once you have figured out the appropriate room addrss, you can join by doing `/join #_discord_guildid_channelid` in your Matrix client. Other configuration options are available via the `matrix_appservice_discord_configuration_extension_yaml` variable. diff --git a/docs/configuring-playbook-bridge-mautrix-signal.md b/docs/configuring-playbook-bridge-mautrix-signal.md new file mode 100644 index 00000000..164b06de --- /dev/null +++ b/docs/configuring-playbook-bridge-mautrix-signal.md @@ -0,0 +1,46 @@ +# Setting up Mautrix Signal (optional) + +The playbook can install and configure [mautrix-signal](https://github.com/tulir/mautrix-signal) for you. + +See the project's [documentation](https://github.com/tulir/mautrix-signal/wiki) to learn what it does and why it might be useful to you. + +**Note/Prerequisite**: If you're running with the Postgres database server integrated by the playbook (which is the default), you don't need to do anything special and can easily proceed with installing. However, if you're [using an external Postgres server](configuring-playbook-external-postgres.md), you'd need to manually prepare a Postgres database for this bridge and adjust the variables related to that (`matrix_mautrix_signal_database_*`). + +Use the following playbook configuration: + +```yaml +matrix_mautrix_signal_enabled: true +``` + +## Set up Double Puppeting + +If you'd like to use [Double Puppeting](https://github.com/tulir/mautrix-whatsapp/wiki/Authentication#replacing-whatsapp-accounts-matrix-puppet-with-matrix-account) (hint: you most likely do), you have 2 ways of going about it. + +### Method 1: automatically, by enabling Shared Secret Auth + +The bridge will automatically perform Double Puppeting if you enable [Shared Secret Auth](configuring-playbook-shared-secret-auth.md) for this playbook. + +This is the recommended way of setting up Double Puppeting, as it's easier to accomplish, works for all your users automatically, and has less of a chance of breaking in the future. + +### Method 2: manually, by asking each user to provide a working access token + +**Note**: This method for enabling Double Puppeting can be configured only after you've already set up bridging (see [Usage](#usage)). + +When using this method, **each user** that wishes to enable Double Puppeting needs to follow the following steps: + +- retrieve a Matrix access token for yourself. You can use the following command: + +``` +curl \ +--data '{"identifier": {"type": "m.id.user", "user": "YOUR_MATRIX_USERNAME" }, "password": "YOUR_MATRIX_PASSWORD", "type": "m.login.password", "device_id": "Mautrix-Signal", "initial_device_display_name": "Mautrix-Signal"}' \ +https://matrix.DOMAIN/_matrix/client/r0/login +``` + +- send the access token to the bot. Example: `login-matrix MATRIX_ACCESS_TOKEN_HERE` + +- make sure you don't log out the `Mautrix-Signal` device some time in the future, as that would break the Double Puppeting feature + + +## Usage + +You then need to start a chat with `@signalbot:YOUR_DOMAIN` (where `YOUR_DOMAIN` is your base domain, not the `matrix.` domain). diff --git a/docs/configuring-playbook-dimension.md b/docs/configuring-playbook-dimension.md index 604e6aa6..d5f0a9e6 100644 --- a/docs/configuring-playbook-dimension.md +++ b/docs/configuring-playbook-dimension.md @@ -3,6 +3,9 @@ **[Dimension](https://dimension.t2bot.io) can only be installed after Matrix services are installed and running.** If you're just installing Matrix services for the first time, please continue with the [Configuration](configuring-playbook.md) / [Installation](installing.md) flow and come back here later. +**Note**: enabling Dimension, means that the `openid` API endpoints will be exposed on the Matrix Federation port (usually `8448`), even if [federation](configuring-playbook-federation.md) is disabled. It's something to be aware of, especially in terms of firewall whitelisting (make sure port `8448` is accessible). + + ## Prerequisites This playbook now supports running [Dimension](https://dimension.t2bot.io) in both a federated and an [unfederated](https://github.com/turt2live/matrix-dimension/blob/master/docs/unfederated.md) environment. This is handled automatically based on the value of `matrix_synapse_federation_enabled`. @@ -48,7 +51,7 @@ To get an access token for the Dimension user, you can follow one of two options 3. Copy the highlighted text to your configuration. 4. Close the private browsing session. **Do not log out**. Logging out will invalidate the token, making it not work. -*With CURL* +*With CURL* ``` curl -X POST --header 'Content-Type: application/json' -d '{ diff --git a/docs/configuring-playbook-federation.md b/docs/configuring-playbook-federation.md index 1e4ad61e..2e6410ec 100644 --- a/docs/configuring-playbook-federation.md +++ b/docs/configuring-playbook-federation.md @@ -37,3 +37,13 @@ matrix_synapse_federation_enabled: false ``` With that, your server's users will only be able to talk among themselves, but not to anyone who is on another server. + +**Disabling federation does not necessarily disable the federation port** (`8448`). Services like [Dimension](configuring-playbook-dimension.md) and [ma1sd](configuring-playbook-ma1sd.md) normally rely on `openid` APIs exposed on that port. Even if you disable federation and only if necessary, we may still be exposing the federation port and serving the `openid` APIs there. To override this and completely disable Synapse's federation port use: + +```yaml +# This stops the federation port on the Synapse side (normally `matrix-synapse:8048` on the container network). +matrix_synapse_federation_port_enabled: false + +# This removes the `8448` virtual host from the matrix-nginx-proxy reverse-proxy server. +matrix_nginx_proxy_proxy_matrix_federation_api_enabled: false +``` diff --git a/docs/configuring-playbook-jitsi.md b/docs/configuring-playbook-jitsi.md index 62edc578..f72241e1 100644 --- a/docs/configuring-playbook-jitsi.md +++ b/docs/configuring-playbook-jitsi.md @@ -99,7 +99,7 @@ matrix_jitsi_web_custom_config_extension: | config.disableAudioLevels = true; - # Limit the number of video feeds forwarded to each client + // Limit the number of video feeds forwarded to each client config.channelLastN = 4; matrix_jitsi_web_config_resolution_width_ideal_and_max: 480 diff --git a/docs/configuring-playbook-ma1sd.md b/docs/configuring-playbook-ma1sd.md index 03208337..70c507cb 100644 --- a/docs/configuring-playbook-ma1sd.md +++ b/docs/configuring-playbook-ma1sd.md @@ -4,7 +4,9 @@ By default, this playbook configures an [ma1sd](https://github.com/ma1uta/ma1sd) This server is private by default, potentially at the expense of user discoverability. -ma1sd is a fork of [mxisd](https://github.com/kamax-io/mxisd) which was pronounced end of life 2019-06-21. +*ma1sd is a fork of [mxisd](https://github.com/kamax-io/mxisd) which was pronounced end of life 2019-06-21.* + +**Note**: enabling ma1sd (which is also the default), means that the `openid` API endpoints will be exposed on the Matrix Federation port (usually `8448`), even if [federation](configuring-playbook-federation.md) is disabled. It's something to be aware of, especially in terms of firewall whitelisting (make sure port `8448` is accessible). ## Disabling ma1sd @@ -50,6 +52,9 @@ To use the [Registration](https://github.com/ma1uta/ma1sd/blob/master/docs/featu - `matrix_ma1sd_configuration_extension_yaml` - to configure ma1sd as required. See the [Registration feature's docs](https://github.com/ma1uta/ma1sd/blob/master/docs/features/registration.md) for inspiration. Also see the [Additional features](#additional-features) section below to learn more about how to use `matrix_ma1sd_configuration_extension_yaml`. +**Note**: For this to work, either the homeserver needs to [federate](configuring-playbook-federation.md) or the `openid` APIs need to exposed on the federation port. When federation is disabled and ma1sd is enabled, we automatically expose the `openid` APIs (only!) on the federation port. Make sure the federation port (usually `https://matrix.DOMAIN:8448`) is whitelisted in your firewall (even if you don't actually use/need federation). + + ## Authentication [Authentication](https://github.com/ma1uta/ma1sd/blob/master/docs/features/authentication.md) provides the possibility to use your own [Identity Stores](https://github.com/ma1uta/ma1sd/blob/master/docs/stores/README.md) (for example LDAP) to authenticate users on your Homeserver. The following configuration can be used to authenticate against an LDAP server: diff --git a/docs/configuring-playbook-nginx.md b/docs/configuring-playbook-nginx.md index 945864e9..5693c569 100644 --- a/docs/configuring-playbook-nginx.md +++ b/docs/configuring-playbook-nginx.md @@ -24,7 +24,6 @@ matrix_nginx_proxy_proxy_matrix_nginx_status_allowed_addresses: - 1.1.1.1 ``` - ## Synapse + OpenID Connect for Single-Sign-On If you want to use OpenID Connect as an SSO provider (as per the [Synapse OpenID docs](https://github.com/matrix-org/synapse/blob/develop/docs/openid.md)), you need to use the following configuration (in your `vars.yml` file) to instruct nginx to forward `/_synapse/oidc` to Synapse: @@ -32,3 +31,11 @@ If you want to use OpenID Connect as an SSO provider (as per the [Synapse OpenID ```yaml matrix_nginx_proxy_proxy_matrix_client_api_forwarded_location_synapse_oidc_api_enabled: true ``` + +## Disable Nginx access logs + +This will disable the access logging for nginx. + +```yaml +matrix_nginx_proxy_access_log_enabled: false +``` diff --git a/docs/configuring-playbook-own-webserver.md b/docs/configuring-playbook-own-webserver.md index 8c671df4..915c2c37 100644 --- a/docs/configuring-playbook-own-webserver.md +++ b/docs/configuring-playbook-own-webserver.md @@ -144,7 +144,7 @@ matrix_nginx_proxy_container_extra_arguments: - '--label "traefik.enable=true"' # The Nginx proxy container will receive traffic from these subdomains - - '--label "traefik.http.routers.matrix-nginx-proxy.rule=Host(`{{ matrix_server_fqn_matrix }}`,`{{ matrix_server_fqn_element }}`,`{{ matrix_server_fqn_dimension }},`{{ matrix_server_fqn_jitsi }}`)"' + - '--label "traefik.http.routers.matrix-nginx-proxy.rule=Host(`{{ matrix_server_fqn_matrix }}`,`{{ matrix_server_fqn_element }}`,`{{ matrix_server_fqn_dimension }}`,`{{ matrix_server_fqn_jitsi }}`)"' # (The 'web-secure' entrypoint must bind to port 443 in Traefik config) - '--label "traefik.http.routers.matrix-nginx-proxy.entrypoints=web-secure"' @@ -219,7 +219,7 @@ services: - "--certificatesresolvers.default.acme.storage=/letsencrypt/acme.json" ports: - "443:443" - - "8080:8080" + - "8448:8448" volumes: - "./letsencrypt:/letsencrypt" - "/var/run/docker.sock:/var/run/docker.sock:ro" diff --git a/docs/configuring-playbook-ssl-certificates.md b/docs/configuring-playbook-ssl-certificates.md index 5b5c7cdc..7f05a5b2 100644 --- a/docs/configuring-playbook-ssl-certificates.md +++ b/docs/configuring-playbook-ssl-certificates.md @@ -67,6 +67,7 @@ By default, it obtains certificates for: - possibly for `element.`, unless you have disabled the [Element client component](configuring-playbook-client-element.md) using `matrix_client_element_enabled: false` - possibly for `riot.`, if you have explicitly enabled Riot to Element redirection (for background compatibility) using `matrix_nginx_proxy_proxy_riot_compat_redirect_enabled: true` - possibly for `dimension.`, if you have explicitly [set up Dimension](configuring-playbook-dimension.md). +- possibly for `jitsi.`, if you have explicitly [set up Jitsi](configuring-playbook-jitsi.md). - possibly for your base domain (``), if you have explicitly configured [Serving the base domain](configuring-playbook-base-domain-serving.md) If you are hosting other domains on the Matrix machine, you can make the playbook obtain and renew certificates for those other domains too. @@ -80,6 +81,7 @@ matrix_ssl_domains_to_obtain_certificates_for: - '{{ matrix_server_fqn_matrix }}' - '{{ matrix_server_fqn_element }}' - '{{ matrix_server_fqn_dimension }}' + - '{{ matrix_server_fqn_jitsi }}' - '{{ matrix_domain }}' ``` diff --git a/docs/configuring-playbook.md b/docs/configuring-playbook.md index 7ef58b2f..3bb28c3f 100644 --- a/docs/configuring-playbook.md +++ b/docs/configuring-playbook.md @@ -94,6 +94,8 @@ When you're done with all the configuration you'd like to do, continue with [Ins - [Setting up Mautrix Hangouts bridging](configuring-playbook-bridge-mautrix-hangouts.md) (optional) +- [Setting up Mautrix Signal bridging](configuring-playbook-bridge-mautrix-signal.md) (optional) + - [Setting up Appservice IRC bridging](configuring-playbook-bridge-appservice-irc.md) (optional) - [Setting up Appservice Discord bridging](configuring-playbook-bridge-appservice-discord.md) (optional) diff --git a/docs/howto-server-delegation.md b/docs/howto-server-delegation.md index 1ea1091a..b89bd2f2 100644 --- a/docs/howto-server-delegation.md +++ b/docs/howto-server-delegation.md @@ -22,20 +22,20 @@ If this is okay with you, feel free to not read ahead. Server Delegation by means of a `/.well-known/matrix/server` file is the most straightforward, but suffers from the following downsides: -- you need to have a working HTTPS server for the base domain (``) +- you need to have a working HTTPS server for the base domain (``). If you don't have any server for the base domain at all, you can easily solve it by making the playbook [serve the base domain from the Matrix server](configuring-playbook-base-domain-serving.md). - any downtime on the base domain (``) or network trouble between the matrix subdomain (`matrix.`) and the base `` may cause Matrix Federation outages. As the [Server-Server spec says](https://matrix.org/docs/spec/server_server/r0.1.0.html#server-discovery): > Errors are recommended to be cached for up to an hour, and servers are encouraged to exponentially back off for repeated failures. -If this is not a concern for you, feel free to not read ahead. +**For most people, this is a reasonable tradeoff** given that it's easy and straightforward to set up. We recommend you stay on this path. -Otherwise, you can decide to go against the default for this playbook, and instead set up [Server Delegation via a DNS SRV record (advanced)](#server-delegation-via-a-dns-srv-record-advanced). +Otherwise, you can decide to go against the default for this playbook, and instead set up [Server Delegation via a DNS SRV record (advanced)](#server-delegation-via-a-dns-srv-record-advanced) (much more complicated). ## Server Delegation via a DNS SRV record (advanced) -**NOTE**: doing Server Delegation via a DNS SRV record is a more advanced way to do it and is not the default for this playbook. +**NOTE**: doing Server Delegation via a DNS SRV record is a more **advanced** way to do it and is not the default for this playbook. This is usually **much more complicated** to set up, so **we don't recommend it**. If you're not an experience sysadmin, you'd better stay away from this. As per the [Server-Server spec](https://matrix.org/docs/spec/server_server/r0.1.0.html#server-discovery), it's possible to do Server Delegation using only a SRV record (without a `/.well-known/matrix/server` file). @@ -47,7 +47,7 @@ To use DNS SRV record validation, you need to: - ensure that you have a `_matrix._tcp` DNS SRV record for your base domain (``) with a value of `10 0 8448 matrix.` -- ensure that you are serving the Matrix Federation API (tcp/8448) with a certificate for `` (not `matrix.`!). See below. +- ensure that you are serving the Matrix Federation API (tcp/8448) with a certificate for `` (not `matrix.`!). Getting this certificate to the `matrix.` server may be complicated. The playbook's automatic SSL obtaining/renewal flow will likely not work and you'll need to copy certificates around manually. See below. ### Obtaining certificates diff --git a/docs/importing-postgres.md b/docs/importing-postgres.md index a88067e1..0dd75cb2 100644 --- a/docs/importing-postgres.md +++ b/docs/importing-postgres.md @@ -1,7 +1,7 @@ # Importing an existing Postgres database from another installation (optional) -Run this if you'd like to import your database from a previous installation of Synapse. -(don't forget to import your `media_store` files as well - see [the importing-media-store guide](importing-media-store.md)). +Run this if you'd like to import your database from a previous installation. +(don't forget to import your Synapse `media_store` files as well - see [the importing-synape-media-store guide](importing-synapse-media-store.md)). ## Prerequisites diff --git a/docs/importing-media-store.md b/docs/importing-synapse-media-store.md similarity index 92% rename from docs/importing-media-store.md rename to docs/importing-synapse-media-store.md index 0d86370b..0ba7bacb 100644 --- a/docs/importing-media-store.md +++ b/docs/importing-synapse-media-store.md @@ -1,4 +1,4 @@ -# Importing `media_store` data files from an existing installation (optional) +# Importing `media_store` data files from an existing Synapse installation (optional) Run this if you'd like to import your `media_store` files from a previous installation of Synapse. @@ -17,6 +17,6 @@ As an alternative, you can perform a manual restore using the [AWS CLI tool](htt Run this command (make sure to replace `` with a path on your server): - ansible-playbook -i inventory/hosts setup.yml --extra-vars='server_path_media_store=' --tags=import-media-store + ansible-playbook -i inventory/hosts setup.yml --extra-vars='server_path_media_store=' --tags=import-synapse-media-store **Note**: `` must be a file path to a `media_store` directory on the server (not on your local machine!). diff --git a/docs/importing-sqlite.md b/docs/importing-synapse-sqlite.md similarity index 86% rename from docs/importing-sqlite.md rename to docs/importing-synapse-sqlite.md index 9e3a910d..aade2226 100644 --- a/docs/importing-sqlite.md +++ b/docs/importing-synapse-sqlite.md @@ -1,7 +1,7 @@ -# Importing an existing SQLite database from another installation (optional) +# Importing an existing SQLite database from another Synapse installation (optional) Run this if you'd like to import your database from a previous default installation of Synapse. -(don't forget to import your `media_store` files as well - see [the importing-media-store guide](importing-media-store.md)). +(don't forget to import your `media_store` files as well - see [the importing-synapse-media-store guide](importing-synapse-media-store.md)). While this playbook always sets up PostgreSQL, by default a Synapse installation would run using an SQLite database. @@ -18,7 +18,7 @@ Before doing the actual import, **you need to upload your SQLite database file t Run this command (make sure to replace `` with a file path on your server): - ansible-playbook -i inventory/hosts setup.yml --extra-vars='server_path_homeserver_db=' --tags=import-sqlite-db + ansible-playbook -i inventory/hosts setup.yml --extra-vars='server_path_homeserver_db=' --tags=import-synapse-sqlite-db **Notes**: diff --git a/docs/installing.md b/docs/installing.md index 43758ffb..a2ce1371 100644 --- a/docs/installing.md +++ b/docs/installing.md @@ -21,11 +21,11 @@ Feel free to **re-run this setup command any time** you think something is off w After installing, but before starting the services, you may want to do additional things like: -- [Importing an existing SQLite database (from another installation)](importing-sqlite.md) (optional) +- [Importing an existing SQLite database (from another Synapse installation)](importing-synapse-sqlite.md) (optional) - [Importing an existing Postgres database (from another installation)](importing-postgres.md) (optional) -- [Importing `media_store` data files from an existing installation](importing-media-store.md) (optional) +- [Importing `media_store` data files from an existing Synapse installation](importing-synapse-media-store.md) (optional) ## Starting the services diff --git a/docs/maintenance-postgres.md b/docs/maintenance-postgres.md index 14d5cdb9..7c936479 100644 --- a/docs/maintenance-postgres.md +++ b/docs/maintenance-postgres.md @@ -45,7 +45,7 @@ docker run \ --log-driver=none \ --network=matrix \ --env-file=/matrix/postgres/env-postgres-psql \ -postgres:13.0-alpine \ +docker.io/postgres:13.1-alpine \ pg_dumpall -h matrix-postgres \ | gzip -c \ > /postgres.sql.gz @@ -69,7 +69,7 @@ This playbook can upgrade your existing Postgres setup with the following comman ansible-playbook -i inventory/hosts setup.yml --tags=upgrade-postgres -**The old Postgres data directory is backed up** automatically, by renaming it to `/matrix/postgres-auto-upgrade-backup`. +**The old Postgres data directory is backed up** automatically, by renaming it to `/matrix/postgres/data-auto-upgrade-backup`. To rename to a different path, pass some extra flags to the command above, like this: `--extra-vars="postgres_auto_upgrade_backup_data_path=/another/disk/matrix-postgres-before-upgrade"` The auto-upgrade-backup directory stays around forever, until you **manually decide to delete it**. diff --git a/docs/maintenance-synapse.md b/docs/maintenance-synapse.md index 79998299..143238c1 100644 --- a/docs/maintenance-synapse.md +++ b/docs/maintenance-synapse.md @@ -4,14 +4,11 @@ This document shows you how to perform various maintenance tasks related to the Table of contents: -- [Purging unused data with synapse-janitor](#purging-unused-data-with-synapse-janitor), for when you wish to delete unused data from the Synapse database - - [Purging old data with the Purge History API](#purging-old-data-with-the-purge-history-api), for when you wish to delete in-use (but old) data from the Synapse database - [Synapse maintenance](#synapse-maintenance) - [Purging old data with the Purge History API](#purging-old-data-with-the-purge-history-api) - [Compressing state with rust-synapse-compress-state](#compressing-state-with-rust-synapse-compress-state) - - [Purging unused data with synapse-janitor](#purging-unused-data-with-synapse-janitor) - [Browse and manipulate the database](#browse-and-manipulate-the-database) - [Browse and manipulate the database](#browse-and-manipulate-the-database), for when you really need to take matters into your own hands @@ -57,27 +54,6 @@ If you need to adjust this, pass: `--extra-vars='matrix_synapse_rust_synapse_com After state compression, you may wish to run a [`FULL` Postgres `VACUUM`](./maintenance-postgres.md#vacuuming-postgresql). -## Purging unused data with synapse-janitor - -**NOTE**: There are [reports](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/465) that **synapse-janitor is dangerous to use and causes database corruption**. You may wish to refrain from using it. - -When you **leave** and **forget** a room, Synapse can clean up its data, but currently doesn't. -This **unused and unreachable data** remains in your database forever. - -There are external tools (like [synapse-janitor](https://github.com/xwiki-labs/synapse_scripts)), which are meant to solve this problem. - -To ask the playbook to run synapse-janitor, execute: - -```bash -ansible-playbook -i inventory/hosts setup.yml --tags=run-postgres-synapse-janitor,start -``` - -**Note**: this will automatically stop Synapse temporarily and restart it later. - -Running synapse-janitor potentially deletes a lot of data from the Postgres database. -You may wish to run a [`FULL` Postgres `VACUUM`](./maintenance-postgres.md#vacuuming-postgresql) after that. - - ## Browse and manipulate the database When the [matrix admin API](https://github.com/matrix-org/synapse/tree/master/docs/admin_api) and the other tools do not provide a more convenient way, having a look at synapse's postgresql database can satisfy a lot of admins' needs. diff --git a/docs/prerequisites.md b/docs/prerequisites.md index 4356081b..488f6c6f 100644 --- a/docs/prerequisites.md +++ b/docs/prerequisites.md @@ -10,7 +10,7 @@ This playbook doesn't support running on ARM (see [this issue](https://github.co - `root` access to your server (or a user capable of elevating to `root` via `sudo`). -- [Python](https://www.python.org/) being installed on the server. Most distributions install Python by default, but some don't (e.g. Ubuntu 18.04) and require manual installation (something like `apt-get install python`). +- [Python](https://www.python.org/) being installed on the server. Most distributions install Python by default, but some don't (e.g. Ubuntu 18.04) and require manual installation (something like `apt-get install python3`). On some distros, Ansible may incorrectly [detect the Python version](https://docs.ansible.com/ansible/latest/reference_appendices/interpreter_discovery.html) (2 vs 3) and you may need to explicitly specify the interpreter path in `inventory/hosts` during installation (e.g. `ansible_python_interpreter=/usr/bin/python3`) - A `cron`-like tool installed on the server such as `cron` or `anacron` to automatically schedule the Let's Encrypt SSL certificates's renewal. *This can be ignored if you use your own SSL certificates.* @@ -22,6 +22,17 @@ This playbook doesn't support running on ARM (see [this issue](https://github.co - Properly configured DNS records for `` (details in [Configuring DNS](configuring-dns.md)). -- Some TCP/UDP ports open. This playbook configures the server's internal firewall for you. In most cases, you don't need to do anything special. But **if your server is running behind another firewall**, you'd need to open these ports: `80/tcp` (HTTP webserver), `443/tcp` (HTTPS webserver), `3478/tcp` (TURN over TCP), `3478/udp` (TURN over UDP), `5349/tcp` (TURN over TCP), `5349/udp` (TURN over UDP), `8448/tcp` (Matrix Federation API HTTPS webserver), the range `49152-49172/udp` (TURN over UDP), `4443/tcp` (Jitsi Harvester fallback), `10000/udp` (Jitsi video RTP). Depending on your firewall/NAT setup, incoming RTP packets on port 10000 may have the external IP of your firewall as destination address, due to the usage of STUN in JVB (see [`matrix_jitsi_jvb_stun_servers`](../roles/matrix-jitsi/defaults/main.yml)). +- Some TCP/UDP ports open. This playbook configures the server's internal firewall for you. In most cases, you don't need to do anything special. But **if your server is running behind another firewall**, you'd need to open these ports: + + - `80/tcp`: HTTP webserver + - `443/tcp`: HTTPS webserver + - `3478/tcp`: TURN over TCP (used by Coturn) + - `3478/udp`: TURN over UDP (used by Coturn) + - `5349/tcp`: TURN over TCP (used by Coturn) + - `5349/udp`: TURN over UDP (used by Coturn) + - `8448/tcp`: Matrix Federation API HTTPS webserver. In some cases, this **may necessary even with federation disabled**. Integration Servers (like Dimension) and Identity Servers (like ma1sd) may need to access `openid` APIs on the federation port. + - the range `49152-49172/udp`: TURN over UDP + - `4443/tcp`: Jitsi Harvester fallback + - `10000/udp`: Jitsi video RTP. Depending on your firewall/NAT setup, incoming RTP packets on port `10000` may have the external IP of your firewall as destination address, due to the usage of STUN in JVB (see [`matrix_jitsi_jvb_stun_servers`](../roles/matrix-jitsi/defaults/main.yml)). When ready to proceed, continue with [Configuring DNS](configuring-dns.md). diff --git a/docs/self-building.md b/docs/self-building.md index 169e4aa5..fa4db222 100644 --- a/docs/self-building.md +++ b/docs/self-building.md @@ -15,6 +15,7 @@ List of roles where self-building the Docker image is currently possible: - `matrix-client-element` - `matrix-registration` - `matrix-coturn` +- `matrix-corporal` - `matrix-ma1sd` - `matrix-mailer` - `matrix-bridge-mautrix-facebook` diff --git a/docs/updating-users-passwords.md b/docs/updating-users-passwords.md index 785bc17c..7d2f2832 100644 --- a/docs/updating-users-passwords.md +++ b/docs/updating-users-passwords.md @@ -26,7 +26,7 @@ and then connecting to the postgres server and executing: ``` UPDATE users SET password_hash = '' WHERE name = '@someone:server.com' ``` -` + where `` is the hash returned by the docker command above. diff --git a/examples/caddy2/Caddyfile b/examples/caddy2/Caddyfile index 1438a50b..d13b9788 100644 --- a/examples/caddy2/Caddyfile +++ b/examples/caddy2/Caddyfile @@ -1,4 +1,8 @@ matrix.DOMAIN.tld { + + # creates letsencrypt certificate + # tls your@email.com + @identity { path /_matrix/identity/* } @@ -93,35 +97,12 @@ matrix.DOMAIN.tld:8448 { } } -dimension.DOMAIN.tld { -header { - # Enable HTTP Strict Transport Security (HSTS) to force clients to always connect via HTTPS - Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" - # Enable cross-site filter (XSS) and tell browser to block detected attacks - X-XSS-Protection "1; mode=block" - # Prevent some browsers from MIME-sniffing a response away from the declared Content-Type - X-Content-Type-Options "nosniff" - # Disallow the site to be rendered within a frame (clickjacking protection) - X-Frame-Options "DENY" - # X-Robots-Tag - X-Robots-Tag "noindex, noarchive, nofollow" - } - - handle { - encode zstd gzip - - reverse_proxy localhost:8184 { - header_up X-Forwarded-Port {http.request.port} - header_up X-Forwarded-Proto {http.request.scheme} - header_up X-Forwarded-TlsProto {tls_protocol} - header_up X-Forwarded-TlsCipher {tls_cipher} - header_up X-Forwarded-HttpsProto {proto} - } - } -} - element.DOMAIN.tld { - header { + + # creates letsencrypt certificate + # tls your@email.com + + header { # Enable HTTP Strict Transport Security (HSTS) to force clients to always connect via HTTPS Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" # Enable cross-site filter (XSS) and tell browser to block detected attacks @@ -145,3 +126,79 @@ element.DOMAIN.tld { header_up X-Forwarded-HttpsProto {proto} } } + +#dimension.DOMAIN.tld { +# +# # creates letsencrypt certificate +# # tls your@email.com +# +# header { +# # Enable HTTP Strict Transport Security (HSTS) to force clients to always connect via HTTPS +# Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" +# # Enable cross-site filter (XSS) and tell browser to block detected attacks +# X-XSS-Protection "1; mode=block" +# # Prevent some browsers from MIME-sniffing a response away from the declared Content-Type +# X-Content-Type-Options "nosniff" +# # Disallow the site to be rendered within a frame (clickjacking protection) +# X-Frame-Options "DENY" +# # X-Robots-Tag +# X-Robots-Tag "noindex, noarchive, nofollow" +# } +# +# handle { +# encode zstd gzip +# +# reverse_proxy localhost:8184 { +# header_up X-Forwarded-Port {http.request.port} +# header_up X-Forwarded-Proto {http.request.scheme} +# header_up X-Forwarded-TlsProto {tls_protocol} +# header_up X-Forwarded-TlsCipher {tls_cipher} +# header_up X-Forwarded-HttpsProto {proto} +# } +# } +#} + + +#jitsi.DOMAIN.tld { +# +# creates letsencrypt certificate +# tls your@email.com +# +# header { +# # Enable HTTP Strict Transport Security (HSTS) to force clients to always connect via HTTPS +# Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" +# +# # Enable cross-site filter (XSS) and tell browser to block detected attacks +# X-XSS-Protection "1; mode=block" +# +# # Prevent some browsers from MIME-sniffing a response away from the declared Content-Type +# X-Content-Type-Options "nosniff" +# +# # Disallow the site to be rendered within a frame (clickjacking protection) +# X-Frame-Options "SAMEORIGIN" +# +# # Disable some features +# Feature-Policy "accelerometer 'none';ambient-light-sensor 'none'; autoplay 'none';camera 'none';encrypted-media 'none';focus-without-user-activation 'none'; geolocation 'none';gyroscope #'none';magnetometer 'none';microphone 'none';midi 'none';payment 'none';picture-in-picture 'none'; speaker 'none';sync-xhr 'none';usb 'none';vr 'none'" +# +# # Referer +# Referrer-Policy "no-referrer" +# +# # X-Robots-Tag +# X-Robots-Tag "none" +# +# # Remove Server header +# -Server +# } +# +# handle { +# encode zstd gzip +# +# reverse_proxy 127.0.0.1:12080 { +# header_up X-Forwarded-Port {http.request.port} +# header_up X-Forwarded-Proto {http.request.scheme} +# header_up X-Forwarded-TlsProto {tls_protocol} +# header_up X-Forwarded-TlsCipher {tls_cipher} +# header_up X-Forwarded-HttpsProto {proto} +# } +# } +#} \ No newline at end of file diff --git a/examples/caddy2/README.md b/examples/caddy2/README.md new file mode 100644 index 00000000..022a4fec --- /dev/null +++ b/examples/caddy2/README.md @@ -0,0 +1,12 @@ +# Caddyfile + +This directory contains sample files that show you how to do reverse-proxying using Caddy2. + +## Config + +| Variable | Function | +| ------------------ | -------- | +| tls your@email.com | Specify an email address for your [ACME account](https://caddyserver.com/docs/caddyfile/directives/tls) (but if only one email is used for all sites, we recommend the email [global option](https://caddyserver.com/docs/caddyfile/options) instead) | +| tls | To enable [tls](https://caddyserver.com/docs/caddyfile/directives/tls) support uncomment the lines for tls | +| Dimnension | To enable Dimension support uncomment the lines for Dimension and set your data | +| Jitsi | To enable Jitsi support uncomment the lines for Jitsi and set your data | \ No newline at end of file diff --git a/examples/hosts b/examples/hosts index 3b14f09e..daf2cfc5 100644 --- a/examples/hosts +++ b/examples/hosts @@ -10,6 +10,9 @@ # # If you're running this Ansible playbook on the same server as the one you're installing to, # consider adding an additional `ansible_connection=local` argument below. +# +# Ansible may fail to discover which Python interpreter to use on the host for some distros (like Ubuntu 20.04). +# You may sometimes need to explicitly add `ansible_python_interpreter=/usr/bin/python3` to lines below. [matrix_servers] matrix. ansible_host= ansible_ssh_user=root diff --git a/group_vars/matrix_servers b/group_vars/matrix_servers index ae98df85..76190bf7 100755 --- a/group_vars/matrix_servers +++ b/group_vars/matrix_servers @@ -24,20 +24,6 @@ matrix_identity_server_url: "{{ ('https://' + matrix_server_fqn_matrix) if matri # ###################################################################### -###################################################################### -# -# matrix-architecture -# -###################################################################### - -matrix_architecture: "amd64" - -###################################################################### -# -# /matrix-architecture -# -###################################################################### - ###################################################################### # @@ -61,12 +47,19 @@ matrix_appservice_discord_systemd_required_services_list: | ['docker.service'] + (['matrix-synapse.service'] if matrix_synapse_enabled else []) + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) }} matrix_appservice_discord_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'discord.as.token') | to_uuid }}" matrix_appservice_discord_homeserver_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'discord.hs.token') | to_uuid }}" +# We only make this use Postgres if our own Postgres server is enabled. +# It's only then (for now) that we can automatically create the necessary database and user for this service. +matrix_appservice_discord_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_appservice_discord_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'as.discord.db') | to_uuid }}" + ###################################################################### # # /matrix-bridge-appservice-discord @@ -135,6 +128,10 @@ matrix_appservice_slack_systemd_required_services_list: | (['matrix-synapse.service'] if matrix_synapse_enabled else []) }} +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_appservice_slack_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'nedb' }}" +matrix_appservice_slack_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'as.slack.db') | to_uuid }}" + ###################################################################### # # /matrix-bridge-appservice-slack @@ -170,6 +167,10 @@ matrix_appservice_irc_appservice_token: "{{ matrix_synapse_macaroon_secret_key | matrix_appservice_irc_homeserver_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'irc.hs.token') | to_uuid }}" +matrix_appservice_irc_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'nedb' }}" +matrix_appservice_irc_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'as.irc.db') | to_uuid }}" + + ###################################################################### # # /matrix-bridge-appservice-irc @@ -193,6 +194,8 @@ matrix_mautrix_facebook_systemd_required_services_list: | ['docker.service'] + (['matrix-synapse.service'] if matrix_synapse_enabled else []) + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) }} matrix_mautrix_facebook_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'fb.as.token') | to_uuid }}" @@ -203,6 +206,10 @@ matrix_mautrix_facebook_login_shared_secret: "{{ matrix_synapse_ext_password_pro matrix_mautrix_facebook_bridge_presence: "{{ matrix_synapse_use_presence if matrix_synapse_enabled else true }}" +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_mautrix_facebook_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_mautrix_facebook_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mau.fb.db') | to_uuid }}" + ###################################################################### # # /matrix-bridge-mautrix-facebook @@ -226,6 +233,8 @@ matrix_mautrix_hangouts_systemd_required_services_list: | ['docker.service'] + (['matrix-synapse.service'] if matrix_synapse_enabled else []) + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) }} matrix_mautrix_hangouts_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'ho.as.token') | to_uuid }}" @@ -236,6 +245,10 @@ matrix_mautrix_hangouts_container_http_host_bind_port: "{{ '' if matrix_nginx_pr matrix_mautrix_hangouts_login_shared_secret: "{{ matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret if matrix_synapse_ext_password_provider_shared_secret_auth_enabled else '' }}" +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_mautrix_hangouts_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_mautrix_hangouts_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mau.hangouts.db') | to_uuid }}" + ###################################################################### # # /matrix-bridge-mautrix-hangouts @@ -243,7 +256,44 @@ matrix_mautrix_hangouts_login_shared_secret: "{{ matrix_synapse_ext_password_pro ###################################################################### +###################################################################### +# +# matrix-bridge-mautrix-signal +# +###################################################################### + +# We don't enable bridges by default. +matrix_mautrix_signal_enabled: false + +matrix_mautrix_signal_systemd_required_services_list: | + {{ + ['docker.service'] + + + (['matrix-synapse.service'] if matrix_synapse_enabled else []) + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) + + + ['matrix-mautrix-signal-daemon.service'] + }} + +matrix_mautrix_signal_homeserver_domain: '{{ matrix_domain }}' + +matrix_mautrix_signal_homeserver_address: "{{ 'http://matrix-synapse:8008' if matrix_synapse_enabled else '' }}" +matrix_mautrix_signal_homeserver_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'si.hs.token') | to_uuid }}" + +matrix_mautrix_signal_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'si.as.token') | to_uuid }}" + +matrix_mautrix_signal_login_shared_secret: "{{ matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret if matrix_synapse_ext_password_provider_shared_secret_auth_enabled else '' }}" + +matrix_mautrix_signal_database_engine: 'postgres' +matrix_mautrix_signal_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mau.signal.db') | to_uuid }}" + +###################################################################### +# +# /matrix-bridge-mautrix-signal +# +###################################################################### ###################################################################### @@ -260,6 +310,8 @@ matrix_mautrix_telegram_systemd_required_services_list: | ['docker.service'] + (['matrix-synapse.service'] if matrix_synapse_enabled else []) + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) }} matrix_mautrix_telegram_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'telegr.as.token') | to_uuid }}" @@ -272,6 +324,10 @@ matrix_mautrix_telegram_container_http_host_bind_port: "{{ '' if matrix_nginx_pr matrix_mautrix_telegram_login_shared_secret: "{{ matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret if matrix_synapse_ext_password_provider_shared_secret_auth_enabled else '' }}" +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_mautrix_telegram_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_mautrix_telegram_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mau.telegram.db') | to_uuid }}" + ###################################################################### # # /matrix-bridge-mautrix-telegram @@ -292,6 +348,8 @@ matrix_mautrix_whatsapp_systemd_required_services_list: | ['docker.service'] + (['matrix-synapse.service'] if matrix_synapse_enabled else []) + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) }} matrix_mautrix_whatsapp_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'whats.as.token') | to_uuid }}" @@ -300,6 +358,10 @@ matrix_mautrix_whatsapp_homeserver_token: "{{ matrix_synapse_macaroon_secret_key matrix_mautrix_whatsapp_login_shared_secret: "{{ matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret if matrix_synapse_ext_password_provider_shared_secret_auth_enabled else '' }}" +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_mautrix_whatsapp_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_mautrix_whatsapp_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mauwhatsapp.db') | to_uuid }}" + ###################################################################### # # /matrix-bridge-mautrix-whatsapp @@ -348,6 +410,8 @@ matrix_mx_puppet_skype_systemd_required_services_list: | ['docker.service'] + (['matrix-synapse.service'] if matrix_synapse_enabled else []) + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) }} matrix_mx_puppet_skype_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'skype.as.tok') | to_uuid }}" @@ -356,6 +420,10 @@ matrix_mx_puppet_skype_homeserver_token: "{{ matrix_synapse_macaroon_secret_key matrix_mx_puppet_skype_login_shared_secret: "{{ matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret if matrix_synapse_ext_password_provider_shared_secret_auth_enabled else '' }}" +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_mx_puppet_skype_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_mx_puppet_skype_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mxpup.skype.db') | to_uuid }}" + ###################################################################### # # /matrix-bridge-mx-puppet-skype @@ -379,6 +447,8 @@ matrix_mx_puppet_slack_systemd_required_services_list: | ['docker.service'] + (['matrix-synapse.service'] if matrix_synapse_enabled else []) + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) }} matrix_mx_puppet_slack_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mxslk.as.tok') | to_uuid }}" @@ -387,6 +457,10 @@ matrix_mx_puppet_slack_homeserver_token: "{{ matrix_synapse_macaroon_secret_key matrix_mx_puppet_slack_login_shared_secret: "{{ matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret if matrix_synapse_ext_password_provider_shared_secret_auth_enabled else '' }}" +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_mx_puppet_slack_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_mx_puppet_slack_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mxpup.slack.db') | to_uuid }}" + ###################################################################### # # /matrix-bridge-mx-puppet-slack @@ -409,6 +483,8 @@ matrix_mx_puppet_twitter_systemd_required_services_list: | ['docker.service'] + (['matrix-synapse.service'] if matrix_synapse_enabled else []) + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) }} matrix_mx_puppet_twitter_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mxtwt.as.tok') | to_uuid }}" @@ -419,6 +495,10 @@ matrix_mx_puppet_twitter_login_shared_secret: "{{ matrix_synapse_ext_password_pr matrix_mx_puppet_twitter_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_enabled else ('127.0.0.1:' ~ matrix_mx_puppet_twitter_appservice_port) }}" +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_mx_puppet_twitter_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_mx_puppet_twitter_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mxpup.twitter.db') | to_uuid }}" + ###################################################################### # # /matrix-bridge-mx-puppet-twitter @@ -442,6 +522,8 @@ matrix_mx_puppet_instagram_systemd_required_services_list: | ['docker.service'] + (['matrix-synapse.service'] if matrix_synapse_enabled else []) + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) }} matrix_mx_puppet_instagram_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mxig.as.tok') | to_uuid }}" @@ -450,6 +532,10 @@ matrix_mx_puppet_instagram_homeserver_token: "{{ matrix_synapse_macaroon_secret_ matrix_mx_puppet_instagram_login_shared_secret: "{{ matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret if matrix_synapse_ext_password_provider_shared_secret_auth_enabled else '' }}" +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_mx_puppet_instagram_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_mx_puppet_instagram_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mxpup.ig.db') | to_uuid }}" + ###################################################################### # # /matrix-bridge-mx-puppet-instagram @@ -472,6 +558,8 @@ matrix_mx_puppet_discord_systemd_required_services_list: | ['docker.service'] + (['matrix-synapse.service'] if matrix_synapse_enabled else []) + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) }} matrix_mx_puppet_discord_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mxdsc.as.tok') | to_uuid }}" @@ -480,6 +568,10 @@ matrix_mx_puppet_discord_homeserver_token: "{{ matrix_synapse_macaroon_secret_ke matrix_mx_puppet_discord_login_shared_secret: "{{ matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret if matrix_synapse_ext_password_provider_shared_secret_auth_enabled else '' }}" +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_mx_puppet_discord_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_mx_puppet_discord_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mxpup.dsc.db') | to_uuid }}" + ###################################################################### # # /matrix-bridge-mx-puppet-discord @@ -502,6 +594,8 @@ matrix_mx_puppet_steam_systemd_required_services_list: | ['docker.service'] + (['matrix-synapse.service'] if matrix_synapse_enabled else []) + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) }} matrix_mx_puppet_steam_appservice_token: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mxste.as.tok') | to_uuid }}" @@ -510,6 +604,10 @@ matrix_mx_puppet_steam_homeserver_token: "{{ matrix_synapse_macaroon_secret_key matrix_mx_puppet_steam_login_shared_secret: "{{ matrix_synapse_ext_password_provider_shared_secret_auth_shared_secret if matrix_synapse_ext_password_provider_shared_secret_auth_enabled else '' }}" +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_mx_puppet_steam_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_mx_puppet_steam_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mxpup.steam.db') | to_uuid }}" + ###################################################################### # # /matrix-bridge-mx-puppet-steam @@ -526,6 +624,17 @@ matrix_mx_puppet_steam_login_shared_secret: "{{ matrix_synapse_ext_password_prov # We don't enable bots by default. matrix_bot_matrix_reminder_bot_enabled: false +matrix_bot_matrix_reminder_bot_systemd_required_services_list: | + {{ + ['docker.service'] + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) + }} + +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_bot_matrix_reminder_bot_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_bot_matrix_reminder_bot_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'reminder.bot.db') | to_uuid }}" + ###################################################################### # # /matrix-bot-matrix-reminder-bot @@ -541,6 +650,8 @@ matrix_bot_matrix_reminder_bot_enabled: false matrix_corporal_enabled: false +matrix_corporal_container_image_self_build: "{{ matrix_architecture != 'amd64' }}" + # Normally, matrix-nginx-proxy is enabled and nginx can reach matrix-corporal over the container network. # If matrix-nginx-proxy is not enabled, or you otherwise have a need for it, you can expose # matrix-corporal's web-server ports to the local host. @@ -618,7 +729,16 @@ matrix_dimension_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_ena matrix_integration_manager_rest_url: "{{ matrix_dimension_integrations_rest_url if matrix_dimension_enabled else None }}" matrix_integration_manager_ui_url: "{{ matrix_dimension_integrations_ui_url if matrix_dimension_enabled else None }}" -matrix_dimension_homeserver_federationUrl: "http://matrix-synapse:{{ 8048 if matrix_synapse_federation_enabled|bool else 8008 }}" +matrix_dimension_systemd_required_services_list: | + {{ + ['docker.service'] + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) + }} + +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_dimension_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_dimension_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'dimension.db') | to_uuid }}" ###################################################################### # @@ -764,6 +884,11 @@ matrix_ma1sd_threepid_medium_email_connectors_smtp_tls: 0 matrix_ma1sd_self_check_validate_certificates: "{{ false if matrix_ssl_retrieval_method == 'self-signed' else true }}" +matrix_ma1sd_systemd_required_services_list: | + {{ + (['matrix-postgres.service'] if matrix_postgres_enabled else []) + }} + matrix_ma1sd_systemd_wanted_services_list: | {{ (['matrix-corporal.service'] if matrix_corporal_enabled else ['matrix-synapse.service']) @@ -773,6 +898,10 @@ matrix_ma1sd_systemd_wanted_services_list: | (['matrix-mailer.service'] if matrix_mailer_enabled else []) }} +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_ma1sd_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_ma1sd_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'ma1sd.db') | to_uuid }}" + ###################################################################### # # /matrix-ma1sd @@ -814,7 +943,8 @@ matrix_nginx_proxy_proxy_matrix_identity_api_addr_with_container: "matrix-ma1sd: matrix_nginx_proxy_proxy_matrix_identity_api_addr_sans_container: "127.0.0.1:8090" # By default, we do TLS termination for the Matrix Federation API (port 8448) at matrix-nginx-proxy. -matrix_nginx_proxy_proxy_matrix_federation_api_enabled: true +# Unless this is handled there OR Synapse's federation listener port is disabled, we'll reverse-proxy. +matrix_nginx_proxy_proxy_matrix_federation_api_enabled: "{{ matrix_synapse_federation_port_enabled and not matrix_synapse_tls_federation_listener_enabled }}" matrix_nginx_proxy_proxy_matrix_federation_api_addr_with_container: "matrix-synapse:8048" matrix_nginx_proxy_proxy_matrix_federation_api_addr_sans_container: "127.0.0.1:8048" @@ -897,6 +1027,133 @@ matrix_postgres_connection_username: "synapse" matrix_postgres_connection_password: "synapse-password" matrix_postgres_db_name: "homeserver" +matrix_postgres_pgloader_container_image_self_build: "{{ matrix_architecture != 'amd64' }}" + +matrix_postgres_additional_databases: | + {{ + ([{ + 'name': matrix_ma1sd_database_name, + 'username': matrix_ma1sd_database_username, + 'password': matrix_ma1sd_database_password, + }] if (matrix_ma1sd_enabled and matrix_ma1sd_database_engine == 'postgres' and matrix_ma1sd_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_bot_matrix_reminder_bot_database_name, + 'username': matrix_bot_matrix_reminder_bot_database_username, + 'password': matrix_bot_matrix_reminder_bot_database_password, + }] if (matrix_bot_matrix_reminder_bot_enabled and matrix_bot_matrix_reminder_bot_database_engine == 'postgres' and matrix_bot_matrix_reminder_bot_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_registration_database_name, + 'username': matrix_registration_database_username, + 'password': matrix_registration_database_password, + }] if (matrix_registration_enabled and matrix_registration_database_engine == 'postgres' and matrix_registration_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_appservice_discord_database_name, + 'username': matrix_appservice_discord_database_username, + 'password': matrix_appservice_discord_database_password, + }] if (matrix_appservice_discord_enabled and matrix_appservice_discord_database_engine == 'postgres' and matrix_appservice_discord_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_appservice_slack_database_name, + 'username': matrix_appservice_slack_database_username, + 'password': matrix_appservice_slack_database_password, + }] if (matrix_appservice_slack_enabled and matrix_appservice_slack_database_engine == 'postgres' and matrix_appservice_slack_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_appservice_irc_database_name, + 'username': matrix_appservice_irc_database_username, + 'password': matrix_appservice_irc_database_password, + }] if (matrix_appservice_irc_enabled and matrix_appservice_irc_database_engine == 'postgres' and matrix_appservice_irc_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_mautrix_facebook_database_name, + 'username': matrix_mautrix_facebook_database_username, + 'password': matrix_mautrix_facebook_database_password, + }] if (matrix_mautrix_facebook_enabled and matrix_mautrix_facebook_database_engine == 'postgres' and matrix_mautrix_facebook_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_mautrix_hangouts_database_name, + 'username': matrix_mautrix_hangouts_database_username, + 'password': matrix_mautrix_hangouts_database_password, + }] if (matrix_mautrix_hangouts_enabled and matrix_mautrix_hangouts_database_engine == 'postgres' and matrix_mautrix_hangouts_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_mautrix_signal_database_name, + 'username': matrix_mautrix_signal_database_username, + 'password': matrix_mautrix_signal_database_password, + }] if (matrix_mautrix_signal_enabled and matrix_mautrix_signal_database_engine == 'postgres' and matrix_mautrix_signal_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_mautrix_telegram_database_name, + 'username': matrix_mautrix_telegram_database_username, + 'password': matrix_mautrix_telegram_database_password, + }] if (matrix_mautrix_telegram_enabled and matrix_mautrix_telegram_database_engine == 'postgres' and matrix_mautrix_telegram_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_mautrix_whatsapp_database_name, + 'username': matrix_mautrix_whatsapp_database_username, + 'password': matrix_mautrix_whatsapp_database_password, + }] if (matrix_mautrix_whatsapp_enabled and matrix_mautrix_whatsapp_database_engine == 'postgres' and matrix_mautrix_whatsapp_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_mx_puppet_skype_database_name, + 'username': matrix_mx_puppet_skype_database_username, + 'password': matrix_mx_puppet_skype_database_password, + }] if (matrix_mx_puppet_skype_enabled and matrix_mx_puppet_skype_database_engine == 'postgres' and matrix_mx_puppet_skype_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_mx_puppet_slack_database_name, + 'username': matrix_mx_puppet_slack_database_username, + 'password': matrix_mx_puppet_slack_database_password, + }] if (matrix_mx_puppet_slack_enabled and matrix_mx_puppet_slack_database_engine == 'postgres' and matrix_mx_puppet_slack_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_mx_puppet_twitter_database_name, + 'username': matrix_mx_puppet_twitter_database_username, + 'password': matrix_mx_puppet_twitter_database_password, + }] if (matrix_mx_puppet_twitter_enabled and matrix_mx_puppet_twitter_database_engine == 'postgres' and matrix_mx_puppet_twitter_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_mx_puppet_instagram_database_name, + 'username': matrix_mx_puppet_instagram_database_username, + 'password': matrix_mx_puppet_instagram_database_password, + }] if (matrix_mx_puppet_instagram_enabled and matrix_mx_puppet_instagram_database_engine == 'postgres' and matrix_mx_puppet_instagram_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_mx_puppet_discord_database_name, + 'username': matrix_mx_puppet_discord_database_username, + 'password': matrix_mx_puppet_discord_database_password, + }] if (matrix_mx_puppet_discord_enabled and matrix_mx_puppet_discord_database_engine == 'postgres' and matrix_mx_puppet_discord_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_mx_puppet_steam_database_name, + 'username': matrix_mx_puppet_steam_database_username, + 'password': matrix_mx_puppet_steam_database_password, + }] if (matrix_mx_puppet_steam_enabled and matrix_mx_puppet_steam_database_engine == 'postgres' and matrix_mx_puppet_steam_database_hostname == 'matrix-postgres') else []) + + + ([{ + 'name': matrix_dimension_database_name, + 'username': matrix_dimension_database_username, + 'password': matrix_dimension_database_password, + }] if (matrix_dimension_enabled and matrix_dimension_database_engine == 'postgres' and matrix_dimension_database_hostname == 'matrix-postgres') else []) + }} + +matrix_postgres_import_roles_to_ignore: | + {{ + [matrix_postgres_connection_username] + + + matrix_postgres_additional_databases|map(attribute='username')|list + }} + +matrix_postgres_import_databases_to_ignore: | + {{ + [matrix_postgres_db_name] + + + matrix_postgres_additional_databases|map(attribute='name')|list + }} + ###################################################################### # # /matrix-postgres @@ -1013,12 +1270,7 @@ matrix_synapse_tls_federation_listener_enabled: false matrix_synapse_tls_certificate_path: ~ matrix_synapse_tls_private_key_path: ~ -matrix_synapse_http_listener_resource_names: | - {{ - ["client"] - + - ( ["openid"] if matrix_dimension_enabled and not matrix_synapse_federation_enabled else [] ) - }} +matrix_synapse_federation_port_openid_resource_required: "{{ not matrix_synapse_federation_enabled and (matrix_dimension_enabled or matrix_ma1sd_enabled) }}" matrix_synapse_email_enabled: "{{ matrix_mailer_enabled }}" matrix_synapse_email_smtp_host: "matrix-mailer" @@ -1119,6 +1371,17 @@ matrix_registration_api_validate_certs: "{{ false if matrix_ssl_retrieval_method matrix_registration_container_image_self_build: "{{ matrix_architecture != 'amd64' }}" +matrix_registration_systemd_required_services_list: | + {{ + ['docker.service'] + + + (['matrix-postgres.service'] if matrix_postgres_enabled else []) + }} + +# Postgres is the default, except if not using `matrix_postgres` (internal postgres) +matrix_registration_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" +matrix_registration_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mx.registr.db') | to_uuid }}" + ###################################################################### # # /matrix-registration diff --git a/roles/matrix-base/defaults/main.yml b/roles/matrix-base/defaults/main.yml index 164509b7..2cc4b42f 100644 --- a/roles/matrix-base/defaults/main.yml +++ b/roles/matrix-base/defaults/main.yml @@ -23,6 +23,17 @@ matrix_server_fqn_jitsi: "jitsi.{{ matrix_domain }}" matrix_federation_public_port: 8448 +# The architecture that your server runs. +# Recognized values by us are 'amd64', 'arm32' and 'arm64'. +# Not all architectures support all services, so your experience (on non-amd64) may vary. +# See docs/alternative-architectures.md +matrix_architecture: amd64 + +# The architecture for Debian packages. +# See: https://wiki.debian.org/SupportedArchitectures +# We just remap from our `matrix_architecture` values to what Debian and possibly other distros call things. +matrix_debian_arch: "{{ 'armhf' if matrix_architecture == 'arm32' else matrix_architecture }}" + matrix_user_username: "matrix" matrix_user_groupname: "matrix" @@ -106,7 +117,6 @@ matrix_docker_package_name: docker-ce run_postgres_import: true run_postgres_upgrade: true run_postgres_import_sqlite_db: true -run_postgres_synapse_janitor: true run_postgres_vacuum: true run_synapse_register_user: true run_synapse_update_user_password: true diff --git a/roles/matrix-base/files/yum.repos.d/docker-ce.repo b/roles/matrix-base/files/yum.repos.d/docker-ce.repo index 56242d98..1abdbe36 100644 --- a/roles/matrix-base/files/yum.repos.d/docker-ce.repo +++ b/roles/matrix-base/files/yum.repos.d/docker-ce.repo @@ -1,62 +1,62 @@ [docker-ce-stable] name=Docker CE Stable - $basearch -baseurl=https://download.docker.com/linux/centos/7/$basearch/stable +baseurl=https://download.docker.com/linux/centos/$releasever/$basearch/stable enabled=1 gpgcheck=1 gpgkey=https://download.docker.com/linux/centos/gpg [docker-ce-stable-debuginfo] name=Docker CE Stable - Debuginfo $basearch -baseurl=https://download.docker.com/linux/centos/7/debug-$basearch/stable +baseurl=https://download.docker.com/linux/centos/$releasever/debug-$basearch/stable enabled=0 gpgcheck=1 gpgkey=https://download.docker.com/linux/centos/gpg [docker-ce-stable-source] name=Docker CE Stable - Sources -baseurl=https://download.docker.com/linux/centos/7/source/stable +baseurl=https://download.docker.com/linux/centos/$releasever/source/stable enabled=0 gpgcheck=1 gpgkey=https://download.docker.com/linux/centos/gpg -[docker-ce-edge] -name=Docker CE Edge - $basearch -baseurl=https://download.docker.com/linux/centos/7/$basearch/edge +[docker-ce-test] +name=Docker CE Test - $basearch +baseurl=https://download.docker.com/linux/centos/$releasever/$basearch/test enabled=0 gpgcheck=1 gpgkey=https://download.docker.com/linux/centos/gpg -[docker-ce-edge-debuginfo] -name=Docker CE Edge - Debuginfo $basearch -baseurl=https://download.docker.com/linux/centos/7/debug-$basearch/edge +[docker-ce-test-debuginfo] +name=Docker CE Test - Debuginfo $basearch +baseurl=https://download.docker.com/linux/centos/$releasever/debug-$basearch/test enabled=0 gpgcheck=1 gpgkey=https://download.docker.com/linux/centos/gpg -[docker-ce-edge-source] -name=Docker CE Edge - Sources -baseurl=https://download.docker.com/linux/centos/7/source/edge +[docker-ce-test-source] +name=Docker CE Test - Sources +baseurl=https://download.docker.com/linux/centos/$releasever/source/test enabled=0 gpgcheck=1 gpgkey=https://download.docker.com/linux/centos/gpg -[docker-ce-test] -name=Docker CE Test - $basearch -baseurl=https://download.docker.com/linux/centos/7/$basearch/test +[docker-ce-nightly] +name=Docker CE Nightly - $basearch +baseurl=https://download.docker.com/linux/centos/$releasever/$basearch/nightly enabled=0 gpgcheck=1 gpgkey=https://download.docker.com/linux/centos/gpg -[docker-ce-test-debuginfo] -name=Docker CE Test - Debuginfo $basearch -baseurl=https://download.docker.com/linux/centos/7/debug-$basearch/test +[docker-ce-nightly-debuginfo] +name=Docker CE Nightly - Debuginfo $basearch +baseurl=https://download.docker.com/linux/centos/$releasever/debug-$basearch/nightly enabled=0 gpgcheck=1 gpgkey=https://download.docker.com/linux/centos/gpg -[docker-ce-test-source] -name=Docker CE Test - Sources -baseurl=https://download.docker.com/linux/centos/7/source/test +[docker-ce-nightly-source] +name=Docker CE Nightly - Sources +baseurl=https://download.docker.com/linux/centos/$releasever/source/nightly enabled=0 gpgcheck=1 gpgkey=https://download.docker.com/linux/centos/gpg diff --git a/roles/matrix-base/tasks/sanity_check.yml b/roles/matrix-base/tasks/sanity_check.yml index e504bfe9..c31d9137 100644 --- a/roles/matrix-base/tasks/sanity_check.yml +++ b/roles/matrix-base/tasks/sanity_check.yml @@ -1,10 +1,19 @@ --- -# We generally support Ansible 2.7.0 and above. -- name: Fail if running on Ansible < 2.7 +# We generally support Ansible 2.7.1 and above. +- name: Fail if running on Ansible < 2.7.1 fail: msg: "You are running on Ansible {{ ansible_version.string }}, which is not supported. See our guide about Ansible: https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/docs/ansible.md" - when: "(ansible_version.major < 2) or (ansible_version.major <= 2 and ansible_version.minor < 7)" + when: + - "(ansible_version.major < 2) or (ansible_version.major == 2 and ansible_version.minor < 7) or (ansible_version.major == 2 and ansible_version.minor == 7 and ansible_version.revision < 1)" + +# Though we do not support Ansible 2.9.6 which is buggy +- name: Fail if running on Ansible 2.9.6 on Ubuntu + fail: + msg: "You are running on Ansible {{ ansible_version.string }}, which is not supported. See our guide about Ansible: https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/docs/ansible.md" + when: + - ansible_distribution == 'Ubuntu' + - "ansible_version.major == 2 and ansible_version.minor == 9 and ansible_version.revision == 6" - name: (Deprecation) Catch and report renamed settings fail: diff --git a/roles/matrix-base/tasks/server_base/setup_debian.yml b/roles/matrix-base/tasks/server_base/setup_debian.yml index 6d8d18fa..37706d1f 100644 --- a/roles/matrix-base/tasks/server_base/setup_debian.yml +++ b/roles/matrix-base/tasks/server_base/setup_debian.yml @@ -11,7 +11,7 @@ - name: Ensure Docker's APT key is trusted apt_key: - url: https://download.docker.com/linux/ubuntu/gpg + url: "https://download.docker.com/linux/{{ ansible_distribution|lower }}/gpg" id: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88 state: present register: add_repository_key @@ -20,7 +20,7 @@ - name: Ensure Docker repository is enabled apt_repository: - repo: "deb [arch=amd64] https://download.docker.com/linux/{{ ansible_distribution|lower }} {{ ansible_distribution_release }} stable" + repo: "deb [arch={{ matrix_debian_arch }}] https://download.docker.com/linux/{{ ansible_distribution|lower }} {{ ansible_distribution_release }} stable" state: present update_cache: yes when: matrix_docker_installation_enabled|bool and matrix_docker_package_name == 'docker-ce' diff --git a/roles/matrix-base/tasks/server_base/setup_raspbian.yml b/roles/matrix-base/tasks/server_base/setup_raspbian.yml index a9a5d20a..421905a0 100644 --- a/roles/matrix-base/tasks/server_base/setup_raspbian.yml +++ b/roles/matrix-base/tasks/server_base/setup_raspbian.yml @@ -5,6 +5,7 @@ name: - apt-transport-https - ca-certificates + - gnupg state: present update_cache: yes @@ -19,7 +20,7 @@ - name: Ensure Docker repository is enabled apt_repository: - repo: "deb [arch=armhf] https://download.docker.com/linux/raspbian {{ ansible_distribution_release }} stable" + repo: "deb [arch={{ matrix_debian_arch }}] https://download.docker.com/linux/raspbian {{ ansible_distribution_release }} stable" state: present update_cache: yes when: matrix_docker_installation_enabled|bool and matrix_docker_package_name == 'docker-ce' @@ -27,7 +28,6 @@ - name: Ensure APT packages are installed apt: name: - - python-docker - "{{ matrix_ntpd_package }}" - fuse state: latest @@ -37,5 +37,6 @@ apt: name: - "{{ matrix_docker_package_name }}" + - "python{{'3' if ansible_python.version.major == 3 else ''}}-docker" state: latest when: matrix_docker_installation_enabled|bool diff --git a/roles/matrix-base/tasks/setup_matrix_base.yml b/roles/matrix-base/tasks/setup_matrix_base.yml index 22d330f2..0fad2b3d 100644 --- a/roles/matrix-base/tasks/setup_matrix_base.yml +++ b/roles/matrix-base/tasks/setup_matrix_base.yml @@ -19,31 +19,10 @@ mode: '0660' when: "matrix_vars_yml_snapshotting_enabled|bool" -# `docker_network` doesn't work as expected when the given network -# is a substring of a network that already exists. -# -# See: -# - https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/12 -# - https://github.com/ansible/ansible/issues/32926 -# -# Due to that, we employ a workaround below. -# -# - name: Ensure Matrix network is created in Docker -# docker_network: -# name: "{{ matrix_docker_network }}" -# driver: bridge - -- name: Check existence of Matrix network in Docker - shell: - cmd: "docker network ls -q --filter='name=^{{ matrix_docker_network }}$'" - register: result_check_docker_network - changed_when: false - check_mode: no - -- name: Create Matrix network in Docker - shell: - cmd: "docker network create --driver=bridge {{ matrix_docker_network }}" - when: "result_check_docker_network.stdout == '' and not ansible_check_mode" +- name: Ensure Matrix network is created in Docker + docker_network: + name: "{{ matrix_docker_network }}" + driver: bridge - name: Ensure matrix-remove-all script created template: diff --git a/roles/matrix-bot-matrix-reminder-bot/defaults/main.yml b/roles/matrix-bot-matrix-reminder-bot/defaults/main.yml index 33028147..29bc8307 100644 --- a/roles/matrix-bot-matrix-reminder-bot/defaults/main.yml +++ b/roles/matrix-bot-matrix-reminder-bot/defaults/main.yml @@ -21,6 +21,34 @@ matrix_bot_matrix_reminder_bot_systemd_required_services_list: ['docker.service' matrix_bot_matrix_reminder_bot_systemd_wanted_services_list: [] +# Database-related configuration fields. +# +# To use SQLite, stick to these defaults. +# +# To use Postgres: +# - change the engine (`matrix_bot_matrix_reminder_bot_database_engine: 'postgres'`) +# - adjust your database credentials via the `matrix_bot_matrix_reminder_bot_database_*` variables +matrix_bot_matrix_reminder_bot_database_engine: 'sqlite' + +matrix_bot_matrix_reminder_bot_sqlite_database_path_local: "{{ matrix_bot_matrix_reminder_bot_data_path }}/bot.db" +matrix_bot_matrix_reminder_bot_sqlite_database_path_in_container: "/data/bot.db" + +matrix_bot_matrix_reminder_bot_database_username: 'matrix_reminder_bot' +matrix_bot_matrix_reminder_bot_database_password: 'some-password' +matrix_bot_matrix_reminder_bot_database_hostname: 'matrix-postgres' +matrix_bot_matrix_reminder_bot_database_port: 5432 +matrix_bot_matrix_reminder_bot_database_name: 'matrix_reminder_bot' + +matrix_bot_matrix_reminder_bot_database_connection_string: 'postgres://{{ matrix_bot_matrix_reminder_bot_database_username }}:{{ matrix_bot_matrix_reminder_bot_database_password }}@{{ matrix_bot_matrix_reminder_bot_database_hostname }}:{{ matrix_bot_matrix_reminder_bot_database_port }}/{{ matrix_bot_matrix_reminder_bot_database_name }}' + +matrix_bot_matrix_reminder_bot_storage_database: "{{ + { + 'sqlite': ('sqlite://' + matrix_bot_matrix_reminder_bot_sqlite_database_path_in_container), + 'postgres': matrix_bot_matrix_reminder_bot_database_connection_string, + }[matrix_bot_matrix_reminder_bot_database_engine] +}}" + + # The bot's username. This user needs to be created manually beforehand. # Also see `matrix_bot_matrix_reminder_bot_user_password`. matrix_bot_matrix_reminder_bot_matrix_user_id_localpart: "bot.matrix-reminder-bot" diff --git a/roles/matrix-bot-matrix-reminder-bot/tasks/main.yml b/roles/matrix-bot-matrix-reminder-bot/tasks/main.yml index ade3d191..fc2afddb 100644 --- a/roles/matrix-bot-matrix-reminder-bot/tasks/main.yml +++ b/roles/matrix-bot-matrix-reminder-bot/tasks/main.yml @@ -8,7 +8,14 @@ - setup-all - setup-bot-matrix-reminder-bot -- import_tasks: "{{ role_path }}/tasks/setup.yml" +- import_tasks: "{{ role_path }}/tasks/setup_install.yml" + when: "run_setup|bool and matrix_bot_matrix_reminder_bot_enabled|bool" + tags: + - setup-all + - setup-bot-matrix-reminder-bot + +- import_tasks: "{{ role_path }}/tasks/setup_uninstall.yml" + when: "run_setup|bool and not matrix_bot_matrix_reminder_bot_enabled|bool" tags: - setup-all - setup-bot-matrix-reminder-bot diff --git a/roles/matrix-bot-matrix-reminder-bot/tasks/setup.yml b/roles/matrix-bot-matrix-reminder-bot/tasks/setup.yml deleted file mode 100644 index bc211e31..00000000 --- a/roles/matrix-bot-matrix-reminder-bot/tasks/setup.yml +++ /dev/null @@ -1,88 +0,0 @@ ---- - -# -# Tasks related to setting up matrix-reminder-bot -# - -- name: Ensure matrix-reminder-bot paths exist - file: - path: "{{ item.path }}" - state: directory - mode: 0750 - owner: "{{ matrix_user_username }}" - group: "{{ matrix_user_groupname }}" - with_items: - - { path: "{{ matrix_bot_matrix_reminder_bot_config_path }}", when: true } - - { path: "{{ matrix_bot_matrix_reminder_bot_data_path }}", when: true } - - { path: "{{ matrix_bot_matrix_reminder_bot_data_store_path }}", when: true } - when: matrix_bot_matrix_reminder_bot_enabled|bool and item.when - -- name: Ensure matrix-reminder-bot image is pulled - docker_image: - name: "{{ matrix_bot_matrix_reminder_bot_docker_image }}" - source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" - force_source: "{{ matrix_bot_matrix_reminder_bot_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" - force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_bot_matrix_reminder_bot_docker_image_force_pull }}" - when: matrix_bot_matrix_reminder_bot_enabled|bool - -- name: Ensure matrix-reminder-bot config installed - copy: - content: "{{ matrix_bot_matrix_reminder_bot_configuration|to_nice_yaml }}" - dest: "{{ matrix_bot_matrix_reminder_bot_config_path }}/config.yaml" - mode: 0644 - owner: "{{ matrix_user_username }}" - group: "{{ matrix_user_groupname }}" - when: matrix_bot_matrix_reminder_bot_enabled|bool - -- name: Ensure matrix-matrix-reminder-bot.service installed - template: - src: "{{ role_path }}/templates/systemd/matrix-bot-matrix-reminder-bot.service.j2" - dest: "{{ matrix_systemd_path }}/matrix-bot-matrix-reminder-bot.service" - mode: 0644 - register: matrix_bot_matrix_reminder_bot_systemd_service_result - when: matrix_bot_matrix_reminder_bot_enabled|bool - -- name: Ensure systemd reloaded after matrix-matrix-reminder-bot.service installation - service: - daemon_reload: yes - when: "matrix_bot_matrix_reminder_bot_enabled|bool and matrix_bot_matrix_reminder_bot_systemd_service_result.changed" - -# -# Tasks related to getting rid of matrix-reminder-bot (if it was previously enabled) -# - -- name: Check existence of matrix-matrix-reminder-bot service - stat: - path: "{{ matrix_systemd_path }}/matrix-matrix-reminder-bot.service" - register: matrix_bot_matrix_reminder_bot_service_stat - -- name: Ensure matrix-matrix-reminder-bot is stopped - service: - name: matrix-matrix-reminder-bot - state: stopped - daemon_reload: yes - register: stopping_result - when: "not matrix_bot_matrix_reminder_bot_enabled|bool and matrix_bot_matrix_reminder_bot_service_stat.stat.exists" - -- name: Ensure matrix-matrix-reminder-bot.service doesn't exist - file: - path: "{{ matrix_systemd_path }}/matrix-matrix-reminder-bot.service" - state: absent - when: "not matrix_bot_matrix_reminder_bot_enabled|bool and matrix_bot_matrix_reminder_bot_service_stat.stat.exists" - -- name: Ensure systemd reloaded after matrix-matrix-reminder-bot.service removal - service: - daemon_reload: yes - when: "not matrix_bot_matrix_reminder_bot_enabled|bool and matrix_bot_matrix_reminder_bot_service_stat.stat.exists" - -- name: Ensure Matrix matrix-reminder-bot paths don't exist - file: - path: "{{ matrix_bot_matrix_reminder_bot_base_path }}" - state: absent - when: "not matrix_bot_matrix_reminder_bot_enabled|bool" - -- name: Ensure matrix-reminder-bot Docker image doesn't exist - docker_image: - name: "{{ matrix_bot_matrix_reminder_bot_docker_image }}" - state: absent - when: "not matrix_bot_matrix_reminder_bot_enabled|bool" diff --git a/roles/matrix-bot-matrix-reminder-bot/tasks/setup_install.yml b/roles/matrix-bot-matrix-reminder-bot/tasks/setup_install.yml new file mode 100644 index 00000000..195485e4 --- /dev/null +++ b/roles/matrix-bot-matrix-reminder-bot/tasks/setup_install.yml @@ -0,0 +1,73 @@ +--- + +- set_fact: + matrix_bot_matrix_reminder_bot_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_bot_matrix_reminder_bot_sqlite_database_path_local }}" + register: matrix_bot_matrix_reminder_bot_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_bot_matrix_reminder_bot_sqlite_database_path_local }}" + dst: "{{ matrix_bot_matrix_reminder_bot_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_bot_matrix_reminder_bot_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-bot-matrix-reminder-bot.service'] + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_bot_matrix_reminder_bot_requires_restart: true + when: "matrix_bot_matrix_reminder_bot_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_bot_matrix_reminder_bot_database_engine == 'postgres'" + +- name: Ensure matrix-reminder-bot paths exist + file: + path: "{{ item.path }}" + state: directory + mode: 0750 + owner: "{{ matrix_user_username }}" + group: "{{ matrix_user_groupname }}" + with_items: + - { path: "{{ matrix_bot_matrix_reminder_bot_config_path }}", when: true } + - { path: "{{ matrix_bot_matrix_reminder_bot_data_path }}", when: true } + - { path: "{{ matrix_bot_matrix_reminder_bot_data_store_path }}", when: true } + when: "item.when|bool" + +- name: Ensure matrix-reminder-bot image is pulled + docker_image: + name: "{{ matrix_bot_matrix_reminder_bot_docker_image }}" + source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" + force_source: "{{ matrix_bot_matrix_reminder_bot_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" + force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_bot_matrix_reminder_bot_docker_image_force_pull }}" + +- name: Ensure matrix-reminder-bot config installed + copy: + content: "{{ matrix_bot_matrix_reminder_bot_configuration|to_nice_yaml }}" + dest: "{{ matrix_bot_matrix_reminder_bot_config_path }}/config.yaml" + mode: 0644 + owner: "{{ matrix_user_username }}" + group: "{{ matrix_user_groupname }}" + +- name: Ensure matrix-bot-matrix-reminder-bot.service installed + template: + src: "{{ role_path }}/templates/systemd/matrix-bot-matrix-reminder-bot.service.j2" + dest: "{{ matrix_systemd_path }}/matrix-bot-matrix-reminder-bot.service" + mode: 0644 + register: matrix_bot_matrix_reminder_bot_systemd_service_result + +- name: Ensure systemd reloaded after matrix-bot-matrix-reminder-bot.service installation + service: + daemon_reload: yes + when: "matrix_bot_matrix_reminder_bot_systemd_service_result.changed|bool" + +- name: Ensure matrix-bot-matrix-reminder-bot.service restarted, if necessary + service: + name: "matrix-bot-matrix-reminder-bot.service" + state: restarted + when: "matrix_bot_matrix_reminder_bot_requires_restart|bool" diff --git a/roles/matrix-bot-matrix-reminder-bot/tasks/setup_uninstall.yml b/roles/matrix-bot-matrix-reminder-bot/tasks/setup_uninstall.yml new file mode 100644 index 00000000..744f474d --- /dev/null +++ b/roles/matrix-bot-matrix-reminder-bot/tasks/setup_uninstall.yml @@ -0,0 +1,35 @@ +--- + +- name: Check existence of matrix-matrix-reminder-bot service + stat: + path: "{{ matrix_systemd_path }}/matrix-bot-matrix-reminder-bot.service" + register: matrix_bot_matrix_reminder_bot_service_stat + +- name: Ensure matrix-matrix-reminder-bot is stopped + service: + name: matrix-matrix-reminder-bot + state: stopped + daemon_reload: yes + register: stopping_result + when: "matrix_bot_matrix_reminder_bot_service_stat.stat.exists|bool" + +- name: Ensure matrix-bot-matrix-reminder-bot.service doesn't exist + file: + path: "{{ matrix_systemd_path }}/matrix-bot-matrix-reminder-bot.service" + state: absent + when: "matrix_bot_matrix_reminder_bot_service_stat.stat.exists|bool" + +- name: Ensure systemd reloaded after matrix-bot-matrix-reminder-bot.service removal + service: + daemon_reload: yes + when: "matrix_bot_matrix_reminder_bot_service_stat.stat.exists|bool" + +- name: Ensure Matrix matrix-reminder-bot paths don't exist + file: + path: "{{ matrix_bot_matrix_reminder_bot_base_path }}" + state: absent + +- name: Ensure matrix-reminder-bot Docker image doesn't exist + docker_image: + name: "{{ matrix_bot_matrix_reminder_bot_docker_image }}" + state: absent diff --git a/roles/matrix-bot-matrix-reminder-bot/templates/config.yaml.j2 b/roles/matrix-bot-matrix-reminder-bot/templates/config.yaml.j2 index e5ba5651..59643958 100644 --- a/roles/matrix-bot-matrix-reminder-bot/templates/config.yaml.j2 +++ b/roles/matrix-bot-matrix-reminder-bot/templates/config.yaml.j2 @@ -23,7 +23,7 @@ storage: # For Postgres, this would look like: # database: "postgres://username:password@localhost/dbname?sslmode=disable" #database: "postgres://matrix-reminder-bot:remindme@localhost/matrix-reminder-bot?sslmode=disable" - database: "sqlite:///data/bot.db" + database: {{ matrix_bot_matrix_reminder_bot_storage_database|to_json }} # The path to a directory for internal bot storage # containing encryption keys, sync tokens, etc. store_path: "/data/store" diff --git a/roles/matrix-bot-matrix-reminder-bot/templates/systemd/matrix-bot-matrix-reminder-bot.service.j2 b/roles/matrix-bot-matrix-reminder-bot/templates/systemd/matrix-bot-matrix-reminder-bot.service.j2 index 23493c54..950242e5 100644 --- a/roles/matrix-bot-matrix-reminder-bot/templates/systemd/matrix-bot-matrix-reminder-bot.service.j2 +++ b/roles/matrix-bot-matrix-reminder-bot/templates/systemd/matrix-bot-matrix-reminder-bot.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_bot_matrix_reminder_bot_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-bridge-appservice-discord/defaults/main.yml b/roles/matrix-bridge-appservice-discord/defaults/main.yml index d03ebb92..c7cdddb6 100644 --- a/roles/matrix-bridge-appservice-discord/defaults/main.yml +++ b/roles/matrix-bridge-appservice-discord/defaults/main.yml @@ -3,7 +3,7 @@ matrix_appservice_discord_enabled: true -matrix_appservice_discord_docker_image: "docker.io/halfshot/matrix-appservice-discord:latest" +matrix_appservice_discord_docker_image: "docker.io/halfshot/matrix-appservice-discord:v1.0.0" matrix_appservice_discord_docker_image_force_pull: "{{ matrix_appservice_discord_docker_image.endswith(':latest') }}" matrix_appservice_discord_base_path: "{{ matrix_base_data_path }}/appservice-discord" @@ -17,6 +17,8 @@ matrix_appservice_discord_bot_token: '' matrix_appservice_discord_appservice_token: '' matrix_appservice_discord_homeserver_token: '' +matrix_appservice_discord_homeserver_domain: "{{ matrix_domain }}" + # Controls whether the matrix-appservice-discord container exposes its HTTP port (tcp/9005 in the container). # # Takes an ":" or "" value (e.g. "127.0.0.1:9005"), or empty string to not expose. @@ -39,6 +41,38 @@ matrix_appservice_discord_bridge_homeserverUrl: "{{ matrix_homeserver_url }}" matrix_appservice_discord_bridge_disablePresence: false matrix_appservice_discord_bridge_enableSelfServiceBridging: false +# Database-related configuration fields. +# +# To use SQLite, stick to these defaults. +# +# To use Postgres: +# - change the engine (`matrix_appservice_discord_database_engine: 'postgres'`) +# - adjust your database credentials via the `matrix_appservice_discord_postgres_*` variables +matrix_appservice_discord_database_engine: 'sqlite' + +matrix_appservice_discord_sqlite_database_path_local: "{{ matrix_appservice_discord_data_path }}/discord.db" +matrix_appservice_discord_sqlite_database_path_in_container: "/data/discord.db" + +matrix_appservice_discord_database_username: 'matrix_appservice_discord' +matrix_appservice_discord_database_password: 'some-password' +matrix_appservice_discord_database_hostname: 'matrix-postgres' +matrix_appservice_discord_database_port: 5432 +matrix_appservice_discord_database_name: 'matrix_appservice_discord' + +# These 2 variables are what actually ends up in the bridge configuration. +# It's best if you don't change them directly, but rather redefine the sub-variables that constitute them. +matrix_appservice_discord_database_filename: "{{ matrix_appservice_discord_sqlite_database_path_in_container }}" +matrix_appservice_discord_database_connString: 'postgresql://{{ matrix_appservice_discord_database_username }}:{{ matrix_appservice_discord_database_password }}@{{ matrix_appservice_discord_database_hostname }}:{{ matrix_appservice_discord_database_port }}/{{ matrix_appservice_discord_database_name }}' + + +# Tells whether the bot should make use of "Privileged Gateway Intents". +# +# Enabling this means that you need to enable it for the bot (Discord application) as well, +# by triggering all Intent checkboxes on a page like this: `https://discord.com/developers/applications/694448564151123988/bot` +# +# Learn more: https://gist.github.com/advaith1/e69bcc1cdd6d0087322734451f15aa2f +matrix_appservice_discord_auth_usePrivilegedIntents: false + matrix_appservice_discord_configuration_yaml: "{{ lookup('template', 'templates/config.yaml.j2') }}" matrix_appservice_discord_configuration_extension_yaml: | @@ -62,10 +96,10 @@ matrix_appservice_discord_registration_yaml: | namespaces: users: - exclusive: true - regex: '^@_discord_.*' + regex: '@_discord_.*:{{ matrix_appservice_discord_homeserver_domain|regex_escape }}' aliases: - exclusive: true - regex: '^#_discord_.*' + regex: '#_discord_.*:{{ matrix_appservice_discord_homeserver_domain|regex_escape }}' url: {{ matrix_appservice_discord_appservice_url }} sender_localpart: _discord_bot rate_limited: false diff --git a/roles/matrix-bridge-appservice-discord/tasks/setup_install.yml b/roles/matrix-bridge-appservice-discord/tasks/setup_install.yml index 935d07eb..6d3fdd0f 100644 --- a/roles/matrix-bridge-appservice-discord/tasks/setup_install.yml +++ b/roles/matrix-bridge-appservice-discord/tasks/setup_install.yml @@ -1,5 +1,31 @@ --- +- set_fact: + matrix_appservice_discord_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_appservice_discord_sqlite_database_path_local }}" + register: matrix_appservice_discord_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_appservice_discord_sqlite_database_path_local }}" + dst: "{{ matrix_appservice_discord_database_connString }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_appservice_discord_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-appservice-discord.service'] + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_appservice_discord_requires_restart: true + when: "matrix_appservice_discord_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_appservice_discord_database_engine == 'postgres'" + - name: Ensure Appservice Discord image is pulled docker_image: name: "{{ matrix_appservice_discord_docker_image }}" @@ -80,3 +106,9 @@ service: daemon_reload: yes when: "matrix_appservice_discord_systemd_service_result.changed" + +- name: Ensure matrix-appservice-discord.service restarted, if necessary + service: + name: "matrix-appservice-discord.service" + state: restarted + when: "matrix_appservice_discord_requires_restart|bool" diff --git a/roles/matrix-bridge-appservice-discord/tasks/validate_config.yml b/roles/matrix-bridge-appservice-discord/tasks/validate_config.yml index 46612ba5..73253ba0 100644 --- a/roles/matrix-bridge-appservice-discord/tasks/validate_config.yml +++ b/roles/matrix-bridge-appservice-discord/tasks/validate_config.yml @@ -10,6 +10,7 @@ - "matrix_appservice_discord_bot_token" - "matrix_appservice_discord_appservice_token" - "matrix_appservice_discord_homeserver_token" + - "matrix_appservice_discord_homeserver_domain" - name: (Deprecation) Catch and report renamed appservice-discord variables fail: @@ -19,3 +20,7 @@ when: "item.old in vars" with_items: - {'old': 'matrix_appservice_discord_container_expose_client_server_api_port', 'new': ''} + +- name: Require a valid database engine + fail: msg="`matrix_appservice_discord_database_engine` needs to be either 'sqlite' or 'postgres'" + when: "matrix_appservice_discord_database_engine not in ['sqlite', 'postgres']" diff --git a/roles/matrix-bridge-appservice-discord/templates/config.yaml.j2 b/roles/matrix-bridge-appservice-discord/templates/config.yaml.j2 index e91c60de..b2ecd198 100644 --- a/roles/matrix-bridge-appservice-discord/templates/config.yaml.j2 +++ b/roles/matrix-bridge-appservice-discord/templates/config.yaml.j2 @@ -26,10 +26,17 @@ bridge: disableReadReceipts: false # Disable Join Leave echos from matrix disableJoinLeaveNotifications: false + # Disable Invite echos from matrix + disableInviteNotifications: false + # Auto-determine the language of code blocks (this can be CPU-intensive) + determineCodeLanguage: false # Authentication configuration for the discord bot. auth: clientID: {{ matrix_appservice_discord_client_id|string|to_json }} botToken: {{ matrix_appservice_discord_bot_token }} + # You must enable "Privileged Gateway Intents" in your bot settings on discord.com (e.g. https://discord.com/developers/applications/12345/bot) + # for this to work + usePrivilegedIntents: {{ matrix_appservice_discord_auth_usePrivilegedIntents|to_json }} logging: # What level should the logger output to the console at. console: "warn" #silly, verbose, info, http, warn, error, silent @@ -45,16 +52,17 @@ logging: # enable: # - "DiscordBot" database: - userStorePath: "/data/user-store.db" - roomStorePath: "/data/room-store.db" # You may either use SQLite or Postgresql for the bridge database, which contains # important mappings for events and user puppeting configurations. # Use the filename option for SQLite, or connString for Postgresql. # If you are migrating, see https://github.com/Half-Shot/matrix-appservice-discord/blob/master/docs/howto.md#migrate-to-postgres-from-sqlite # WARNING: You will almost certainly be fine with sqlite unless your bridge # is in heavy demand and you suffer from IO slowness. - filename: "/data/discord.db" - # connString: "postgresql://user:password@localhost/database_name" + {% if matrix_appservice_discord_database_engine == 'sqlite' %} + filename: {{ matrix_appservice_discord_database_filename|to_json }} + {% else %} + connString: {{ matrix_appservice_discord_database_connString|to_json }} + {% endif %} room: # Set the default visibility of alias rooms, defaults to "public". # One of: "public", "private" @@ -82,10 +90,12 @@ channel: limits: # Delay in milliseconds between discord users joining a room. roomGhostJoinDelay: 6000 - # Delay in milliseconds before sending messages to discord to avoid echos. - # (Copies of a sent message may arrive from discord before we've + # Lock timeout in milliseconds before sending messages to discord to avoid + # echos. Default is rather high as the lock will most likely time out + # before anyways. + # echos = (Copies of a sent message may arrive from discord before we've # fininished handling it, causing us to echo it back to the room) - discordSendDelay: 750 + discordSendDelay: 1500 ghosts: # Pattern for the ghosts nick, available is :nick, :username, :tag and :id nickPattern: ":nick" diff --git a/roles/matrix-bridge-appservice-discord/templates/systemd/matrix-appservice-discord.service.j2 b/roles/matrix-bridge-appservice-discord/templates/systemd/matrix-appservice-discord.service.j2 index 6a44a3e6..7ec15752 100644 --- a/roles/matrix-bridge-appservice-discord/templates/systemd/matrix-appservice-discord.service.j2 +++ b/roles/matrix-bridge-appservice-discord/templates/systemd/matrix-appservice-discord.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_appservice_discord_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-bridge-appservice-irc/defaults/main.yml b/roles/matrix-bridge-appservice-irc/defaults/main.yml index 7ff2d46a..0b671e76 100644 --- a/roles/matrix-bridge-appservice-irc/defaults/main.yml +++ b/roles/matrix-bridge-appservice-irc/defaults/main.yml @@ -16,6 +16,25 @@ matrix_appservice_irc_homeserver_domain: '{{ matrix_domain }}' matrix_appservice_irc_homeserver_enablePresence: true matrix_appservice_irc_appservice_address: 'http://matrix-appservice-irc:9999' +matrix_appservice_irc_database_engine: nedb +matrix_appservice_irc_database_username: matrix_appservice_irc +matrix_appservice_irc_database_password: ~ +matrix_appservice_irc_database_hostname: 'matrix-postgres' +matrix_appservice_irc_database_port: 5432 +matrix_appservice_irc_database_name: matrix_appservice_irc + +# This is just the Postgres connection string, if Postgres is used. +# Naming clashes with `matrix_appservice_irc_database_connectionString` somewhat. +matrix_appservice_irc_database_connection_string: 'postgresql://{{ matrix_appservice_irc_database_username }}:{{ matrix_appservice_irc_database_password }}@{{ matrix_appservice_irc_database_hostname }}:{{ matrix_appservice_irc_database_port }}/{{ matrix_appservice_irc_database_name }}?sslmode=disable' + +# This is what actually goes into `database.connectionString` for the bridge. +matrix_appservice_irc_database_connectionString: "{{ + { + 'nedb': 'nedb:///data', + 'postgres': matrix_appservice_irc_database_connection_string, + }[matrix_appservice_irc_database_engine] +}}" + matrix_appservice_irc_ircService_servers: [] # Example of `matrix_appservice_irc_ircService_servers` with one server (and all its options): diff --git a/roles/matrix-bridge-appservice-irc/tasks/migrate_nedb_to_postgres.yml b/roles/matrix-bridge-appservice-irc/tasks/migrate_nedb_to_postgres.yml new file mode 100644 index 00000000..3fab195a --- /dev/null +++ b/roles/matrix-bridge-appservice-irc/tasks/migrate_nedb_to_postgres.yml @@ -0,0 +1,64 @@ +- name: Fail if Postgres not enabled + fail: + msg: "Postgres via the matrix-postgres role is not enabled (`matrix_postgres_enabled`). Cannot migrate." + when: "not matrix_postgres_enabled|bool" + +# Defaults + +- name: Set postgres_start_wait_time, if not provided + set_fact: + postgres_start_wait_time: 15 + when: "postgres_start_wait_time|default('') == ''" + +# Actual import work + +- name: Ensure matrix-postgres is started + service: + name: matrix-postgres + state: started + daemon_reload: yes + register: matrix_postgres_service_start_result + +- name: Wait a bit, so that Postgres can start + wait_for: + timeout: "{{ postgres_start_wait_time }}" + delegate_to: 127.0.0.1 + become: false + when: "matrix_postgres_service_start_result.changed|bool" + +- name: Ensure matrix-appservice-irc is stopped + service: + name: matrix-appservice-irc + state: stopped + +- name: Import appservice-irc NeDB database into Postgres + command: + cmd: >- + {{ matrix_host_command_docker }} run + --rm + --user={{ matrix_user_uid }}:{{ matrix_user_gid }} + --cap-drop=ALL + --network={{ matrix_docker_network }} + --mount type=bind,src={{ matrix_appservice_irc_data_path }},dst=/data + --entrypoint=/bin/sh + {{ matrix_appservice_irc_docker_image }} + -c + '/usr/local/bin/node /app/lib/scripts/migrate-db-to-pgres.js --dbdir /data --privateKey /data/passkey.pem --connectionString {{ matrix_appservice_irc_database_connection_string }}' + +- name: Archive NeDB database files + command: + cmd: "mv {{ matrix_appservice_irc_data_path }}/{{ item }} {{ matrix_appservice_irc_data_path }}/{{ item }}.backup" + with_items: + - rooms.db + - users.db + +- name: Inject result + set_fact: + matrix_playbook_runtime_results: | + {{ + matrix_playbook_runtime_results|default([]) + + + [ + "NOTE: Your appservice-irc database files have been imported into Postgres. The original database files have been moved from `{{ matrix_appservice_irc_data_path }}/*.db` to `{{ matrix_appservice_irc_data_path }}/*.db.backup`. When you've confirmed that the import went well and everything works, you should be able to safely delete these files." + ] + }} diff --git a/roles/matrix-bridge-appservice-irc/tasks/setup_install.yml b/roles/matrix-bridge-appservice-irc/tasks/setup_install.yml index 5e313347..00568c0d 100644 --- a/roles/matrix-bridge-appservice-irc/tasks/setup_install.yml +++ b/roles/matrix-bridge-appservice-irc/tasks/setup_install.yml @@ -1,12 +1,5 @@ --- -- name: Ensure Appservice IRC image is pulled - docker_image: - name: "{{ matrix_appservice_irc_docker_image }}" - source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" - force_source: "{{ matrix_appservice_irc_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" - force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_appservice_irc_docker_image_force_pull }}" - - name: Ensure Appservice IRC paths exist file: path: "{{ item }}" @@ -24,25 +17,48 @@ path: "{{ matrix_appservice_irc_base_path }}/passkey.pem" register: matrix_appservice_irc_stat_passkey -- name: (Data relocation) Ensure matrix-appservice-irc.service is stopped - service: - name: matrix-appservice-irc - state: stopped - daemon_reload: yes - failed_when: false +- block: + - name: (Data relocation) Ensure matrix-appservice-irc.service is stopped + service: + name: matrix-appservice-irc + state: stopped + daemon_reload: yes + failed_when: false + + - name: (Data relocation) Move AppService IRC passkey.pem file to ./data directory + command: "mv {{ matrix_appservice_irc_base_path }}/passkey.pem {{ matrix_appservice_irc_data_path }}/passkey.pem" + + - name: (Data relocation) Move AppService IRC database files to ./data directory + command: "mv {{ matrix_appservice_irc_base_path }}/{{ item }} {{ matrix_appservice_irc_data_path }}/{{ item }}" + with_items: + - rooms.db + - users.db + failed_when: false when: "matrix_appservice_irc_stat_passkey.stat.exists" -- name: (Data relocation) Move AppService IRC passkey.pem file to ./data directory - command: "mv {{ matrix_appservice_irc_base_path }}/passkey.pem {{ matrix_appservice_irc_data_path }}/passkey.pem" - when: "matrix_appservice_irc_stat_passkey.stat.exists" +- set_fact: + matrix_appservice_irc_requires_restart: false -- name: (Data relocation) Move AppService IRC database files to ./data directory - command: "mv {{ matrix_appservice_irc_base_path }}/{{ item }} {{ matrix_appservice_irc_data_path }}/{{ item }}" - with_items: - - rooms.db - - users.db - failed_when: false - when: "matrix_appservice_irc_stat_passkey.stat.exists" +- block: + - name: Check if a nedb database already exists + stat: + path: "{{ matrix_appservice_irc_data_path }}/users.db" + register: matrix_appservice_irc_nedb_database_path_local_stat_result + + - block: + - import_tasks: "{{ role_path }}/tasks/migrate_nedb_to_postgres.yml" + + - set_fact: + matrix_appservice_irc_requires_restart: true + when: "matrix_appservice_irc_nedb_database_path_local_stat_result.stat.exists|bool" + when: "matrix_appservice_irc_database_engine == 'postgres'" + +- name: Ensure Appservice IRC image is pulled + docker_image: + name: "{{ matrix_appservice_irc_docker_image }}" + source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" + force_source: "{{ matrix_appservice_irc_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" + force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_appservice_irc_docker_image_force_pull }}" - name: Ensure Matrix Appservice IRC config installed copy: @@ -147,3 +163,9 @@ service: daemon_reload: yes when: "matrix_appservice_irc_systemd_service_result.changed" + +- name: Ensure matrix-appservice-irc.service restarted, if necessary + service: + name: "matrix-appservice-irc.service" + state: restarted + when: "matrix_appservice_irc_requires_restart|bool" diff --git a/roles/matrix-bridge-appservice-irc/templates/config.yaml.j2 b/roles/matrix-bridge-appservice-irc/templates/config.yaml.j2 index 3daa1898..94bbda7b 100644 --- a/roles/matrix-bridge-appservice-irc/templates/config.yaml.j2 +++ b/roles/matrix-bridge-appservice-irc/templates/config.yaml.j2 @@ -127,8 +127,8 @@ advanced: # Use an external database to store bridge state. database: # database engine (must be 'postgres' or 'nedb'). Default: nedb - engine: "nedb" + engine: {{ matrix_appservice_irc_database_engine|to_json }} # Either a PostgreSQL connection string, or a path to the NeDB storage directory. # For postgres, it must start with postgres:// # For NeDB, it must start with nedb://. The path is relative to the project directory. - connectionString: "nedb:///data" + connectionString: {{ matrix_appservice_irc_database_connectionString|to_json }} diff --git a/roles/matrix-bridge-appservice-irc/templates/systemd/matrix-appservice-irc.service.j2 b/roles/matrix-bridge-appservice-irc/templates/systemd/matrix-appservice-irc.service.j2 index 95638612..c86eb835 100644 --- a/roles/matrix-bridge-appservice-irc/templates/systemd/matrix-appservice-irc.service.j2 +++ b/roles/matrix-bridge-appservice-irc/templates/systemd/matrix-appservice-irc.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_appservice_irc_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-bridge-appservice-slack/defaults/main.yml b/roles/matrix-bridge-appservice-slack/defaults/main.yml index f2cfd8f9..28646a3c 100644 --- a/roles/matrix-bridge-appservice-slack/defaults/main.yml +++ b/roles/matrix-bridge-appservice-slack/defaults/main.yml @@ -45,6 +45,26 @@ matrix_appservice_slack_appservice_token: '' matrix_appservice_slack_homeserver_token: '' matrix_appservice_slack_id_token: '' +matrix_appservice_slack_database_engine: nedb +matrix_appservice_slack_database_username: matrix_appservice_slack +matrix_appservice_slack_database_password: ~ +matrix_appservice_slack_database_hostname: 'matrix-postgres' +matrix_appservice_slack_database_port: 5432 +matrix_appservice_slack_database_name: matrix_appservice_slack + +# This is just the Postgres connection string, if Postgres is used. +# Naming clashes with `matrix_appservice_slack_database_connectionString` somewhat. +matrix_appservice_slack_database_connection_string: 'postgresql://{{ matrix_appservice_slack_database_username }}:{{ matrix_appservice_slack_database_password }}@{{ matrix_appservice_slack_database_hostname }}:{{ matrix_appservice_slack_database_port }}/{{ matrix_appservice_slack_database_name }}?sslmode=disable' + +# This is what actually goes into `database.connectionString` for the bridge. +matrix_appservice_slack_database_connectionString: "{{ + { + 'nedb': 'nedb:///data', + 'postgres': matrix_appservice_slack_database_connection_string, + }[matrix_appservice_slack_database_engine] +}}" + + matrix_appservice_slack_configuration_yaml: "{{ lookup('template', 'templates/config.yaml.j2') }}" matrix_appservice_slack_configuration_extension_yaml: | diff --git a/roles/matrix-bridge-appservice-slack/tasks/migrate_nedb_to_postgres.yml b/roles/matrix-bridge-appservice-slack/tasks/migrate_nedb_to_postgres.yml new file mode 100644 index 00000000..fedad977 --- /dev/null +++ b/roles/matrix-bridge-appservice-slack/tasks/migrate_nedb_to_postgres.yml @@ -0,0 +1,66 @@ +- name: Fail if Postgres not enabled + fail: + msg: "Postgres via the matrix-postgres role is not enabled (`matrix_postgres_enabled`). Cannot migrate." + when: "not matrix_postgres_enabled|bool" + +# Defaults + +- name: Set postgres_start_wait_time, if not provided + set_fact: + postgres_start_wait_time: 15 + when: "postgres_start_wait_time|default('') == ''" + +# Actual import work + +- name: Ensure matrix-postgres is started + service: + name: matrix-postgres + state: started + daemon_reload: yes + register: matrix_postgres_service_start_result + +- name: Wait a bit, so that Postgres can start + wait_for: + timeout: "{{ postgres_start_wait_time }}" + delegate_to: 127.0.0.1 + become: false + when: "matrix_postgres_service_start_result.changed|bool" + +- name: Ensure matrix-appservice-slack is stopped + service: + name: matrix-appservice-slack + state: stopped + +- name: Import appservice-slack NeDB database into Postgres + command: + cmd: >- + {{ matrix_host_command_docker }} run + --rm + --user={{ matrix_user_uid }}:{{ matrix_user_gid }} + --cap-drop=ALL + --network={{ matrix_docker_network }} + --mount type=bind,src={{ matrix_appservice_slack_data_path }},dst=/data + --entrypoint=/bin/sh + {{ matrix_appservice_slack_docker_image }} + -c + '/usr/local/bin/node /usr/src/app/lib/scripts/migrateToPostgres.js --dbdir /data --connectionString {{ matrix_appservice_slack_database_connection_string }}' + +- name: Archive NeDB database files + command: + cmd: "mv {{ matrix_appservice_slack_data_path }}/{{ item }} {{ matrix_appservice_slack_data_path }}/{{ item }}.backup" + with_items: + - teams.db + - room-store.db + - user-store.db + - event-store.db + +- name: Inject result + set_fact: + matrix_playbook_runtime_results: | + {{ + matrix_playbook_runtime_results|default([]) + + + [ + "NOTE: Your appservice-slack database files have been imported into Postgres. The original database files have been moved from `{{ matrix_appservice_slack_data_path }}/*.db` to `{{ matrix_appservice_slack_data_path }}/*.db.backup`. When you've confirmed that the import went well and everything works, you should be able to safely delete these files." + ] + }} diff --git a/roles/matrix-bridge-appservice-slack/tasks/setup_install.yml b/roles/matrix-bridge-appservice-slack/tasks/setup_install.yml index 94e0fedf..29b0f39e 100644 --- a/roles/matrix-bridge-appservice-slack/tasks/setup_install.yml +++ b/roles/matrix-bridge-appservice-slack/tasks/setup_install.yml @@ -1,12 +1,5 @@ --- -- name: Ensure Appservice Slack image is pulled - docker_image: - name: "{{ matrix_appservice_slack_docker_image }}" - source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" - force_source: "{{ matrix_appservice_slack_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" - force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_appservice_slack_docker_image_force_pull }}" - - name: Ensure AppService Slack paths exist file: path: "{{ item }}" @@ -19,6 +12,30 @@ - "{{ matrix_appservice_slack_config_path }}" - "{{ matrix_appservice_slack_data_path }}" +- set_fact: + matrix_appservice_slack_requires_restart: false + +- block: + - name: Check if a nedb database already exists + stat: + path: "{{ matrix_appservice_slack_data_path }}/teams.db" + register: matrix_appservice_slack_nedb_database_path_local_stat_result + + - block: + - import_tasks: "{{ role_path }}/tasks/migrate_nedb_to_postgres.yml" + + - set_fact: + matrix_appservice_slack_requires_restart: true + when: "matrix_appservice_slack_nedb_database_path_local_stat_result.stat.exists|bool" + when: "matrix_appservice_slack_database_engine == 'postgres'" + +- name: Ensure Appservice Slack image is pulled + docker_image: + name: "{{ matrix_appservice_slack_docker_image }}" + source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" + force_source: "{{ matrix_appservice_slack_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" + force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_appservice_slack_docker_image_force_pull }}" + - name: Ensure Matrix Appservice Slack config installed copy: content: "{{ matrix_appservice_slack_configuration|to_nice_yaml }}" @@ -46,3 +63,9 @@ service: daemon_reload: yes when: "matrix_appservice_slack_systemd_service_result.changed" + +- name: Ensure matrix-appservice-slack.service restarted, if necessary + service: + name: "matrix-appservice-slack.service" + state: restarted + when: "matrix_appservice_slack_requires_restart|bool" diff --git a/roles/matrix-bridge-appservice-slack/tasks/validate_config.yml b/roles/matrix-bridge-appservice-slack/tasks/validate_config.yml index 5da5f947..8af10f2f 100644 --- a/roles/matrix-bridge-appservice-slack/tasks/validate_config.yml +++ b/roles/matrix-bridge-appservice-slack/tasks/validate_config.yml @@ -9,4 +9,4 @@ - "matrix_appservice_slack_control_room_id" - "matrix_appservice_slack_appservice_token" - "matrix_appservice_slack_homeserver_token" - - "matrix_appservice_slack_id_token" \ No newline at end of file + - "matrix_appservice_slack_id_token" diff --git a/roles/matrix-bridge-appservice-slack/templates/config.yaml.j2 b/roles/matrix-bridge-appservice-slack/templates/config.yaml.j2 index 8f48d317..bf8072c1 100644 --- a/roles/matrix-bridge-appservice-slack/templates/config.yaml.j2 +++ b/roles/matrix-bridge-appservice-slack/templates/config.yaml.j2 @@ -9,6 +9,12 @@ homeserver: url: "{{ matrix_appservice_slack_homeserver_url }}" server_name: "{{ matrix_domain }}" +{% if matrix_appservice_slack_database_engine == 'nedb' %} dbdir: "/data" +{% else %} +db: + engine: {{ matrix_appservice_slack_database_engine|to_json }} + connectionString: {{ matrix_appservice_slack_database_connectionString|to_json }} +{% endif %} matrix_admin_room: "{{ matrix_appservice_slack_control_room_id }}" diff --git a/roles/matrix-bridge-appservice-slack/templates/systemd/matrix-appservice-slack.service.j2 b/roles/matrix-bridge-appservice-slack/templates/systemd/matrix-appservice-slack.service.j2 index 1c68294f..b16b2fed 100644 --- a/roles/matrix-bridge-appservice-slack/templates/systemd/matrix-appservice-slack.service.j2 +++ b/roles/matrix-bridge-appservice-slack/templates/systemd/matrix-appservice-slack.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_appservice_slack_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-bridge-appservice-webhooks/templates/systemd/matrix-appservice-webhooks.service.j2 b/roles/matrix-bridge-appservice-webhooks/templates/systemd/matrix-appservice-webhooks.service.j2 index 08f5813f..2049ee65 100644 --- a/roles/matrix-bridge-appservice-webhooks/templates/systemd/matrix-appservice-webhooks.service.j2 +++ b/roles/matrix-bridge-appservice-webhooks/templates/systemd/matrix-appservice-webhooks.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_appservice_webhooks_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-bridge-mautrix-facebook/defaults/main.yml b/roles/matrix-bridge-mautrix-facebook/defaults/main.yml index 1547ad71..da9b1889 100644 --- a/roles/matrix-bridge-mautrix-facebook/defaults/main.yml +++ b/roles/matrix-bridge-mautrix-facebook/defaults/main.yml @@ -32,6 +32,35 @@ matrix_mautrix_facebook_systemd_wanted_services_list: [] matrix_mautrix_facebook_appservice_token: '' matrix_mautrix_facebook_homeserver_token: '' + +# Database-related configuration fields. +# +# To use SQLite, stick to these defaults. +# +# To use Postgres: +# - change the engine (`matrix_mautrix_facebook_database_engine: 'postgres'`) +# - adjust your database credentials via the `matrix_mautrix_facebook_postgres_*` variables +matrix_mautrix_facebook_database_engine: 'sqlite' + +matrix_mautrix_facebook_sqlite_database_path_local: "{{ matrix_mautrix_facebook_data_path }}/mautrix-facebook.db" +matrix_mautrix_facebook_sqlite_database_path_in_container: "/data/mautrix-facebook.db" + +matrix_mautrix_facebook_database_username: 'matrix_mautrix_facebook' +matrix_mautrix_facebook_database_password: 'some-password' +matrix_mautrix_facebook_database_hostname: 'matrix-postgres' +matrix_mautrix_facebook_database_port: 5432 +matrix_mautrix_facebook_database_name: 'matrix_mautrix_facebook' + +matrix_mautrix_facebook_database_connection_string: 'postgres://{{ matrix_mautrix_facebook_database_username }}:{{ matrix_mautrix_facebook_database_password }}@{{ matrix_mautrix_facebook_database_hostname }}:{{ matrix_mautrix_facebook_database_port }}/{{ matrix_mautrix_facebook_database_name }}' + +matrix_mautrix_facebook_appservice_database: "{{ + { + 'sqlite': ('sqlite:///' + matrix_mautrix_facebook_sqlite_database_path_in_container), + 'postgres': matrix_mautrix_facebook_database_connection_string, + }[matrix_mautrix_facebook_database_engine] +}}" + + # Can be set to enable automatic double-puppeting via Shared Secret Auth (https://github.com/devture/matrix-synapse-shared-secret-auth). matrix_mautrix_facebook_login_shared_secret: '' diff --git a/roles/matrix-bridge-mautrix-facebook/tasks/setup_install.yml b/roles/matrix-bridge-mautrix-facebook/tasks/setup_install.yml index 638854e2..59998463 100644 --- a/roles/matrix-bridge-mautrix-facebook/tasks/setup_install.yml +++ b/roles/matrix-bridge-mautrix-facebook/tasks/setup_install.yml @@ -8,6 +8,32 @@ The matrix-bridge-mautrix-facebook role needs to execute before the matrix-synapse role. when: "matrix_synapse_role_executed|default(False)" +- set_fact: + matrix_mautrix_facebook_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_mautrix_facebook_sqlite_database_path_local }}" + register: matrix_mautrix_facebook_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_mautrix_facebook_sqlite_database_path_local }}" + dst: "{{ matrix_mautrix_facebook_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_mautrix_facebook_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-mautrix-facebook.service'] + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_mautrix_facebook_requires_restart: true + when: "matrix_mautrix_facebook_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_mautrix_facebook_database_engine == 'postgres'" + - name: Ensure Mautrix Facebook image is pulled docker_image: name: "{{ matrix_mautrix_facebook_docker_image }}" @@ -94,3 +120,9 @@ service: daemon_reload: yes when: "matrix_mautrix_facebook_systemd_service_result.changed" + +- name: Ensure matrix-mautrix-facebook.service restarted, if necessary + service: + name: "matrix-mautrix-facebook.service" + state: restarted + when: "matrix_mautrix_facebook_requires_restart|bool" diff --git a/roles/matrix-bridge-mautrix-facebook/templates/config.yaml.j2 b/roles/matrix-bridge-mautrix-facebook/templates/config.yaml.j2 index 9d8de2d5..09287362 100644 --- a/roles/matrix-bridge-mautrix-facebook/templates/config.yaml.j2 +++ b/roles/matrix-bridge-mautrix-facebook/templates/config.yaml.j2 @@ -27,7 +27,7 @@ appservice: # Format examples: # SQLite: sqlite:///filename.db # Postgres: postgres://username:password@hostname/dbname - database: sqlite:////data/mautrix-facebook.db + database: {{ matrix_mautrix_facebook_appservice_database|to_json }} # Public part of web server for out-of-Matrix interaction with the bridge. public: diff --git a/roles/matrix-bridge-mautrix-facebook/templates/systemd/matrix-mautrix-facebook.service.j2 b/roles/matrix-bridge-mautrix-facebook/templates/systemd/matrix-mautrix-facebook.service.j2 index b593a2e3..caa52eb5 100644 --- a/roles/matrix-bridge-mautrix-facebook/templates/systemd/matrix-mautrix-facebook.service.j2 +++ b/roles/matrix-bridge-mautrix-facebook/templates/systemd/matrix-mautrix-facebook.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_mautrix_facebook_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple @@ -17,6 +18,7 @@ ExecStartPre={{ matrix_host_command_docker }} run --rm --name matrix-mautrix-fac --log-driver=none \ --user={{ matrix_user_uid }}:{{ matrix_user_gid }} \ --cap-drop=ALL \ + --network={{ matrix_docker_network }} \ -v {{ matrix_mautrix_facebook_config_path }}:/config:z \ -v {{ matrix_mautrix_facebook_data_path }}:/data:z \ {{ matrix_mautrix_facebook_docker_image }} \ diff --git a/roles/matrix-bridge-mautrix-hangouts/defaults/main.yml b/roles/matrix-bridge-mautrix-hangouts/defaults/main.yml index ab670052..8dfee030 100644 --- a/roles/matrix-bridge-mautrix-hangouts/defaults/main.yml +++ b/roles/matrix-bridge-mautrix-hangouts/defaults/main.yml @@ -39,6 +39,35 @@ matrix_mautrix_hangouts_systemd_wanted_services_list: [] matrix_mautrix_hangouts_appservice_token: '' matrix_mautrix_hangouts_homeserver_token: '' + +# Database-related configuration fields. +# +# To use SQLite, stick to these defaults. +# +# To use Postgres: +# - change the engine (`matrix_mautrix_hangouts_database_engine: 'postgres'`) +# - adjust your database credentials via the `matrix_mautrix_hangouts_postgres_*` variables +matrix_mautrix_hangouts_database_engine: 'sqlite' + +matrix_mautrix_hangouts_sqlite_database_path_local: "{{ matrix_mautrix_hangouts_data_path }}/mautrix-hangouts.db" +matrix_mautrix_hangouts_sqlite_database_path_in_container: "/data/mautrix-hangouts.db" + +matrix_mautrix_hangouts_database_username: 'matrix_mautrix_hangouts' +matrix_mautrix_hangouts_database_password: 'some-password' +matrix_mautrix_hangouts_database_hostname: 'matrix-postgres' +matrix_mautrix_hangouts_database_port: 5432 +matrix_mautrix_hangouts_database_name: 'matrix_mautrix_hangouts' + +matrix_mautrix_hangouts_database_connection_string: 'postgres://{{ matrix_mautrix_hangouts_database_username }}:{{ matrix_mautrix_hangouts_database_password }}@{{ matrix_mautrix_hangouts_database_hostname }}:{{ matrix_mautrix_hangouts_database_port }}/{{ matrix_mautrix_hangouts_database_name }}' + +matrix_mautrix_hangouts_appservice_database: "{{ + { + 'sqlite': ('sqlite:///' + matrix_mautrix_hangouts_sqlite_database_path_in_container), + 'postgres': matrix_mautrix_hangouts_database_connection_string, + }[matrix_mautrix_hangouts_database_engine] +}}" + + # Can be set to enable automatic double-puppeting via Shared Secret Auth (https://github.com/devture/matrix-synapse-shared-secret-auth). matrix_mautrix_hangouts_login_shared_secret: '' diff --git a/roles/matrix-bridge-mautrix-hangouts/tasks/setup_install.yml b/roles/matrix-bridge-mautrix-hangouts/tasks/setup_install.yml index eec5e006..2ce8a441 100644 --- a/roles/matrix-bridge-mautrix-hangouts/tasks/setup_install.yml +++ b/roles/matrix-bridge-mautrix-hangouts/tasks/setup_install.yml @@ -8,6 +8,32 @@ The matrix-bridge-mautrix-hangouts role needs to execute before the matrix-synapse role. when: "matrix_synapse_role_executed|default(False)" +- set_fact: + matrix_mautrix_hangouts_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_mautrix_hangouts_sqlite_database_path_local }}" + register: matrix_mautrix_hangouts_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_mautrix_hangouts_sqlite_database_path_local }}" + dst: "{{ matrix_mautrix_hangouts_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_mautrix_hangouts_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-mautrix-hangouts.service'] + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_mautrix_hangouts_requires_restart: true + when: "matrix_mautrix_hangouts_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_mautrix_hangouts_database_engine == 'postgres'" + - name: Ensure Mautrix Hangouts image is pulled docker_image: name: "{{ matrix_mautrix_hangouts_docker_image }}" @@ -93,3 +119,9 @@ service: daemon_reload: yes when: "matrix_mautrix_hangouts_systemd_service_result.changed" + +- name: Ensure matrix-mautrix-hangouts.service restarted, if necessary + service: + name: "matrix-mautrix-hangouts.service" + state: restarted + when: "matrix_mautrix_hangouts_requires_restart|bool" diff --git a/roles/matrix-bridge-mautrix-hangouts/templates/config.yaml.j2 b/roles/matrix-bridge-mautrix-hangouts/templates/config.yaml.j2 index f274b203..cc2ca90b 100644 --- a/roles/matrix-bridge-mautrix-hangouts/templates/config.yaml.j2 +++ b/roles/matrix-bridge-mautrix-hangouts/templates/config.yaml.j2 @@ -27,7 +27,7 @@ appservice: # Format examples: # SQLite: sqlite:///filename.db # Postgres: postgres://username:password@hostname/dbname - database: sqlite:////data/mautrix-hangouts.db + database: {{ matrix_mautrix_hangouts_appservice_database|to_json }} # The unique ID of this appservice. id: hangouts diff --git a/roles/matrix-bridge-mautrix-hangouts/templates/systemd/matrix-mautrix-hangouts.service.j2 b/roles/matrix-bridge-mautrix-hangouts/templates/systemd/matrix-mautrix-hangouts.service.j2 index 58433982..39559190 100644 --- a/roles/matrix-bridge-mautrix-hangouts/templates/systemd/matrix-mautrix-hangouts.service.j2 +++ b/roles/matrix-bridge-mautrix-hangouts/templates/systemd/matrix-mautrix-hangouts.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_mautrix_hangouts_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple @@ -17,6 +18,7 @@ ExecStartPre={{ matrix_host_command_docker }} run --rm --name matrix-mautrix-han --log-driver=none \ --user={{ matrix_user_uid }}:{{ matrix_user_gid }} \ --cap-drop=ALL \ + --network={{ matrix_docker_network }} \ -v {{ matrix_mautrix_hangouts_config_path }}:/config:z \ -v {{ matrix_mautrix_hangouts_data_path }}:/data:z \ {{ matrix_mautrix_hangouts_docker_image }} \ diff --git a/roles/matrix-bridge-mautrix-signal/defaults/main.yml b/roles/matrix-bridge-mautrix-signal/defaults/main.yml new file mode 100644 index 00000000..962140c9 --- /dev/null +++ b/roles/matrix-bridge-mautrix-signal/defaults/main.yml @@ -0,0 +1,95 @@ +# mautrix-signal is a Matrix <-> Signal bridge +# See: https://github.com/tulir/mautrix-signal + +matrix_mautrix_signal_enabled: true + +# See: https://mau.dev/tulir/mautrix-signal/container_registry +matrix_mautrix_signal_docker_image: "dock.mau.dev/tulir/mautrix-signal:latest" +matrix_mautrix_signal_docker_image_force_pull: "{{ matrix_mautrix_signal_docker_image.endswith(':latest') }}" + +matrix_mautrix_signal_daemon_docker_image: "dock.mau.dev/maunium/signald:latest" +matrix_mautrix_signal_daemon_docker_image_force_pull: "{{ matrix_mautrix_signal_daemon_docker_image.endswith(':latest') }}" + +matrix_mautrix_signal_base_path: "{{ matrix_base_data_path }}/mautrix-signal" +matrix_mautrix_signal_config_path: "{{ matrix_mautrix_signal_base_path }}/bridge" +matrix_mautrix_signal_daemon_path: "{{ matrix_mautrix_signal_base_path }}/signald" + +matrix_mautrix_signal_homeserver_address: '' +matrix_mautrix_signal_homeserver_domain: '' +matrix_mautrix_signal_appservice_address: 'http://matrix-mautrix-signal:29328' + +# Controls whether the matrix-mautrix-signal container exposes its port (tcp/29328 in the container). +# +# Takes an ":" or "" value (e.g. "127.0.0.1:9006"), or empty string to not expose. +matrix_mautrix_signal_container_http_host_bind_port: '' + +# A list of extra arguments to pass to the container +matrix_mautrix_signal_container_extra_arguments: [] + +# List of systemd services that matrix-mautrix-signal.service depends on. +matrix_mautrix_signal_systemd_required_services_list: + - 'docker.service' + - 'matrix-mautrix-signal-daemon.service' + +# List of systemd services that matrix-mautrix-signal.service wants +matrix_mautrix_signal_systemd_wanted_services_list: [] + +# List of systemd services that matrix-mautrix-signal-daemon.service depends on. +matrix_mautrix_signal_daemon_systemd_required_services_list: ['docker.service'] + +# List of systemd services that matrix-mautrix-signal-daemon.service wants +matrix_mautrix_signal_daemon_systemd_wanted_services_list: [] + +matrix_mautrix_signal_appservice_token: '' +matrix_mautrix_signal_homeserver_token: '' + +# Database-related configuration fields +# +# This bridge only supports postgres. +# +matrix_mautrix_signal_database_engine: 'postgres' + +matrix_mautrix_signal_database_username: 'matrix_mautrix_signal' +matrix_mautrix_signal_database_password: 'some-password' +matrix_mautrix_signal_database_hostname: 'matrix-postgres' +matrix_mautrix_signal_database_port: 5432 +matrix_mautrix_signal_database_name: 'matrix_mautrix_signal' + +matrix_mautrix_signal_database_connection_string: 'postgres://{{ matrix_mautrix_signal_database_username }}:{{ matrix_mautrix_signal_database_password }}@{{ matrix_mautrix_signal_database_hostname }}:{{ matrix_mautrix_signal_database_port }}/{{ matrix_mautrix_signal_database_name }}' + +matrix_mautrix_signal_appservice_database: "{{ + { + 'postgres': matrix_mautrix_facebook_database_connection_string, + }[matrix_mautrix_signal_database_engine] + }}" + +# Can be set to enable automatic double-puppeting via Shared Secret Auth (https://github.com/devture/matrix-synapse-shared-secret-auth). +matrix_mautrix_signal_login_shared_secret: '' + +# Default configuration template which covers the generic use case. +# You can customize it by controlling the various variables inside it. +# +# For a more advanced customization, you can extend the default (see `matrix_mautrix_signal_configuration_extension_yaml`) +# or completely replace this variable with your own template. +matrix_mautrix_signal_configuration_yaml: "{{ lookup('template', 'templates/config.yaml.j2') }}" + +matrix_mautrix_signal_configuration_extension_yaml: | + # Your custom YAML configuration goes here. + # This configuration extends the default starting configuration (`matrix_mautrix_signal_configuration_yaml`). + # + # You can override individual variables from the default configuration, or introduce new ones. + # + # If you need something more special, you can take full control by + # completely redefining `matrix_mautrix_signal_configuration_yaml`. + +matrix_mautrix_signal_configuration_extension: "{{ matrix_mautrix_signal_configuration_extension_yaml|from_yaml if matrix_mautrix_signal_configuration_extension_yaml|from_yaml is mapping else {} }}" + +# Holds the final configuration (a combination of the default and its extension). +# You most likely don't need to touch this variable. Instead, see `matrix_mautrix_signal_configuration_yaml`. +matrix_mautrix_signal_configuration: "{{ matrix_mautrix_signal_configuration_yaml|from_yaml|combine(matrix_mautrix_signal_configuration_extension, recursive=True) }}" + +matrix_mautrix_signal_registration_yaml: "{{ lookup('template', 'templates/registration.yaml.j2') }}" + +matrix_mautrix_signal_registration: "{{ matrix_mautrix_signal_registration_yaml|from_yaml }}" + +matrix_mautrix_signal_log_level: 'DEBUG' diff --git a/roles/matrix-bridge-mautrix-signal/tasks/init.yml b/roles/matrix-bridge-mautrix-signal/tasks/init.yml new file mode 100644 index 00000000..c2b37841 --- /dev/null +++ b/roles/matrix-bridge-mautrix-signal/tasks/init.yml @@ -0,0 +1,16 @@ +- set_fact: + matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-mautrix-signal', 'matrix-mautrix-signal-daemon'] }}" + when: matrix_mautrix_signal_enabled|bool + +# If the matrix-synapse role is not used, these variables may not exist. +- set_fact: + matrix_synapse_container_extra_arguments: > + {{ matrix_synapse_container_extra_arguments|default([]) }} + + + ["--mount type=bind,src={{ matrix_mautrix_signal_config_path }}/registration.yaml,dst=/matrix-mautrix-signal-registration.yaml,ro"] + + matrix_synapse_app_service_config_files: > + {{ matrix_synapse_app_service_config_files|default([]) }} + + + {{ ["/matrix-mautrix-signal-registration.yaml"] }} + when: matrix_mautrix_signal_enabled|bool diff --git a/roles/matrix-bridge-mautrix-signal/tasks/main.yml b/roles/matrix-bridge-mautrix-signal/tasks/main.yml new file mode 100644 index 00000000..edca20e6 --- /dev/null +++ b/roles/matrix-bridge-mautrix-signal/tasks/main.yml @@ -0,0 +1,21 @@ +- import_tasks: "{{ role_path }}/tasks/init.yml" + tags: + - always + +- import_tasks: "{{ role_path }}/tasks/validate_config.yml" + when: "run_setup|bool and matrix_mautrix_signal_enabled|bool" + tags: + - setup-all + - setup-mautrix-signal + +- import_tasks: "{{ role_path }}/tasks/setup_install.yml" + when: "run_setup|bool and matrix_mautrix_signal_enabled|bool" + tags: + - setup-all + - setup-mautrix-signal + +- import_tasks: "{{ role_path }}/tasks/setup_uninstall.yml" + when: "run_setup|bool and not matrix_mautrix_signal_enabled|bool" + tags: + - setup-all + - setup-mautrix-signal diff --git a/roles/matrix-bridge-mautrix-signal/tasks/setup_install.yml b/roles/matrix-bridge-mautrix-signal/tasks/setup_install.yml new file mode 100644 index 00000000..29555116 --- /dev/null +++ b/roles/matrix-bridge-mautrix-signal/tasks/setup_install.yml @@ -0,0 +1,72 @@ +--- + +# If the matrix-synapse role is not used, `matrix_synapse_role_executed` won't exist. +# We don't want to fail in such cases. +- name: Fail if matrix-synapse role already executed + fail: + msg: >- + The matrix-bridge-mautrix-signal role needs to execute before the matrix-synapse role. + when: "matrix_synapse_role_executed|default(False)" + +- name: Ensure Mautrix Signal image is pulled + docker_image: + name: "{{ matrix_mautrix_signal_docker_image }}" + source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" + force_source: "{{ matrix_mautrix_signal_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" + force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mautrix_signal_docker_image_force_pull }}" + when: matrix_mautrix_signal_enabled|bool + +- name: Ensure Mautrix Signal Daemon image is pulled + docker_image: + name: "{{ matrix_mautrix_signal_daemon_docker_image }}" + source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" + force_source: "{{ matrix_mautrix_signal_daemon_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" + force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mautrix_signal_docker_image_force_pull }}" + when: matrix_mautrix_signal_enabled|bool + +- name: Ensure Mautrix Signal paths exist + file: + path: "{{ item }}" + state: directory + mode: 0750 + owner: "{{ matrix_user_username }}" + group: "{{ matrix_user_groupname }}" + with_items: + - "{{ matrix_mautrix_signal_base_path }}" + - "{{ matrix_mautrix_signal_config_path }}" + - "{{ matrix_mautrix_signal_daemon_path }}" + +- name: Ensure mautrix-signal config.yaml installed + copy: + content: "{{ matrix_mautrix_signal_configuration|to_nice_yaml }}" + dest: "{{ matrix_mautrix_signal_config_path }}/config.yaml" + mode: 0644 + owner: "{{ matrix_user_username }}" + group: "{{ matrix_user_groupname }}" + +- name: Ensure mautrix-signal registration.yaml installed + copy: + content: "{{ matrix_mautrix_signal_registration|to_nice_yaml }}" + dest: "{{ matrix_mautrix_signal_config_path }}/registration.yaml" + mode: 0644 + owner: "{{ matrix_user_username }}" + group: "{{ matrix_user_groupname }}" + +- name: Ensure matrix-mautrix-signal-daemon.service installed + template: + src: "{{ role_path }}/templates/systemd/matrix-mautrix-signal-daemon.service.j2" + dest: "{{ matrix_systemd_path }}/matrix-mautrix-signal-daemon.service" + mode: 0644 + register: matrix_mautrix_signal_daemon_systemd_service_result + +- name: Ensure matrix-mautrix-signal.service installed + template: + src: "{{ role_path }}/templates/systemd/matrix-mautrix-signal.service.j2" + dest: "{{ matrix_systemd_path }}/matrix-mautrix-signal.service" + mode: 0644 + register: matrix_mautrix_signal_systemd_service_result + +- name: Ensure systemd reloaded after matrix-mautrix-signal.service installation + service: + daemon_reload: yes + when: "matrix_mautrix_signal_systemd_service_result.changed or matrix_mautrix_signal_daemon_systemd_service_result.changed" diff --git a/roles/matrix-bridge-mautrix-signal/tasks/setup_uninstall.yml b/roles/matrix-bridge-mautrix-signal/tasks/setup_uninstall.yml new file mode 100644 index 00000000..2ca6a9a9 --- /dev/null +++ b/roles/matrix-bridge-mautrix-signal/tasks/setup_uninstall.yml @@ -0,0 +1,45 @@ +--- + +# Signal daemon service +- name: Check existence of matrix-mautrix-signal-daemon service + stat: + path: "{{ matrix_systemd_path }}/matrix-mautrix-signal-daemon.service" + register: matrix_mautrix_signal_daemon_service_stat + +- name: Ensure matrix-mautrix-signal-daemon is stopped + service: + name: matrix-mautrix-signal-daemon + state: stopped + daemon_reload: yes + when: "matrix_mautrix_signal_daemon_service_stat.stat.exists" + +- name: Ensure matrix-mautrix-signal-daemon.service doesn't exist + file: + path: "{{ matrix_systemd_path }}/matrix-mautrix-signal-daemon.service" + state: absent + when: "matrix_mautrix_signal_daemon_service_stat.stat.exists" + +# Bridge service +- name: Check existence of matrix-mautrix-signal service + stat: + path: "{{ matrix_systemd_path }}/matrix-mautrix-signal.service" + register: matrix_mautrix_signal_service_stat + +- name: Ensure matrix-mautrix-signal is stopped + service: + name: matrix-mautrix-signal + state: stopped + daemon_reload: yes + when: "matrix_mautrix_signal_service_stat.stat.exists" + +- name: Ensure matrix-mautrix-signal.service doesn't exist + file: + path: "{{ matrix_systemd_path }}/matrix-mautrix-signal.service" + state: absent + when: "matrix_mautrix_signal_service_stat.stat.exists" + +# All services +- name: Ensure systemd reloaded after matrix-mautrix-signal_X.service removal + service: + daemon_reload: yes + when: "matrix_mautrix_signal_service_stat.stat.exists or matrix_mautrix_signal_daemon_service_stat.stat.exists" diff --git a/roles/matrix-bridge-mautrix-signal/tasks/validate_config.yml b/roles/matrix-bridge-mautrix-signal/tasks/validate_config.yml new file mode 100644 index 00000000..100af3f8 --- /dev/null +++ b/roles/matrix-bridge-mautrix-signal/tasks/validate_config.yml @@ -0,0 +1,28 @@ +--- + +- name: Fail if required settings not defined + fail: + msg: >- + You need to define a required configuration setting (`{{ item }}`). + when: "vars[item] == ''" + with_items: + - "matrix_mautrix_signal_homeserver_domain" + - "matrix_mautrix_signal_homeserver_address" + - "matrix_mautrix_signal_homeserver_token" + - "matrix_mautrix_signal_appservice_token" + +- name: (Deprecation) Catch and report renamed Signal variables + fail: + msg: >- + Your configuration contains a variable, which now has a different name. + Please change your configuration to rename the variable (`{{ item.old }}` -> `{{ item.new }}`). + when: "item.old in vars" + with_items: + - {'old': 'matrix_mautrix_signal_container_exposed_port_number', 'new': ''} + - {'old': 'matrix_mautrix_signal_db_user', 'new': 'matrix_mautrix_signal_database_username'} + - {'old': 'matrix_mautrix_signal_db_password', 'new': 'matrix_mautrix_signal_database_password'} + - {'old': 'matrix_mautrix_signal_db_database', 'new': 'matrix_mautrix_signal_database_name'} + - {'old': 'matrix_mautrix_signal_db_host', 'new': 'matrix_mautrix_signal_database_hostname'} + - {'old': 'matrix_mautrix_signal_db_port', 'new': 'matrix_mautrix_signal_database_port'} + - {'old': 'matrix_mautrix_signal_db_url', 'new': 'matrix_mautrix_signal_database_connection_string'} + - {'old': 'matrix_mautrix_signal_configuration_permissions', 'new': ''} diff --git a/roles/matrix-bridge-mautrix-signal/templates/config.yaml.j2 b/roles/matrix-bridge-mautrix-signal/templates/config.yaml.j2 new file mode 100644 index 00000000..28fff6f0 --- /dev/null +++ b/roles/matrix-bridge-mautrix-signal/templates/config.yaml.j2 @@ -0,0 +1,202 @@ +#jinja2: lstrip_blocks: "True" +# Homeserver details +homeserver: + # The address that this appservice can use to connect to the homeserver. + address: {{ matrix_mautrix_signal_homeserver_address }} + # The domain of the homeserver (for MXIDs, etc). + domain: {{ matrix_mautrix_signal_homeserver_domain }} + # Whether or not to verify the SSL certificate of the homeserver. + # Only applies if address starts with https:// + verify_ssl: true + asmux: false + +# Application service host/registration related details +# Changing these values requires regeneration of the registration. +appservice: + # The address that the homeserver can use to connect to this appservice. + address: {{ matrix_mautrix_signal_appservice_address }} + # When using https:// the TLS certificate and key files for the address. + tls_cert: false + tls_key: false + + # The hostname and port where this appservice should listen. + hostname: 0.0.0.0 + port: 29328 + # The maximum body size of appservice API requests (from the homeserver) in mebibytes + # Usually 1 is enough, but on high-traffic bridges you might need to increase this to avoid 413s + max_body_size: 1 + + # The full URI to the database. Only Postgres is currently supported. + database: {{ matrix_mautrix_signal_database_connection_string }} + + # Provisioning API part of the web server for automated portal creation and fetching information. + # Used by things like mautrix-manager (https://github.com/tulir/mautrix-manager). + provisioning: + # Whether or not the provisioning API should be enabled. + enabled: true + # The prefix to use in the provisioning API endpoints. + prefix: /_matrix/provision/v1 + # The shared secret to authorize users of the API. + # Set to "generate" to generate and save a new token. + shared_secret: generate + + # The unique ID of this appservice. + id: signal + # Username of the appservice bot. + bot_username: signalbot + # Display name and avatar for bot. Set to "remove" to remove display name/avatar, leave empty + # to leave display name/avatar as-is. + bot_displayname: Signal bridge bot + bot_avatar: mxc://maunium.net/wPJgTQbZOtpBFmDNkiNEMDUp + + # Community ID for bridged users (changes registration file) and rooms. + # Must be created manually. + # + # Example: "+signal:example.com". Set to false to disable. + community_id: false + + # Authentication tokens for AS <-> HS communication. + as_token: "{{ matrix_mautrix_signal_appservice_token }}" + hs_token: "{{ matrix_mautrix_signal_homeserver_token }}" + +# Prometheus telemetry config. Requires prometheus-client to be installed. +metrics: + enabled: false + listen_port: 8000 + +signal: + # Path to signald unix socket + socket_path: /signald/signald.sock + # Directory for temp files when sending files to Signal. This should be an + # absolute path that signald can read. For attachments in the other direction, + # make sure signald is configured to use an absolute path as the data directory. + outgoing_attachment_dir: /signald/attachments + # Directory where signald stores avatars for groups. + avatar_dir: /signald/avatars + # Directory where signald stores auth data. Used to delete data when logging out. + data_dir: /signald/data + # Whether or not message attachments should be removed from disk after they're bridged. + remove_file_after_handling: true + +# Bridge config +bridge: + # Localpart template of MXIDs for Signal users. + # {userid} is replaced with an identifier for the Signal user. + username_template: "signal_{userid}" + # Displayname template for Signal users. + # {displayname} is replaced with the displayname of the Signal user, which is the first + # available variable in displayname_preference. The variables in displayname_preference + # can also be used here directly. + displayname_template: "{displayname} (Signal)" + # Whether or not contact list displaynames should be used. + # Using this isn't recommended on multi-user instances. + allow_contact_list_name_updates: false + # Available variables: full_name, first_name, last_name, phone, uuid + displayname_preference: + - full_name + - phone + + # Whether or not to create portals for all groups on login/connect. + autocreate_group_portal: true + # Whether or not to create portals for all contacts on login/connect. + autocreate_contact_portal: false + # Whether or not to use /sync to get read receipts and typing notifications + # when double puppeting is enabled + sync_with_custom_puppets: true + # Whether or not to update the m.direct account data event when double puppeting is enabled. + # Note that updating the m.direct event is not atomic (except with mautrix-asmux) + # and is therefore prone to race conditions. + sync_direct_chat_list: false + # Allow using double puppeting from any server with a valid client .well-known file. + double_puppet_allow_discovery: false + # Servers to allow double puppeting from, even if double_puppet_allow_discovery is false. + double_puppet_server_map: {} + # Shared secret for https://github.com/devture/matrix-synapse-shared-secret-auth + # + # If set, custom puppets will be enabled automatically for local users + # instead of users having to find an access token and run `login-matrix` + # manually. + # If using this for other servers than the bridge's server, + # you must also set the URL in the double_puppet_server_map. + login_shared_secret_map: + {{ matrix_mautrix_signal_homeserver_domain }}: {{ matrix_mautrix_signal_login_shared_secret|to_json }} + # Whether or not created rooms should have federation enabled. + # If false, created portal rooms will never be federated. + federate_rooms: true + # End-to-bridge encryption support options. These require matrix-nio to be installed with pip + # and login_shared_secret to be configured in order to get a device for the bridge bot. + # + # Additionally, https://github.com/matrix-org/synapse/pull/5758 is required if using a normal + # application service. + encryption: + # Allow encryption, work in group chat rooms with e2ee enabled + allow: false + # Default to encryption, force-enable encryption in all portals the bridge creates + # This will cause the bridge bot to be in private chats for the encryption to work properly. + default: false + # Options for automatic key sharing. + key_sharing: + # Enable key sharing? If enabled, key requests for rooms where users are in will be fulfilled. + # You must use a client that supports requesting keys from other users to use this feature. + allow: false + # Require the requesting device to have a valid cross-signing signature? + # This doesn't require that the bridge has verified the device, only that the user has verified it. + # Not yet implemented. + require_cross_signing: false + # Require devices to be verified by the bridge? + # Verification by the bridge is not yet implemented. + require_verification: true + # Whether or not to explicitly set the avatar and room name for private + # chat portal rooms. This will be implicitly enabled if encryption.default is true. + private_chat_portal_meta: false + # Whether or not the bridge should send a read receipt from the bridge bot when a message has + # been sent to Signal. This let's you check manually whether the bridge is receiving your + # messages. + # Note that this is not related to Signal delivery receipts. + delivery_receipts: false + # Whether or not delivery errors should be reported as messages in the Matrix room. (not yet implemented) + delivery_error_reports: false + # Set this to true to tell the bridge to re-send m.bridge events to all rooms on the next run. + # This field will automatically be changed back to false after it, + # except if the config file is not writable. + resend_bridge_info: false + + # The prefix for commands. Only required in non-management rooms. + command_prefix: "!signal" + + # Permissions for using the bridge. + # Permitted values: + # user - Use the bridge with puppeting. + # admin - Use and administrate the bridge. + # Permitted keys: + # * - All Matrix users + # domain - All users on that homeserver + # mxid - Specific user + permissions: + '{{ matrix_mautrix_signal_homeserver_domain }}': user + + +# Python logging configuration. +# +# See section 16.7.2 of the Python documentation for more info: +# https://docs.python.org/3.6/library/logging.config.html#configuration-dictionary-schema +logging: + version: 1 + formatters: + colored: + (): mautrix_signal.util.ColorFormatter + format: "[%(asctime)s] [%(levelname)s@%(name)s] %(message)s" + normal: + format: "[%(asctime)s] [%(levelname)s@%(name)s] %(message)s" + handlers: + console: + class: logging.StreamHandler + formatter: colored + loggers: + mau: + level: {{ matrix_mautrix_signal_log_level }} + aiohttp: + level: INFO + root: + level: {{ matrix_mautrix_signal_log_level }} + handlers: [console] diff --git a/roles/matrix-bridge-mautrix-signal/templates/registration.yaml.j2 b/roles/matrix-bridge-mautrix-signal/templates/registration.yaml.j2 new file mode 100644 index 00000000..45cc5a0f --- /dev/null +++ b/roles/matrix-bridge-mautrix-signal/templates/registration.yaml.j2 @@ -0,0 +1,14 @@ +#jinja2: lstrip_blocks: "True" +id: signal +as_token: "{{ matrix_mautrix_signal_appservice_token }}" +hs_token: "{{ matrix_mautrix_signal_homeserver_token }}" +namespaces: + users: + - exclusive: true + regex: '^@signal_.+:{{ matrix_mautrix_signal_homeserver_domain|regex_escape }}$' + aliases: + - exclusive: true + regex: '^#signal_.+:{{ matrix_mautrix_signal_homeserver_domain|regex_escape }}$' +url: {{ matrix_mautrix_signal_appservice_address }} +sender_localpart: signalbot +rate_limited: false diff --git a/roles/matrix-bridge-mautrix-signal/templates/systemd/matrix-mautrix-signal-daemon.service.j2 b/roles/matrix-bridge-mautrix-signal/templates/systemd/matrix-mautrix-signal-daemon.service.j2 new file mode 100644 index 00000000..d8952992 --- /dev/null +++ b/roles/matrix-bridge-mautrix-signal/templates/systemd/matrix-mautrix-signal-daemon.service.j2 @@ -0,0 +1,38 @@ +#jinja2: lstrip_blocks: "True" +[Unit] +Description=Matrix Mautrix Signal daemon + +{% for service in matrix_mautrix_signal_daemon_systemd_required_services_list %} +Requires={{ service }} +After={{ service }} +{% endfor %} + +{% for service in matrix_mautrix_signal_daemon_systemd_wanted_services_list %} +Wants={{ service }} +{% endfor %} + +[Service] +Type=simple + +ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mautrix-signal-daemon +ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mautrix-signal-daemon + +# Intentional delay, so that the homeserver (we likely depend on) can manage to start. +ExecStartPre={{ matrix_host_command_sleep }} 5 + +ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-mautrix-signal-daemon \ + --log-driver=none \ + --user={{ matrix_user_uid }}:{{ matrix_user_gid }} \ + --network={{ matrix_docker_network }} \ + -v {{ matrix_mautrix_signal_daemon_path }}:/signald:z \ + {{ matrix_mautrix_signal_daemon_docker_image }} + +ExecStop=-{{ matrix_host_command_docker }} kill matrix-mautrix-signal-daemon +ExecStop=-{{ matrix_host_command_docker }} rm matrix-mautrix-signal-daemon + +Restart=always +RestartSec=30 +SyslogIdentifier=matrix-mautrix-signal-daemon + +[Install] +WantedBy=multi-user.target diff --git a/roles/matrix-bridge-mautrix-signal/templates/systemd/matrix-mautrix-signal.service.j2 b/roles/matrix-bridge-mautrix-signal/templates/systemd/matrix-mautrix-signal.service.j2 new file mode 100644 index 00000000..c1c835a2 --- /dev/null +++ b/roles/matrix-bridge-mautrix-signal/templates/systemd/matrix-mautrix-signal.service.j2 @@ -0,0 +1,44 @@ +#jinja2: lstrip_blocks: "True" +[Unit] +Description=Matrix Mautrix Signal server + +{% for service in matrix_mautrix_signal_systemd_required_services_list %} +Requires={{ service }} +After={{ service }} +{% endfor %} + +{% for service in matrix_mautrix_signal_systemd_wanted_services_list %} +Wants={{ service }} +{% endfor %} + +[Service] +Type=simple +ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-mautrix-signal +ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-mautrix-signal + +# Intentional delay, so that the homeserver (we likely depend on) can manage to start. +ExecStartPre={{ matrix_host_command_sleep }} 5 + +ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-mautrix-signal \ + --log-driver=none \ + --network={{ matrix_docker_network }} \ + {% if matrix_mautrix_signal_container_http_host_bind_port %} + -p {{ matrix_mautrix_signal_container_http_host_bind_port }}:29328 \ + {% endif %} + -v {{ matrix_mautrix_signal_daemon_path }}:/signald:z \ + -v {{ matrix_mautrix_signal_config_path }}:/data:z \ + {% for arg in matrix_mautrix_signal_container_extra_arguments %} + {{ arg }} \ + {% endfor %} + {{ matrix_mautrix_signal_docker_image }} \ + python3 -m mautrix_signal -c /data/config.yaml + +ExecStop=-{{ matrix_host_command_docker }} kill matrix-mautrix-signal +ExecStop=-{{ matrix_host_command_docker }} rm matrix-mautrix-signal + +Restart=always +RestartSec=30 +SyslogIdentifier=matrix-mautrix-signal + +[Install] +WantedBy=multi-user.target diff --git a/roles/matrix-bridge-mautrix-telegram/defaults/main.yml b/roles/matrix-bridge-mautrix-telegram/defaults/main.yml index b6f4ef5b..3f81617a 100644 --- a/roles/matrix-bridge-mautrix-telegram/defaults/main.yml +++ b/roles/matrix-bridge-mautrix-telegram/defaults/main.yml @@ -43,6 +43,35 @@ matrix_mautrix_telegram_systemd_wanted_services_list: [] matrix_mautrix_telegram_appservice_token: '' matrix_mautrix_telegram_homeserver_token: '' + +# Database-related configuration fields. +# +# To use SQLite, stick to these defaults. +# +# To use Postgres: +# - change the engine (`matrix_mautrix_telegram_database_engine: 'postgres'`) +# - adjust your database credentials via the `matrix_mautrix_telegram_postgres_*` variables +matrix_mautrix_telegram_database_engine: 'sqlite' + +matrix_mautrix_telegram_sqlite_database_path_local: "{{ matrix_mautrix_telegram_data_path }}/mautrix-telegram.db" +matrix_mautrix_telegram_sqlite_database_path_in_container: "/data/mautrix-telegram.db" + +matrix_mautrix_telegram_database_username: 'matrix_mautrix_telegram' +matrix_mautrix_telegram_database_password: 'some-password' +matrix_mautrix_telegram_database_hostname: 'matrix-postgres' +matrix_mautrix_telegram_database_port: 5432 +matrix_mautrix_telegram_database_name: 'matrix_mautrix_telegram' + +matrix_mautrix_telegram_database_connection_string: 'postgres://{{ matrix_mautrix_telegram_database_username }}:{{ matrix_mautrix_telegram_database_password }}@{{ matrix_mautrix_telegram_database_hostname }}:{{ matrix_mautrix_telegram_database_port }}/{{ matrix_mautrix_telegram_database_name }}' + +matrix_mautrix_telegram_appservice_database: "{{ + { + 'sqlite': ('sqlite:///' + matrix_mautrix_telegram_sqlite_database_path_in_container), + 'postgres': matrix_mautrix_telegram_database_connection_string, + }[matrix_mautrix_telegram_database_engine] +}}" + + # Can be set to enable automatic double-puppeting via Shared Secret Auth (https://github.com/devture/matrix-synapse-shared-secret-auth). matrix_mautrix_telegram_login_shared_secret: '' diff --git a/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml b/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml index 9dc38ec2..e9a93c72 100644 --- a/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml +++ b/roles/matrix-bridge-mautrix-telegram/tasks/setup_install.yml @@ -8,6 +8,32 @@ The matrix-bridge-mautrix-telegram role needs to execute before the matrix-synapse role. when: "matrix_synapse_role_executed|default(False)" +- set_fact: + matrix_mautrix_telegram_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_mautrix_telegram_sqlite_database_path_local }}" + register: matrix_mautrix_telegram_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_mautrix_telegram_sqlite_database_path_local }}" + dst: "{{ matrix_mautrix_telegram_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_mautrix_telegram_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-mautrix-telegram.service'] + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_mautrix_telegram_requires_restart: true + when: "matrix_mautrix_telegram_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_mautrix_telegram_database_engine == 'postgres'" + - name: Ensure Mautrix Telegram image is pulled docker_image: name: "{{ matrix_mautrix_telegram_docker_image }}" @@ -71,3 +97,9 @@ service: daemon_reload: yes when: "matrix_mautrix_telegram_systemd_service_result.changed" + +- name: Ensure matrix-mautrix-telegram.service restarted, if necessary + service: + name: "matrix-mautrix-telegram.service" + state: restarted + when: "matrix_mautrix_telegram_requires_restart|bool" diff --git a/roles/matrix-bridge-mautrix-telegram/templates/config.yaml.j2 b/roles/matrix-bridge-mautrix-telegram/templates/config.yaml.j2 index 1a4ac43e..52efba02 100644 --- a/roles/matrix-bridge-mautrix-telegram/templates/config.yaml.j2 +++ b/roles/matrix-bridge-mautrix-telegram/templates/config.yaml.j2 @@ -27,7 +27,7 @@ appservice: # Format examples: # SQLite: sqlite:///filename.db # Postgres: postgres://username:password@hostname/dbname - database: sqlite:////data/mautrix-telegram.db + database: {{ matrix_mautrix_telegram_appservice_database|to_json }} # Public part of web server for out-of-Matrix interaction with the bridge. # Used for things like login if the user wants to make sure the 2FA password isn't stored in diff --git a/roles/matrix-bridge-mautrix-telegram/templates/systemd/matrix-mautrix-telegram.service.j2 b/roles/matrix-bridge-mautrix-telegram/templates/systemd/matrix-mautrix-telegram.service.j2 index c0fb8938..d61cdaa4 100644 --- a/roles/matrix-bridge-mautrix-telegram/templates/systemd/matrix-mautrix-telegram.service.j2 +++ b/roles/matrix-bridge-mautrix-telegram/templates/systemd/matrix-mautrix-telegram.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_mautrix_telegram_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-bridge-mautrix-whatsapp/defaults/main.yml b/roles/matrix-bridge-mautrix-whatsapp/defaults/main.yml index e9929f96..beda6d7d 100644 --- a/roles/matrix-bridge-mautrix-whatsapp/defaults/main.yml +++ b/roles/matrix-bridge-mautrix-whatsapp/defaults/main.yml @@ -27,6 +27,42 @@ matrix_mautrix_whatsapp_systemd_wanted_services_list: [] matrix_mautrix_whatsapp_appservice_token: '' matrix_mautrix_whatsapp_homeserver_token: '' + +# Database-related configuration fields. +# +# To use SQLite, stick to these defaults. +# +# To use Postgres: +# - change the engine (`matrix_mautrix_whatsapp_database_engine: 'postgres'`) +# - adjust your database credentials via the `matrix_mautrix_whatsapp_postgres_*` variables +matrix_mautrix_whatsapp_database_engine: 'sqlite' + +matrix_mautrix_whatsapp_sqlite_database_path_local: "{{ matrix_mautrix_whatsapp_data_path }}/mautrix-whatsapp.db" +matrix_mautrix_whatsapp_sqlite_database_path_in_container: "/data/mautrix-whatsapp.db" + +matrix_mautrix_whatsapp_database_username: 'matrix_mautrix_whatsapp' +matrix_mautrix_whatsapp_database_password: 'some-password' +matrix_mautrix_whatsapp_database_hostname: 'matrix-postgres' +matrix_mautrix_whatsapp_database_port: 5432 +matrix_mautrix_whatsapp_database_name: 'matrix_mautrix_whatsapp' + +matrix_mautrix_whatsapp_database_connection_string: 'postgresql://{{ matrix_mautrix_whatsapp_database_username }}:{{ matrix_mautrix_whatsapp_database_password }}@{{ matrix_mautrix_whatsapp_database_hostname }}:{{ matrix_mautrix_whatsapp_database_port }}/{{ matrix_mautrix_whatsapp_database_name }}?sslmode=disable' + +matrix_mautrix_whatsapp_appservice_database_type: "{{ + { + 'sqlite': 'sqlite3', + 'postgres':'postgres', + }[matrix_mautrix_whatsapp_database_engine] +}}" + +matrix_mautrix_whatsapp_appservice_database_uri: "{{ + { + 'sqlite': matrix_mautrix_whatsapp_sqlite_database_path_in_container, + 'postgres': matrix_mautrix_whatsapp_database_connection_string, + }[matrix_mautrix_whatsapp_database_engine] +}}" + + # Can be set to enable automatic double-puppeting via Shared Secret Auth (https://github.com/devture/matrix-synapse-shared-secret-auth). matrix_mautrix_whatsapp_login_shared_secret: '' diff --git a/roles/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml b/roles/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml index 6bd87bbe..1cfa60f8 100644 --- a/roles/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml +++ b/roles/matrix-bridge-mautrix-whatsapp/tasks/setup_install.yml @@ -8,6 +8,32 @@ The matrix-bridge-mautrix-whatsapp role needs to execute before the matrix-synapse role. when: "matrix_synapse_role_executed|default(False)" +- set_fact: + matrix_mautrix_whatsapp_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_mautrix_whatsapp_sqlite_database_path_local }}" + register: matrix_mautrix_whatsapp_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_mautrix_whatsapp_sqlite_database_path_local }}" + dst: "{{ matrix_mautrix_whatsapp_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_mautrix_whatsapp_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-mautrix-whatsapp.service'] + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_mautrix_whatsapp_requires_restart: true + when: "matrix_mautrix_whatsapp_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_mautrix_whatsapp_database_engine == 'postgres'" + - name: Ensure Mautrix Whatsapp image is pulled docker_image: name: "{{ matrix_mautrix_whatsapp_docker_image }}" @@ -26,12 +52,12 @@ - "{{ matrix_mautrix_whatsapp_base_path }}" - "{{ matrix_mautrix_whatsapp_config_path }}" - "{{ matrix_mautrix_whatsapp_data_path }}" - + - name: Check if an old database file exists stat: path: "{{ matrix_mautrix_whatsapp_base_path }}/mautrix-whatsapp.db" register: matrix_mautrix_whatsapp_stat_database - + - name: Check if an old matrix state file exists stat: path: "{{ matrix_mautrix_whatsapp_base_path }}/mx-state.json" @@ -48,7 +74,7 @@ - name: (Data relocation) Move mautrix-whatsapp database file to ./data directory command: "mv {{ matrix_mautrix_whatsapp_base_path }}/mautrix-whatsapp.db {{ matrix_mautrix_whatsapp_data_path }}/mautrix-whatsapp.db" when: "matrix_mautrix_whatsapp_stat_database.stat.exists" - + - name: (Data relocation) Move mautrix-whatsapp mx-state file to ./data directory command: "mv {{ matrix_mautrix_whatsapp_base_path }}/mx-state.json {{ matrix_mautrix_whatsapp_data_path }}/mx-state.json" when: "matrix_mautrix_whatsapp_stat_mx_state.stat.exists" @@ -80,3 +106,9 @@ service: daemon_reload: yes when: "matrix_mautrix_whatsapp_systemd_service_result.changed" + +- name: Ensure matrix-mautrix-whatsapp.service restarted, if necessary + service: + name: "matrix-mautrix-whatsapp.service" + state: restarted + when: "matrix_mautrix_whatsapp_requires_restart|bool" diff --git a/roles/matrix-bridge-mautrix-whatsapp/templates/config.yaml.j2 b/roles/matrix-bridge-mautrix-whatsapp/templates/config.yaml.j2 index a527a188..89216695 100644 --- a/roles/matrix-bridge-mautrix-whatsapp/templates/config.yaml.j2 +++ b/roles/matrix-bridge-mautrix-whatsapp/templates/config.yaml.j2 @@ -19,11 +19,11 @@ appservice: # Database config. database: # The database type. "sqlite3" and "postgres" are supported. - type: sqlite3 + type: {{ matrix_mautrix_whatsapp_appservice_database_type|to_json }} # The database URI. # SQLite: File name is enough. https://github.com/mattn/go-sqlite3#connection-string # Postgres: Connection string. For example, postgres://user:password@host/database - uri: mautrix-whatsapp.db + uri: {{ matrix_mautrix_whatsapp_appservice_database_uri|to_json }} # Maximum number of connections. Mostly relevant for Postgres. max_open_conns: 20 max_idle_conns: 2 diff --git a/roles/matrix-bridge-mautrix-whatsapp/templates/systemd/matrix-mautrix-whatsapp.service.j2 b/roles/matrix-bridge-mautrix-whatsapp/templates/systemd/matrix-mautrix-whatsapp.service.j2 index ac2b961e..972d4e01 100644 --- a/roles/matrix-bridge-mautrix-whatsapp/templates/systemd/matrix-mautrix-whatsapp.service.j2 +++ b/roles/matrix-bridge-mautrix-whatsapp/templates/systemd/matrix-mautrix-whatsapp.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_mautrix_whatsapp_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-bridge-mx-puppet-discord/defaults/main.yml b/roles/matrix-bridge-mx-puppet-discord/defaults/main.yml index 268bbf8f..97b20313 100644 --- a/roles/matrix-bridge-mx-puppet-discord/defaults/main.yml +++ b/roles/matrix-bridge-mx-puppet-discord/defaults/main.yml @@ -55,6 +55,20 @@ matrix_mx_puppet_discord_homeserver_token: '' # Can be set to enable automatic double-puppeting via Shared Secret Auth (https://github.com/devture/matrix-synapse-shared-secret-auth). matrix_mx_puppet_discord_login_shared_secret: '' +# Database configuration +matrix_mx_puppet_discord_database_engine: 'sqlite' + +matrix_mx_puppet_discord_sqlite_database_path_local: "{{ matrix_mx_puppet_discord_data_path }}/database.db" +matrix_mx_puppet_discord_sqlite_database_path_in_container: "/data/database.db" + +matrix_mx_puppet_discord_database_username: matrix_mx_puppet_discord +matrix_mx_puppet_discord_database_password: ~ +matrix_mx_puppet_discord_database_hostname: 'matrix-postgres' +matrix_mx_puppet_discord_database_port: 5432 +matrix_mx_puppet_discord_database_name: matrix_mx_puppet_discord + +matrix_mx_puppet_discord_database_connection_string: 'postgresql://{{ matrix_mx_puppet_discord_database_username }}:{{ matrix_mx_puppet_discord_database_password }}@{{ matrix_mx_puppet_discord_database_hostname }}:{{ matrix_mx_puppet_discord_database_port }}/{{ matrix_mx_puppet_discord_database_name }}?sslmode=disable' + # Default configuration template which covers the generic use case. # You can customize it by controlling the various variables inside it. # diff --git a/roles/matrix-bridge-mx-puppet-discord/tasks/setup_install.yml b/roles/matrix-bridge-mx-puppet-discord/tasks/setup_install.yml index c3f2b01f..c7865e98 100644 --- a/roles/matrix-bridge-mx-puppet-discord/tasks/setup_install.yml +++ b/roles/matrix-bridge-mx-puppet-discord/tasks/setup_install.yml @@ -8,14 +8,6 @@ The matrix-bridge-mx-puppet-discord role needs to execute before the matrix-synapse role. when: "matrix_synapse_role_executed|default(False)" -- name: Ensure MX Puppet Discord image is pulled - docker_image: - name: "{{ matrix_mx_puppet_discord_docker_image }}" - source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" - force_source: "{{ matrix_mx_puppet_discord_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" - force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mx_puppet_discord_docker_image_force_pull }}" - when: matrix_mx_puppet_discord_enabled|bool and not matrix_mx_puppet_discord_container_image_self_build - - name: Ensure MX Puppet Discord paths exist file: path: "{{ item.path }}" @@ -30,6 +22,57 @@ - { path: "{{ matrix_mx_puppet_discord_docker_src_files_path }}", when: "{{ matrix_mx_puppet_discord_container_image_self_build }}" } when: matrix_mx_puppet_discord_enabled|bool and item.when|bool +- name: Check if an old database file already exists + stat: + path: "{{ matrix_mx_puppet_discord_base_path }}/database.db" + register: matrix_mx_puppet_discord_stat_database + +- block: + - name: (Data relocation) Ensure matrix-mx-puppet-discord.service is stopped + service: + name: matrix-mx-puppet-discord + state: stopped + daemon_reload: yes + failed_when: False + + - name: (Data relocation) Move mx-puppet-discord database file to ./data directory + command: "mv {{ matrix_mx_puppet_discord_base_path }}/database.db {{ matrix_mx_puppet_discord_data_path }}/database.db" + when: "matrix_mx_puppet_discord_stat_database.stat.exists" + +- set_fact: + matrix_mx_puppet_discord_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_mx_puppet_discord_sqlite_database_path_local }}" + register: matrix_mx_puppet_discord_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_mx_puppet_discord_sqlite_database_path_local }}" + dst: "{{ matrix_mx_puppet_discord_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_mx_puppet_discord_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-mx-puppet-discord.service'] + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_mx_puppet_discord_requires_restart: true + when: "matrix_mx_puppet_discord_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_mx_puppet_discord_database_engine == 'postgres'" + +- name: Ensure MX Puppet Discord image is pulled + docker_image: + name: "{{ matrix_mx_puppet_discord_docker_image }}" + source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" + force_source: "{{ matrix_mx_puppet_discord_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" + force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mx_puppet_discord_docker_image_force_pull }}" + when: matrix_mx_puppet_discord_enabled|bool and not matrix_mx_puppet_discord_container_image_self_build + - name: Ensure MX Puppet Discord repository is present on self build git: repo: "{{ matrix_mx_puppet_discord_container_image_self_build_repo }}" @@ -49,23 +92,6 @@ pull: yes when: "matrix_mx_puppet_discord_enabled|bool and matrix_mx_puppet_discord_container_image_self_build|bool" -- name: Check if an old database file already exists - stat: - path: "{{ matrix_mx_puppet_discord_base_path }}/database.db" - register: matrix_mx_puppet_discord_stat_database - -- name: (Data relocation) Ensure matrix-mx-puppet-discord.service is stopped - service: - name: matrix-mx-puppet-discord - state: stopped - daemon_reload: yes - failed_when: false - when: "matrix_mx_puppet_discord_stat_database.stat.exists" - -- name: (Data relocation) Move mx-puppet-discord database file to ./data directory - command: "mv {{ matrix_mx_puppet_discord_base_path }}/database.db {{ matrix_mx_puppet_discord_data_path }}/database.db" - when: "matrix_mx_puppet_discord_stat_database.stat.exists" - - name: Ensure mx-puppet-discord config.yaml installed copy: content: "{{ matrix_mx_puppet_discord_configuration|to_nice_yaml }}" @@ -93,3 +119,9 @@ service: daemon_reload: yes when: "matrix_mx_puppet_discord_systemd_service_result.changed" + +- name: Ensure matrix-mx-puppet-discord.service restarted, if necessary + service: + name: "matrix-mx-puppet-discord.service" + state: restarted + when: "matrix_mx_puppet_discord_requires_restart|bool" diff --git a/roles/matrix-bridge-mx-puppet-discord/templates/config.yaml.j2 b/roles/matrix-bridge-mx-puppet-discord/templates/config.yaml.j2 index 2c703796..1f4548d8 100644 --- a/roles/matrix-bridge-mx-puppet-discord/templates/config.yaml.j2 +++ b/roles/matrix-bridge-mx-puppet-discord/templates/config.yaml.j2 @@ -100,15 +100,18 @@ namePatterns: group: :name database: +{% if matrix_mx_puppet_discord_database_engine == 'sqlite' %} + # Use SQLite3 as a database backend + # The name of the database file + filename: {{ matrix_mx_puppet_discord_sqlite_database_path_in_container|to_json }} +{% else %} # Use Postgres as a database backend # If set, will be used instead of SQLite3 # Connection string to connect to the Postgres instance # with username "user", password "pass", host "localhost" and database name "dbname". # Modify each value as necessary - #connString: "postgres://user:pass@localhost/dbname?sslmode=disable" - # Use SQLite3 as a database backend - # The name of the database file - filename: /data/database.db + connString: {{ matrix_mx_puppet_discord_database_connection_string|to_json }} +{% endif %} logging: # Log level of console output diff --git a/roles/matrix-bridge-mx-puppet-discord/templates/systemd/matrix-mx-puppet-discord.service.j2 b/roles/matrix-bridge-mx-puppet-discord/templates/systemd/matrix-mx-puppet-discord.service.j2 index 78737a17..36a4da8b 100644 --- a/roles/matrix-bridge-mx-puppet-discord/templates/systemd/matrix-mx-puppet-discord.service.j2 +++ b/roles/matrix-bridge-mx-puppet-discord/templates/systemd/matrix-mx-puppet-discord.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_mx_puppet_discord_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-bridge-mx-puppet-instagram/defaults/main.yml b/roles/matrix-bridge-mx-puppet-instagram/defaults/main.yml index 21cde53f..cd08c010 100644 --- a/roles/matrix-bridge-mx-puppet-instagram/defaults/main.yml +++ b/roles/matrix-bridge-mx-puppet-instagram/defaults/main.yml @@ -46,6 +46,19 @@ matrix_mx_puppet_instagram_homeserver_token: '' # Can be set to enable automatic double-puppeting via Shared Secret Auth (https://github.com/devture/matrix-synapse-shared-secret-auth). matrix_mx_puppet_instagram_login_shared_secret: '' +matrix_mx_puppet_instagram_database_engine: sqlite + +matrix_mx_puppet_instagram_sqlite_database_path_local: "{{ matrix_mx_puppet_instagram_data_path }}/database.db" +matrix_mx_puppet_instagram_sqlite_database_path_in_container: "/data/database.db" + +matrix_mx_puppet_instagram_database_username: matrix_mx_puppet_instagram +matrix_mx_puppet_instagram_database_password: ~ +matrix_mx_puppet_instagram_database_hostname: 'matrix-postgres' +matrix_mx_puppet_instagram_database_port: 5432 +matrix_mx_puppet_instagram_database_name: matrix_mx_puppet_instagram + +matrix_mx_puppet_instagram_database_connection_string: 'postgresql://{{ matrix_mx_puppet_instagram_database_username }}:{{ matrix_mx_puppet_instagram_database_password }}@{{ matrix_mx_puppet_instagram_database_hostname }}:{{ matrix_mx_puppet_instagram_database_port }}/{{ matrix_mx_puppet_instagram_database_name }}?sslmode=disable' + # Default configuration template which covers the generic use case. # You can customize it by controlling the various variables inside it. # diff --git a/roles/matrix-bridge-mx-puppet-instagram/tasks/setup_install.yml b/roles/matrix-bridge-mx-puppet-instagram/tasks/setup_install.yml index 78bd78c1..5701a916 100644 --- a/roles/matrix-bridge-mx-puppet-instagram/tasks/setup_install.yml +++ b/roles/matrix-bridge-mx-puppet-instagram/tasks/setup_install.yml @@ -8,6 +8,33 @@ The matrix-bridge-mx-puppet-instagram role needs to execute before the matrix-synapse role. when: "matrix_synapse_role_executed|default(False)" + +- set_fact: + matrix_mx_puppet_instagram_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_mx_puppet_instagram_sqlite_database_path_local }}" + register: matrix_mx_puppet_instagram_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_mx_puppet_instagram_sqlite_database_path_local }}" + dst: "{{ matrix_mx_puppet_instagram_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_mx_puppet_instagram_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-mx-puppet-instagram.service'] + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_mx_puppet_instagram_requires_restart: true + when: "matrix_mx_puppet_instagram_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_mx_puppet_instagram_database_engine == 'postgres'" + - name: Ensure mx-puppet-instagram image is pulled docker_image: name: "{{ matrix_mx_puppet_instagram_docker_image }}" @@ -77,4 +104,8 @@ daemon_reload: yes when: "matrix_mx_puppet_instagram_systemd_service_result.changed" - +- name: Ensure matrix-mx-puppet-instagram.service restarted, if necessary + service: + name: "matrix-mx-puppet-instagram.service" + state: restarted + when: "matrix_mx_puppet_instagram_requires_restart|bool" diff --git a/roles/matrix-bridge-mx-puppet-instagram/templates/config.yaml.j2 b/roles/matrix-bridge-mx-puppet-instagram/templates/config.yaml.j2 index 634fbaec..b830da2b 100644 --- a/roles/matrix-bridge-mx-puppet-instagram/templates/config.yaml.j2 +++ b/roles/matrix-bridge-mx-puppet-instagram/templates/config.yaml.j2 @@ -44,15 +44,18 @@ provisioning: apiPrefix: /_matrix/provision database: +{% if matrix_mx_puppet_instagram_database_engine == 'postgres' %} # Use Postgres as a database backend # If set, will be used instead of SQLite3 # Connection string to connect to the Postgres instance # with username "user", password "pass", host "localhost" and database name "dbname". # Modify each value as necessary - #connString: "postgres://user:pass@localhost/dbname?sslmode=disable" + connString: {{ matrix_mx_puppet_instagram_database_connection_string|to_json }} +{% else %} # Use SQLite3 as a database backend # The name of the database file - filename: /data/database.db + filename: {{ matrix_mx_puppet_instagram_sqlite_database_path_in_container|to_json }} +{% endif %} logging: # Log level of console output diff --git a/roles/matrix-bridge-mx-puppet-instagram/templates/systemd/matrix-mx-puppet-instagram.service.j2 b/roles/matrix-bridge-mx-puppet-instagram/templates/systemd/matrix-mx-puppet-instagram.service.j2 index 81e3e081..4c94c84a 100644 --- a/roles/matrix-bridge-mx-puppet-instagram/templates/systemd/matrix-mx-puppet-instagram.service.j2 +++ b/roles/matrix-bridge-mx-puppet-instagram/templates/systemd/matrix-mx-puppet-instagram.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_mx_puppet_instagram_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-bridge-mx-puppet-skype/defaults/main.yml b/roles/matrix-bridge-mx-puppet-skype/defaults/main.yml index 0c16a8ba..83cd3dc5 100644 --- a/roles/matrix-bridge-mx-puppet-skype/defaults/main.yml +++ b/roles/matrix-bridge-mx-puppet-skype/defaults/main.yml @@ -53,6 +53,20 @@ matrix_mx_puppet_skype_homeserver_token: '' # Can be set to enable automatic double-puppeting via Shared Secret Auth (https://github.com/devture/matrix-synapse-shared-secret-auth). matrix_mx_puppet_skype_login_shared_secret: '' +# Database configuration, role default is `sqlite` but playbook default is `postgres` +matrix_mx_puppet_skype_database_engine: sqlite + +matrix_mx_puppet_skype_sqlite_database_path_local: "{{ matrix_mx_puppet_skype_data_path }}/database.db" +matrix_mx_puppet_skype_sqlite_database_path_in_container: "/data/database.db" + +matrix_mx_puppet_skype_database_username: matrix_mx_puppet_skype +matrix_mx_puppet_skype_database_password: ~ +matrix_mx_puppet_skype_database_hostname: 'matrix-postgres' +matrix_mx_puppet_skype_database_port: 5432 +matrix_mx_puppet_skype_database_name: matrix_mx_puppet_skype + +matrix_mx_puppet_skype_database_connection_string: 'postgresql://{{ matrix_mx_puppet_skype_database_username }}:{{ matrix_mx_puppet_skype_database_password }}@{{ matrix_mx_puppet_skype_database_hostname }}:{{ matrix_mx_puppet_skype_database_port }}/{{ matrix_mx_puppet_skype_database_name }}?sslmode=disable' + # Default configuration template which covers the generic use case. # You can customize it by controlling the various variables inside it. # diff --git a/roles/matrix-bridge-mx-puppet-skype/tasks/setup_install.yml b/roles/matrix-bridge-mx-puppet-skype/tasks/setup_install.yml index fb5185ed..68a1d7f4 100644 --- a/roles/matrix-bridge-mx-puppet-skype/tasks/setup_install.yml +++ b/roles/matrix-bridge-mx-puppet-skype/tasks/setup_install.yml @@ -8,14 +8,6 @@ The matrix-bridge-mx-puppet-skype role needs to execute before the matrix-synapse role. when: "matrix_synapse_role_executed|default(False)" -- name: Ensure MX Puppet Skype image is pulled - docker_image: - name: "{{ matrix_mx_puppet_skype_docker_image }}" - source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" - force_source: "{{ matrix_mx_puppet_skype_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" - force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mx_puppet_skype_docker_image_force_pull }}" - when: matrix_mx_puppet_skype_enabled|bool and not matrix_mx_puppet_skype_container_image_self_build - - name: Ensure MX Puppet Skype paths exist file: path: "{{ item.path }}" @@ -30,6 +22,57 @@ - { path: "{{ matrix_mx_puppet_skype_docker_src_files_path }}", when: "{{ matrix_mx_puppet_skype_container_image_self_build }}" } when: matrix_mx_puppet_skype_enabled|bool and item.when|bool +- name: Check if an old database file already exists + stat: + path: "{{ matrix_mx_puppet_skype_base_path }}/database.db" + register: matrix_mx_puppet_skype_stat_database + +- name: (Data relocation) Ensure matrix-mx-puppet-skype.service is stopped + service: + name: matrix-mx-puppet-skype + state: stopped + daemon_reload: yes + failed_when: false + when: "matrix_mx_puppet_skype_stat_database.stat.exists" + +- name: (Data relocation) Move mx-puppet-skype database file to ./data directory + command: "mv {{ matrix_mx_puppet_skype_base_path }}/database.db {{ matrix_mx_puppet_skype_data_path }}/database.db" + when: "matrix_mx_puppet_skype_stat_database.stat.exists" + +- set_fact: + matrix_mx_puppet_skype_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_mx_puppet_skype_sqlite_database_path_local }}" + register: matrix_mx_puppet_skype_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_mx_puppet_skype_sqlite_database_path_local }}" + dst: "{{ matrix_mx_puppet_skype_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_mx_puppet_skype_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-mx-puppet-skype.service'] + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_mx_puppet_skype_requires_restart: true + when: "matrix_mx_puppet_skype_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_mx_puppet_skype_database_engine == 'postgres'" + +- name: Ensure MX Puppet Skype image is pulled + docker_image: + name: "{{ matrix_mx_puppet_skype_docker_image }}" + source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" + force_source: "{{ matrix_mx_puppet_skype_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" + force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mx_puppet_skype_docker_image_force_pull }}" + when: matrix_mx_puppet_skype_enabled|bool and not matrix_mx_puppet_skype_container_image_self_build + - name: Ensure MX Puppet Skype repository is present on self build git: repo: "{{ matrix_mx_puppet_skype_container_image_self_build_repo }}" @@ -49,23 +92,6 @@ pull: yes when: "matrix_mx_puppet_skype_enabled|bool and matrix_mx_puppet_skype_container_image_self_build|bool" -- name: Check if an old database file already exists - stat: - path: "{{ matrix_mx_puppet_skype_base_path }}/database.db" - register: matrix_mx_puppet_skype_stat_database - -- name: (Data relocation) Ensure matrix-mx-puppet-skype.service is stopped - service: - name: matrix-mx-puppet-skype - state: stopped - daemon_reload: yes - failed_when: false - when: "matrix_mx_puppet_skype_stat_database.stat.exists" - -- name: (Data relocation) Move mx-puppet-skype database file to ./data directory - command: "mv {{ matrix_mx_puppet_skype_base_path }}/database.db {{ matrix_mx_puppet_skype_data_path }}/database.db" - when: "matrix_mx_puppet_skype_stat_database.stat.exists" - - name: Ensure mx-puppet-skype config.yaml installed copy: content: "{{ matrix_mx_puppet_skype_configuration|to_nice_yaml }}" @@ -93,3 +119,9 @@ service: daemon_reload: yes when: "matrix_mx_puppet_skype_systemd_service_result.changed" + +- name: Ensure matrix-mx-puppet-skype.service restarted, if necessary + service: + name: "matrix-mx-puppet-skype.service" + state: restarted + when: "matrix_mx_puppet_skype_requires_restart|bool" diff --git a/roles/matrix-bridge-mx-puppet-skype/templates/config.yaml.j2 b/roles/matrix-bridge-mx-puppet-skype/templates/config.yaml.j2 index c7b5c870..d41d3a23 100644 --- a/roles/matrix-bridge-mx-puppet-skype/templates/config.yaml.j2 +++ b/roles/matrix-bridge-mx-puppet-skype/templates/config.yaml.j2 @@ -68,15 +68,18 @@ logging: # - Store database: +{% if matrix_mx_puppet_skype_database_engine == 'postgres' %} # Use Postgres as a database backend # If set, will be used instead of SQLite3 # Connection string to connect to the Postgres instance # with username "user", password "pass", host "localhost" and database name "dbname". # Modify each value as necessary - #connString: "postgres://user:pass@localhost/dbname?sslmode=disable" + connString: {{ matrix_mx_puppet_skype_database_connection_string|to_json }} +{% else %} # Use SQLite3 as a database backend # The name of the database file - filename: /data/database.db + filename: {{ matrix_mx_puppet_skype_sqlite_database_path_in_container|to_json }} +{% endif %} provisioning: # Regex of Matrix IDs allowed to use the puppet bridge diff --git a/roles/matrix-bridge-mx-puppet-skype/templates/systemd/matrix-mx-puppet-skype.service.j2 b/roles/matrix-bridge-mx-puppet-skype/templates/systemd/matrix-mx-puppet-skype.service.j2 index 8a46cb25..cd958662 100644 --- a/roles/matrix-bridge-mx-puppet-skype/templates/systemd/matrix-mx-puppet-skype.service.j2 +++ b/roles/matrix-bridge-mx-puppet-skype/templates/systemd/matrix-mx-puppet-skype.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_mx_puppet_skype_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-bridge-mx-puppet-slack/defaults/main.yml b/roles/matrix-bridge-mx-puppet-slack/defaults/main.yml index 96ab7833..70b98ece 100644 --- a/roles/matrix-bridge-mx-puppet-slack/defaults/main.yml +++ b/roles/matrix-bridge-mx-puppet-slack/defaults/main.yml @@ -57,6 +57,20 @@ matrix_mx_puppet_slack_homeserver_token: '' # Can be set to enable automatic double-puppeting via Shared Secret Auth (https://github.com/devture/matrix-synapse-shared-secret-auth). matrix_mx_puppet_slack_login_shared_secret: '' +# Database configuration, role uses 'sqlite' per default but playbook sets up postgres by default +matrix_mx_puppet_slack_database_engine: sqlite + +matrix_mx_puppet_slack_sqlite_database_path_local: "{{ matrix_mx_puppet_slack_data_path }}/database.db" +matrix_mx_puppet_slack_sqlite_database_path_in_container: "/data/database.db" + +matrix_mx_puppet_slack_database_username: matrix_mx_puppet_slack +matrix_mx_puppet_slack_database_password: ~ +matrix_mx_puppet_slack_database_hostname: 'matrix-postgres' +matrix_mx_puppet_slack_database_port: 5432 +matrix_mx_puppet_slack_database_name: matrix_mx_puppet_slack + +matrix_mx_puppet_slack_database_connection_string: 'postgresql://{{ matrix_mx_puppet_slack_database_username }}:{{ matrix_mx_puppet_slack_database_password }}@{{ matrix_mx_puppet_slack_database_hostname }}:{{ matrix_mx_puppet_slack_database_port }}/{{ matrix_mx_puppet_slack_database_name }}?sslmode=disable' + # Default configuration template which covers the generic use case. # You can customize it by controlling the various variables inside it. # diff --git a/roles/matrix-bridge-mx-puppet-slack/tasks/setup_install.yml b/roles/matrix-bridge-mx-puppet-slack/tasks/setup_install.yml index 2e45ecf6..04eab20a 100644 --- a/roles/matrix-bridge-mx-puppet-slack/tasks/setup_install.yml +++ b/roles/matrix-bridge-mx-puppet-slack/tasks/setup_install.yml @@ -8,14 +8,6 @@ The matrix-bridge-mx-puppet-slack role needs to execute before the matrix-synapse role. when: "matrix_synapse_role_executed|default(False)" -- name: Ensure MX Puppet Slack image is pulled - docker_image: - name: "{{ matrix_mx_puppet_slack_docker_image }}" - source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" - force_source: "{{ matrix_mx_puppet_slack_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" - force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mx_puppet_slack_docker_image_force_pull }}" - when: matrix_mx_puppet_slack_enabled|bool and not matrix_mx_puppet_slack_container_image_self_build - - name: Ensure MX Puppet Slack paths exist file: path: "{{ item.path }}" @@ -30,6 +22,53 @@ - { path: "{{ matrix_mx_puppet_slack_docker_src_files_path }}", when: "{{ matrix_mx_puppet_slack_container_image_self_build }}" } when: matrix_mx_puppet_slack_enabled|bool and item.when|bool +- name: Check if an old database file already exists + stat: + path: "{{ matrix_mx_puppet_slack_base_path }}/database.db" + register: matrix_mx_puppet_slack_stat_database + +- name: (Data relocation) Ensure matrix-mx-puppet-slack.service is stopped + service: + name: matrix-mx-puppet-slack + state: stopped + daemon_reload: yes + failed_when: false + when: "matrix_mx_puppet_slack_stat_database.stat.exists" + +- set_fact: + matrix_mx_puppet_slack_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_mx_puppet_slack_sqlite_database_path_local }}" + register: matrix_mx_puppet_slack_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_mx_puppet_slack_sqlite_database_path_local }}" + dst: "{{ matrix_mx_puppet_slack_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_mx_puppet_slack_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-mx-puppet-slack.service'] + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_mx_puppet_slack_requires_restart: true + when: "matrix_mx_puppet_slack_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_mx_puppet_slack_database_engine == 'postgres'" + +- name: Ensure MX Puppet Slack image is pulled + docker_image: + name: "{{ matrix_mx_puppet_slack_docker_image }}" + source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" + force_source: "{{ matrix_mx_puppet_slack_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" + force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mx_puppet_slack_docker_image_force_pull }}" + when: matrix_mx_puppet_slack_enabled|bool and not matrix_mx_puppet_slack_container_image_self_build + - name: Ensure MX Puppet Slack repository is present on self build git: repo: "{{ matrix_mx_puppet_slack_container_image_self_build_repo }}" @@ -49,19 +88,6 @@ pull: yes when: "matrix_mx_puppet_slack_enabled|bool and matrix_mx_puppet_slack_container_image_self_build" -- name: Check if an old database file already exists - stat: - path: "{{ matrix_mx_puppet_slack_base_path }}/database.db" - register: matrix_mx_puppet_slack_stat_database - -- name: (Data relocation) Ensure matrix-mx-puppet-slack.service is stopped - service: - name: matrix-mx-puppet-slack - state: stopped - daemon_reload: yes - failed_when: false - when: "matrix_mx_puppet_slack_stat_database.stat.exists" - - name: (Data relocation) Move mx-puppet-slack database file to ./data directory command: "mv {{ matrix_mx_puppet_slack_base_path }}/database.db {{ matrix_mx_puppet_slack_data_path }}/database.db" when: "matrix_mx_puppet_slack_stat_database.stat.exists" @@ -93,3 +119,9 @@ service: daemon_reload: yes when: "matrix_mx_puppet_slack_systemd_service_result.changed" + +- name: Ensure matrix-mx-puppet-slack.service restarted, if necessary + service: + name: "matrix-mx-puppet-slack.service" + state: restarted + when: "matrix_mx_puppet_slack_requires_restart|bool" diff --git a/roles/matrix-bridge-mx-puppet-slack/templates/config.yaml.j2 b/roles/matrix-bridge-mx-puppet-slack/templates/config.yaml.j2 index b6e88784..af6b5cb8 100644 --- a/roles/matrix-bridge-mx-puppet-slack/templates/config.yaml.j2 +++ b/roles/matrix-bridge-mx-puppet-slack/templates/config.yaml.j2 @@ -58,15 +58,18 @@ provisioning: apiPrefix: /_matrix/provision database: +{% if matrix_mx_puppet_slack_database_engine == 'postgres' %} # Use Postgres as a database backend # If set, will be used instead of SQLite3 # Connection string to connect to the Postgres instance # with username "user", password "pass", host "localhost" and database name "dbname". # Modify each value as necessary - #connString: "postgres://user:pass@localhost/dbname?sslmode=disable" + connString: {{ matrix_mx_puppet_slack_database_connection_string|to_json }} +{% else %} # Use SQLite3 as a database backend # The name of the database file - filename: /data/database.db + filename: {{ matrix_mx_puppet_slack_sqlite_database_path_in_container|to_json }} +{% endif %} logging: # Log level of console output diff --git a/roles/matrix-bridge-mx-puppet-slack/templates/systemd/matrix-mx-puppet-slack.service.j2 b/roles/matrix-bridge-mx-puppet-slack/templates/systemd/matrix-mx-puppet-slack.service.j2 index 23c2504f..2b1456f5 100644 --- a/roles/matrix-bridge-mx-puppet-slack/templates/systemd/matrix-mx-puppet-slack.service.j2 +++ b/roles/matrix-bridge-mx-puppet-slack/templates/systemd/matrix-mx-puppet-slack.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_mx_puppet_slack_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-bridge-mx-puppet-steam/defaults/main.yml b/roles/matrix-bridge-mx-puppet-steam/defaults/main.yml index b5eb8473..15fa889f 100644 --- a/roles/matrix-bridge-mx-puppet-steam/defaults/main.yml +++ b/roles/matrix-bridge-mx-puppet-steam/defaults/main.yml @@ -55,6 +55,19 @@ matrix_mx_puppet_steam_homeserver_token: '' # Can be set to enable automatic double-puppeting via Shared Secret Auth (https://github.com/devture/matrix-synapse-shared-secret-auth). matrix_mx_puppet_steam_login_shared_secret: '' +matrix_mx_puppet_steam_database_engine: sqlite + +matrix_mx_puppet_steam_sqlite_database_path_local: "{{ matrix_mx_puppet_steam_data_path }}/database.db" +matrix_mx_puppet_steam_sqlite_database_path_in_container: "/data/database.db" + +matrix_mx_puppet_steam_database_username: matrix_mx_puppet_steam +matrix_mx_puppet_steam_database_password: ~ +matrix_mx_puppet_steam_database_hostname: 'matrix-postgres' +matrix_mx_puppet_steam_database_port: 5432 +matrix_mx_puppet_steam_database_name: matrix_mx_puppet_steam + +matrix_mx_puppet_steam_database_connection_string: 'postgresql://{{ matrix_mx_puppet_steam_database_username }}:{{ matrix_mx_puppet_steam_database_password }}@{{ matrix_mx_puppet_steam_database_hostname }}:{{ matrix_mx_puppet_steam_database_port }}/{{ matrix_mx_puppet_steam_database_name }}?sslmode=disable' + # Default configuration template which covers the generic use case. # You can customize it by controlling the various variables inside it. # diff --git a/roles/matrix-bridge-mx-puppet-steam/tasks/setup_install.yml b/roles/matrix-bridge-mx-puppet-steam/tasks/setup_install.yml index 7b7f8211..6b574656 100644 --- a/roles/matrix-bridge-mx-puppet-steam/tasks/setup_install.yml +++ b/roles/matrix-bridge-mx-puppet-steam/tasks/setup_install.yml @@ -8,14 +8,6 @@ The matrix-bridge-mx-puppet-steam role needs to execute before the matrix-synapse role. when: "matrix_synapse_role_executed|default(False)" -- name: Ensure MX Puppet Steam image is pulled - docker_image: - name: "{{ matrix_mx_puppet_steam_docker_image }}" - source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" - force_source: "{{ matrix_mx_puppet_steam_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" - force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mx_puppet_steam_docker_image_force_pull }}" - when: matrix_mx_puppet_steam_enabled|bool and not matrix_mx_puppet_steam_container_image_self_build - - name: Ensure MX Puppet Steam paths exist file: path: "{{ item.path }}" @@ -30,6 +22,57 @@ - { path: "{{ matrix_mx_puppet_steam_docker_src_files_path }}", when: "{{ matrix_mx_puppet_steam_container_image_self_build }}" } when: matrix_mx_puppet_steam_enabled|bool and item.when|bool +- name: Check if an old database file already exists + stat: + path: "{{ matrix_mx_puppet_steam_base_path }}/database.db" + register: matrix_mx_puppet_steam_stat_database + +- name: (Data relocation) Ensure matrix-mx-puppet-steam.service is stopped + service: + name: matrix-mx-puppet-steam + state: stopped + daemon_reload: yes + failed_when: false + when: "matrix_mx_puppet_steam_stat_database.stat.exists" + +- name: (Data relocation) Move mx-puppet-steam database file to ./data directory + command: "mv {{ matrix_mx_puppet_steam_base_path }}/database.db {{ matrix_mx_puppet_steam_data_path }}/database.db" + when: "matrix_mx_puppet_steam_stat_database.stat.exists" + +- set_fact: + matrix_mx_puppet_steam_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_mx_puppet_steam_sqlite_database_path_local }}" + register: matrix_mx_puppet_steam_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_mx_puppet_steam_sqlite_database_path_local }}" + dst: "{{ matrix_mx_puppet_steam_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_mx_puppet_steam_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-mx-puppet-steam.service'] + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_mx_puppet_steam_requires_restart: true + when: "matrix_mx_puppet_steam_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_mx_puppet_steam_database_engine == 'postgres'" + +- name: Ensure MX Puppet Steam image is pulled + docker_image: + name: "{{ matrix_mx_puppet_steam_docker_image }}" + source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" + force_source: "{{ matrix_mx_puppet_steam_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" + force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mx_puppet_steam_docker_image_force_pull }}" + when: matrix_mx_puppet_steam_enabled|bool and not matrix_mx_puppet_steam_container_image_self_build + - name: Ensure MX Puppet Steam repository is present on self build git: repo: "{{ matrix_mx_puppet_steam_container_image_self_build_repo }}" @@ -49,23 +92,6 @@ pull: yes when: "matrix_mx_puppet_steam_enabled|bool and matrix_mx_puppet_steam_container_image_self_build" -- name: Check if an old database file already exists - stat: - path: "{{ matrix_mx_puppet_steam_base_path }}/database.db" - register: matrix_mx_puppet_steam_stat_database - -- name: (Data relocation) Ensure matrix-mx-puppet-steam.service is stopped - service: - name: matrix-mx-puppet-steam - state: stopped - daemon_reload: yes - failed_when: false - when: "matrix_mx_puppet_steam_stat_database.stat.exists" - -- name: (Data relocation) Move mx-puppet-steam database file to ./data directory - command: "mv {{ matrix_mx_puppet_steam_base_path }}/database.db {{ matrix_mx_puppet_steam_data_path }}/database.db" - when: "matrix_mx_puppet_steam_stat_database.stat.exists" - - name: Ensure mx-puppet-steam config.yaml installed copy: content: "{{ matrix_mx_puppet_steam_configuration|to_nice_yaml }}" @@ -93,3 +119,9 @@ service: daemon_reload: yes when: "matrix_mx_puppet_steam_systemd_service_result.changed" + +- name: Ensure matrix-mx-puppet-steam.service restarted, if necessary + service: + name: "matrix-mx-puppet-steam.service" + state: restarted + when: "matrix_mx_puppet_steam_requires_restart|bool" diff --git a/roles/matrix-bridge-mx-puppet-steam/templates/config.yaml.j2 b/roles/matrix-bridge-mx-puppet-steam/templates/config.yaml.j2 index d08982ca..149e08b6 100644 --- a/roles/matrix-bridge-mx-puppet-steam/templates/config.yaml.j2 +++ b/roles/matrix-bridge-mx-puppet-steam/templates/config.yaml.j2 @@ -61,15 +61,18 @@ selfService: blacklist: {{ matrix_mx_puppet_steam_provisioning_blacklist|to_json }} database: +{% if matrix_mx_puppet_steam_database_engine == 'postgres' %} # Use Postgres as a database backend # If set, will be used instead of SQLite3 # Connection string to connect to the Postgres instance # with username "user", password "pass", host "localhost" and database name "dbname". # Modify each value as necessary - #connString: "postgres://user:pass@localhost/dbname?sslmode=disable" + connString: {{ matrix_mx_puppet_steam_database_connection_string|to_json }} +{% else %} # Use SQLite3 as a database backend # The name of the database file - filename: /data/database.db + filename: {{ matrix_mx_puppet_steam_sqlite_database_path_in_container|to_json }} +{% endif %} logging: # Log level of console output diff --git a/roles/matrix-bridge-mx-puppet-steam/templates/systemd/matrix-mx-puppet-steam.service.j2 b/roles/matrix-bridge-mx-puppet-steam/templates/systemd/matrix-mx-puppet-steam.service.j2 index 986c5d2c..e263154b 100644 --- a/roles/matrix-bridge-mx-puppet-steam/templates/systemd/matrix-mx-puppet-steam.service.j2 +++ b/roles/matrix-bridge-mx-puppet-steam/templates/systemd/matrix-mx-puppet-steam.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_mx_puppet_steam_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-bridge-mx-puppet-twitter/defaults/main.yml b/roles/matrix-bridge-mx-puppet-twitter/defaults/main.yml index 13438a21..28639fda 100644 --- a/roles/matrix-bridge-mx-puppet-twitter/defaults/main.yml +++ b/roles/matrix-bridge-mx-puppet-twitter/defaults/main.yml @@ -60,6 +60,20 @@ matrix_mx_puppet_twitter_homeserver_token: '' # Can be set to enable automatic double-puppeting via Shared Secret Auth (https://github.com/devture/matrix-synapse-shared-secret-auth). matrix_mx_puppet_twitter_login_shared_secret: '' +# Database configuration +matrix_mx_puppet_twitter_database_engine: sqlite + +matrix_mx_puppet_twitter_sqlite_database_path_local: "{{ matrix_mx_puppet_twitter_data_path }}/database.db" +matrix_mx_puppet_twitter_sqlite_database_path_in_container: "/data/database.db" + +matrix_mx_puppet_twitter_database_username: mx_puppet_twitter +matrix_mx_puppet_twitter_database_password: ~ +matrix_mx_puppet_twitter_database_hostname: 'matrix-postgres' +matrix_mx_puppet_twitter_database_port: 5432 +matrix_mx_puppet_twitter_database_name: matrix_mx_puppet_twitter + +matrix_mx_puppet_twitter_database_connection_string: 'postgresql://{{ matrix_mx_puppet_twitter_database_username }}:{{ matrix_mx_puppet_twitter_database_password }}@{{ matrix_mx_puppet_twitter_database_hostname }}:{{ matrix_mx_puppet_twitter_database_port }}/{{ matrix_mx_puppet_twitter_database_name }}?sslmode=disable' + # Default configuration template which covers the generic use case. # You can customize it by controlling the various variables inside it. # diff --git a/roles/matrix-bridge-mx-puppet-twitter/tasks/setup_install.yml b/roles/matrix-bridge-mx-puppet-twitter/tasks/setup_install.yml index 663f822c..1c48c030 100644 --- a/roles/matrix-bridge-mx-puppet-twitter/tasks/setup_install.yml +++ b/roles/matrix-bridge-mx-puppet-twitter/tasks/setup_install.yml @@ -8,14 +8,6 @@ The matrix-bridge-mx-puppet-twitter role needs to execute before the matrix-synapse role. when: "matrix_synapse_role_executed|default(False)" -- name: Ensure MX Puppet Twitter image is pulled - docker_image: - name: "{{ matrix_mx_puppet_twitter_docker_image }}" - source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" - force_source: "{{ matrix_mx_puppet_twitter_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" - force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mx_puppet_twitter_docker_image_force_pull }}" - when: matrix_mx_puppet_twitter_enabled|bool and not matrix_mx_puppet_twitter_container_image_self_build - - name: Ensure MX Puppet Twitter paths exist file: path: "{{ item.path }}" @@ -30,6 +22,57 @@ - { path: "{{ matrix_mx_puppet_twitter_docker_src_files_path }}", when: "{{ matrix_mx_puppet_twitter_container_image_self_build }}" } when: matrix_mx_puppet_twitter_enabled|bool and item.when|bool +- name: Check if an old database file already exists + stat: + path: "{{ matrix_mx_puppet_twitter_base_path }}/database.db" + register: matrix_mx_puppet_twitter_stat_database + +- name: (Data relocation) Ensure matrix-mx-puppet-twitter.service is stopped + service: + name: matrix-mx-puppet-twitter + state: stopped + daemon_reload: yes + failed_when: false + when: "matrix_mx_puppet_twitter_stat_database.stat.exists" + +- name: (Data relocation) Move mx-puppet-twitter database file to ./data directory + command: "mv {{ matrix_mx_puppet_twitter_base_path }}/database.db {{ matrix_mx_puppet_twitter_data_path }}/database.db" + when: "matrix_mx_puppet_twitter_stat_database.stat.exists" + +- set_fact: + matrix_mx_puppet_twitter_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_mx_puppet_twitter_sqlite_database_path_local }}" + register: matrix_mx_puppet_twitter_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_mx_puppet_twitter_sqlite_database_path_local }}" + dst: "{{ matrix_mx_puppet_twitter_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_mx_puppet_twitter_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-mx-puppet-twitter.service'] + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_mx_puppet_twitter_requires_restart: true + when: "matrix_mx_puppet_twitter_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_mx_puppet_twitter_database_engine == 'postgres'" + +- name: Ensure MX Puppet Twitter image is pulled + docker_image: + name: "{{ matrix_mx_puppet_twitter_docker_image }}" + source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" + force_source: "{{ matrix_mx_puppet_twitter_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" + force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_mx_puppet_twitter_docker_image_force_pull }}" + when: matrix_mx_puppet_twitter_enabled|bool and not matrix_mx_puppet_twitter_container_image_self_build + - name: Ensure MX Puppet Twitter repository is present on self build git: repo: "{{ matrix_mx_puppet_twitter_container_image_self_build_repo }}" @@ -49,23 +92,6 @@ pull: yes when: "matrix_mx_puppet_twitter_enabled|bool and matrix_mx_puppet_twitter_container_image_self_build" -- name: Check if an old database file already exists - stat: - path: "{{ matrix_mx_puppet_twitter_base_path }}/database.db" - register: matrix_mx_puppet_twitter_stat_database - -- name: (Data relocation) Ensure matrix-mx-puppet-twitter.service is stopped - service: - name: matrix-mx-puppet-twitter - state: stopped - daemon_reload: yes - failed_when: false - when: "matrix_mx_puppet_twitter_stat_database.stat.exists" - -- name: (Data relocation) Move mx-puppet-twitter database file to ./data directory - command: "mv {{ matrix_mx_puppet_twitter_base_path }}/database.db {{ matrix_mx_puppet_twitter_data_path }}/database.db" - when: "matrix_mx_puppet_twitter_stat_database.stat.exists" - - name: Ensure mx-puppet-twitter config.yaml installed copy: content: "{{ matrix_mx_puppet_twitter_configuration|to_nice_yaml }}" @@ -93,3 +119,9 @@ service: daemon_reload: yes when: "matrix_mx_puppet_twitter_systemd_service_result.changed" + +- name: Ensure matrix-mx-puppet-twitter.service restarted, if necessary + service: + name: "matrix-mx-puppet-twitter.service" + state: restarted + when: "matrix_mx_puppet_twitter_requires_restart|bool" diff --git a/roles/matrix-bridge-mx-puppet-twitter/templates/config.yaml.j2 b/roles/matrix-bridge-mx-puppet-twitter/templates/config.yaml.j2 index 7d3033b3..bdecf1dc 100644 --- a/roles/matrix-bridge-mx-puppet-twitter/templates/config.yaml.j2 +++ b/roles/matrix-bridge-mx-puppet-twitter/templates/config.yaml.j2 @@ -54,15 +54,18 @@ provisioning: apiPrefix: /_matrix/provision database: +{% if matrix_mx_puppet_twitter_database_engine == 'postgres' %} # Use Postgres as a database backend # If set, will be used instead of SQLite3 # Connection string to connect to the Postgres instance # with username "user", password "pass", host "localhost" and database name "dbname". # Modify each value as necessary - #connString: "postgres://user:pass@localhost/dbname?sslmode=disable" + connString: {{ matrix_mx_puppet_twitter_database_connection_string|to_json }} +{% else %} # Use SQLite3 as a database backend # The name of the database file - filename: /data/database.db + filename: {{ matrix_mx_puppet_twitter_sqlite_database_path_in_container|to_json }} +{% endif %} logging: # Log level of console output diff --git a/roles/matrix-bridge-mx-puppet-twitter/templates/systemd/matrix-mx-puppet-twitter.service.j2 b/roles/matrix-bridge-mx-puppet-twitter/templates/systemd/matrix-mx-puppet-twitter.service.j2 index 6afb6fb4..8d7898eb 100644 --- a/roles/matrix-bridge-mx-puppet-twitter/templates/systemd/matrix-mx-puppet-twitter.service.j2 +++ b/roles/matrix-bridge-mx-puppet-twitter/templates/systemd/matrix-mx-puppet-twitter.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_mx_puppet_twitter_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-bridge-sms/defaults/main.yml b/roles/matrix-bridge-sms/defaults/main.yml index 82f9b841..3c6b4c61 100644 --- a/roles/matrix-bridge-sms/defaults/main.yml +++ b/roles/matrix-bridge-sms/defaults/main.yml @@ -3,7 +3,7 @@ matrix_sms_bridge_enabled: true -matrix_sms_bridge_docker_image: "docker.io/folivonet/matrix-sms-bridge:0.5.4" +matrix_sms_bridge_docker_image: "docker.io/folivonet/matrix-sms-bridge:0.5.5" matrix_sms_bridge_base_path: "{{ matrix_base_data_path }}/matrix-sms-bridge" matrix_sms_bridge_config_path: "{{ matrix_base_data_path }}/matrix-sms-bridge/config" diff --git a/roles/matrix-bridge-sms/templates/systemd/matrix-sms-bridge.service.j2 b/roles/matrix-bridge-sms/templates/systemd/matrix-sms-bridge.service.j2 index 7c96f9f4..0eb0eb58 100644 --- a/roles/matrix-bridge-sms/templates/systemd/matrix-sms-bridge.service.j2 +++ b/roles/matrix-bridge-sms/templates/systemd/matrix-sms-bridge.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_sms_bridge_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-client-element/defaults/main.yml b/roles/matrix-client-element/defaults/main.yml index cbe70503..d0297193 100644 --- a/roles/matrix-client-element/defaults/main.yml +++ b/roles/matrix-client-element/defaults/main.yml @@ -3,7 +3,7 @@ matrix_client_element_enabled: true matrix_client_element_container_image_self_build: false matrix_client_element_container_image_self_build_repo: "https://github.com/vector-im/riot-web.git" -matrix_client_element_docker_image: "{{ matrix_client_element_docker_image_name_prefix }}vectorim/element-web:v1.7.14" +matrix_client_element_docker_image: "{{ matrix_client_element_docker_image_name_prefix }}vectorim/element-web:v1.7.16" matrix_client_element_docker_image_name_prefix: "{{ 'localhost/' if matrix_client_element_container_image_self_build else 'docker.io/' }}" matrix_client_element_docker_image_force_pull: "{{ matrix_client_element_docker_image.endswith(':latest') }}" diff --git a/roles/matrix-client-element/templates/systemd/matrix-client-element.service.j2 b/roles/matrix-client-element/templates/systemd/matrix-client-element.service.j2 index 39b7aa52..e0dd2e7e 100644 --- a/roles/matrix-client-element/templates/systemd/matrix-client-element.service.j2 +++ b/roles/matrix-client-element/templates/systemd/matrix-client-element.service.j2 @@ -5,6 +5,7 @@ Description=Matrix Element server Requires={{ service }} After={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-corporal/defaults/main.yml b/roles/matrix-corporal/defaults/main.yml index def3fcc9..cccaadd0 100644 --- a/roles/matrix-corporal/defaults/main.yml +++ b/roles/matrix-corporal/defaults/main.yml @@ -3,6 +3,9 @@ matrix_corporal_enabled: true +matrix_corporal_container_image_self_build: false +matrix_corporal_container_image_self_build_repo: "https://github.com/devture/matrix-corporal.git" + # Controls whether the matrix-corporal container exposes its gateway HTTP port (tcp/41080 in the container). # # Takes an ":" or "" value (e.g. "127.0.0.1:41080"), or empty string to not expose. @@ -19,10 +22,13 @@ matrix_corporal_container_extra_arguments: [] # List of systemd services that matrix-corporal.service depends on matrix_corporal_systemd_required_services_list: ['docker.service'] -matrix_corporal_docker_image: "docker.io/devture/matrix-corporal:1.11.0" +matrix_corporal_docker_image: "{{ matrix_corporal_docker_image_name_prefix }}devture/matrix-corporal:{{ matrix_corporal_docker_image_tag }}" +matrix_corporal_docker_image_name_prefix: "{{ 'localhost/' if matrix_corporal_container_image_self_build else 'docker.io/' }}" +matrix_corporal_docker_image_tag: "1.11.0" matrix_corporal_docker_image_force_pull: "{{ matrix_corporal_docker_image.endswith(':latest') }}" matrix_corporal_base_path: "{{ matrix_base_data_path }}/corporal" +matrix_corporal_container_src_files_path: "{{ matrix_corporal_base_path }}/container-src" matrix_corporal_config_dir_path: "{{ matrix_corporal_base_path }}/config" matrix_corporal_cache_dir_path: "{{ matrix_corporal_base_path }}/cache" matrix_corporal_var_dir_path: "{{ matrix_corporal_base_path }}/var" diff --git a/roles/matrix-corporal/tasks/setup_corporal.yml b/roles/matrix-corporal/tasks/setup_corporal.yml index 72b6a5b2..188f09bf 100644 --- a/roles/matrix-corporal/tasks/setup_corporal.yml +++ b/roles/matrix-corporal/tasks/setup_corporal.yml @@ -17,13 +17,33 @@ - "{{ matrix_corporal_var_dir_path }}" when: matrix_corporal_enabled|bool +- name: Ensure Matrix Corporal repository is present on self-build + git: + repo: "{{ matrix_corporal_container_image_self_build_repo }}" + dest: "{{ matrix_corporal_container_src_files_path }}" + version: "{{ matrix_corporal_docker_image.split(':')[1] }}" + force: "yes" + register: matrix_corporal_git_pull_results + when: "matrix_corporal_enabled|bool and matrix_corporal_container_image_self_build|bool" + +- name: Ensure Matrix Corporal Docker image is built + docker_image: + name: "{{ matrix_corporal_docker_image }}" + source: build + force_source: "{{ matrix_corporal_git_pull_results.changed }}" + build: + dockerfile: etc/docker/Dockerfile + path: "{{ matrix_corporal_container_src_files_path }}" + pull: yes + when: "matrix_corporal_enabled|bool and matrix_corporal_container_image_self_build|bool" + - name: Ensure Matrix Corporal Docker image is pulled docker_image: name: "{{ matrix_corporal_docker_image }}" source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" force_source: "{{ matrix_corporal_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_corporal_docker_image_force_pull }}" - when: matrix_corporal_enabled|bool + when: "matrix_corporal_enabled|bool and not matrix_corporal_container_image_self_build|bool" - name: Ensure Matrix Corporal config installed copy: diff --git a/roles/matrix-corporal/templates/systemd/matrix-corporal.service.j2 b/roles/matrix-corporal/templates/systemd/matrix-corporal.service.j2 index 97ec7d89..e8ce8c0a 100644 --- a/roles/matrix-corporal/templates/systemd/matrix-corporal.service.j2 +++ b/roles/matrix-corporal/templates/systemd/matrix-corporal.service.j2 @@ -5,6 +5,7 @@ Description=Matrix Corporal Requires={{ service }} After={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-coturn/tasks/setup_coturn.yml b/roles/matrix-coturn/tasks/setup_coturn.yml index 1f13da03..f3c1048f 100644 --- a/roles/matrix-coturn/tasks/setup_coturn.yml +++ b/roles/matrix-coturn/tasks/setup_coturn.yml @@ -59,22 +59,11 @@ mode: 0644 when: matrix_coturn_enabled|bool -# `docker_network` doesn't work as expected when the given network -# is a substring of a network that already exists. -# -# See our other comments in `roles/matrix-base/tasks/setup_matrix_base.yml` -- name: Check existence of Coturn network in Docker - shell: - cmd: "docker network ls -q --filter='name=^{{ matrix_coturn_docker_network }}$'" - register: matrix_coturn_result_docker_network - changed_when: false +- name: Ensure Coturn network is created in Docker + docker_network: + name: "{{ matrix_coturn_docker_network }}" + driver: bridge when: matrix_coturn_enabled|bool - check_mode: no - -- name: Create Coturn network in Docker - shell: - cmd: "docker network create --driver=bridge {{ matrix_coturn_docker_network }}" - when: "matrix_coturn_enabled|bool and matrix_coturn_result_docker_network.stdout == '' and not ansible_check_mode" - name: Ensure matrix-coturn.service installed template: diff --git a/roles/matrix-coturn/templates/systemd/matrix-coturn.service.j2 b/roles/matrix-coturn/templates/systemd/matrix-coturn.service.j2 index e484b59b..16ca5d2a 100644 --- a/roles/matrix-coturn/templates/systemd/matrix-coturn.service.j2 +++ b/roles/matrix-coturn/templates/systemd/matrix-coturn.service.j2 @@ -5,6 +5,7 @@ Description=Matrix Coturn server Requires={{ service }} After={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-dimension/defaults/main.yml b/roles/matrix-dimension/defaults/main.yml index 3b69227a..8a691946 100644 --- a/roles/matrix-dimension/defaults/main.yml +++ b/roles/matrix-dimension/defaults/main.yml @@ -15,6 +15,12 @@ matrix_dimension_base_path: "{{ matrix_base_data_path }}/dimension" matrix_dimension_docker_image: "docker.io/turt2live/matrix-dimension:latest" matrix_dimension_docker_image_force_pull: "{{ matrix_dimension_docker_image.endswith(':latest') }}" +# List of systemd services that matrix-dimension.service depends on. +matrix_dimension_systemd_required_services_list: ['docker.service'] + +# List of systemd services that matrix-dimension.service wants +matrix_dimension_systemd_wanted_services_list: [] + # The user and group id correspond to the node user in the `turt2live/matrix-dimension` image. matrix_dimension_user_uid: '1000' matrix_dimension_user_gid: '1000' @@ -34,6 +40,28 @@ matrix_dimension_integrations_jitsi_widget_url: "https://{{ matrix_server_fqn_di matrix_dimension_homeserver_federationUrl: "http://matrix-synapse:8048" + +# Database-related configuration fields. +# +# To use SQLite, stick to these defaults. +# +# To use Postgres: +# - change the engine (`matrix_dimension_database_engine: 'postgres'`) +# - adjust your database credentials via the `matrix_dimension_postgres_*` variables +matrix_dimension_database_engine: 'sqlite' + +matrix_dimension_sqlite_database_path_local: "{{ matrix_dimension_base_path }}/dimension.db" +matrix_dimension_sqlite_database_path_in_container: "dimension.db" + +matrix_dimension_database_username: 'matrix_dimension' +matrix_dimension_database_password: 'some-password' +matrix_dimension_database_hostname: 'matrix-postgres' +matrix_dimension_database_port: 5432 +matrix_dimension_database_name: 'matrix_dimension' + +matrix_dimension_database_connection_string: 'postgres://{{ matrix_dimension_database_username }}:{{ matrix_dimension_database_password }}@{{ matrix_dimension_database_hostname }}:{{ matrix_dimension_database_port }}/{{ matrix_dimension_database_name }}' + + # Default Dimension configuration template which covers the generic use case. # You can customize it by controlling the various variables inside it. # diff --git a/roles/matrix-dimension/tasks/main.yml b/roles/matrix-dimension/tasks/main.yml index 1888f945..aad55286 100644 --- a/roles/matrix-dimension/tasks/main.yml +++ b/roles/matrix-dimension/tasks/main.yml @@ -8,8 +8,14 @@ - setup-all - setup-dimension -- import_tasks: "{{ role_path }}/tasks/setup_dimension.yml" - when: run_setup|bool +- import_tasks: "{{ role_path }}/tasks/setup_install.yml" + when: run_setup|bool and matrix_dimension_enabled|bool + tags: + - setup-all + - setup-dimension + +- import_tasks: "{{ role_path }}/tasks/setup_uninstall.yml" + when: run_setup|bool and not matrix_dimension_enabled|bool tags: - setup-all - setup-dimension diff --git a/roles/matrix-dimension/tasks/setup_dimension.yml b/roles/matrix-dimension/tasks/setup_dimension.yml deleted file mode 100644 index 2437a547..00000000 --- a/roles/matrix-dimension/tasks/setup_dimension.yml +++ /dev/null @@ -1,85 +0,0 @@ ---- - -# -# Tasks related to setting up the dimension -# - -- name: Ensure Dimension base path exists - file: - path: "{{ matrix_dimension_base_path }}" - state: directory - mode: 0770 - owner: "{{ matrix_user_username }}" - group: "{{ matrix_dimension_user_gid }}" - when: matrix_dimension_enabled|bool - -- name: Ensure Dimension config installed - copy: - content: "{{ matrix_dimension_configuration|to_nice_yaml }}" - dest: "{{ matrix_dimension_base_path }}/config.yaml" - mode: 0640 - owner: "{{ matrix_user_username }}" - group: "{{ matrix_dimension_user_gid }}" - when: matrix_dimension_enabled|bool - -- name: Ensure Dimension image is pulled - docker_image: - name: "{{ matrix_dimension_docker_image }}" - source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" - force_source: "{{ matrix_dimension_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" - force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_dimension_docker_image_force_pull }}" - when: matrix_dimension_enabled|bool - -- name: Ensure matrix-dimension.service installed - template: - src: "{{ role_path }}/templates/systemd/matrix-dimension.service.j2" - dest: "{{ matrix_systemd_path }}/matrix-dimension.service" - mode: 0644 - register: matrix_dimension_systemd_service_result - when: matrix_dimension_enabled|bool - -- name: Ensure systemd reloaded after matrix-dimension.service installation - service: - daemon_reload: yes - when: "matrix_dimension_enabled|bool and matrix_dimension_systemd_service_result.changed" - -# -# Tasks related to getting rid of the dimension (if it was previously enabled) -# - -- name: Check existence of matrix-dimension service - stat: - path: "{{ matrix_systemd_path }}/matrix-dimension.service" - register: matrix_dimension_service_stat - when: "not matrix_dimension_enabled|bool" - -- name: Ensure matrix-dimension is stopped - service: - name: matrix-dimension - state: stopped - daemon_reload: yes - register: stopping_result - when: "not matrix_dimension_enabled|bool and matrix_dimension_service_stat.stat.exists" - -- name: Ensure matrix-dimension.service doesn't exist - file: - path: "{{ matrix_systemd_path }}/matrix-dimension.service" - state: absent - when: "not matrix_dimension_enabled|bool and matrix_dimension_service_stat.stat.exists" - -- name: Ensure systemd reloaded after matrix-dimension.service removal - service: - daemon_reload: yes - when: "not matrix_dimension_enabled|bool and matrix_dimension_service_stat.stat.exists" - -- name: Ensure Dimension environment variables path doesn't exist - file: - path: "{{ matrix_dimension_base_path }}" - state: absent - when: "not matrix_dimension_enabled|bool" - -- name: Ensure Dimension Docker image doesn't exist - docker_image: - name: "{{ matrix_dimension_docker_image }}" - state: absent - when: "not matrix_dimension_enabled|bool" diff --git a/roles/matrix-dimension/tasks/setup_install.yml b/roles/matrix-dimension/tasks/setup_install.yml new file mode 100644 index 00000000..26a75bcb --- /dev/null +++ b/roles/matrix-dimension/tasks/setup_install.yml @@ -0,0 +1,109 @@ +--- + +- set_fact: + matrix_dimension_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_dimension_sqlite_database_path_local }}" + register: matrix_dimension_sqlite_database_path_local_stat_result + + - block: + # pgloader makes a few columns `smallint`, instead of `boolean`. + # We need to fix them up. + - set_fact: + matrix_dimension_pgloader_additional_psql_statements_list: [] + + - set_fact: + matrix_dimension_pgloader_additional_psql_statements_list: | + {{ + matrix_dimension_pgloader_additional_psql_statements_list + + + ([] if item.default == '' else ['ALTER TABLE ' + item.table + ' ALTER COLUMN "' + item.column + '" DROP default;']) + + + (['ALTER TABLE ' + item.table + ' ALTER COLUMN "' + item.column + '" TYPE BOOLEAN USING("' + item.column + '"::text::boolean);']) + + + ([] if item.default == '' else ['ALTER TABLE ' + item.table + ' ALTER COLUMN "' + item.column + '" SET default ' + item.default + ';']) + }} + with_items: + - {'table': 'dimension_widgets', 'column': 'isEnabled', 'default': ''} + - {'table': 'dimension_widgets', 'column': 'isPublic', 'default': ''} + - {'table': 'dimension_webhook_bridges', 'column': 'isEnabled', 'default': ''} + - {'table': 'dimension_user_sticker_packs', 'column': 'isSelected', 'default': ''} + - {'table': 'dimension_scalar_tokens', 'column': 'isDimensionToken', 'default': ''} + - {'table': 'dimension_users', 'column': 'isSelfBot', 'default': 'false'} + - {'table': 'dimension_telegram_bridges', 'column': 'allowTgPuppets', 'default': ''} + - {'table': 'dimension_telegram_bridges', 'column': 'allowMxPuppets', 'default': ''} + - {'table': 'dimension_telegram_bridges', 'column': 'isEnabled', 'default': ''} + - {'table': 'dimension_sticker_packs', 'column': 'isEnabled', 'default': ''} + - {'table': 'dimension_sticker_packs', 'column': 'isPublic', 'default': ''} + - {'table': 'dimension_slack_bridges', 'column': 'isEnabled', 'default': ''} + - {'table': 'dimension_neb_integrations', 'column': 'isPublic', 'default': ''} + - {'table': 'dimension_irc_bridges', 'column': 'isEnabled', 'default': ''} + - {'table': 'dimension_irc_bridge_networks', 'column': 'isEnabled', 'default': ''} + - {'table': 'dimension_gitter_bridges', 'column': 'isEnabled', 'default': ''} + - {'table': 'dimension_custom_simple_bots', 'column': 'isEnabled', 'default': ''} + - {'table': 'dimension_custom_simple_bots', 'column': 'isPublic', 'default': ''} + - {'table': 'dimension_bridges', 'column': 'isEnabled', 'default': ''} + - {'table': 'dimension_bridges', 'column': 'isPublic', 'default': ''} + + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_dimension_sqlite_database_path_local }}" + dst: "{{ matrix_dimension_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_dimension_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-dimension.service'] + pgloader_options: ['--with "quote identifiers"'] + additional_psql_statements_list: "{{ matrix_dimension_pgloader_additional_psql_statements_list }}" + additional_psql_statements_db_name: "{{ matrix_dimension_database_name }}" + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_dimension_requires_restart: true + when: "matrix_dimension_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_dimension_database_engine == 'postgres'" + +- name: Ensure Dimension base path exists + file: + path: "{{ matrix_dimension_base_path }}" + state: directory + mode: 0770 + owner: "{{ matrix_user_username }}" + group: "{{ matrix_dimension_user_gid }}" + +- name: Ensure Dimension config installed + copy: + content: "{{ matrix_dimension_configuration|to_nice_yaml }}" + dest: "{{ matrix_dimension_base_path }}/config.yaml" + mode: 0640 + owner: "{{ matrix_user_username }}" + group: "{{ matrix_dimension_user_gid }}" + +- name: Ensure Dimension image is pulled + docker_image: + name: "{{ matrix_dimension_docker_image }}" + source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" + force_source: "{{ matrix_dimension_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" + force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_dimension_docker_image_force_pull }}" + +- name: Ensure matrix-dimension.service installed + template: + src: "{{ role_path }}/templates/systemd/matrix-dimension.service.j2" + dest: "{{ matrix_systemd_path }}/matrix-dimension.service" + mode: 0644 + register: matrix_dimension_systemd_service_result + +- name: Ensure systemd reloaded after matrix-dimension.service installation + service: + daemon_reload: yes + when: "matrix_dimension_systemd_service_result.changed|bool" + +- name: Ensure matrix-dimension.service restarted, if necessary + service: + name: "matrix-dimension.service" + state: restarted + when: "matrix_dimension_requires_restart|bool" diff --git a/roles/matrix-dimension/tasks/setup_uninstall.yml b/roles/matrix-dimension/tasks/setup_uninstall.yml new file mode 100644 index 00000000..9bc4ac8b --- /dev/null +++ b/roles/matrix-dimension/tasks/setup_uninstall.yml @@ -0,0 +1,35 @@ +--- + +- name: Check existence of matrix-dimension service + stat: + path: "{{ matrix_systemd_path }}/matrix-dimension.service" + register: matrix_dimension_service_stat + +- name: Ensure matrix-dimension is stopped + service: + name: matrix-dimension + state: stopped + daemon_reload: yes + register: stopping_result + when: "matrix_dimension_service_stat.stat.exists|bool" + +- name: Ensure matrix-dimension.service doesn't exist + file: + path: "{{ matrix_systemd_path }}/matrix-dimension.service" + state: absent + when: "matrix_dimension_service_stat.stat.exists|bool" + +- name: Ensure systemd reloaded after matrix-dimension.service removal + service: + daemon_reload: yes + when: "matrix_dimension_service_stat.stat.exists|bool" + +- name: Ensure Dimension base directory doesn't exist + file: + path: "{{ matrix_dimension_base_path }}" + state: absent + +- name: Ensure Dimension Docker image doesn't exist + docker_image: + name: "{{ matrix_dimension_docker_image }}" + state: absent diff --git a/roles/matrix-dimension/templates/config.yaml.j2 b/roles/matrix-dimension/templates/config.yaml.j2 index a05b6c35..200871e7 100644 --- a/roles/matrix-dimension/templates/config.yaml.j2 +++ b/roles/matrix-dimension/templates/config.yaml.j2 @@ -44,7 +44,11 @@ widgetBlacklist: # Where the database for Dimension is database: - file: "dimension.db" +{% if matrix_dimension_database_engine == 'sqlite' %} + file: {{ matrix_dimension_sqlite_database_path_in_container|to_json }} +{% elif matrix_dimension_database_engine == 'postgres' %} + uri: {{ matrix_dimension_database_connection_string|to_json }} +{% endif %} # Display settings that apply to self-hosted go-neb instances goneb: diff --git a/roles/matrix-dimension/templates/systemd/matrix-dimension.service.j2 b/roles/matrix-dimension/templates/systemd/matrix-dimension.service.j2 index bad94f64..ff10224a 100644 --- a/roles/matrix-dimension/templates/systemd/matrix-dimension.service.j2 +++ b/roles/matrix-dimension/templates/systemd/matrix-dimension.service.j2 @@ -1,8 +1,14 @@ #jinja2: lstrip_blocks: "True" [Unit] Description=Matrix Dimension -After=docker.service -Requires=docker.service +{% for service in matrix_dimension_systemd_required_services_list %} +Requires={{ service }} +After={{ service }} +{% endfor %} +{% for service in matrix_dimension_systemd_wanted_services_list %} +Wants={{ service }} +{% endfor %} +DefaultDependencies=no [Service] Type=simple @@ -10,7 +16,9 @@ ExecStartPre=-{{ matrix_host_command_docker }} kill matrix-dimension ExecStartPre=-{{ matrix_host_command_docker }} rm matrix-dimension # Fixup database ownership if it got changed somehow (during a server migration, etc.) -ExecStartPre=-{{ matrix_host_command_chown }} {{ matrix_dimension_user_uid }}:{{ matrix_dimension_user_gid }} {{ matrix_dimension_base_path }}/dimension.db +{% if matrix_dimension_database_engine == 'sqlite' %} +ExecStartPre=-{{ matrix_host_command_chown }} {{ matrix_dimension_user_uid }}:{{ matrix_dimension_user_gid }} {{ matrix_dimension_sqlite_database_path_local }} +{% endif %} ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-dimension \ --log-driver=none \ diff --git a/roles/matrix-dynamic-dns/templates/systemd/matrix-dynamic-dns.service.j2 b/roles/matrix-dynamic-dns/templates/systemd/matrix-dynamic-dns.service.j2 index 7e73b587..df7d810a 100644 --- a/roles/matrix-dynamic-dns/templates/systemd/matrix-dynamic-dns.service.j2 +++ b/roles/matrix-dynamic-dns/templates/systemd/matrix-dynamic-dns.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_dynamic_dns_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-email2matrix/templates/systemd/matrix-email2matrix.service.j2 b/roles/matrix-email2matrix/templates/systemd/matrix-email2matrix.service.j2 index 52806f35..1577877b 100644 --- a/roles/matrix-email2matrix/templates/systemd/matrix-email2matrix.service.j2 +++ b/roles/matrix-email2matrix/templates/systemd/matrix-email2matrix.service.j2 @@ -3,6 +3,7 @@ Description=Email2Matrix After=docker.service Requires=docker.service +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-jitsi/templates/jicofo/matrix-jitsi-jicofo.service.j2 b/roles/matrix-jitsi/templates/jicofo/matrix-jitsi-jicofo.service.j2 index f19d08ff..6b5cc941 100644 --- a/roles/matrix-jitsi/templates/jicofo/matrix-jitsi-jicofo.service.j2 +++ b/roles/matrix-jitsi/templates/jicofo/matrix-jitsi-jicofo.service.j2 @@ -5,6 +5,7 @@ Description=Matrix jitsi-jicofo server Requires={{ service }} After={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-jitsi/templates/jvb/matrix-jitsi-jvb.service.j2 b/roles/matrix-jitsi/templates/jvb/matrix-jitsi-jvb.service.j2 index 7fcfeec6..2931133f 100644 --- a/roles/matrix-jitsi/templates/jvb/matrix-jitsi-jvb.service.j2 +++ b/roles/matrix-jitsi/templates/jvb/matrix-jitsi-jvb.service.j2 @@ -5,6 +5,7 @@ Description=Matrix jitsi-jvb server Requires={{ service }} After={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-jitsi/templates/prosody/matrix-jitsi-prosody.service.j2 b/roles/matrix-jitsi/templates/prosody/matrix-jitsi-prosody.service.j2 index 74b13df3..4f532d89 100644 --- a/roles/matrix-jitsi/templates/prosody/matrix-jitsi-prosody.service.j2 +++ b/roles/matrix-jitsi/templates/prosody/matrix-jitsi-prosody.service.j2 @@ -5,6 +5,7 @@ Description=Matrix jitsi-prosody server Requires={{ service }} After={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-jitsi/templates/web/interface_config.js.j2 b/roles/matrix-jitsi/templates/web/interface_config.js.j2 index 60fac4d5..a12ca973 100644 --- a/roles/matrix-jitsi/templates/web/interface_config.js.j2 +++ b/roles/matrix-jitsi/templates/web/interface_config.js.j2 @@ -205,9 +205,11 @@ var interfaceConfig = { {% if matrix_jitsi_enable_transcriptions %} 'closedcaptions', {% endif %} - + {% if matrix_jitsi_enable_recording %} + 'recording', + {% endif %} 'microphone', 'camera', 'desktop', 'embedmeeting', 'fullscreen', - 'fodeviceselection', 'hangup', 'profile', 'chat', 'recording', + 'fodeviceselection', 'hangup', 'profile', 'chat', 'livestreaming', 'etherpad', 'sharedvideo', 'settings', 'raisehand', 'videoquality', 'filmstrip', 'invite', 'feedback', 'stats', 'shortcuts', 'tileview', 'videobackgroundblur', 'download', 'help', 'mute-everyone', 'security' diff --git a/roles/matrix-jitsi/templates/web/matrix-jitsi-web.service.j2 b/roles/matrix-jitsi/templates/web/matrix-jitsi-web.service.j2 index e13f9633..1978fb0e 100644 --- a/roles/matrix-jitsi/templates/web/matrix-jitsi-web.service.j2 +++ b/roles/matrix-jitsi/templates/web/matrix-jitsi-web.service.j2 @@ -5,6 +5,7 @@ Description=Matrix jitsi-web server Requires={{ service }} After={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-ma1sd/defaults/main.yml b/roles/matrix-ma1sd/defaults/main.yml index f7a0782d..28e3405c 100644 --- a/roles/matrix-ma1sd/defaults/main.yml +++ b/roles/matrix-ma1sd/defaults/main.yml @@ -39,6 +39,28 @@ matrix_ma1sd_systemd_wanted_services_list: [] # Enabling this is discouraged. Learn more here: https://github.com/ma1uta/ma1sd/blob/master/docs/features/identity.md#lookups matrix_ma1sd_matrixorg_forwarding_enabled: false + +# Database-related configuration fields. +# +# To use SQLite, stick to these defaults. +# +# To use Postgres: +# - change the engine (`matrix_ma1sd_database_engine: 'postgres'`) +# - adjust your database credentials via the `matrix_ma1sd_postgres_*` variables +matrix_ma1sd_database_engine: 'sqlite' + +matrix_ma1sd_sqlite_database_path_local: "{{ matrix_ma1sd_data_path }}/ma1sd.db" +matrix_ma1sd_sqlite_database_path_in_container: "/var/ma1sd/ma1sd.db" + +matrix_ma1sd_database_username: 'matrix_ma1sd' +matrix_ma1sd_database_password: 'some-password' +matrix_ma1sd_database_hostname: 'matrix-postgres' +matrix_ma1sd_database_port: 5432 +matrix_ma1sd_database_name: 'matrix_ma1sd' + +matrix_ma1sd_database_connection_string: 'postgresql://{{ matrix_ma1sd_database_username }}:{{ matrix_ma1sd_database_password }}@{{ matrix_ma1sd_database_hostname }}:{{ matrix_ma1sd_database_port }}/{{ matrix_ma1sd_database_name }}' + + # ma1sd has serveral supported identity stores. # One of them is storing identities directly in Synapse's database. # Learn more here: https://github.com/ma1uta/ma1sd/blob/master/docs/stores/synapse.md @@ -61,6 +83,17 @@ matrix_ma1sd_dns_overwrite_enabled: false matrix_ma1sd_dns_overwrite_homeserver_client_name: "{{ matrix_server_fqn_matrix }}" matrix_ma1sd_dns_overwrite_homeserver_client_value: "http://matrix-synapse:8008" +# Override the default session templates +# To use this, fill in the template variables with the full desired template as a multi-line YAML variable +# +# More info: +# https://github.com/ma1uta/ma1sd/blob/master/docs/threepids/session/session-views.md +matrix_ma1sd_view_session_custom_templates_enabled: false +# Defaults to: https://github.com/ma1uta/ma1sd/blob/master/src/main/resources/templates/session/tokenSubmitSuccess.html +matrix_ma1sd_view_session_custom_onTokenSubmit_success_template: "" +# Defaults to: https://github.com/ma1uta/ma1sd/blob/master/src/main/resources/templates/session/tokenSubmitFailure.html +matrix_ma1sd_view_session_custom_onTokenSubmit_failure_template: "" + # Override the default email templates # To use this, fill in the template variables with the full desired template as a multi-line YAML variable # @@ -68,9 +101,13 @@ matrix_ma1sd_dns_overwrite_homeserver_client_value: "http://matrix-synapse:8008" # https://github.com/ma1uta/ma1sd/blob/master/docs/threepids/notification/template-generator.md # https://github.com/ma1uta/ma1sd/tree/master/src/main/resources/threepids/email matrix_ma1sd_threepid_medium_email_custom_templates_enabled: false +# Defaults to: https://github.com/ma1uta/ma1sd/blob/master/src/main/resources/threepids/email/invite-template.eml matrix_ma1sd_threepid_medium_email_custom_invite_template: "" +# Defaults to: https://github.com/ma1uta/ma1sd/blob/master/src/main/resources/threepids/email/validate-template.eml matrix_ma1sd_threepid_medium_email_custom_session_validation_template: "" -matrix_ma1sd_threepid_medium_email_custom_unbind_fraudulent_template: "" +# Defaults to: https://github.com/ma1uta/ma1sd/blob/master/src/main/resources/threepids/email/unbind-notification.eml +matrix_ma1sd_threepid_medium_email_custom_session_unbind_notification_template: "" +# Defaults to: https://github.com/ma1uta/ma1sd/blob/master/src/main/resources/threepids/email/mxid-template.eml matrix_ma1sd_threepid_medium_email_custom_matrixid_template: "" # Controls whether the self-check feature should validate SSL certificates. diff --git a/roles/matrix-ma1sd/tasks/main.yml b/roles/matrix-ma1sd/tasks/main.yml index f5ac34d6..0b8a114e 100644 --- a/roles/matrix-ma1sd/tasks/main.yml +++ b/roles/matrix-ma1sd/tasks/main.yml @@ -8,7 +8,14 @@ - setup-all - setup-ma1sd -- import_tasks: "{{ role_path }}/tasks/setup_ma1sd.yml" +- import_tasks: "{{ role_path }}/tasks/setup_install.yml" + when: "run_setup|bool and matrix_ma1sd_enabled|bool" + tags: + - setup-all + - setup-ma1sd + +- import_tasks: "{{ role_path }}/tasks/setup_uninstall.yml" + when: "run_setup|bool and not matrix_ma1sd_enabled|bool" tags: - setup-all - setup-ma1sd diff --git a/roles/matrix-ma1sd/tasks/setup_ma1sd.yml b/roles/matrix-ma1sd/tasks/setup_install.yml similarity index 53% rename from roles/matrix-ma1sd/tasks/setup_ma1sd.yml rename to roles/matrix-ma1sd/tasks/setup_install.yml index 46acb428..a4a70ea4 100644 --- a/roles/matrix-ma1sd/tasks/setup_ma1sd.yml +++ b/roles/matrix-ma1sd/tasks/setup_install.yml @@ -1,9 +1,5 @@ --- -# -# Tasks related to setting up ma1sd -# - - name: Ensure ma1sd paths exist file: path: "{{ item.path }}" @@ -15,10 +11,39 @@ - { path: "{{ matrix_ma1sd_config_path }}", when: true } - { path: "{{ matrix_ma1sd_data_path }}", when: true } - { path: "{{ matrix_ma1sd_docker_src_files_path }}", when: "{{ matrix_ma1sd_container_image_self_build }}"} - when: matrix_ma1sd_enabled|bool and item.when + when: "item.when|bool" - import_tasks: "{{ role_path }}/tasks/migrate_mxisd.yml" - when: matrix_ma1sd_enabled|bool + + +# These (SQLite -> Postgres) migration tasks are usually at the top, +# but we'd like to run them after `migrate_mxisd.yml`, which requires the ma1sd paths to exist. +- set_fact: + matrix_ma1sd_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_ma1sd_sqlite_database_path_local }}" + register: matrix_ma1sd_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_ma1sd_sqlite_database_path_local }}" + dst: "{{ matrix_ma1sd_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_ma1sd_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-ma1sd.service'] + pgloader_options: ['--with "quote identifiers"'] + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_ma1sd_requires_restart: true + when: "matrix_ma1sd_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_ma1sd_database_engine == 'postgres'" - name: Ensure ma1sd image is pulled docker_image: @@ -26,7 +51,7 @@ source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" force_source: "{{ matrix_ma1sd_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_ma1sd_docker_image_force_pull }}" - when: matrix_ma1sd_enabled|bool and not matrix_ma1sd_container_image_self_build + when: "not matrix_ma1sd_container_image_self_build|bool" - block: - name: Ensure gradle is installed for self-building (Debian) @@ -72,7 +97,7 @@ repository: "{{ matrix_ma1sd_docker_image }}" force_tag: yes source: local - when: "matrix_ma1sd_enabled|bool and matrix_ma1sd_container_image_self_build" + when: "matrix_ma1sd_container_image_self_build|bool" - name: Ensure ma1sd config installed copy: @@ -81,21 +106,44 @@ mode: 0644 owner: "{{ matrix_user_username }}" group: "{{ matrix_user_groupname }}" - when: matrix_ma1sd_enabled|bool -- name: Ensure custom templates are installed if any +- name: Ensure custom view templates are installed, if any + copy: + content: "{{ item.value }}" + dest: "{{ matrix_ma1sd_config_path }}/{{ item.location }}" + mode: 0644 + owner: "{{ matrix_user_username }}" + group: "{{ matrix_user_groupname }}" + with_items: + - {value: "{{ matrix_ma1sd_view_session_custom_onTokenSubmit_success_template }}", location: 'tokenSubmitSuccess.html'} + - {value: "{{ matrix_ma1sd_view_session_custom_onTokenSubmit_failure_template }}", location: 'tokenSubmitFailure.html'} + when: "matrix_ma1sd_view_session_custom_templates_enabled|bool and item.value" + +- name: Ensure custom email templates are installed, if any copy: content: "{{ item.value }}" - dest: "{{ matrix_ma1sd_data_path }}/{{ item.location }}" + dest: "{{ matrix_ma1sd_config_path }}/{{ item.location }}" mode: 0644 owner: "{{ matrix_user_username }}" group: "{{ matrix_user_groupname }}" with_items: - {value: "{{ matrix_ma1sd_threepid_medium_email_custom_invite_template }}", location: 'invite-template.eml'} - {value: "{{ matrix_ma1sd_threepid_medium_email_custom_session_validation_template }}", location: 'validate-template.eml'} - - {value: "{{ matrix_ma1sd_threepid_medium_email_custom_unbind_fraudulent_template }}", location: 'unbind-fraudulent.eml'} + - {value: "{{ matrix_ma1sd_threepid_medium_email_custom_session_unbind_notification_template }}", location: 'unbind-notification.eml'} - {value: "{{ matrix_ma1sd_threepid_medium_email_custom_matrixid_template }}", location: 'mxid-template.eml'} - when: "matrix_ma1sd_enabled|bool and matrix_ma1sd_threepid_medium_email_custom_templates_enabled|bool and item.value" + when: "matrix_ma1sd_threepid_medium_email_custom_templates_enabled|bool and item.value" + +# Only cleaning up for people who define the respective templates +- name: (Cleanup) Ensure custom email templates are not in data/ anymore (we've put them in config/) + file: + path: "{{ matrix_ma1sd_data_path }}/{{ item.location }}" + state: absent + with_items: + - {value: "{{ matrix_ma1sd_threepid_medium_email_custom_invite_template }}", location: 'invite-template.eml'} + - {value: "{{ matrix_ma1sd_threepid_medium_email_custom_session_validation_template }}", location: 'validate-template.eml'} + - {value: "{{ matrix_ma1sd_threepid_medium_email_custom_session_unbind_notification_template }}", location: 'unbind-notification.eml'} + - {value: "{{ matrix_ma1sd_threepid_medium_email_custom_matrixid_template }}", location: 'mxid-template.eml'} + when: "matrix_ma1sd_threepid_medium_email_custom_templates_enabled|bool and item.value" - name: Ensure matrix-ma1sd.service installed template: @@ -103,49 +151,14 @@ dest: "{{ matrix_systemd_path }}/matrix-ma1sd.service" mode: 0644 register: matrix_ma1sd_systemd_service_result - when: matrix_ma1sd_enabled|bool - name: Ensure systemd reloaded after matrix-ma1sd.service installation service: daemon_reload: yes - when: "matrix_ma1sd_enabled|bool and matrix_ma1sd_systemd_service_result.changed" - -# -# Tasks related to getting rid of ma1sd (if it was previously enabled) -# - -- name: Check existence of matrix-ma1sd service - stat: - path: "{{ matrix_systemd_path }}/matrix-ma1sd.service" - register: matrix_ma1sd_service_stat - -- name: Ensure matrix-ma1sd is stopped - service: - name: matrix-ma1sd - state: stopped - daemon_reload: yes - register: stopping_result - when: "not matrix_ma1sd_enabled|bool and matrix_ma1sd_service_stat.stat.exists" - -- name: Ensure matrix-ma1sd.service doesn't exist - file: - path: "{{ matrix_systemd_path }}/matrix-ma1sd.service" - state: absent - when: "not matrix_ma1sd_enabled|bool and matrix_ma1sd_service_stat.stat.exists" + when: "matrix_ma1sd_systemd_service_result.changed|bool" -- name: Ensure systemd reloaded after matrix-ma1sd.service removal +- name: Ensure matrix-ma1sd.service restarted, if necessary service: - daemon_reload: yes - when: "not matrix_ma1sd_enabled|bool and matrix_ma1sd_service_stat.stat.exists" - -- name: Ensure Matrix ma1sd paths don't exist - file: - path: "{{ matrix_ma1sd_base_path }}" - state: absent - when: "not matrix_ma1sd_enabled|bool" - -- name: Ensure ma1sd Docker image doesn't exist - docker_image: - name: "{{ matrix_ma1sd_docker_image }}" - state: absent - when: "not matrix_ma1sd_enabled|bool" + name: "matrix-ma1sd.service" + state: restarted + when: "matrix_ma1sd_requires_restart|bool" diff --git a/roles/matrix-ma1sd/tasks/setup_uninstall.yml b/roles/matrix-ma1sd/tasks/setup_uninstall.yml new file mode 100644 index 00000000..b36ab508 --- /dev/null +++ b/roles/matrix-ma1sd/tasks/setup_uninstall.yml @@ -0,0 +1,35 @@ +--- + +- name: Check existence of matrix-ma1sd service + stat: + path: "{{ matrix_systemd_path }}/matrix-ma1sd.service" + register: matrix_ma1sd_service_stat + +- name: Ensure matrix-ma1sd is stopped + service: + name: matrix-ma1sd + state: stopped + daemon_reload: yes + register: stopping_result + when: "matrix_ma1sd_service_stat.stat.exists|bool" + +- name: Ensure matrix-ma1sd.service doesn't exist + file: + path: "{{ matrix_systemd_path }}/matrix-ma1sd.service" + state: absent + when: "matrix_ma1sd_service_stat.stat.exists|bool" + +- name: Ensure systemd reloaded after matrix-ma1sd.service removal + service: + daemon_reload: yes + when: "matrix_ma1sd_service_stat.stat.exists|bool" + +- name: Ensure Matrix ma1sd paths don't exist + file: + path: "{{ matrix_ma1sd_base_path }}" + state: absent + +- name: Ensure ma1sd Docker image doesn't exist + docker_image: + name: "{{ matrix_ma1sd_docker_image }}" + state: absent diff --git a/roles/matrix-ma1sd/tasks/validate_config.yml b/roles/matrix-ma1sd/tasks/validate_config.yml index 691fe0ac..4ca25e7e 100644 --- a/roles/matrix-ma1sd/tasks/validate_config.yml +++ b/roles/matrix-ma1sd/tasks/validate_config.yml @@ -54,6 +54,7 @@ when: "vars | dict2items | selectattr('key', 'match', item.old) | list | items2dict" with_items: - {'old': 'matrix_ma1sd_container_expose_port', 'new': ''} + - {'old': 'matrix_ma1sd_threepid_medium_email_custom_unbind_fraudulent_template', 'new': 'matrix_ma1sd_threepid_medium_email_custom_session_unbind_notification_template'} - name: (Deprecation) Catch and report mxisd variables fail: diff --git a/roles/matrix-ma1sd/templates/ma1sd.yaml.j2 b/roles/matrix-ma1sd/templates/ma1sd.yaml.j2 index 8f3569b1..a4100adc 100644 --- a/roles/matrix-ma1sd/templates/ma1sd.yaml.j2 +++ b/roles/matrix-ma1sd/templates/ma1sd.yaml.j2 @@ -11,9 +11,19 @@ key: path: /var/ma1sd/sign.key storage: - provider: - sqlite: - database: /var/ma1sd/ma1sd.db + {% if matrix_ma1sd_database_engine == 'sqlite' %} + backend: sqlite + provider: + sqlite: + database: {{ matrix_ma1sd_sqlite_database_path_in_container|to_json }} + {% elif matrix_ma1sd_database_engine == 'postgres' %} + backend: postgresql + provider: + postgresql: + database: //{{ matrix_ma1sd_database_hostname }}:{{ matrix_ma1sd_database_port }}/{{ matrix_ma1sd_database_name }} + username: {{ matrix_ma1sd_database_username|to_json }} + password: {{ matrix_ma1sd_database_password|to_json }} + {% endif %} {% if matrix_ma1sd_dns_overwrite_enabled %} dns: @@ -45,24 +55,36 @@ threepid: generators: template: {% if matrix_ma1sd_threepid_medium_email_custom_invite_template %} - invite: '/var/ma1sd/invite-template.eml' + invite: '/etc/ma1sd/invite-template.eml' {% endif %} - {% if matrix_ma1sd_threepid_medium_email_custom_session_validation_template or matrix_ma1sd_threepid_medium_email_custom_unbind_fraudulent_template %} + {% if matrix_ma1sd_threepid_medium_email_custom_session_validation_template or matrix_ma1sd_threepid_medium_email_custom_session_unbind_notification_template %} session: {% if matrix_ma1sd_threepid_medium_email_custom_session_validation_template %} - validation: '/var/ma1sd/validate-template.eml' + validation: '/etc/ma1sd/validate-template.eml' {% endif %} - {% if matrix_ma1sd_threepid_medium_email_custom_unbind_fraudulent_template %} + {% if matrix_ma1sd_threepid_medium_email_custom_session_unbind_notification_template %} unbind: - frandulent: '/var/ma1sd/unbind-fraudulent.eml' + notification: '/etc/ma1sd/unbind-notification.eml' {% endif %} {% endif %} {% if matrix_ma1sd_threepid_medium_email_custom_matrixid_template %} generic: - matrixId: '/var/ma1sd/mxid-template.eml' + matrixId: '/etc/ma1sd/mxid-template.eml' {% endif %} {% endif %} +{% if matrix_ma1sd_view_session_custom_templates_enabled %} +view: + session: + onTokenSubmit: + {% if matrix_ma1sd_view_session_custom_onTokenSubmit_success_template %} + success: '/etc/ma1sd/tokenSubmitSuccess.html' + {% endif %} + {% if matrix_ma1sd_view_session_custom_onTokenSubmit_failure_template %} + failure: '/etc/ma1sd/tokenSubmitFailure.html' + {% endif %} +{% endif %} + {% if matrix_ma1sd_hashing_enabled %} hashing: enabled: true # enable or disable the hash lookup MSC2140 (default is false) @@ -73,10 +95,10 @@ hashing: - none # the same as v1 bulk lookup - sha256 # hash the 3PID and pepper. delay: 2m # how often hashes will be updated if rotation policy = per_seconds (default is 10s) - requests: 10 + requests: 10 {% endif %} synapseSql: - enabled: {{ matrix_ma1sd_synapsesql_enabled }} - type: {{ matrix_ma1sd_synapsesql_type }} - connection: {{ matrix_ma1sd_synapsesql_connection }} + enabled: {{ matrix_ma1sd_synapsesql_enabled|to_json }} + type: {{ matrix_ma1sd_synapsesql_type|to_json }} + connection: {{ matrix_ma1sd_synapsesql_connection|to_json }} diff --git a/roles/matrix-ma1sd/templates/systemd/matrix-ma1sd.service.j2 b/roles/matrix-ma1sd/templates/systemd/matrix-ma1sd.service.j2 index 3c49ec75..95f15254 100644 --- a/roles/matrix-ma1sd/templates/systemd/matrix-ma1sd.service.j2 +++ b/roles/matrix-ma1sd/templates/systemd/matrix-ma1sd.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_ma1sd_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-mailer/templates/systemd/matrix-mailer.service.j2 b/roles/matrix-mailer/templates/systemd/matrix-mailer.service.j2 index 1371a861..9345a1d6 100644 --- a/roles/matrix-mailer/templates/systemd/matrix-mailer.service.j2 +++ b/roles/matrix-mailer/templates/systemd/matrix-mailer.service.j2 @@ -3,6 +3,7 @@ Description=Matrix mailer After=docker.service Requires=docker.service +DefaultDependencies=no [Service] Type=simple @@ -18,7 +19,7 @@ ExecStart={{ matrix_host_command_docker }} run --rm --name matrix-mailer \ --tmpfs=/var/spool/exim:rw,noexec,nosuid,size=100m \ --network={{ matrix_docker_network }} \ --env-file={{ matrix_mailer_base_path }}/env-mailer \ - --hostname={{ matrix_server_fqn_matrix }} \ + --hostname=matrix-mailer \ {% for arg in matrix_mailer_container_extra_arguments %} {{ arg }} \ {% endfor %} diff --git a/roles/matrix-nginx-proxy/defaults/main.yml b/roles/matrix-nginx-proxy/defaults/main.yml index 408a5f18..ddacf222 100644 --- a/roles/matrix-nginx-proxy/defaults/main.yml +++ b/roles/matrix-nginx-proxy/defaults/main.yml @@ -3,7 +3,7 @@ matrix_nginx_proxy_enabled: true # We use an official nginx image, which we fix-up to run unprivileged. # An alternative would be an `nginxinc/nginx-unprivileged` image, but # that is frequently out of date. -matrix_nginx_proxy_docker_image: "docker.io/nginx:1.19.4-alpine" +matrix_nginx_proxy_docker_image: "docker.io/nginx:1.19.6-alpine" matrix_nginx_proxy_docker_image_force_pull: "{{ matrix_nginx_proxy_docker_image.endswith(':latest') }}" matrix_nginx_proxy_base_path: "{{ matrix_base_data_path }}/nginx-proxy" @@ -92,6 +92,8 @@ matrix_nginx_proxy_base_domain_homepage_template: |- +# Option to disable the access log +matrix_nginx_proxy_access_log_enabled: true # Controls whether proxying the riot domain should be done. matrix_nginx_proxy_proxy_riot_compat_redirect_enabled: false @@ -259,7 +261,7 @@ matrix_ssl_domains_to_obtain_certificates_for: [] # Controls whether to obtain production or staging certificates from Let's Encrypt. matrix_ssl_lets_encrypt_staging: false -matrix_ssl_lets_encrypt_certbot_docker_image: "docker.io/certbot/certbot:{{ matrix_ssl_architecture }}-v1.9.0" +matrix_ssl_lets_encrypt_certbot_docker_image: "docker.io/certbot/certbot:{{ matrix_ssl_architecture }}-v1.10.1" matrix_ssl_lets_encrypt_certbot_docker_image_force_pull: "{{ matrix_ssl_lets_encrypt_certbot_docker_image.endswith(':latest') }}" matrix_ssl_lets_encrypt_certbot_standalone_http_port: 2402 matrix_ssl_lets_encrypt_support_email: ~ diff --git a/roles/matrix-nginx-proxy/templates/nginx/nginx.conf.j2 b/roles/matrix-nginx-proxy/templates/nginx/nginx.conf.j2 index 51aa8a00..975c8b4f 100644 --- a/roles/matrix-nginx-proxy/templates/nginx/nginx.conf.j2 +++ b/roles/matrix-nginx-proxy/templates/nginx/nginx.conf.j2 @@ -33,7 +33,11 @@ http { '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; + {% if matrix_nginx_proxy_access_log_enabled %} access_log /var/log/nginx/access.log main; + {% else %} + access_log off; + {% endif %} sendfile on; #tcp_nopush on; diff --git a/roles/matrix-nginx-proxy/templates/systemd/matrix-nginx-proxy.service.j2 b/roles/matrix-nginx-proxy/templates/systemd/matrix-nginx-proxy.service.j2 index 0945c25e..58f5c953 100644 --- a/roles/matrix-nginx-proxy/templates/systemd/matrix-nginx-proxy.service.j2 +++ b/roles/matrix-nginx-proxy/templates/systemd/matrix-nginx-proxy.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_nginx_proxy_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-postgres/defaults/main.yml b/roles/matrix-postgres/defaults/main.yml index 37538721..1dfe2026 100644 --- a/roles/matrix-postgres/defaults/main.yml +++ b/roles/matrix-postgres/defaults/main.yml @@ -35,4 +35,49 @@ matrix_postgres_process_extra_arguments: [] # Takes an ":" or "" value (e.g. "127.0.0.1:5432"), or empty string to not expose. matrix_postgres_container_postgres_bind_port: "" -matrix_postgres_tool_synapse_janitor: "https://raw.githubusercontent.com/xwiki-labs/synapse_scripts/a9188ff175ae581610f92d58ea6eac9a114d854b/synapse_janitor.sql" +# A list of additional (databases and their credentials) to create. +# +# Example: +# matrix_postgres_additional_databases: +# - name: matrix_appservice_discord +# username: matrix_appservice_discord +# password: some_password +# - name: matrix_appservice_slack +# username: matrix_appservice_slack +# password: some_password +matrix_postgres_additional_databases: [] + +# A list of roles/users to avoid creating when importing (or upgrading) the database. +# If a dump file contains the roles and they've also been created beforehand (see `matrix_postgres_additional_databases`), +# importing would fail. +# We either need to not create them or to ignore the `CREATE ROLE` statements in the dump. +matrix_postgres_import_roles_to_ignore: [matrix_postgres_connection_username] + +matrix_postgres_import_roles_ignore_regex: "^CREATE ROLE ({{ matrix_postgres_import_roles_to_ignore|join('|') }});" + +# A list of databases to avoid creating when importing (or upgrading) the database. +# If a dump file contains the databases and they've also been created beforehand (see `matrix_postgres_additional_databases`), +# importing would fail. +# We either need to not create them or to ignore the `CREATE DATABASE` statements in the dump. +matrix_postgres_import_databases_to_ignore: [matrix_postgres_db_name] + +matrix_postgres_import_databases_ignore_regex: "^CREATE DATABASE ({{ matrix_postgres_import_databases_to_ignore|join('|') }})\\s" + +# The number of seconds to wait after starting `matrix-postgres.service` +# and before trying to run queries for creating additional databases/users against it. +# +# For most (subsequent) runs, Postgres would already be running, so no waiting will be happening at all. +matrix_postgres_additional_databases_postgres_start_wait_timeout_seconds: 15 + + +matrix_postgres_pgloader_container_image_self_build: false +matrix_postgres_pgloader_container_image_self_build_repo: "https://github.com/illagrenan/pgloader-docker.git" +matrix_postgres_pgloader_container_image_self_build_repo_branch: "v{{ matrix_postgres_pgloader_docker_image_tag }}" +matrix_postgres_pgloader_container_image_self_build_src_path: "{{ matrix_postgres_base_path }}/pgloader-container-src" + +# We use illagrenan/pgloader, instead of the more official dimitri/pgloader image, +# because the official one only provides a `latest` tag. +matrix_postgres_pgloader_docker_image: "{{ matrix_postgres_pgloader_docker_image_name_prefix }}illagrenan/pgloader:{{ matrix_postgres_pgloader_docker_image_tag }}" +matrix_postgres_pgloader_docker_image_name_prefix: "{{ 'localhost/' if matrix_postgres_pgloader_container_image_self_build else 'docker.io/' }}" +matrix_postgres_pgloader_docker_image_tag: "3.6.2" +matrix_postgres_pgloader_docker_image_force_pull: "{{ matrix_postgres_pgloader_docker_image.endswith(':latest') }}" diff --git a/roles/matrix-postgres/tasks/import_generic_sqlite_db.yml b/roles/matrix-postgres/tasks/import_generic_sqlite_db.yml new file mode 100644 index 00000000..a42c6f55 --- /dev/null +++ b/roles/matrix-postgres/tasks/import_generic_sqlite_db.yml @@ -0,0 +1,97 @@ +--- + +# Pre-checks + +- name: Fail if Postgres not enabled + fail: + msg: "Postgres via the matrix-postgres role is not enabled (`matrix_postgres_enabled`). Cannot import." + when: "not matrix_postgres_enabled|bool" + +- name: Fail if playbook called incorrectly + fail: + msg: "The `sqlite_database_path` variable needs to be provided to this playbook, via --extra-vars" + when: "sqlite_database_path is not defined or sqlite_database_path.startswith('<')" + +- name: Check if the provided SQLite database file exists + stat: + path: "{{ sqlite_database_path }}" + register: sqlite_database_path_stat_result + +- name: Fail if provided SQLite database file doesn't exist + fail: + msg: "File cannot be found on the server at {{ sqlite_database_path }}" + when: "not sqlite_database_path_stat_result.stat.exists" + +# We either expect `postgres_db_connection_string` specifying a full Postgres database connection string, +# or `postgres_connection_string_variable_name`, specifying a name of a variable, which contains a valid connection string. + +- block: + - name: Fail if postgres_connection_string_variable_name points to an undefined variable + fail: msg="postgres_connection_string_variable_name is defined, but there is no variable with the name `{{ postgres_connection_string_variable_name }}`" + when: "postgres_connection_string_variable_name not in vars" + + - name: Get Postgres connection string from variable + set_fact: + postgres_db_connection_string: "{{ lookup('vars', postgres_connection_string_variable_name) }}" + when: 'postgres_connection_string_variable_name is defined' + +- name: Fail if playbook called incorrectly + fail: + msg: >- + Either a `postgres_db_connection_string` variable or a `postgres_connection_string_variable_name` needs to be provided to this playbook, via `--extra-vars`. + Example: `--extra-vars="postgres_db_connection_string=postgresql://username:password@localhost:/database_name"` or `--extra-vars="postgres_connection_string_variable_name=matrix_appservice_discord_database_connString"` + when: "postgres_db_connection_string is not defined or not postgres_db_connection_string.startswith('postgresql://')" + + +# Defaults + +- name: Set postgres_start_wait_time, if not provided + set_fact: + postgres_start_wait_time: 15 + when: "postgres_start_wait_time|default('') == ''" + + +# Actual import work + +- name: Ensure matrix-postgres is started + service: + name: matrix-postgres + state: started + daemon_reload: yes + register: matrix_postgres_service_start_result + +- name: Wait a bit, so that Postgres can start + wait_for: + timeout: "{{ postgres_start_wait_time }}" + delegate_to: 127.0.0.1 + become: false + when: "matrix_postgres_service_start_result.changed|bool" + +- name: Import SQLite database from {{ sqlite_database_path }} into Postgres + command: + cmd: >- + {{ matrix_host_command_docker }} run + --rm + --user={{ matrix_user_uid }}:{{ matrix_user_gid }} + --cap-drop=ALL + --network={{ matrix_docker_network }} + --mount type=bind,src={{ sqlite_database_path }},dst=/in.db,ro + --entrypoint=/bin/sh + {{ matrix_postgres_pgloader_docker_image }} + -c + 'pgloader /in.db {{ postgres_db_connection_string }}' + +- name: Archive SQLite database ({{ sqlite_database_path }} -> {{ sqlite_database_path }}.backup) + command: + cmd: "mv {{ sqlite_database_path }} {{ sqlite_database_path }}.backup" + +- name: Inject result + set_fact: + matrix_playbook_runtime_results: | + {{ + matrix_playbook_runtime_results|default([]) + + + [ + "NOTE: Your SQLite database file has been imported into Postgres. The original file has been moved from `{{ sqlite_database_path }}` to `{{ sqlite_database_path }}.backup`. When you've confirmed that the import went well and everything works, you should be able to safely delete this file." + ] + }} diff --git a/roles/matrix-postgres/tasks/import_postgres.yml b/roles/matrix-postgres/tasks/import_postgres.yml index 33d98691..c26affbb 100644 --- a/roles/matrix-postgres/tasks/import_postgres.yml +++ b/roles/matrix-postgres/tasks/import_postgres.yml @@ -74,8 +74,8 @@ {{ matrix_postgres_docker_image_latest }} -c "cat /{{ server_path_postgres_dump|basename }} | {{ 'gunzip |' if server_path_postgres_dump.endswith('.gz') else '' }} - grep -vE '^CREATE ROLE {{ matrix_postgres_connection_username }}' | - grep -vE '^CREATE DATABASE {{ matrix_postgres_db_name }}' | + grep -vE '{{ matrix_postgres_import_roles_ignore_regex }}' | + grep -vE '{{ matrix_postgres_import_databases_ignore_regex }}' | psql -v ON_ERROR_STOP=1 -h matrix-postgres" # This is a hack. diff --git a/roles/matrix-postgres/tasks/import_sqlite_db.yml b/roles/matrix-postgres/tasks/import_synapse_sqlite_db.yml similarity index 100% rename from roles/matrix-postgres/tasks/import_sqlite_db.yml rename to roles/matrix-postgres/tasks/import_synapse_sqlite_db.yml diff --git a/roles/matrix-postgres/tasks/main.yml b/roles/matrix-postgres/tasks/main.yml index 41b9c861..b9c2ae7c 100644 --- a/roles/matrix-postgres/tasks/main.yml +++ b/roles/matrix-postgres/tasks/main.yml @@ -19,21 +19,24 @@ tags: - import-postgres -- import_tasks: "{{ role_path }}/tasks/import_sqlite_db.yml" +# The `run_postgres_import_sqlite_db` variable had better be renamed to be consistent, +# but that's a breaking change which may cause trouble for people. +- import_tasks: "{{ role_path }}/tasks/import_synapse_sqlite_db.yml" when: run_postgres_import_sqlite_db|bool tags: - - import-sqlite-db + - import-synapse-sqlite-db + +# Perhaps we need a new variable here, instead of `run_postgres_import_sqlite_db`. +- import_tasks: "{{ role_path }}/tasks/import_generic_sqlite_db.yml" + when: run_postgres_import_sqlite_db|bool + tags: + - import-generic-sqlite-db - import_tasks: "{{ role_path }}/tasks/upgrade_postgres.yml" when: run_postgres_upgrade|bool tags: - upgrade-postgres -- import_tasks: "{{ role_path }}/tasks/run_synapse_janitor.yml" - when: run_postgres_synapse_janitor|bool - tags: - - run-postgres-synapse-janitor - - import_tasks: "{{ role_path }}/tasks/run_vacuum.yml" when: run_postgres_vacuum|bool tags: diff --git a/roles/matrix-postgres/tasks/run_synapse_janitor.yml b/roles/matrix-postgres/tasks/run_synapse_janitor.yml deleted file mode 100644 index d7f283be..00000000 --- a/roles/matrix-postgres/tasks/run_synapse_janitor.yml +++ /dev/null @@ -1,117 +0,0 @@ ---- - -# Pre-checks - -- name: Fail if Postgres not enabled - fail: - msg: "Postgres via the matrix-postgres role is not enabled (`matrix_postgres_enabled`). Cannot run synapse-janitor." - when: "not matrix_postgres_enabled|bool" - -- name: Fail if not aware of the risks - fail: - msg: >- - Using Synapse Janitor is considered dangerous and may break your database. - See https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/465. - If you'd like to run it anyway, add `--extra-vars='i_know_synapse_janitor_is_dangerous=1'` to your command. - when: "i_know_synapse_janitor_is_dangerous|default('') == ''" - -# Defaults - -- name: Set postgres_start_wait_time, if not provided - set_fact: - postgres_start_wait_time: 15 - when: "postgres_start_wait_time|default('') == ''" - -- name: Set postgres_synapse_janitor_wait_time, if not provided - set_fact: - postgres_synapse_janitor_wait_time: "{{ 7 * 86400 }}" - when: "postgres_synapse_janitor_wait_time|default('') == ''" - -- name: Set postgres_synapse_janitor_tool_path, if not provided - set_fact: - postgres_synapse_janitor_tool_path: "{{ matrix_postgres_base_path }}/synapse_janitor.sql" - when: "postgres_synapse_janitor_tool_path|default('') == ''" - - -# Actual janitor work - -- name: Download synapse-janitor tool - get_url: - url: "{{ matrix_postgres_tool_synapse_janitor }}" - dest: "{{ postgres_synapse_janitor_tool_path }}" - force: true - mode: 0550 - owner: "{{ matrix_user_username }}" - group: "{{ matrix_user_groupname }}" - -- name: Ensure matrix-postgres is started - service: - name: matrix-postgres - state: started - daemon_reload: yes - -- name: Wait a bit, so that Postgres can start - wait_for: - timeout: "{{ postgres_start_wait_time }}" - delegate_to: 127.0.0.1 - become: false - -- import_tasks: tasks/util/detect_existing_postgres_version.yml - -- name: Abort, if no existing Postgres version detected - fail: - msg: "Could not find existing Postgres installation" - when: "not matrix_postgres_detected_existing|bool" - -- name: Generate Postgres database synapse-janitor command - set_fact: - matrix_postgres_synapse_janitor_command: >- - {{ matrix_host_command_docker }} run --rm --name matrix-postgres-synapse-janitor - --user={{ matrix_user_uid }}:{{ matrix_user_gid }} - --cap-drop=ALL - --network={{ matrix_docker_network }} - --env-file={{ matrix_postgres_base_path }}/env-postgres-psql - --mount type=bind,src={{ postgres_synapse_janitor_tool_path }},dst=/synapse_janitor.sql,ro=true - {{ matrix_postgres_docker_image_latest }} - psql -v ON_ERROR_STOP=1 -h matrix-postgres {{ matrix_synapse_database_database }} -f /synapse_janitor.sql - -- name: Note about Postgres purging alternative - debug: - msg: >- - Running synapse-janitor with the following Postgres command: `{{ matrix_postgres_synapse_janitor_command }}`. - If this crashes, you can stop all processes (`systemctl stop matrix-*`), - start Postgres only (`systemctl start matrix-postgres`) - and manually run the above command directly on the server. - -- name: Populate service facts - service_facts: - -- set_fact: - matrix_postgres_synapse_was_running: "{{ ansible_facts.services['matrix-synapse.service']|default(none) is not none and ansible_facts.services['matrix-synapse.service'].state == 'running' }}" - -- name: Ensure matrix-synapse is stopped - service: - name: matrix-synapse - state: stopped - daemon_reload: yes - -- name: Run synapse-janitor - command: "{{ matrix_postgres_synapse_janitor_command }}" - async: "{{ postgres_synapse_janitor_wait_time }}" - poll: 10 - register: matrix_postgres_synapse_janitor_result - -# Intentionally show the results -- debug: var="matrix_postgres_synapse_janitor_result" - -- name: Ensure matrix-synapse is started, if it previously was - service: - name: matrix-synapse - state: started - daemon_reload: yes - when: "matrix_postgres_synapse_was_running|bool" - -- name: Delete synapse-janitor tool - file: - path: "{{ postgres_synapse_janitor_tool_path }}" - state: absent diff --git a/roles/matrix-postgres/tasks/setup_postgres.yml b/roles/matrix-postgres/tasks/setup_postgres.yml index f186bdca..518d1a5f 100644 --- a/roles/matrix-postgres/tasks/setup_postgres.yml +++ b/roles/matrix-postgres/tasks/setup_postgres.yml @@ -113,6 +113,13 @@ daemon_reload: yes when: "matrix_postgres_enabled|bool and matrix_postgres_systemd_service_result.changed" +- include_tasks: + file: "{{ role_path }}/tasks/util/create_additional_databases.yml" + apply: + tags: + - always + when: "matrix_postgres_enabled|bool and matrix_postgres_additional_databases|length > 0" + # # Tasks related to getting rid of the internal postgres server (if it was previously enabled) # diff --git a/roles/matrix-postgres/tasks/upgrade_postgres.yml b/roles/matrix-postgres/tasks/upgrade_postgres.yml index 72f327b3..564265d8 100644 --- a/roles/matrix-postgres/tasks/upgrade_postgres.yml +++ b/roles/matrix-postgres/tasks/upgrade_postgres.yml @@ -135,8 +135,8 @@ {{ matrix_postgres_docker_image_latest }} -c "cat /in/{{ postgres_dump_name }} | {{ 'gunzip |' if postgres_dump_name.endswith('.gz') else '' }} - grep -vE '^CREATE ROLE {{ matrix_postgres_connection_username }}' | - grep -vE '^CREATE DATABASE {{ matrix_postgres_db_name }}' | + grep -vE '{{ matrix_postgres_import_roles_ignore_regex }}' | + grep -vE '{{ matrix_postgres_import_databases_ignore_regex }}' | psql -v ON_ERROR_STOP=1 -h matrix-postgres" # This is a hack. diff --git a/roles/matrix-postgres/tasks/util/create_additional_database.yml b/roles/matrix-postgres/tasks/util/create_additional_database.yml new file mode 100644 index 00000000..22b3c9a2 --- /dev/null +++ b/roles/matrix-postgres/tasks/util/create_additional_database.yml @@ -0,0 +1,40 @@ +--- + +# It'd be better if this is belonged to `validate_config.yml`, but it would have to be some loop-within-a-loop there, +# and that's ugly. We also don't expect this to catch errors often. It's more of a defensive last-minute check. +- name: Fail if additional database data appears invalid + fail: + msg: "Additional database definition ({{ additional_db }} lacks a required key: {{ item }}" + when: "item not in additional_db" + with_items: "{{ ['name', 'username', 'password'] }}" + +# The SQL statements that we'll run against Postgres are stored in a file that others can't read. +# This file will be mounted into the container and fed to Postgres. +# This way, we avoid passing sensitive data around in CLI commands that other users on the system can see. +- name: Create additional database initialization SQL file for {{ additional_db.name }} + template: + src: "{{ role_path }}/templates/sql/init-additional-db-user-and-role.sql.j2" + dest: "/tmp/matrix-postgres-init-additional-db-user-and-role.sql" + mode: 0600 + owner: "{{ matrix_user_uid }}" + group: "{{ matrix_user_gid }}" + +- name: Execute Postgres additional database initialization SQL file for {{ additional_db.name }} + command: + cmd: >- + {{ matrix_host_command_docker }} run + --rm + --user={{ matrix_user_uid }}:{{ matrix_user_gid }} + --cap-drop=ALL + --env-file={{ matrix_postgres_base_path }}/env-postgres-psql + --network {{ matrix_docker_network }} + --mount type=bind,src=/tmp/matrix-postgres-init-additional-db-user-and-role.sql,dst=/matrix-postgres-init-additional-db-user-and-role.sql,ro + --entrypoint=/bin/sh + {{ matrix_postgres_docker_image_to_use }} + -c + 'psql -h {{ matrix_postgres_connection_hostname }} --file=/matrix-postgres-init-additional-db-user-and-role.sql' + +- name: Delete additional database initialization SQL file for {{ additional_db.name }} + file: + path: /tmp/matrix-postgres-init-additional-db-user-and-role.sql + state: absent diff --git a/roles/matrix-postgres/tasks/util/create_additional_databases.yml b/roles/matrix-postgres/tasks/util/create_additional_databases.yml new file mode 100644 index 00000000..0ad460dd --- /dev/null +++ b/roles/matrix-postgres/tasks/util/create_additional_databases.yml @@ -0,0 +1,23 @@ +--- + +- name: Ensure matrix-postgres is started + service: + name: matrix-postgres + state: started + daemon_reload: yes + register: matrix_postgres_service_start_result + +- name: Wait a bit, so that Postgres can start + wait_for: + timeout: "{{ matrix_postgres_additional_databases_postgres_start_wait_timeout_seconds }}" + delegate_to: 127.0.0.1 + become: false + when: "matrix_postgres_service_start_result.changed|bool" + +- name: Create additional Postgres user and database + include_tasks: "{{ role_path }}/tasks/util/create_additional_database.yml" + with_items: "{{ matrix_postgres_additional_databases }}" + loop_control: + loop_var: additional_db + # Suppress logging to avoid dumping the credentials to the shell + no_log: true diff --git a/roles/matrix-postgres/tasks/util/migrate_db_to_postgres.yml b/roles/matrix-postgres/tasks/util/migrate_db_to_postgres.yml new file mode 100644 index 00000000..d3429c97 --- /dev/null +++ b/roles/matrix-postgres/tasks/util/migrate_db_to_postgres.yml @@ -0,0 +1,168 @@ +--- + +- name: Fail if Postgres not enabled + fail: + msg: "Postgres via the matrix-postgres role is not enabled (`matrix_postgres_enabled`). Cannot migrate." + when: "not matrix_postgres_enabled|bool" + +- name: Fail if util called incorrectly (missing matrix_postgres_db_migration_request) + fail: + msg: "The `matrix_postgres_db_migration_request` variable needs to be provided to this util." + when: "matrix_postgres_db_migration_request is not defined" + +- name: Fail if util called incorrectly (invalid matrix_postgres_db_migration_request) + fail: + msg: "The `matrix_postgres_db_migration_request` variable needs to contain `{{ item }}`." + with_items: + - src + - dst + - caller + - engine_variable_name + - systemd_services_to_stop + when: "item not in matrix_postgres_db_migration_request" + +- name: Check if the provided source database file exists + stat: + path: "{{ matrix_postgres_db_migration_request.src }}" + register: matrix_postgres_db_migration_request_src_stat_result + +- name: Fail if provided source database file doesn't exist + fail: + msg: "File cannot be found on the server at {{ matrix_postgres_db_migration_request.src }}" + when: "not matrix_postgres_db_migration_request_src_stat_result.stat.exists" + +- block: + - name: Ensure pgloader repository is present on self-build + git: + repo: "{{ matrix_postgres_pgloader_container_image_self_build_repo }}" + dest: "{{ matrix_postgres_pgloader_container_image_self_build_src_path }}" + version: "{{ matrix_postgres_pgloader_container_image_self_build_repo_branch }}" + force: "yes" + register: matrix_postgres_pgloader_git_pull_results + + # If `stable` is used, we hit an error when processing /opt/src/pgloader/build/quicklisp/dists/quicklisp/software/uax-15-20201220-git/data/CompositionExclusions.txt: + # > the octet sequence #(194) cannot be decoded + # + # The issue is described here and is not getting fixed for months: https://github.com/dimitri/pgloader/pull/1179 + # + # Although we're not using the dimitri/pgloader image, the one we're using suffers from the same problem. + - name: Switch pgloader base image from Debian stable (likely 10.x/Buster) to Bullseye + lineinfile: + path: "{{ matrix_postgres_pgloader_container_image_self_build_src_path }}/Dockerfile" + regexp: "{{ item.match }}" + line: "{{ item.replace }}" + with_items: + - match: '^FROM debian:stable-slim as builder$' + replace: 'FROM debian:bullseye-slim as builder' + - match: '^FROM debian:stable-slim$' + replace: 'FROM debian:bullseye-slim' + + - name: Ensure pgloader Docker image is built + docker_image: + name: "{{ matrix_postgres_pgloader_docker_image }}" + source: build + force_source: "{{ matrix_postgres_pgloader_git_pull_results.changed }}" + build: + dockerfile: Dockerfile + path: "{{ matrix_postgres_pgloader_container_image_self_build_src_path }}" + pull: yes + when: "matrix_postgres_pgloader_container_image_self_build|bool" + +- name: Ensure pgloader Docker image is pulled + docker_image: + name: "{{ matrix_postgres_pgloader_docker_image }}" + source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" + force_source: "{{ matrix_postgres_pgloader_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" + force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_postgres_pgloader_docker_image_force_pull }}" + when: "not matrix_postgres_pgloader_container_image_self_build" + +# Defaults + +- name: Set postgres_start_wait_time, if not provided + set_fact: + postgres_start_wait_time: 15 + when: "postgres_start_wait_time|default('') == ''" + +# Actual import work + +# matrix-postgres is most likely started already +- name: Ensure matrix-postgres is started + service: + name: matrix-postgres + state: started + daemon_reload: yes + register: matrix_postgres_service_start_result + +- name: Wait a bit, so that Postgres can start + wait_for: + timeout: "{{ postgres_start_wait_time }}" + delegate_to: 127.0.0.1 + become: false + when: "matrix_postgres_service_start_result.changed|bool" + +# We only stop services here, leaving it to the caller to start them later. +# +# We can't start them, because they probably need to be reconfigured too (changing the configuration from using SQLite to Postgres, etc.), +# before starting. +# +# Since the caller will be starting them, it might make sense to leave stopping to it as well. +# However, we don't do it, because it's simpler having it here, and it also gets to happen only if we'll be doing an import. +# If we bailed out (somewhere above), nothing would have gotten stopped. It's nice to leave this running in such cases. +- name: Ensure systemd services blocking the database import are stopped + service: + name: "{{ item }}" + state: stopped + failed_when: false + with_items: "{{ matrix_postgres_db_migration_request.systemd_services_to_stop }}" + +- name: Import {{ matrix_postgres_db_migration_request.engine_old }} database from {{ matrix_postgres_db_migration_request.src }} into Postgres + command: + cmd: >- + {{ matrix_host_command_docker }} run + --rm + --user={{ matrix_user_uid }}:{{ matrix_user_gid }} + --cap-drop=ALL + --network={{ matrix_docker_network }} + --mount type=bind,src={{ matrix_postgres_db_migration_request.src }},dst=/in.db,ro + --entrypoint=/bin/sh + {{ matrix_postgres_pgloader_docker_image }} + -c + 'pgloader {{ matrix_postgres_db_migration_request.pgloader_options|default([])|join(' ') }} /in.db {{ matrix_postgres_db_migration_request.dst }}' + +- block: + # We can't use `{{ role_path }}` here, neither with `import_tasks`, nor with `include_tasks`, + # because it refers to the role that included this util, and not to the role this file belongs to. + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/detect_existing_postgres_version.yml" + + - set_fact: + matrix_postgres_docker_image_to_use: "{{ matrix_postgres_docker_image_latest if matrix_postgres_detected_version_corresponding_docker_image == '' else matrix_postgres_detected_version_corresponding_docker_image }}" + + - name: Execute additional Postgres SQL migration statements + command: + cmd: >- + {{ matrix_host_command_docker }} run + --rm + --user={{ matrix_user_uid }}:{{ matrix_user_gid }} + --cap-drop=ALL + --env-file={{ matrix_postgres_base_path }}/env-postgres-psql + --network={{ matrix_docker_network }} + {{ matrix_postgres_docker_image_to_use }} + psql --host=matrix-postgres --dbname={{ matrix_postgres_db_migration_request.additional_psql_statements_db_name }} --command='{{ item }}' + with_items: "{{ matrix_postgres_db_migration_request.additional_psql_statements_list }}" + + when: "matrix_postgres_db_migration_request.additional_psql_statements_list|default([])|length > 0" + +- name: Archive {{ matrix_postgres_db_migration_request.engine_old }} database ({{ matrix_postgres_db_migration_request.src }} -> {{ matrix_postgres_db_migration_request.src }}.backup) + command: + cmd: "mv {{ matrix_postgres_db_migration_request.src }} {{ matrix_postgres_db_migration_request.src }}.backup" + +- name: Inject result + set_fact: + matrix_playbook_runtime_results: | + {{ + matrix_playbook_runtime_results|default([]) + + + [ + "NOTE: Your {{ matrix_postgres_db_migration_request.engine_old }} database file has been imported into Postgres. The original database file has been moved from `{{ matrix_postgres_db_migration_request.src }}` to `{{ matrix_postgres_db_migration_request.src }}.backup`. When you've confirmed that the import went well and everything works, you should be able to safely delete this file." + ] + }} diff --git a/roles/matrix-postgres/templates/sql/init-additional-db-user-and-role.sql.j2 b/roles/matrix-postgres/templates/sql/init-additional-db-user-and-role.sql.j2 new file mode 100644 index 00000000..609a1344 --- /dev/null +++ b/roles/matrix-postgres/templates/sql/init-additional-db-user-and-role.sql.j2 @@ -0,0 +1,19 @@ +-- `CREATE USER` does not support `IF NOT EXISTS`, so we use this workaround to prevent an error and raise a notice instead. +-- Seen here: https://stackoverflow.com/a/49858797 +DO $$ +BEGIN + CREATE USER {{ additional_db.username }}; + EXCEPTION WHEN DUPLICATE_OBJECT THEN + RAISE NOTICE 'not creating user {{ additional_db.username }}, since it already exists'; +END +$$; + +-- This is useful for initial user creation (since we don't assign a password above) and for handling subsequent password changes +-- TODO - we should escape quotes in the password. +ALTER ROLE {{ additional_db.username }} PASSWORD '{{ additional_db.password }}'; + +-- This will generate an error on subsequent execution +CREATE DATABASE {{ additional_db.name }} WITH LC_CTYPE 'C' LC_COLLATE 'C' OWNER {{ additional_db.username }}; + +-- This is useful for changing the database owner subsequently +ALTER DATABASE {{ additional_db.name }} OWNER TO {{ additional_db.username }}; diff --git a/roles/matrix-postgres/templates/systemd/matrix-postgres.service.j2 b/roles/matrix-postgres/templates/systemd/matrix-postgres.service.j2 index 8ab110f1..00afc64e 100644 --- a/roles/matrix-postgres/templates/systemd/matrix-postgres.service.j2 +++ b/roles/matrix-postgres/templates/systemd/matrix-postgres.service.j2 @@ -3,6 +3,7 @@ Description=Matrix Postgres server After=docker.service Requires=docker.service +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-registration/defaults/main.yml b/roles/matrix-registration/defaults/main.yml index 680e8bfe..065e0c48 100644 --- a/roles/matrix-registration/defaults/main.yml +++ b/roles/matrix-registration/defaults/main.yml @@ -5,15 +5,16 @@ matrix_registration_enabled: true matrix_registration_container_image_self_build: false matrix_registration_container_image_self_build_repo: "https://github.com/ZerataX/matrix-registration" +matrix_registration_container_image_self_build_branch: "{{ 'master' if matrix_registration_version == 'latest' else matrix_registration_version }}" matrix_registration_base_path: "{{ matrix_base_data_path }}/matrix-registration" matrix_registration_config_path: "{{ matrix_registration_base_path }}/config" matrix_registration_data_path: "{{ matrix_registration_base_path }}/data" matrix_registration_docker_src_files_path: "{{ matrix_registration_base_path }}/docker-src" -matrix_registration_version: "v0.7.0" +matrix_registration_version: "v0.7.2" -matrix_registration_docker_image: "{{ matrix_registration_docker_image_name_prefix }}devture/zeratax-matrix-registration:{{ matrix_registration_version }}" +matrix_registration_docker_image: "{{ matrix_registration_docker_image_name_prefix }}zeratax/matrix-registration:{{ matrix_registration_version }}" matrix_registration_docker_image_name_prefix: "{{ 'localhost/' if matrix_registration_container_image_self_build else 'docker.io/' }}" matrix_registration_docker_image_force_pull: "{{ matrix_registration_docker_image.endswith(':latest') }}" @@ -31,10 +32,42 @@ matrix_registration_systemd_wanted_services_list: [] # Takes an ":" or "" value (e.g. "127.0.0.1:8767"), or empty string to not expose. matrix_registration_container_http_host_bind_port: '' +# Database-related configuration fields. +# +# To use SQLite, stick to these defaults. +# +# To use Postgres: +# - change the engine (`matrix_registration_database_engine: 'postgres'`) +# - adjust your database credentials via the `matrix_registration_postgres_*` variables +matrix_registration_database_engine: 'sqlite' + +matrix_registration_sqlite_database_path_local: "{{ matrix_registration_data_path }}/db.sqlite3" +matrix_registration_sqlite_database_path_in_container: "/data/db.sqlite3" + +matrix_registration_database_username: 'matrix_registration' +matrix_registration_database_password: 'some-password' +matrix_registration_database_hostname: 'matrix-postgres' +matrix_registration_database_port: 5432 +matrix_registration_database_name: 'matrix_registration' + +matrix_registration_database_connection_string: 'postgresql://{{ matrix_registration_database_username }}:{{ matrix_registration_database_password }}@{{ matrix_registration_database_hostname }}:{{ matrix_registration_database_port }}/{{ matrix_registration_database_name }}' + +# For some reason, matrix-registraiton expects the `db` field to be like this: `sqlite:////data/db.sqlite3`. +# (seems like one too many slashes, but..) +matrix_registration_db: "{{ + { + 'sqlite': ('sqlite:///' + matrix_registration_sqlite_database_path_in_container), + 'postgres': matrix_registration_database_connection_string, + }[matrix_registration_database_engine] +}}" + + # The path at which Matrix Registration will be exposed on `matrix.DOMAIN` # (only applies when matrix-nginx-proxy is used). matrix_registration_public_endpoint: /matrix-registration +matrix_registration_base_url: "{{ matrix_registration_public_endpoint }}" + matrix_registration_api_register_endpoint: "{{ matrix_homeserver_url }}{{ matrix_registration_public_endpoint }}/register" matrix_registration_api_token_endpoint: "{{ matrix_homeserver_url }}{{ matrix_registration_public_endpoint }}/token" @@ -55,7 +88,6 @@ matrix_registration_admin_secret: "" matrix_registration_riot_instance: "https://riot.im/app/" - # Default matrix-registration configuration template which covers the generic use case. # You can customize it by controlling the various variables inside it. # diff --git a/roles/matrix-registration/tasks/init.yml b/roles/matrix-registration/tasks/init.yml index 8a7cdc75..bdb3928e 100644 --- a/roles/matrix-registration/tasks/init.yml +++ b/roles/matrix-registration/tasks/init.yml @@ -30,15 +30,12 @@ {% endif %} {# - Workaround matrix-registration serving static files at /static - (see https://github.com/ZerataX/matrix-registration/issues/29) - - Also fixing the form, which goes to /register. + Workaround matrix-registration serving the background image at /static + (see https://github.com/ZerataX/matrix-registration/issues/47) #} sub_filter_once off; - sub_filter_types text/html text/css; + sub_filter_types text/css; sub_filter "/static/" "{{ matrix_registration_public_endpoint }}/static/"; - sub_filter "/register" "{{ matrix_registration_public_endpoint }}/register"; } - name: Register matrix-registration proxying configuration with matrix-nginx-proxy diff --git a/roles/matrix-registration/tasks/main.yml b/roles/matrix-registration/tasks/main.yml index 4a884ccd..4fef6abe 100644 --- a/roles/matrix-registration/tasks/main.yml +++ b/roles/matrix-registration/tasks/main.yml @@ -8,7 +8,14 @@ - setup-all - setup-matrix-registration -- import_tasks: "{{ role_path }}/tasks/setup.yml" +- import_tasks: "{{ role_path }}/tasks/setup_install.yml" + when: "run_setup|bool and matrix_registration_enabled|bool" + tags: + - setup-all + - setup-matrix-registration + +- import_tasks: "{{ role_path }}/tasks/setup_uninstall.yml" + when: "run_setup|bool and not matrix_registration_enabled|bool" tags: - setup-all - setup-matrix-registration diff --git a/roles/matrix-registration/tasks/setup.yml b/roles/matrix-registration/tasks/setup_install.yml similarity index 54% rename from roles/matrix-registration/tasks/setup.yml rename to roles/matrix-registration/tasks/setup_install.yml index 8427d950..2b806fe0 100644 --- a/roles/matrix-registration/tasks/setup.yml +++ b/roles/matrix-registration/tasks/setup_install.yml @@ -1,8 +1,35 @@ --- -# -# Tasks related to setting up matrix-registration -# +- set_fact: + matrix_registration_requires_restart: false + +- block: + - name: Check if an SQLite database already exists + stat: + path: "{{ matrix_registration_sqlite_database_path_local }}" + register: matrix_registration_sqlite_database_path_local_stat_result + + - block: + - set_fact: + matrix_postgres_db_migration_request: + src: "{{ matrix_registration_sqlite_database_path_local }}" + dst: "{{ matrix_registration_database_connection_string }}" + caller: "{{ role_path|basename }}" + engine_variable_name: 'matrix_registration_database_engine' + engine_old: 'sqlite' + systemd_services_to_stop: ['matrix-registration.service'] + # pgloader makes `ex_date` of type `TIMESTAMP WITH TIMEZONE`, + # which makes matrix-registration choke on it later on when comparing dates. + additional_psql_statements_list: + - ALTER TABLE tokens ALTER COLUMN ex_date TYPE TIMESTAMP WITHOUT TIME ZONE; + additional_psql_statements_db_name: "{{ matrix_registration_database_name }}" + + - import_tasks: "{{ role_path }}/../matrix-postgres/tasks/util/migrate_db_to_postgres.yml" + + - set_fact: + matrix_registration_requires_restart: true + when: "matrix_registration_sqlite_database_path_local_stat_result.stat.exists|bool" + when: "matrix_registration_database_engine == 'postgres'" - name: Ensure matrix-registration paths exist file: @@ -16,7 +43,7 @@ - { path: "{{ matrix_registration_config_path }}", when: true } - { path: "{{ matrix_registration_data_path }}", when: true } - { path: "{{ matrix_registration_docker_src_files_path }}", when: "{{ matrix_registration_container_image_self_build }}"} - when: matrix_registration_enabled|bool and item.when + when: "item.when|bool" - name: Ensure matrix-registration image is pulled docker_image: @@ -24,16 +51,16 @@ source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" force_source: "{{ matrix_registration_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_registration_docker_image_force_pull }}" - when: "matrix_registration_enabled|bool and not matrix_registration_container_image_self_build|bool" + when: "not matrix_registration_container_image_self_build|bool" - name: Ensure matrix-registration repository is present when self-building git: repo: "{{ matrix_registration_container_image_self_build_repo }}" dest: "{{ matrix_registration_docker_src_files_path }}" - version: "{{ matrix_registration_version }}" + version: "{{ matrix_registration_container_image_self_build_branch }}" force: "yes" register: matrix_registration_git_pull_results - when: "matrix_registration_enabled|bool and matrix_registration_container_image_self_build|bool" + when: "matrix_registration_container_image_self_build|bool" - name: Ensure matrix-registration Docker image is built docker_image: @@ -44,7 +71,7 @@ dockerfile: Dockerfile path: "{{ matrix_registration_docker_src_files_path }}" pull: yes - when: "matrix_registration_enabled|bool and matrix_registration_container_image_self_build|bool" + when: "matrix_registration_container_image_self_build|bool" - name: Ensure matrix-registration config installed copy: @@ -53,7 +80,6 @@ mode: 0644 owner: "{{ matrix_user_username }}" group: "{{ matrix_user_groupname }}" - when: matrix_registration_enabled|bool - name: Ensure matrix-registration.service installed template: @@ -61,43 +87,14 @@ dest: "{{ matrix_systemd_path }}/matrix-registration.service" mode: 0644 register: matrix_registration_systemd_service_result - when: matrix_registration_enabled|bool - name: Ensure systemd reloaded after matrix-registration.service installation service: daemon_reload: yes - when: "matrix_registration_enabled|bool and matrix_registration_systemd_service_result.changed" + when: "matrix_registration_systemd_service_result.changed|bool" -# -# Tasks related to getting rid of matrix-registration (if it was previously enabled) -# - -- name: Check existence of matrix-registration service - stat: - path: "{{ matrix_systemd_path }}/matrix-registration.service" - register: matrix_registration_service_stat - -- name: Ensure matrix-registration is stopped +- name: Ensure matrix-registration.service restarted, if necessary service: - name: matrix-registration - state: stopped - daemon_reload: yes - register: stopping_result - when: "not matrix_registration_enabled|bool and matrix_registration_service_stat.stat.exists" - -- name: Ensure matrix-registration.service doesn't exist - file: - path: "{{ matrix_systemd_path }}/matrix-registration.service" - state: absent - when: "not matrix_registration_enabled|bool and matrix_registration_service_stat.stat.exists" - -- name: Ensure systemd reloaded after matrix-registration.service removal - service: - daemon_reload: yes - when: "not matrix_registration_enabled|bool and matrix_registration_service_stat.stat.exists" - -- name: Ensure matrix-registration Docker image doesn't exist - docker_image: - name: "{{ matrix_registration_docker_image }}" - state: absent - when: "not matrix_registration_enabled|bool" + name: "matrix-registration.service" + state: restarted + when: "matrix_registration_requires_restart|bool" diff --git a/roles/matrix-registration/tasks/setup_uninstall.yml b/roles/matrix-registration/tasks/setup_uninstall.yml new file mode 100644 index 00000000..573f8170 --- /dev/null +++ b/roles/matrix-registration/tasks/setup_uninstall.yml @@ -0,0 +1,30 @@ +--- + +- name: Check existence of matrix-registration service + stat: + path: "{{ matrix_systemd_path }}/matrix-registration.service" + register: matrix_registration_service_stat + +- name: Ensure matrix-registration is stopped + service: + name: matrix-registration + state: stopped + daemon_reload: yes + register: stopping_result + when: "matrix_registration_service_stat.stat.exists|bool" + +- name: Ensure matrix-registration.service doesn't exist + file: + path: "{{ matrix_systemd_path }}/matrix-registration.service" + state: absent + when: "matrix_registration_service_stat.stat.exists|bool" + +- name: Ensure systemd reloaded after matrix-registration.service removal + service: + daemon_reload: yes + when: "matrix_registration_service_stat.stat.exists|bool" + +- name: Ensure matrix-registration Docker image doesn't exist + docker_image: + name: "{{ matrix_registration_docker_image }}" + state: absent diff --git a/roles/matrix-registration/templates/config.yaml.j2 b/roles/matrix-registration/templates/config.yaml.j2 index f3b1c57b..39211b24 100644 --- a/roles/matrix-registration/templates/config.yaml.j2 +++ b/roles/matrix-registration/templates/config.yaml.j2 @@ -3,7 +3,7 @@ server_name: {{ matrix_registration_server_name|to_json }} shared_secret: {{ matrix_registration_shared_secret|to_json }} admin_secret: {{ matrix_registration_admin_secret|to_json }} riot_instance: {{ matrix_registration_riot_instance|to_json }} -db: 'sqlite:////data/db.sqlite3' +db: {{ matrix_registration_db|to_json }} host: '0.0.0.0' port: 5000 rate_limit: ["100 per day", "10 per minute"] @@ -28,3 +28,4 @@ logging: # password requirements password: min_length: 8 +base_url: {{ matrix_registration_base_url|to_json }} diff --git a/roles/matrix-registration/templates/systemd/matrix-registration.service.j2 b/roles/matrix-registration/templates/systemd/matrix-registration.service.j2 index 052b7d95..3744c2de 100644 --- a/roles/matrix-registration/templates/systemd/matrix-registration.service.j2 +++ b/roles/matrix-registration/templates/systemd/matrix-registration.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_registration_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-synapse-admin/defaults/main.yml b/roles/matrix-synapse-admin/defaults/main.yml index ce149dfd..1dbf0ad2 100644 --- a/roles/matrix-synapse-admin/defaults/main.yml +++ b/roles/matrix-synapse-admin/defaults/main.yml @@ -8,7 +8,7 @@ matrix_synapse_admin_container_self_build_repo: "https://github.com/Awesome-Tech matrix_synapse_admin_docker_src_files_path: "{{ matrix_base_data_path }}/synapse-admin/docker-src" -matrix_synapse_admin_docker_image: "{{ matrix_synapse_admin_docker_image_name_prefix }}awesometechnologies/synapse-admin:0.5.0" +matrix_synapse_admin_docker_image: "{{ matrix_synapse_admin_docker_image_name_prefix }}awesometechnologies/synapse-admin:0.6.1" matrix_synapse_admin_docker_image_name_prefix: "{{ 'localhost/' if matrix_synapse_admin_container_self_build else 'docker.io/' }}" matrix_synapse_admin_docker_image_force_pull: "{{ matrix_synapse_admin_docker_image.endswith(':latest') }}" diff --git a/roles/matrix-synapse-admin/templates/systemd/matrix-synapse-admin.service.j2 b/roles/matrix-synapse-admin/templates/systemd/matrix-synapse-admin.service.j2 index c03c627b..7b1e40de 100644 --- a/roles/matrix-synapse-admin/templates/systemd/matrix-synapse-admin.service.j2 +++ b/roles/matrix-synapse-admin/templates/systemd/matrix-synapse-admin.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_synapse_admin_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-synapse/defaults/main.yml b/roles/matrix-synapse/defaults/main.yml index 1a87c149..682349f0 100644 --- a/roles/matrix-synapse/defaults/main.yml +++ b/roles/matrix-synapse/defaults/main.yml @@ -6,8 +6,12 @@ matrix_synapse_enabled: true matrix_synapse_container_image_self_build: false matrix_synapse_container_image_self_build_repo: "https://github.com/matrix-org/synapse.git" -matrix_synapse_docker_image: "{{ matrix_synapse_docker_image_name_prefix }}matrixdotorg/synapse:v1.23.0" +matrix_synapse_docker_image: "{{ matrix_synapse_docker_image_name_prefix }}matrixdotorg/synapse:{{ matrix_synapse_docker_image_tag }}" matrix_synapse_docker_image_name_prefix: "{{ 'localhost/' if matrix_synapse_container_image_self_build else 'docker.io/' }}" +# The if statement below may look silly at times (leading to the same version being returned), +# but ARM-compatible container images are only released 1-7 hours after a release, +# so we may often be on different versions for different architectures when new Synapse releases come out. +matrix_synapse_docker_image_tag: "{{ 'v1.24.0' if matrix_architecture == 'amd64' else 'v1.24.0' }}" matrix_synapse_docker_image_force_pull: "{{ matrix_synapse_docker_image.endswith(':latest') }}" matrix_synapse_base_path: "{{ matrix_base_data_path }}/synapse" @@ -148,6 +152,11 @@ matrix_synapse_tls_private_key_path: "/data/{{ matrix_server_fqn_matrix }}.tls.k # names. matrix_synapse_http_listener_resource_names: ["client"] +# Resources served on Synapse's federation port. +# When disabling federation, we may wish to serve the `openid` resource here, +# so that services like Dimension and ma1sd can work. +matrix_synapse_federation_listener_resource_names: "{{ ['federation'] if matrix_synapse_federation_enabled else (['openid'] if matrix_synapse_federation_port_openid_resource_required else []) }}" + # Enable this to allow Synapse to report utilization statistics about your server to matrix.org # (things like number of users, number of messages sent, uptime, load, etc.) matrix_synapse_report_stats: false @@ -223,10 +232,25 @@ matrix_synapse_caches_global_factor: 0.5 # Controls whether Synapse will federate at all. # Disable this to completely isolate your server from the rest of the Matrix network. -# Also see: `matrix_synapse_tls_federation_listener_enabled` if you wish to keep federation enabled, +# +# Disabling this still keeps the federation port exposed, because it may be used for other services (`openid`). +# +# Also see: +# - `matrix_synapse_tls_federation_listener_enabled` if you wish to keep federation enabled, # but want to stop the TLS listener (port 8448). +# - `matrix_synapse_federation_port_enabled` to avoid exposing the federation ports matrix_synapse_federation_enabled: true +# Controls whether the federation ports are used at all. +# One may wish to disable federation (`matrix_synapse_federation_enabled: true`), +# but still run other resources (like `openid`) on the federation port +# by enabling them in `matrix_synapse_federation_listener_resource_names`. +matrix_synapse_federation_port_enabled: "{{ matrix_synapse_federation_enabled or matrix_synapse_federation_port_openid_resource_required }}" + +# Controls whether an `openid` listener is to be enabled. Useful when disabling federation, +# but needing the `openid` APIs for Dimension or an identity server like ma1sd. +matrix_synapse_federation_port_openid_resource_required: false + # A list of domain names that are allowed to federate with the given Synapse server. # An empty list value (`[]`) will also effectively stop federation, but if that's the desired # result, it's better to accomplish it by changing `matrix_synapse_federation_enabled`. diff --git a/roles/matrix-synapse/tasks/main.yml b/roles/matrix-synapse/tasks/main.yml index 48e72ace..e366a13f 100644 --- a/roles/matrix-synapse/tasks/main.yml +++ b/roles/matrix-synapse/tasks/main.yml @@ -17,7 +17,7 @@ - import_tasks: "{{ role_path }}/tasks/import_media_store.yml" when: run_synapse_import_media_store|bool tags: - - import-media-store + - import-synapse-media-store - import_tasks: "{{ role_path }}/tasks/register_user.yml" when: run_synapse_register_user|bool diff --git a/roles/matrix-synapse/templates/goofys/systemd/matrix-goofys.service.j2 b/roles/matrix-synapse/templates/goofys/systemd/matrix-goofys.service.j2 index c3ec9249..0bbfde99 100644 --- a/roles/matrix-synapse/templates/goofys/systemd/matrix-goofys.service.j2 +++ b/roles/matrix-synapse/templates/goofys/systemd/matrix-goofys.service.j2 @@ -3,6 +3,7 @@ Description=Matrix Goofys media store After=docker.service Requires=docker.service +DefaultDependencies=no [Service] Type=simple diff --git a/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2 b/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2 index c35f526f..8c6f5cb4 100644 --- a/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2 +++ b/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2 @@ -188,7 +188,7 @@ listeners: - '0.0.0.0' {% endif %} -{% if matrix_synapse_federation_enabled and matrix_synapse_tls_federation_listener_enabled %} +{% if matrix_synapse_federation_port_enabled and matrix_synapse_tls_federation_listener_enabled %} # TLS-enabled listener: for when matrix traffic is sent directly to synapse. - port: 8448 tls: true @@ -197,7 +197,7 @@ listeners: x_forwarded: false resources: - - names: [federation] + - names: {{ matrix_synapse_federation_listener_resource_names|to_json }} compress: false {% endif %} @@ -213,7 +213,7 @@ listeners: - names: {{ matrix_synapse_http_listener_resource_names|to_json }} compress: false -{% if matrix_synapse_federation_enabled %} +{% if matrix_synapse_federation_port_enabled %} # Unsecure HTTP listener (Federation API): for when matrix traffic passes through a reverse proxy # that unwraps TLS. - port: 8048 @@ -223,7 +223,7 @@ listeners: x_forwarded: true resources: - - names: [federation] + - names: {{ matrix_synapse_federation_listener_resource_names|to_json }} compress: false {% endif %} @@ -1236,8 +1236,9 @@ allow_guest_access: {{ matrix_synapse_allow_guest_access|to_json }} # email will be globally disabled. # # Additionally, if `msisdn` is not set, registration and password resets via msisdn -# will be disabled regardless. This is due to Synapse currently not supporting any -# method of sending SMS messages on its own. +# will be disabled regardless, and users will not be able to associate an msisdn +# identifier to their account. This is due to Synapse currently not supporting +# any method of sending SMS messages on its own. # # To enable using an identity server for operations regarding a particular third-party # identifier type, set the value to the URL of that identity server as shown in the @@ -1556,6 +1557,12 @@ saml2_config: # remote: # - url: https://our_idp/metadata.xml + # Allowed clock difference in seconds between the homeserver and IdP. + # + # Uncomment the below to increase the accepted time difference from 0 to 3 seconds. + # + #accepted_time_diff: 3 + # By default, the user has to go to our login page first. If you'd like # to allow IdP-initiated login, set 'allow_unsolicited: true' in a # 'service.sp' section: @@ -1678,6 +1685,14 @@ saml2_config: # - attribute: department # value: "sales" + # If the metadata XML contains multiple IdP entities then the `idp_entityid` + # option must be set to the entity to redirect users to. + # + # Most deployments only have a single IdP entity and so should omit this + # option. + # + #idp_entityid: 'https://our_idp/entityid' + # Enable OpenID Connect (OIDC) / OAuth 2.0 for registration and login. # @@ -2256,21 +2271,35 @@ password_providers: {% endif %} +## Push ## -# Clients requesting push notifications can either have the body of -# the message sent in the notification poke along with other details -# like the sender, or just the event ID and room ID (`event_id_only`). -# If clients choose the former, this option controls whether the -# notification request includes the content of the event (other details -# like the sender are still included). For `event_id_only` push, it -# has no effect. -# -# For modern android devices the notification content will still appear -# because it is loaded by the app. iPhone, however will send a -# notification saying only that a message arrived and who it came from. -# push: - include_content: {{ matrix_synapse_push_include_content|to_json }} + # Clients requesting push notifications can either have the body of + # the message sent in the notification poke along with other details + # like the sender, or just the event ID and room ID (`event_id_only`). + # If clients choose the former, this option controls whether the + # notification request includes the content of the event (other details + # like the sender are still included). For `event_id_only` push, it + # has no effect. + # + # For modern android devices the notification content will still appear + # because it is loaded by the app. iPhone, however will send a + # notification saying only that a message arrived and who it came from. + # + # The default value is "true" to include message details. Uncomment to only + # include the event ID and room ID in push notification payloads. + # + include_content: {{ matrix_synapse_push_include_content|to_json }} + + # When a push notification is received, an unread count is also sent. + # This number can either be calculated as the number of unread messages + # for the user, or the number of *rooms* the user has unread messages in. + # + # The default value is "true", meaning push clients will see the number of + # rooms with unread messages in them. Uncomment to instead send the number + # of unread messages. + # + #group_unread_count_by_room: false # Spam checkers are third-party modules that can block specific actions diff --git a/roles/matrix-synapse/templates/synapse/systemd/matrix-synapse.service.j2 b/roles/matrix-synapse/templates/synapse/systemd/matrix-synapse.service.j2 index 216d1aef..e4b45094 100644 --- a/roles/matrix-synapse/templates/synapse/systemd/matrix-synapse.service.j2 +++ b/roles/matrix-synapse/templates/synapse/systemd/matrix-synapse.service.j2 @@ -8,6 +8,7 @@ After={{ service }} {% for service in matrix_synapse_systemd_wanted_services_list %} Wants={{ service }} {% endfor %} +DefaultDependencies=no [Service] Type=simple diff --git a/setup.yml b/setup.yml index c67e53f0..f4ebe568 100755 --- a/setup.yml +++ b/setup.yml @@ -16,6 +16,7 @@ - matrix-bridge-appservice-irc - matrix-bridge-mautrix-facebook - matrix-bridge-mautrix-hangouts + - matrix-bridge-mautrix-signal - matrix-bridge-mautrix-telegram - matrix-bridge-mautrix-whatsapp - matrix-bridge-mx-puppet-discord