## matrix-postgres-backup has been replaced by the com.devture.ansible.role.postgres_backup external role
Just like we've [replaced Postgres with an external role](#matrix-postgres-has-been-replaced-by-the-comdevtureansiblerolepostgres-external-role) on 2022-11-28, we're now replacing `matrix-postgres-backup` with an external role - [com.devture.ansible.role.postgres_backup](https://github.com/devture/com.devture.ansible.role.postgres_backup).
You'll need to rename your `matrix_postgres_backup`-prefixed variables such that they use a `devture_postgres_backup` prefix.
# 2022-11-28
## matrix-postgres has been replaced by the com.devture.ansible.role.postgres external role
**TLDR**: the tasks that install the integrated Postgres server now live in an external role - [com.devture.ansible.role.postgres](https://github.com/devture/com.devture.ansible.role.postgres). You'll need to run `make roles` to install it, and to also rename your `matrix_postgres`-prefixed variables to use a `devture_postgres` prefix (e.g. `matrix_postgres_connection_password` -> `devture_postgres_connection_password`). All your data will still be there! Some scripts have moved (`/usr/local/bin/matrix-postgres-cli` -> `/matrix/postgres/bin/cli`).
The `matrix-postgres` role that has been part of the playbook for a long time has been replaced with the [com.devture.ansible.role.postgres](https://github.com/devture/com.devture.ansible.role.postgres) role. This was done as part of our work to [use external roles for some things](#the-playbook-now-uses-external-roles-for-some-things) for better code re-use and maintainability.
The new role is an upgraded version of the old `matrix-postgres` role with these notable differences:
- it uses different names for its variables (`matrix_postgres` -> `devture_postgres`)
- when [Vacuuming PostgreSQL](docs/maintenance-postgres.md#vacuuming-postgresql), it will vacuum all your databases, not just the Synapse one
You'll need to run `make roles` to install the new role. You would also need to rename your `matrix_postgres`-prefixed variables to use a `devture_postgres` prefix.
Note: the systemd service still remains the same - `matrix-postgres.service`. Your data will still be in `/matrix/postgres`, etc.
Postgres-related scripts will be moved to `/matrix/postgres/bin` (`/usr/local/bin/matrix-postgres-cli` -> `/matrix/postgres/bin/cli`, etc). Also see [The playbook no longer installs scripts in /usr/local/bin](#the-playbook-no-longer-installs-scripts-in-usrlocalbin).
## The playbook no longer installs scripts to /usr/local/bin
The locations of various scripts installed by the playbook have changed.
The playbook no longer contaminates your `/usr/local/bin` directory.
All scripts installed by the playbook now live in `bin/` directories under `/matrix`. Some examples are below:
## 2x-5x performance improvements in playbook runtime
**TLDR**: the playbook is 2x faster for running `--tags=setup-all` (and various other tags). It also has new `--tags=install-*` tags (like `--tags=install-all`), which skip uninstallation tasks and bring an additional 2.5x speedup. In total, the playbook can maintain your server 5 times faster.
Our [etke.cc managed Matrix hosting service](https://etke.cc) runs maintenance against hundreds of servers, so the playbook being fast means a lot.
The [etke.cc Ansible playbook](https://gitlab.com/etke.cc/ansible) (which is an extension of this one) is growing to support more and more services (besides just Matrix), so the Matrix playbook being leaner prevents runtimes from becoming too slow and improves the customer experience.
Even when running `ansible-playbook` manually (as most of us here do), it's beneficial not to waste time and CPU resources.
Recently, a few large optimizations have been done to this playbook and its external roles (see [The playbook now uses external roles for some things](#the-playbook-now-uses-external-roles-for-some-things) and don't forget to run `make roles`):
1. Replacing Ansible `import_tasks` calls with `include_tasks`, which decreased runtime in half. Using `import_tasks` is slower and causes Ansible to go through and skip way too many tasks (tasks which could have been skipped altogether by not having Ansible include them in the first place). On an experimental VM, **deployment time was decreased from ~530 seconds to ~250 seconds**.
2. Introducing new `install-*` tags (`install-all` and `install-COMPONENT`, e.g. `install-synapse`, `install-bot-postmoogle`), which only run Ansible tasks pertaining to installation, while skipping uninstallation tasks. In most cases, people are maintaining the same setup or they're *adding* new components. Removing components is rare. Running thousands of uninstallation tasks each time is wasteful. On an experimental VM, **deployment time was decreased from ~250 seconds (`--tags=setup-all`) to ~100 seconds (`--tags=install-all`)**.
You can still use `--tags=setup-all`. In fact, that's the best way to ensure your server is reconciled with the `vars.yml` configuration.
If you know you haven't uninstalled any services since the last time you ran the playbook, you could run `--tags=install-all` instead and benefit from quicker runtimes.
It should be noted that a service may become "eligible for uninstallation" even if your `vars.yml` file remains the same. In rare cases, we toggle services from being auto-installed to being optional, like we did on the 17th of March 2022 when we made [ma1sd not get installed by default](https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/CHANGELOG.md#compatibility-break-ma1sd-identity-server-no-longer-installed-by-default). In such rare cases, you'd also need to run `--tags=setup-all`.
# 2022-11-22
# Automatic `matrix_architecture` determination
From now on, the playbook automatically determines your server's architecture and sets the `matrix_architecture` variable accordingly.
You no longer need to set this variable manually in your `vars.yml` file.
# Docker and the Docker SDK for Python are now installed via external roles
We're continuing our effort to make [the playbook use external roles for some things](#the-playbook-now-uses-external-roles-for-some-things), so as to avoid doing everything ourselves and to facilitate code re-use.
Docker will now be installed on the server via the [geerlingguy.docker](https://github.com/geerlingguy/ansible-role-docker) Ansible role.
If you'd like to manage the Docker installation yourself, you can disable the playbook's installation of Docker by setting `matrix_playbook_docker_installation_enabled: false`.
The Docker SDK for Python (named `docker-python`, `python-docker`, etc. on the different platforms) is now also installed by another role ([com.devture.ansible.role.docker_sdk_for_python](https://github.com/devture/com.devture.ansible.role.docker_sdk_for_python)). To disable this role and install the necessary tools yourself, use `devture_docker_sdk_for_python_installation_enabled: false`.
If you're hitting issues with Docker installation or Docker SDK for Python installation, consider reporting bugs or contributing to these other projects.
These additional roles are downloaded into the playbook directory (to `roles/galaxy`) via an `ansible-galaxy ..` command. `make roles` is an easy shortcut for invoking the `ansible-galaxy` command to download these roles.
# 2022-11-20
## (Backward Compatibility Break) Changing how reverse-proxying to Synapse works - now via a `matrix-synapse-reverse-proxy-companion` service
**TLDR**: There's now a `matrix-synapse-reverse-proxy-companion` nginx service, which helps with reverse-proxying to Synapse and its various worker processes (if workers are enabled), so that `matrix-nginx-proxy` can be relieved of this role. `matrix-nginx-proxy` still remains as the public SSL-terminating reverse-proxy in the playbook. `matrix-synapse-reverse-proxy-companion` is just one more reverse-proxy thrown into the mix for convenience. People with a more custom reverse-proxying configuration may be affected - see [Webserver configuration](#webserver-configuration) below.
### Background
Previously, `matrix-nginx-proxy` forwarded requests to Synapse directly. When Synapse is running in worker mode, the reverse-proxying configuration is more complicated (different requests need to go to different Synapse worker processes). `matrix-nginx-proxy` had configuration for sending each URL endpoint to the correct Synapse worker responsible for handling it. However, sometimes people like to disable `matrix-nginx-proxy` (for whatever reason) as detailed in [Using your own webserver, instead of this playbook's nginx proxy](docs/configuring-playbook-own-webserver.md).
Because `matrix-nginx-proxy` was so central to request forwarding, when it was disabled and Synapse was running with workers enabled, there was nothing which could forward requests to the correct place anymore.. which caused [problems such as this one affecting Dimension](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2090).
### Solution
From now on, `matrix-nginx-proxy` is relieved of its function of reverse-proxying to Synapse and its various worker processes.
This role is now handled by the new `matrix-synapse-reverse-proxy-companion` nginx service and works even if `matrix-nginx-proxy` is disabled.
The purpose of the new `matrix-synapse-reverse-proxy-companion` service is to:
- serve as a companion to Synapse and know how to reverse-proxy to Synapse correctly (no matter if workers are enabled or not)
- provide a unified container address for reaching Synapse (no matter if workers are enabled or not)
- `matrix-synapse-reverse-proxy-companion:8008` for Synapse Client-Server API traffic
- `matrix-synapse-reverse-proxy-companion:8048` for Synapse Server-Server (Federation) API traffic
- simplify `matrix-nginx-proxy` configuration - it now only needs to send requests to `matrix-synapse-reverse-proxy-companion` or `matrix-dendrite`, etc., without having to worry about workers
- allow reverse-proxying to Synapse, even if `matrix-nginx-proxy` is disabled
`matrix-nginx-proxy` still remains as the public SSL-terminating reverse-proxy in the playbook. All traffic goes through it before reaching any of the services.
It's just that now the Synapse traffic is routed through `matrix-synapse-reverse-proxy-companion` like this:
(`matrix-nginx-proxy` -> `matrix-synapse-reverse-proxy-companion` -> (`matrix-synapse` or some Synapse worker)).
Various services (like Dimension, etc.) still talk to Synapse via `matrix-nginx-proxy` (e.g. `http://matrix-nginx-proxy:12080`) preferentially. They only talk to Synapse via the reverse-proxy companion (e.g. `http://matrix-synapse-reverse-proxy-companion:8008`) if `matrix-nginx-proxy` is disabled. Services should not be talking to Synapse (e.g. `https://matrix-synapse:8008` directly anymore), because when workers are enabled, that's the Synapse `master` process and may not be serving all URL endpoints needed by the service.
### Webserver configuration
- if you're using `matrix-nginx-proxy` (`matrix_nginx_proxy_enabled: true`, which is the default for the playbook), you don't need to do anything
- if you're using your own `nginx` webserver running on the server, you shouldn't be affected. The `/matrix/nginx/conf.d` configuration and exposed ports that you're relying on will automatically be updated in a way that should work
- if you're using another local webserver (e.g. Apache, etc.) and haven't changed any ports (`matrix_*_host_bind_port` definitions), you shouldn't be affected. You're likely sending Matrix traffic to `127.0.0.1:8008` and `127.0.0.1:8048`. These ports (`8008` and `8048`) will still be exposed on `127.0.0.1` by default - just not by the `matrix-synapse` container from now on, but by the `matrix-synapse-reverse-proxy-companion` container instead
- if you've been exposing `matrix-synapse` ports (`matrix_synapse_container_client_api_host_bind_port`, etc.) manually, you should consider exposing `matrix-synapse-reverse-proxy-companion` ports instead
- if you're running Traefik and reverse-proxying directly to the `matrix-synapse` container, you should start reverse-proxying to the `matrix-synapse-reverse-proxy-companion` container instead. See [our updated Traefik example configuration](docs/configuring-playbook-own-webserver.md#sample-configuration-for-running-behind-traefik-20). Note: we now recommend calling the federation entry point `federation` (instead of `synapse`) and reverse-proxying the federation traffic via `matrix-nginx-proxy`, instead of sending it directly to Synapse (or `matrix-synapse-reverse-proxy-companion`). This makes the configuration simpler.
# 2022-11-05
# 2022-11-05
## (Backward Compatibility Break) A new default standalone mode for Etherpad
## (Backward Compatibility Break) A new default standalone mode for Etherpad
@ -15,137 +15,144 @@ We run all services in [Docker](https://www.docker.com/) containers (see [the co
## Supported services
## Supported services
Using this playbook, you can get the following services configured on your server:
Using this playbook, you can get the following list of services configured on your server. Basically, this playbook aims to get you up-and-running with all the necessities around Matrix, without you having to do anything else.
- (optional, default) a [Synapse](https://github.com/matrix-org/synapse) homeserver - storing your data and managing your presence in the [Matrix](http://matrix.org/) network
**Note**: the list below is exhaustive. It includes optional or even some advanced components that you will most likely not need.
Sticking with the defaults (which install a subset of the above components) is the best choice, especially for a new installation.
- (optional) a [Conduit](https://conduit.rs) homeserver - storing your data and managing your presence in the [Matrix](http://matrix.org/) network. Conduit is a lightweight open-source server implementation of the Matrix Specification with a focus on easy setup and low system requirements
You can always re-run the playbook later to add or remove components.
- (optional) a [Dendrite](https://github.com/matrix-org/dendrite) homeserver - storing your data and managing your presence in the [Matrix](http://matrix.org/) network. Dendrite is a second-generation Matrix homeserver written in Go, an alternative to Synapse.
- (optional) [Amazon S3](https://aws.amazon.com/s3/) (or other S3-compatible object store) storage for Synapse's content repository (`media_store`) files using [Goofys](https://github.com/kahing/goofys) or [`synapse-s3-storage-provider`](https://github.com/matrix-org/synapse-s3-storage-provider)
- (optional, default) [PostgreSQL](https://www.postgresql.org/) database for Synapse. [Using an external PostgreSQL server](docs/configuring-playbook-external-postgres.md) is also possible.
- (optional, default) a [coturn](https://github.com/coturn/coturn) STUN/TURN server for WebRTC audio/video calls
- (optional, default) free [Let's Encrypt](https://letsencrypt.org/) SSL certificate, which secures the connection to the Synapse server and the Element web UI
- (optional, default) an [Element](https://app.element.io/) ([formerly Riot](https://element.io/previously-riot)) web UI, which is configured to connect to your own Synapse server by default
- (optional) a [ma1sd](https://github.com/ma1uta/ma1sd) Matrix Identity server
- (optional, default) an [Exim](https://www.exim.org/) mail server, through which all Matrix services send outgoing email (can be configured to relay through another SMTP server)
- (optional, default) an [nginx](http://nginx.org/) web server, listening on ports 80 and 443 - standing in front of all the other services. Using your own webserver [is possible](docs/configuring-playbook-own-webserver.md)
- (optional, advanced) the [matrix-synapse-rest-auth](https://github.com/ma1uta/matrix-synapse-rest-password-provider) REST authentication password provider module
- (optional, advanced) the [matrix-synapse-shared-secret-auth](https://github.com/devture/matrix-synapse-shared-secret-auth) password provider module
- (optional, advanced) the [matrix-synapse-ldap3](https://github.com/matrix-org/matrix-synapse-ldap3) LDAP Auth password provider module
- (optional, advanced) the [matrix-ldap-registration-proxy](https://gitlab.com/activism.international/matrix_ldap_registration_proxy) a proxy that handles Matrix registration requests and forwards them to LDAP.
- (optional, advanced) the [synapse-simple-antispam](https://github.com/t2bot/synapse-simple-antispam) spam checker module
- (optional, advanced) the [Matrix Corporal](https://github.com/devture/matrix-corporal) reconciliator and gateway for a managed Matrix server
- (optional) the [mautrix-discord](https://github.com/mautrix/discord) bridge for bridging your Matrix server to [Discord](https://discord.com/) - see [docs/configuring-playbook-bridge-mautrix-discord.md](docs/configuring-playbook-bridge-mautrix-discord.md) for setup documentation
- (optional) the [mautrix-telegram](https://github.com/mautrix/telegram) bridge for bridging your Matrix server to [Telegram](https://telegram.org/)
- (optional) the [mautrix-whatsapp](https://github.com/mautrix/whatsapp) bridge for bridging your Matrix server to [WhatsApp](https://www.whatsapp.com/)
- (optional) the [mautrix-facebook](https://github.com/mautrix/facebook) bridge for bridging your Matrix server to [Facebook](https://facebook.com/)
- (optional) the [mautrix-twitter](https://github.com/mautrix/twitter) bridge for bridging your Matrix server to [Twitter](https://twitter.com/)
- (optional) the [mautrix-hangouts](https://github.com/mautrix/hangouts) bridge for bridging your Matrix server to [Google Hangouts](https://en.wikipedia.org/wiki/Google_Hangouts)
- (optional) the [mautrix-googlechat](https://github.com/mautrix/googlechat) bridge for bridging your Matrix server to [Google Chat](https://en.wikipedia.org/wiki/Google_Chat)
- (optional) the [mautrix-instagram](https://github.com/mautrix/instagram) bridge for bridging your Matrix server to [Instagram](https://instagram.com/)
- (optional) the [mautrix-signal](https://github.com/mautrix/signal) bridge for bridging your Matrix server to [Signal](https://www.signal.org/)
- (optional) the [beeper-linkedin](https://github.com/beeper/linkedin) bridge for bridging your Matrix server to [LinkedIn](https://www.linkedin.com/)
- (optional) the [matrix-appservice-irc](https://github.com/matrix-org/matrix-appservice-irc) bridge for bridging your Matrix server to [IRC](https://wikipedia.org/wiki/Internet_Relay_Chat)
- (optional) the [matrix-appservice-discord](https://github.com/Half-Shot/matrix-appservice-discord) bridge for bridging your Matrix server to [Discord](https://discordapp.com/)
- (optional) the [matrix-appservice-slack](https://github.com/matrix-org/matrix-appservice-slack) bridge for bridging your Matrix server to [Slack](https://slack.com/)
- (optional) the [matrix-appservice-webhooks](https://github.com/turt2live/matrix-appservice-webhooks) bridge for slack compatible webhooks ([ConcourseCI](https://concourse-ci.org/), [Slack](https://slack.com/) etc. pp.)
- (optional) the [matrix-hookshot](https://github.com/Half-Shot/matrix-hookshot) bridge for bridging Matrix to generic webhooks and multiple project management services, such as GitHub, GitLab, Figma, and Jira in particular
- (optional) the [matrix-sms-bridge](https://github.com/benkuly/matrix-sms-bridge) for bridging your Matrix server to SMS - see [docs/configuring-playbook-bridge-matrix-bridge-sms.md](docs/configuring-playbook-bridge-matrix-bridge-sms.md) for setup documentation
- (optional) the [Heisenbridge](https://github.com/hifi/heisenbridge) for bridging your Matrix server to IRC bouncer-style - see [docs/configuring-playbook-bridge-heisenbridge.md](docs/configuring-playbook-bridge-heisenbridge.md) for setup documentation
- (optional) the [go-skype-bridge](https://github.com/kelaresg/go-skype-bridge) for bridging your Matrix server to [Skype](https://www.skype.com) - see [docs/configuring-playbook-bridge-go-skype-bridge.md](docs/configuring-playbook-bridge-go-skype-bridge.md) for setup documentation
- (optional) the [mx-puppet-slack](https://hub.docker.com/r/sorunome/mx-puppet-slack) for bridging your Matrix server to [Slack](https://slack.com) - see [docs/configuring-playbook-bridge-mx-puppet-slack.md](docs/configuring-playbook-bridge-mx-puppet-slack.md) for setup documentation
- (optional) the [mx-puppet-instagram](https://github.com/Sorunome/mx-puppet-instagram) bridge for Instagram-DMs ([Instagram](https://www.instagram.com/)) - see [docs/configuring-playbook-bridge-mx-puppet-instagram.md](docs/configuring-playbook-bridge-mx-puppet-instagram.md) for setup documentation
- (optional) the [mx-puppet-twitter](https://github.com/Sorunome/mx-puppet-twitter) bridge for Twitter-DMs ([Twitter](https://twitter.com/)) - see [docs/configuring-playbook-bridge-mx-puppet-twitter.md](docs/configuring-playbook-bridge-mx-puppet-twitter.md) for setup documentation
### Homeserver
- (optional) the [mx-puppet-discord](https://github.com/matrix-discord/mx-puppet-discord) bridge for [Discord](https://discordapp.com/) - see [docs/configuring-playbook-bridge-mx-puppet-discord.md](docs/configuring-playbook-bridge-mx-puppet-discord.md) for setup documentation
The homeserver is the backbone of your matrix system. Choose one from the following list.
- (optional) the [mx-puppet-groupme](https://gitlab.com/xangelix-pub/matrix/mx-puppet-groupme) bridge for [GroupMe](https://groupme.com/) - see [docs/configuring-playbook-bridge-mx-puppet-groupme.md](docs/configuring-playbook-bridge-mx-puppet-groupme.md) for setup documentation
| Name | Default? | Description | Documentation |
| ---- | -------- | ----------- | ------------- |
| [Synapse](https://github.com/matrix-org/synapse) | ✓ | Storing your data and managing your presence in the [Matrix](http://matrix.org/) network | [Link](docs/configuring-playbook-synapse.md) |
| [Conduit](https://conduit.rs) | x | Storing your data and managing your presence in the [Matrix](http://matrix.org/) network. Conduit is a lightweight open-source server implementation of the Matrix Specification with a focus on easy setup and low system requirements | [Link](docs/configuring-playbook-conduit.md) |
| [Dendrite](https://github.com/matrix-org/dendrite) | x | Storing your data and managing your presence in the [Matrix](http://matrix.org/) network. Dendrite is a second-generation Matrix homeserver written in Go, an alternative to Synapse. | [Link](docs/configuring-playbook-dendrite.md) |
- (optional) the [mx-puppet-steam](https://github.com/icewind1991/mx-puppet-steam) bridge for [Steam](https://steamapp.com/) - see [docs/configuring-playbook-bridge-mx-puppet-steam.md](docs/configuring-playbook-bridge-mx-puppet-steam.md) for setup documentation
### Clients
- (optional) [Email2Matrix](https://github.com/devture/email2matrix) for relaying email messages to Matrix rooms - see [docs/configuring-playbook-email2matrix.md](docs/configuring-playbook-email2matrix.md) for setup documentation
Web clients for matrix that you can host on your own domains.
- (optional) [Dimension](https://github.com/turt2live/matrix-dimension), an open source integrations manager for matrix clients - see [docs/configuring-playbook-dimension.md](docs/configuring-playbook-dimension.md) for setup documentation
| Name | Default? | Description | Documentation |
| ---- | -------- | ----------- | ------------- |
[Element](https://app.element.io/) | ✓ | Web UI, which is configured to connect to your own Synapse server by default | [Link](docs/configuring-playbook-client-element.md) |
| [Hydrogen](https://github.com/vector-im/hydrogen-web) | x | Web client | [Link](docs/configuring-playbook-client-hydrogen.md) |
| [Cinny](https://github.com/ajbura/cinny) | x | Web client | [Link](docs/configuring-playbook-client-cinny.md) |
- (optional) [Etherpad](https://etherpad.org), an open source collaborative text editor - see [docs/configuring-playbook-etherpad.md](docs/configuring-playbook-etherpad.md) for setup documentation
- (optional) [Jitsi](https://jitsi.org/), an open source video-conferencing platform - see [docs/configuring-playbook-jitsi.md](docs/configuring-playbook-jitsi.md) for setup documentation
- (optional) [matrix-reminder-bot](https://github.com/anoadragon453/matrix-reminder-bot) for scheduling one-off & recurring reminders and alarms - see [docs/configuring-playbook-bot-matrix-reminder-bot.md](docs/configuring-playbook-bot-matrix-reminder-bot.md) for setup documentation
### Server Components
- (optional) [matrix-registration-bot](https://github.com/moan0s/matrix-registration-bot) for invitations by creating and managing registration tokens - see [docs/configuring-playbook-bot-matrix-registration-bot.md](docs/configuring-playbook-bot-matrix-registration-bot.md) for setup documentation
Services that run on the server to make the various parts of your installation work.
- (optional) [maubot](https://github.com/maubot/maubot) a plugin-based Matrix bot system - see [docs/configuring-playbook-bot-maubot.md](docs/configuring-playbook-bot-maubot.md) for setup documentation
| Name | Default? | Description | Documentation |
| ---- | -------- | ----------- | ------------- |
| [PostgreSQL](https://www.postgresql.org/)| ✓ | Database for Synapse. [Using an external PostgreSQL server](docs/configuring-playbook-external-postgres.md) is also possible. | [Link](docs/configuring-playbook-external-postgres.md) |
| [Coturn](https://github.com/coturn/coturn) | ✓ | STUN/TURN server for WebRTC audio/video calls | [Link](docs/configuring-playbook-turn.md) |
| [nginx](http://nginx.org/) | ✓ | Web server, listening on ports 80 and 443 - standing in front of all the other services. Using your own webserver [is possible](docs/configuring-playbook-own-webserver.md) | [Link](docs/configuring-playbook-nginx.md) |
| [Let's Encrypt](https://letsencrypt.org/) | ✓ | Free SSL certificate, which secures the connection to the Synapse server and the Element web UI | [Link](docs/configuring-playbook-ssl-certificates.md) |
| [ma1sd](https://github.com/ma1uta/ma1sd) | x | Matrix Identity Server | [Link](docs/configuring-playbook-ma1sd.md)
| [Exim](https://www.exim.org/) | ✓ | Mail server, through which all Matrix services send outgoing email (can be configured to relay through another SMTP server) | - |
| [Dimension](https://github.com/turt2live/matrix-dimension) | x | An open source integrations manager for matrix clients | [Link](docs/configuring-playbook-dimension.md) |
| [Sygnal](https://github.com/matrix-org/sygnal) | x | Push gateway | [Link](docs/configuring-playbook-sygnal.md) |
| [ntfy](https://ntfy.sh) | x | Push notifications server | [Link](docs/configuring-playbook-ntfy.md) |
- (optional) [honoroit](https://gitlab.com/etke.cc/honoroit) helpdesk bot - see [docs/configuring-playbook-bot-honoroit.md](docs/configuring-playbook-bot-honoroit.md) for setup documentation
- (optional) [Postmoogle](https://gitlab.com/etke.cc/postmoogle) email to matrix bot - see [docs/configuring-playbook-bot-postmoogle.md](docs/configuring-playbook-bot-postmoogle.md) for setup documentation
### Authentication
- (optional) [Go-NEB](https://github.com/matrix-org/go-neb) multi functional bot written in Go - see [docs/configuring-playbook-bot-go-neb.md](docs/configuring-playbook-bot-go-neb.md) for setup documentation
Extend and modify how users are authenticated on your homeserver.
- (optional) [Mjolnir](https://github.com/matrix-org/mjolnir), a moderation tool for Matrix - see [docs/configuring-playbook-bot-mjolnir.md](docs/configuring-playbook-bot-mjolnir.md) for setup documentation
| [matrix-ldap-registration-proxy](https://gitlab.com/activism.international/matrix_ldap_registration_proxy) (advanced) | x | A proxy that handles Matrix registration requests and forwards them to LDAP. | [Link](docs/configuring-playbook-matrix-ldap-registration-proxy.md) |
| [matrix-registration](https://github.com/ZerataX/matrix-registration) | x | A simple python application to have a token based matrix registration | [Link](docs/configuring-playbook-matrix-registration.md) |
- (optional) [synapse-admin](https://github.com/Awesome-Technologies/synapse-admin), a web UI tool for administrating users and rooms on your Matrix server - see [docs/configuring-playbook-synapse-admin.md](docs/configuring-playbook-synapse-admin.md) for setup documentation
- (optional) [matrix-registration](https://github.com/ZerataX/matrix-registration), a simple python application to have a token based matrix registration - see [docs/configuring-playbook-matrix-registration.md](docs/configuring-playbook-matrix-registration.md) for setup documentation
### File Storage
- (optional) the [Prometheus](https://prometheus.io) time-series database server, the Prometheus [node-exporter](https://prometheus.io/docs/guides/node-exporter/) host metrics exporter, and the [Grafana](https://grafana.com/) web UI - see [Enabling metrics and graphs (Prometheus, Grafana) for your Matrix server](docs/configuring-playbook-prometheus-grafana.md) for setup documentation
Use alternative file storage to the default `media_store` folder.
- (optional) the [Sygnal](https://github.com/matrix-org/sygnal) push gateway - see [Setting up the Sygnal push gateway](docs/configuring-playbook-sygnal.md) for setup documentation
| Name | Default? | Description | Documentation |
| ---- | -------- | ----------- | ------------- |
| [Goofys](https://github.com/kahing/goofys) | x | [Amazon S3](https://aws.amazon.com/s3/) (or other S3-compatible object store) storage for Synapse's content repository (`media_store`) files | [Link](docs/configuring-playbook-s3-goofys.md) |
| [synapse-s3-storage-provider](https://github.com/matrix-org/synapse-s3-storage-provider) | x | [Amazon S3](https://aws.amazon.com/s3/) (or other S3-compatible object store) storage for Synapse's content repository (`media_store`) files | [Link](docs/configuring-playbook-s3.md) |
- (optional) the [ntfy](https://ntfy.sh) push notifications server - see [docs/configuring-playbook-ntfy.md](docs/configuring-playbook-ntfy.md) for setup documentation
### Bridges
- (optional) the [Hydrogen](https://github.com/vector-im/hydrogen-web) web client - see [docs/configuring-playbook-client-hydrogen.md](docs/configuring-playbook-client-hydrogen.md) for setup documentation
Bridges can be used to connect your matrix installation with third-party communication networks.
- (optional) the [Cinny](https://github.com/ajbura/cinny) web client - see [docs/configuring-playbook-client-cinny.md](docs/configuring-playbook-client-cinny.md) for setup documentation
| Name | Default? | Description | Documentation |
| ---- | -------- | ----------- | ------------- |
[mautrix-discord](https://github.com/mautrix/discord) | x | Bridge for bridging your Matrix server to [Discord](https://discord.com/) | [Link](docs/configuring-playbook-bridge-mautrix-discord.md) |
| [mautrix-telegram](https://github.com/mautrix/telegram) | x | Bridge for bridging your Matrix server to [Telegram](https://telegram.org/) | [Link](docs/configuring-playbook-bridge-mautrix-telegram.md) |
| [mautrix-whatsapp](https://github.com/mautrix/whatsapp) | x | Bridge for bridging your Matrix server to [WhatsApp](https://www.whatsapp.com/) | [Link](docs/configuring-playbook-bridge-mautrix-whatsapp.md) |
| [mautrix-facebook](https://github.com/mautrix/facebook) | x | Bridge for bridging your Matrix server to [Facebook](https://facebook.com/) | [Link](docs/configuring-playbook-bridge-mautrix-facebook.md) |
| [mautrix-twitter](https://github.com/mautrix/twitter) | x | Bridge for bridging your Matrix server to [Twitter](https://twitter.com/) | [Link](docs/configuring-playbook-bridge-mautrix-twitter.md) |
| [mautrix-hangouts](https://github.com/mautrix/hangouts) | x | Bridge for bridging your Matrix server to [Google Hangouts](https://en.wikipedia.org/wiki/Google_Hangouts) | [Link](docs/configuring-playbook-bridge-mautrix-hangouts.md) |
| [mautrix-googlechat](https://github.com/mautrix/googlechat) | x | Bridge for bridging your Matrix server to [Google Chat](https://en.wikipedia.org/wiki/Google_Chat) | [Link](docs/configuring-playbook-bridge-mautrix-googlechat.md) |
| [mautrix-instagram](https://github.com/mautrix/instagram) | x | Bridge for bridging your Matrix server to [Instagram](https://instagram.com/) | [Link](docs/configuring-playbook-bridge-mautrix-instagram.md) |
| [mautrix-signal](https://github.com/mautrix/signal) | x | Bridge for bridging your Matrix server to [Signal](https://www.signal.org/) | [Link](docs/configuring-playbook-bridge-mautrix-signal.md) |
| [beeper-linkedin](https://github.com/beeper/linkedin) | x | Bridge for bridging your Matrix server to [LinkedIn](https://www.linkedin.com/) | [Link](docs/configuring-playbook-bridge-beeper-linkedin.md) |
| [matrix-appservice-irc](https://github.com/matrix-org/matrix-appservice-irc) | x | Bridge for bridging your Matrix server to [IRC](https://wikipedia.org/wiki/Internet_Relay_Chat) | [Link](docs/configuring-playbook-bridge-appservice-irc.md) |
| [matrix-appservice-discord](https://github.com/Half-Shot/matrix-appservice-discord) | x | Bridge for bridging your Matrix server to [Discord](https://discordapp.com/) | [Link](docs/configuring-playbook-bridge-appservice-discord.md) |
| [matrix-appservice-slack](https://github.com/matrix-org/matrix-appservice-slack) | x | Bridge for bridging your Matrix server to [Slack](https://slack.com/) | [Link](docs/configuring-playbook-bridge-appservice-slack.md) |
| [matrix-appservice-webhooks](https://github.com/turt2live/matrix-appservice-webhooks) | x | Bridge for slack compatible webhooks ([ConcourseCI](https://concourse-ci.org/), [Slack](https://slack.com/) etc. pp.) | [Link](docs/configuring-playbook-bridge-appservice-webhooks.md) |
| [matrix-hookshot](https://github.com/Half-Shot/matrix-hookshot) | x | Bridge for bridging Matrix to generic webhooks and multiple project management services, such as GitHub, GitLab, Figma, and Jira in particular | [Link](docs/configuring-playbook-bridge-hookshot.md) |
| [matrix-sms-bridge](https://github.com/benkuly/matrix-sms-bridge) | x | Bridge for bridging your Matrix server to SMS | [Link](docs/configuring-playbook-bridge-matrix-bridge-sms.md) |
| [Heisenbridge](https://github.com/hifi/heisenbridge) | x | Bridge for bridging your Matrix server to IRC bouncer-style | [Link](docs/configuring-playbook-bridge-heisenbridge.md) |
| [go-skype-bridge](https://github.com/kelaresg/go-skype-bridge) | x | Bridge for bridging your Matrix server to [Skype](https://www.skype.com) | [Link](docs/configuring-playbook-bridge-go-skype-bridge.md) |
| [mx-puppet-slack](https://hub.docker.com/r/sorunome/mx-puppet-slack) | x | Bridge for bridging your Matrix server to [Slack](https://slack.com) | [Link](docs/configuring-playbook-bridge-mx-puppet-slack.md) |
| [mx-puppet-instagram](https://github.com/Sorunome/mx-puppet-instagram) | x | Bridge for Instagram-DMs ([Instagram](https://www.instagram.com/)) | [Link](docs/configuring-playbook-bridge-mx-puppet-instagram.md) |
| [mx-puppet-twitter](https://github.com/Sorunome/mx-puppet-twitter) | x | Bridge for Twitter-DMs ([Twitter](https://twitter.com/)) | [Link](docs/configuring-playbook-bridge-mx-puppet-twitter.md) |
| [mx-puppet-discord](https://github.com/matrix-discord/mx-puppet-discord) | x | Bridge for [Discord](https://discordapp.com/) | [Link](docs/configuring-playbook-bridge-mx-puppet-discord.md) |
| [mx-puppet-groupme](https://gitlab.com/xangelix-pub/matrix/mx-puppet-groupme) | x | Bridge for [GroupMe](https://groupme.com/) | [Link](docs/configuring-playbook-bridge-mx-puppet-groupme.md) |
| [mx-puppet-steam](https://github.com/icewind1991/mx-puppet-steam) | x | Bridge for [Steam](https://steamapp.com/) | [Link](docs/configuring-playbook-bridge-mx-puppet-steam.md) |
| [Email2Matrix](https://github.com/devture/email2matrix) | x | Bridge for relaying email messages to Matrix rooms | [Link](docs/configuring-playbook-email2matrix.md) |
- (optional) the [Borg](https://borgbackup.org) backup - see [docs/configuring-playbook-backup-borg.md](docs/configuring-playbook-backup-borg.md) for setup documentation
- (optional) the [Buscarron](https://gitlab.com/etke.cc/buscarron) bot - see [docs/configuring-playbook-bot-buscarron.md](docs/configuring-playbook-bot-buscarron.md) for setup documentation
### Bots
- (optional) [Cactus Comments](https://cactus.chat), a federated comment system built on matrix - see [docs/configuring-playbook-cactus-comments.md](docs/configuring-playbook-cactus-comments.md) for setup documentation
Bots provide various additional functionality to your installation.
Basically, this playbook aims to get you up-and-running with all the necessities around Matrix, without you having to do anything else.
| Name | Default? | Description | Documentation |
| ---- | -------- | ----------- | ------------- |
| [matrix-reminder-bot](https://github.com/anoadragon453/matrix-reminder-bot) | x | Bot for scheduling one-off & recurring reminders and alarms | [Link](docs/configuring-playbook-bot-matrix-reminder-bot.md) |
| [matrix-registration-bot](https://github.com/moan0s/matrix-registration-bot) | x | Bot for invitations by creating and managing registration tokens | [Link](docs/configuring-playbook-bot-matrix-registration-bot.md) |
| [maubot](https://github.com/maubot/maubot) | x | A plugin-based Matrix bot system | [Link](docs/configuring-playbook-bot-maubot.md) |
| [honoroit](https://gitlab.com/etke.cc/honoroit) | x | A helpdesk bot | [Link](docs/configuring-playbook-bot-honoroit.md) |
| [Postmoogle](https://gitlab.com/etke.cc/postmoogle) | x | Email to matrix bot | [Link](docs/configuring-playbook-bot-postmoogle.md) |
| [Go-NEB](https://github.com/matrix-org/go-neb) | x | A multi functional bot written in Go | [Link](docs/configuring-playbook-bot-go-neb.md) |
| [Mjolnir](https://github.com/matrix-org/mjolnir) | x | A moderation tool for Matrix | [Link](docs/configuring-playbook-bot-mjolnir.md) |
| [Buscarron](https://gitlab.com/etke.cc/buscarron) | x | Web forms (HTTP POST) to matrix | [Link](docs/configuring-playbook-bot-buscarron.md) |
**Note**: the list above is exhaustive. It includes optional or even some advanced components that you will most likely not need.
### Administration
Sticking with the defaults (which install a subset of the above components) is the best choice, especially for a new installation.
You can always re-run the playbook later to add or remove components.
Services that help you in administrating and monitoring your matrix installation.
| Name | Default? | Description | Documentation |
| ---- | -------- | ----------- | ------------- |
| [synapse-admin](https://github.com/Awesome-Technologies/synapse-admin) | x | A web UI tool for administrating users and rooms on your Matrix server | [Link](docs/configuring-playbook-synapse-admin.md) |
| Metrics and Graphs | x | Consists of the [Prometheus](https://prometheus.io) time-series database server, the Prometheus [node-exporter](https://prometheus.io/docs/guides/node-exporter/) host metrics exporter, and the [Grafana](https://grafana.com/) web UI | [Link](docs/configuring-playbook-prometheus-grafana.md) |
| [Borg](https://borgbackup.org) | x | Backups | [Link](docs/configuring-playbook-backup-borg.md) |
### Misc
Various services that don't fit any other category.
| Name | Default? | Description | Documentation |
| ---- | -------- | ----------- | ------------- |
| [synapse-simple-antispam](https://github.com/t2bot/synapse-simple-antispam) (advanced) | x | A spam checker module | [Link](docs/configuring-playbook-synapse-simple-antispam.md) |
| [Matrix Corporal](https://github.com/devture/matrix-corporal) (advanced) | x | Reconciliator and gateway for a managed Matrix server | [Link](docs/configuring-playbook-matrix-corporal.md) |
| [Etherpad](https://etherpad.org) | x | An open source collaborative text editor | [Link](docs/configuring-playbook-etherpad.md) |
| [Jitsi](https://jitsi.org/) | x | An open source video-conferencing platform | [Link](docs/configuring-playbook-jitsi.md) |
| [Cactus Comments](https://cactus.chat) | x | A federated comment system built on matrix | [Link](docs/configuring-playbook-cactus-comments.md) |
As stated in the [Prerequisites](prerequisites.md), currently only `x86_64` is fully supported. However, it is possible to set the target architecture, and some tools can be built on the host or other measures can be used.
As stated in the [Prerequisites](prerequisites.md), currently only `amd64` (`x86_64`) is fully supported.
To that end add the following variable to your `vars.yml` file (see [Configuring playbook](configuring-playbook.md)):
The playbook automatically determines the target server's architecture (the `matrix_architecture` variable) to be one of the following:
Currently supported architectures are the following:
- `amd64` (the default)
- `arm64`
- `arm32`
- `arm32`
- `arm64`
so for the Raspberry Pi, the following should be in your `vars.yml` file:
Some tools and container images can be built on the host or other measures can be used to install on that architecture.
```yaml
matrix_architecture: "arm32"
```
## Implementation details
## Implementation details
For `amd64`, prebuilt container images (see the [container images we use](container-images.md)) are used for all components (except [Hydrogen](configuring-playbook-client-hydrogen.md), which goes through self-building).
For `amd64`, prebuilt container images (see the [container images we use](container-images.md)) are used for all components (except [Hydrogen](configuring-playbook-client-hydrogen.md), which goes through self-building).
For other architectures, components which have a prebuilt image make use of it. If the component is not available for the specific architecture, [self-building](self-building.md) will be used. Not all components support self-building though, so your mileage may vary.
For other architecture (`arm64`, `arm32`), components which have a prebuilt image make use of it. If the component is not available for the specific architecture, [self-building](self-building.md) will be used. Not all components support self-building though, so your mileage may vary.
@ -9,19 +9,14 @@ If your local computer cannot run Ansible, you can also run Ansible on some serv
## Supported Ansible versions
## Supported Ansible versions
Ansible 2.7.1 or newer is required ([last discussion about Ansible versions](https://github.com/spantaleev/matrix-docker-ansible-deploy/pull/743)).
To manually check which version of Ansible you're on, run: `ansible --version`.
Note: Ubuntu 20.04 ships with Ansible 2.9.6 which is a buggy version (see this [bug](https://bugs.launchpad.net/ubuntu/+source/ansible/+bug/1880359)), which can't be used in combination with a host running new systemd (more details in [#517](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/517), [#669](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/669)). If this problem affects you, you can: avoid running Ubuntu 20.04 on your host; run Ansible from another machine targeting your host; or try to upgrade to a newer Ansible version (see below).
## Checking your Ansible version
In most cases, you won't need to worry about the Ansible version.
For the **best experience**, we recommend getting the **latest version of Ansible available**.
The playbook will try to detect it and tell you if you're on an unsupported version.
To manually check which version of Ansible you're on, run: `ansible --version`.
We're not sure what's the minimum version of Ansible that can run this playbook successfully.
The lowest version that we've confirmed (on 2022-11-26) to be working fine is: `ansible-core` (`2.11.7`) combined with `ansible` (`4.10.0`).
If you're on an old version of Ansible, you should [upgrade Ansible to a newer version](#upgrading-ansible) or [use Ansible via Docker](#using-ansible-via-docker).
If your distro ships with an Ansible version older than this, you may run into issues. Consider [Upgrading Ansible](#upgrading-ansible) or [using Ansible via Docker](#using-ansible-via-docker).
## Upgrading Ansible
## Upgrading Ansible
@ -53,7 +48,7 @@ You can either [run Ansible in a container on the Matrix server itself](#running
To run Ansible in a (Docker) container on the Matrix server itself, you need to have a working Docker installation.
To run Ansible in a (Docker) container on the Matrix server itself, you need to have a working Docker installation.
Docker is normally installed by the playbook, so this may be a bit of a chicken and egg problem. To solve it:
Docker is normally installed by the playbook, so this may be a bit of a chicken and egg problem. To solve it:
- you **either** need to install Docker manually first. Follow [the upstream instructions](https://docs.docker.com/engine/install/) for your distribution and consider setting `matrix_docker_installation_enabled: false` in your `vars.yml` file, to prevent the playbook from installing Docker
- you **either** need to install Docker manually first. Follow [the upstream instructions](https://docs.docker.com/engine/install/) for your distribution and consider setting `matrix_playbook_docker_installation_enabled: false` in your `vars.yml` file, to prevent the playbook from installing Docker
- **or** you need to run the playbook in another way (e.g. [Running Ansible in a container on another computer (not the Matrix server)](#running-ansible-in-a-container-on-another-computer-not-the-matrix-server)) at least the first time around
- **or** you need to run the playbook in another way (e.g. [Running Ansible in a container on another computer (not the Matrix server)](#running-ansible-in-a-container-on-another-computer-not-the-matrix-server)) at least the first time around
Once you have a working Docker installation on the server, **clone the playbook** somewhere on the server and configure it as per usual (`inventory/hosts`, `inventory/host_vars/..`, etc.), as described in [configuring the playbook](configuring-playbook.md).
Once you have a working Docker installation on the server, **clone the playbook** somewhere on the server and configure it as per usual (`inventory/hosts`, `inventory/host_vars/..`, etc.), as described in [configuring the playbook](configuring-playbook.md).
@ -70,7 +65,7 @@ docker run -it --rm \
-w /work \
-w /work \
-v `pwd`:/work \
-v `pwd`:/work \
--entrypoint=/bin/sh \
--entrypoint=/bin/sh \
docker.io/devture/ansible:2.13.6-r0
docker.io/devture/ansible:2.13.6-r0-1
```
```
Once you execute the above command, you'll be dropped into a `/work` directory inside a Docker container.
Once you execute the above command, you'll be dropped into a `/work` directory inside a Docker container.
Captcha can be enabled for this home server. This file explains how to do that.
Captcha can be enabled for this home server. This file explains how to do that.
The captcha mechanism used is Google's [ReCaptcha](https://www.google.com/recaptcha/). This requires API keys from Google.
The captcha mechanism used is Google's [ReCaptcha](https://www.google.com/recaptcha/). This requires API keys from Google. If your homeserver is Dendrite then [hCapcha](https://www.hcaptcha.com) can be used instead.
## Getting keys
## ReCaptcha
### Getting keys
Requires a site/secret key pair from:
Requires a site/secret key pair from:
@ -12,12 +14,39 @@ Requires a site/secret key pair from:
Must be a reCAPTCHA **v2** key using the "I'm not a robot" Checkbox option
Must be a reCAPTCHA **v2** key using the "I'm not a robot" Checkbox option
## Setting ReCaptcha Keys
### Setting ReCaptcha keys
Once registered as above, set the following values:
Once registered as above, set the following values:
| metrics | `/metrics/hookshot` | `matrix_hookshot_metrics_enabled` and `matrix_hookshot_metrics_proxying_enabled`. Requires `/metrics/*` endpoints to also be enabled via `matrix_nginx_proxy_proxy_matrix_metrics_enabled` (see the `matrix-nginx-proxy` role). Read more in the [Metrics section](#metrics) below. | Prometheus |
| metrics | `/metrics/hookshot` | `matrix_hookshot_metrics_enabled` and `matrix_hookshot_metrics_proxying_enabled`. Requires `/metrics/*` endpoints to also be enabled via `matrix_nginx_proxy_proxy_matrix_metrics_enabled` (see the `matrix-nginx-proxy` role). Read more in the [Metrics section](#metrics) below. | Prometheus |
See also `matrix_hookshot_matrix_nginx_proxy_configuration` in [init.yml](/roles/custom/matrix-bridge-hookshot/tasks/init.yml).
See also `matrix_hookshot_matrix_nginx_proxy_configuration` in [init.yml](/roles/custom/matrix-bridge-hookshot/tasks/inject_into_nginx_proxy.yml).
The different listeners are also reachable *internally* in the docker-network via the container's name (configured by `matrix_hookshot_container_url`) and on different ports (e.g. `matrix_hookshot_appservice_port`). Read [main.yml](/roles/custom/matrix-bridge-hookshot/defaults/main.yml) in detail for more info.
The different listeners are also reachable *internally* in the docker-network via the container's name (configured by `matrix_hookshot_container_url`) and on different ports (e.g. `matrix_hookshot_appservice_port`). Read [main.yml](/roles/custom/matrix-bridge-hookshot/defaults/main.yml) in detail for more info.
@ -93,4 +93,4 @@ To explicitly enable metrics, use `matrix_hookshot_metrics_enabled: true`. This
### Collision with matrix-appservice-webhooks
### Collision with matrix-appservice-webhooks
If you are also running [matrix-appservice-webhooks](configuring-playbook-bridge-appservice-webhooks.md), it reserves its namespace by the default setting `matrix_appservice_webhooks_user_prefix: '_webhook_'`. You should take care if you modify its or hookshot's prefix that they do not collide with each other's namespace (default `matrix_hookshot_generic_user_id_prefix: '_webhooks_'`).
If you are also running [matrix-appservice-webhooks](configuring-playbook-bridge-appservice-webhooks.md), it reserves its namespace by the default setting `matrix_appservice_webhooks_user_prefix: '_webhook_'`. You should take care if you modify its or hookshot's prefix that they do not collide with each other's namespace (default `matrix_hookshot_generic_userIdPrefix: '_webhooks_'`).
By default, this playbook configures the [Synapse](https://github.com/matrix-org/synapse) Matrix server, but you can also use [Dendrite](https://github.com/matrix-org/dendrite).
**NOTES**:
- **You can't switch an existing Matrix server's implementation** (e.g. Synapse -> Dendrite). Proceed below only if you're OK with losing data or you're dealing with a server on a new domain name, which hasn't participated in the Matrix federation yet.
- **homeserver implementations other than Synapse may not be fully functional**. The playbook may also not assist you in an optimal way (like it does with Synapse). Make yourself familiar with the downsides before proceeding
The playbook provided settings for Dendrite are defined in [`roles/custom/matrix-dendrite/defaults/main.yml`](../roles/custom/matrix-dendrite/defaults/main.yml) and they ultimately end up in the generated `/matrix/dendrite/config/dendrite.yaml` file (on the server). This file is generated from the [`roles/custom/matrix-dendrite/templates/dendrite/dendrite.yaml.j2`](../roles/custom/matrix-dendrite/templates/dendrite/dendrite.yaml.j2) template.
**If there's an existing variable** which controls a setting you wish to change, you can simply define that variable in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`) and [re-run the playbook](installing.md) to apply the changes.
Alternatively, **if there is no pre-defined variable** for a Dendrite setting you wish to change:
- you can either **request a variable to be created** (or you can submit such a contribution yourself). Keep in mind that it's **probably not a good idea** to create variables for each one of Dendrite's various settings that rarely get used.
- or, you can **extend and override the default configuration** ([`dendrite.yaml.j2`](../roles/custom/matrix-dendrite/templates/dendrite/dendrite.yaml.j2)) by making use of the `matrix_dendrite_configuration_extension_yaml` variable. You can find information about this in [`roles/custom/matrix-dendrite/defaults/main.yml`](../roles/custom/matrix-dendrite/defaults/main.yml).
- or, if extending the configuration is still not powerful enough for your needs, you can **override the configuration completely** using `matrix_dendrite_configuration` (or `matrix_dendrite_configuration_yaml`). You can find information about this in [`roles/custom/matrix-dendrite/defaults/main.yml`](../roles/custom/matrix-dendrite/defaults/main.yml).
## Installation
To use Dendrite, you **generally** need the following additional `vars.yml` configuration:
@ -41,6 +41,13 @@ If you want to manage and remove old unused pads from Etherpad, you will first n
Then from the plugin manager page (`https://etherpad.<your-domain>/admin/plugins` or `https://dimension.<your-domain>/etherpad/admin/plugins`), install the `adminpads2` plugin. Once installed, you should have a "Manage pads" section in the Admin web-UI.
Then from the plugin manager page (`https://etherpad.<your-domain>/admin/plugins` or `https://dimension.<your-domain>/etherpad/admin/plugins`), install the `adminpads2` plugin. Once installed, you should have a "Manage pads" section in the Admin web-UI.
## How to use Etherpad widgets without an Integration Manager (like Dimension)
This is how it works in Element, it might work quite similar with other clients:
To integrate a standalone etherpad in a room, create your pad by visiting `https://etherpad.DOMAIN`. When the pad opens, copy the URL and send a command like this to the room: `/addwidget URL`. You will then find your integrated Etherpad within the right sidebar in the `Widgets` section.
## Set Dimension default to the self-hosted Etherpad (optional)
## Set Dimension default to the self-hosted Etherpad (optional)
If you decided to install [Dimension integration manager](configuring-playbook-dimension.md) alongside Etherpad, the Dimension administrator users can configure the default URL template.
If you decided to install [Dimension integration manager](configuring-playbook-dimension.md) alongside Etherpad, the Dimension administrator users can configure the default URL template.
# This stops the federation port on the synapse-reverse-proxy-companion side (normally `matrix-synapse-reverse-proxy-companion:8048` on the container network).
By default, a single JVB ([Jitsi VideoBridge](https://github.com/jitsi/jitsi-videobridge)) is deployed on the same host as the Matrix server. To allow more video-conferences to happen at the same time, you may need to provision additional JVB services on other hosts.
There is an ansible playbook that can be run with the following tag:
Each JVB will require a server id to be set so that it can be uniquely identified and this allows Jitsi to keep track of which conferences are on which JVB.
The server id is set with the variable `matrix_jitsi_jvb_server_id` which ends up as the JVB_WS_SERVER_ID environment variables in the JVB docker container.
This variable can be set via the host file, a parameter to the ansible command or in the `vars.yaml` for the host which will have the additional JVB. For example:
Note that the server id `jvb-1` is reserved for the JVB instance running on the Matrix host and therefore should not be used as the id of an additional jvb host.
The additional JVB will also need to expose the colibri web socket port and this can be done with the following variable:
The JVB will also need to know where the prosody xmpp server is located, similar to the server id this can be set in the vars for the JVB by using the variable
`matrix_jitsi_xmpp_server`. The Jitsi prosody container is deployed on the matrix server by default so the value can be set to the matrix domain. For example:
```yaml
matrix_jitsi_xmpp_server: "{{ matrix_domain }}"
```
However, it can also be set the ip address of the matrix server. This can be useful if you wish to use a private ip. For example:
```yaml
matrix_jitsi_xmpp_server: "192.168.0.1"
```
The nginx configuration will also need to be updated in order to deal with the additional JVB servers. This is achieved via its own configuration variable
`matrix_nginx_proxy_proxy_jitsi_additional_jvbs`, which contains a dictionary of server ids to ip addresses.
For example,
``` yaml
matrix_nginx_proxy_proxy_jitsi_additional_jvbs:
jvb-2: 192.168.0.2
jvb-3: 192.168.0.3
```
Applied together this will allow you to provision extra JVB instances which will register themselves with the prosody service and be available for jicofo
@ -4,6 +4,8 @@ The playbook can install and configure [matrix-registration](https://github.com/
**WARNING**: this is a poorly maintained and buggy project. It's better to avoid using it.
**WARNING**: this is a poorly maintained and buggy project. It's better to avoid using it.
**WARNING**: this is not related to [matrix-registration-bot](configuring-playbook-bot-matrix-registration-bot.md)
> matrix-registration is a simple python application to have a token based matrix registration.
> matrix-registration is a simple python application to have a token based matrix registration.
Use matrix-registration to **create unique registration links**, which people can use to register on your Matrix server. It allows you to **keep your server's registration closed (private)**, but still allow certain people (these having a special link) to register a user account.
Use matrix-registration to **create unique registration links**, which people can use to register on your Matrix server. It allows you to **keep your server's registration closed (private)**, but still allow certain people (these having a special link) to register a user account.
@ -6,9 +6,6 @@ If that's alright, you can skip this.
If you don't want this playbook's nginx webserver to take over your server's 80/443 ports like that,
If you don't want this playbook's nginx webserver to take over your server's 80/443 ports like that,
and you'd like to use your own webserver (be it nginx, Apache, Varnish Cache, etc.), you can.
and you'd like to use your own webserver (be it nginx, Apache, Varnish Cache, etc.), you can.
You should note, however, that the playbook's services work best when you keep using the integrated `matrix-nginx-proxy` webserver.
For example, disabling `matrix-nginx-proxy` when running a [Synapse worker setup for load-balancing](configuring-playbook-synapse.md#load-balancing-with-workers) (a more advanced, non-default configuration) is likely to cause various troubles (see [this issue](https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2090)). If you need a such more scalable setup, disabling `matrix-nginx-proxy` will be a bad idea. If yours will be a simple (default, non-worker-load-balancing) deployment, disabling `matrix-nginx-proxy` may be fine.
There are **2 ways you can go about it**, if you'd like to use your own webserver:
There are **2 ways you can go about it**, if you'd like to use your own webserver:
- [Method 1: Disabling the integrated nginx reverse-proxy webserver](#method-1-disabling-the-integrated-nginx-reverse-proxy-webserver)
- [Method 1: Disabling the integrated nginx reverse-proxy webserver](#method-1-disabling-the-integrated-nginx-reverse-proxy-webserver)
@ -19,7 +16,12 @@ There are **2 ways you can go about it**, if you'd like to use your own webserve
## Method 1: Disabling the integrated nginx reverse-proxy webserver
## Method 1: Disabling the integrated nginx reverse-proxy webserver
This method is about completely disabling the integrated nginx reverse-proxy webserver and replicating its behavior using another webserver.
This method is about completely disabling the integrated nginx reverse-proxy webserver and replicating its behavior using another webserver.
For an alternative, make sure to check Method #2 as well.
If that other webserver is `nginx`, you'd be able to include configuration files generated by the playbook into your `nginx` webserver.
If you'd like to use another webserver (not `nginx`), you'd need to do things manually. We have examples for other webservers below.
For an alternative (which keeps `matrix-nginx-proxy` around and connects your other reverse-proxy with it), make sure to check Method #2.
### Preparation
### Preparation
@ -40,8 +42,8 @@ No matter which external webserver you decide to go with, you'll need to:
Here are the variables required for the default configuration (Synapse and Element)
Here are the variables required for the default configuration (Synapse and Element)
- '--label "traefik.http.services.matrix-nginx-proxy-federation.loadbalancer.server.scheme={{ "https" if matrix_nginx_proxy_https_enabled else "http" }}"'
```
```
This method uses labels attached to the Nginx and Synapse containers to provide the Traefik Docker provider with the information it needs to proxy `matrix.DOMAIN`, `element.DOMAIN`, `dimension.DOMAIN` and `jitsi.DOMAIN`. Some [static configuration](https://docs.traefik.io/v2.0/reference/static-configuration/file/) is required in Traefik; namely, having endpoints on ports 443 and 8448 and having a certificate resolver.
This method uses labels attached to the Nginx and Synapse containers to provide the Traefik Docker provider with the information it needs to proxy `matrix.DOMAIN`, `element.DOMAIN`, `dimension.DOMAIN` and `jitsi.DOMAIN`. Some [static configuration](https://docs.traefik.io/v2.0/reference/static-configuration/file/) is required in Traefik; namely, having endpoints on ports 443 and 8448 and having a certificate resolver.
The playbook can install and configure [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) for you.
The playbook can install and configure [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) for you via the [com.devture.ansible.role.postgres_backup](https://github.com/devture/com.devture.ansible.role.postgres_backup) Ansible role.
For a more complete backup solution (one that includes not only Postgres, but also other configuration/data files), you may wish to look into [borg backup](configuring-playbook-backup-borg.md) instead.
For a more complete backup solution (one that includes not only Postgres, but also other configuration/data files), you may wish to look into [borg backup](configuring-playbook-backup-borg.md) instead.
@ -10,7 +10,7 @@ For a more complete backup solution (one that includes not only Postgres, but al
Minimal working configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`) to enable Postgres backup:
Minimal working configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`) to enable Postgres backup:
```yaml
```yaml
matrix_postgres_backup_enabled: true
devture_postgres_backup_enabled: true
```
```
Refer to the table below for additional configuration variables and their default values.
Refer to the table below for additional configuration variables and their default values.
@ -18,12 +18,13 @@ Refer to the table below for additional configuration variables and their defaul
|`matrix_postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups|
|`devture_postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups|
|`matrix_postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.|
|`devture_postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.|
|`matrix_postgres_backup_keep_days`|`7`|Number of daily backups to keep|
|`devture_postgres_backup_keep_days`|`7`|Number of daily backups to keep|
|`matrix_postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep|
|`devture_postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep|
|`matrix_postgres_backup_keep_months`|`12`|Number of monthly backups to keep|
|`devture_postgres_backup_keep_months`|`12`|Number of monthly backups to keep|
|`matrix_postgres_backup_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Storagepath for the database backups|
|`devture_postgres_base_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Base path for postgres-backup. Also see `devture_postgres_data_path` |
`matrix_prometheus_enabled`|[Prometheus](https://prometheus.io) is a time series database. It holds all the data we're going to talk about.
`matrix_prometheus_enabled`|[Prometheus](https://prometheus.io) is a time series database. It holds all the data we're going to talk about.
`matrix_prometheus_node_exporter_enabled`|[Node Exporter](https://prometheus.io/docs/guides/node-exporter/) is an addon of sorts to Prometheus that collects generic system information such as CPU, memory, filesystem, and even system temperatures
`matrix_prometheus_node_exporter_enabled`|[Node Exporter](https://prometheus.io/docs/guides/node-exporter/) is an addon of sorts to Prometheus that collects generic system information such as CPU, memory, filesystem, and even system temperatures
`matrix_prometheus_postgres_exporter_enabled`|[Postgres Exporter](configuring-playbook-prometheus-postgres.md) is an addon of sorts to expose Postgres database metrics to Prometheus.
`matrix_prometheus_postgres_exporter_enabled`|[Postgres Exporter](configuring-playbook-prometheus-postgres.md) is an addon of sorts to expose Postgres database metrics to Prometheus.
`matrix_prometheus_nginxlog_exporter_enabled`|[NGINX Log Exporter](configuring-playbook-prometheus-nginxlog.md) is an addon of sorts to expose NGINX logs to Prometheus.
`matrix_grafana_enabled`|[Grafana](https://grafana.com/) is the visual component. It shows (on the `stats.<your-domain>` subdomain) the dashboards with the graphs that we're interested in
`matrix_grafana_enabled`|[Grafana](https://grafana.com/) is the visual component. It shows (on the `stats.<your-domain>` subdomain) the dashboards with the graphs that we're interested in
`matrix_grafana_anonymous_access`|By default you need to log in to see graphs. If you want to publicly share your graphs (e.g. when asking for help in [`#synapse:matrix.org`](https://matrix.to/#/#synapse:matrix.org?via=matrix.org&via=privacytools.io&via=mozilla.org)) you'll want to enable this option.
`matrix_grafana_anonymous_access`|By default you need to log in to see graphs. If you want to publicly share your graphs (e.g. when asking for help in [`#synapse:matrix.org`](https://matrix.to/#/#synapse:matrix.org?via=matrix.org&via=privacytools.io&via=mozilla.org)) you'll want to enable this option.
`matrix_grafana_default_admin_user`<br>`matrix_grafana_default_admin_password`|By default Grafana creates a user with `admin` as the username and password. If you feel this is insecure and you want to change it beforehand, you can do that here
`matrix_grafana_default_admin_user`<br>`matrix_grafana_default_admin_password`|By default Grafana creates a user with `admin` as the username and password. If you feel this is insecure and you want to change it beforehand, you can do that here
@ -73,6 +77,7 @@ Name | Description
`matrix_prometheus_node_exporter_enabled`|Set this to `true` to enable the node (general system stats) exporter (locally, on the container network)
`matrix_prometheus_node_exporter_enabled`|Set this to `true` to enable the node (general system stats) exporter (locally, on the container network)
`matrix_prometheus_node_exporter_metrics_proxying_enabled`|Set this to `true` to expose the node (general system stats) metrics on `https://matrix.DOMAIN/metrics/node-exporter` (only takes effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`)
`matrix_prometheus_node_exporter_metrics_proxying_enabled`|Set this to `true` to expose the node (general system stats) metrics on `https://matrix.DOMAIN/metrics/node-exporter` (only takes effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`)
`matrix_prometheus_postgres_exporter_enabled`|Set this to `true` to enable the [Postgres exporter](configuring-playbook-prometheus-postgres.md) (locally, on the container network)
`matrix_prometheus_postgres_exporter_enabled`|Set this to `true` to enable the [Postgres exporter](configuring-playbook-prometheus-postgres.md) (locally, on the container network)
`matrix_prometheus_nginxlog_exporter_enabled`|Set this to `true` to enable the [NGINX Log exporter](configuring-playbook-prometheus-nginxlog.md) (locally, on the container network)
`matrix_prometheus_postgres_exporter_metrics_proxying_enabled`|Set this to `true` to expose the [Postgres exporter](configuring-playbook-prometheus-postgres.md) metrics on `https://matrix.DOMAIN/metrics/postgres-exporter` (only takes effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`)
`matrix_prometheus_postgres_exporter_metrics_proxying_enabled`|Set this to `true` to expose the [Postgres exporter](configuring-playbook-prometheus-postgres.md) metrics on `https://matrix.DOMAIN/metrics/postgres-exporter` (only takes effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`)
`matrix_bridge_hookshot_metrics_enabled`|Set this to `true` to make [Hookshot](configuring-playbook-bridge-hookshot.md) expose metrics (locally, on the container network)
`matrix_bridge_hookshot_metrics_enabled`|Set this to `true` to make [Hookshot](configuring-playbook-bridge-hookshot.md) expose metrics (locally, on the container network)
`matrix_bridge_hookshot_metrics_proxying_enabled`|Set this to `true` to expose the [Hookshot](configuring-playbook-bridge-hookshot.md) metrics on `https://matrix.DOMAIN/metrics/hookshot` (only takes effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`)
`matrix_bridge_hookshot_metrics_proxying_enabled`|Set this to `true` to expose the [Hookshot](configuring-playbook-bridge-hookshot.md) metrics on `https://matrix.DOMAIN/metrics/hookshot` (only takes effect if `matrix_nginx_proxy_proxy_matrix_metrics_enabled: true`)
# Enabling metrics and graphs for NginX logs (optional)
It can be useful to have some (visual) insight into NignX logs.
This adds [prometheus-nginxlog-exporter](https://github.com/martin-helmich/prometheus-nginxlog-exporter/) to your matrix deployment.
It will provide a prometheus 'metrics' endpoint exposing data from both the `matrix-nginx-proxy` and `matrix-synapse-reverse-proxy-companion` logs and automatically aggregates the data with prometheus.
Optionally it visualizes the data, if [`matrix-grafana`](configuring-playbook-prometheus-grafana.md) is enabled, by means of a dedicated Grafana dashboard named `NGINX PROXY`
You can enable this role by adding the following settings in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
matrix_prometheus_nginxlog_exporter_enabled: true
# required depency
matrix_prometheus_enabled: true
# optional for visualization
matrix_grafana_enabled: true
```
x | Prerequisites | Variable | Description
|:--:|:--:|:--:|:--|
**REQUIRED** | `matrix-prometheus`| `matrix_prometheus_enabled`|[Prometheus](https://prometheus.io) is a time series database. It holds all the data we're going to talk about.
_Optional_ | [`matrix-grafana`](configuring-playbook-prometheus-grafana.md) | [`matrix_grafana_enabled`](configuring-playbook-prometheus-grafana.md)|[Grafana](https://grafana.com) is the visual component. It shows (on the `stats.<your-domain>` subdomain) graphs that we're interested in. When enabled the `NGINX PROXY` dashboard is automatically added.
## Docker Image Compatibility
At the moment of writing only images for `amd64` and `arm64` architectures are available
The playbook currently does not support building an image.
You can however use a custom-build image by setting
Metrics and resulting graphs can contain a lot of information. NginX logs contain information like IP address, URLs, UserAgents and more. This information can reveal usage patterns and could be considered Personally Identifiable Information (PII). Think about this before enabling (anonymous) access.
Please make sure you change the default Grafana password.
## Save metrics on an external Prometheus server
The playbook will automatically integrate the metrics into the Prometheus server provided with this playbook. You can choose to save data on an external Prometheus instance.
The metrics of this role will be exposed on `https://matrix.DOMAIN/metrics/nginxlog` when setting
The playbook can provide a single endpoint (`https://matrix.DOMAIN/metrics/*`), under which various services may expose their metrics (e.g. `/metrics/node-exporter`, `/metrics/postgres-exporter`, `/metrics/nginxlog`, etc). To enable this `/metrics/*` feature, use `matrix_nginx_proxy_proxy_matrix_metrics_enabled`. To protect access using [Basic Authentication](https://en.wikipedia.org/wiki/Basic_access_authentication), see `matrix_nginx_proxy_proxy_matrix_metrics_basic_auth_enabled`.
The following variables may be of interest:
Name | Description
-----|----------
`matrix_nginx_proxy_proxy_matrix_metrics_enabled`|Set this to `true` to enable metrics exposure for various services on `https://matrix.DOMAIN/metrics/*`. Refer to the individual `matrix_SERVICE_metrics_proxying_enabled` variables below for exposing metrics for each individual service.
@ -28,13 +28,13 @@ While you will need some local disk space around, it's only to accommodate usage
## Installing
## Installing
After [creating the S3 bucket and configuring it](configuring-playbook-s3.md#bucket-creation-and-security-configuration), you can proceed to configure Goofys in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
After [creating the S3 bucket and configuring it](configuring-playbook-s3.md#bucket-creation-and-security-configuration), you can proceed to configure `s3-storage-provider` in your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
matrix_synapse_ext_synapse_s3_storage_provider_config_storage_class: STANDARD # or STANDARD_IA, etc.
matrix_synapse_ext_synapse_s3_storage_provider_config_storage_class: STANDARD # or STANDARD_IA, etc.
@ -62,7 +62,7 @@ Migrating your existing data can happen in multiple ways:
Instead of using `s3_media_upload` directly, which is very slow and painful for an initial data migration, we recommend [using another tool in combination with `s3_media_upload`](#using-another-tool-in-combination-with-s3_media_upload).
Instead of using `s3_media_upload` directly, which is very slow and painful for an initial data migration, we recommend [using another tool in combination with `s3_media_upload`](#using-another-tool-in-combination-with-s3_media_upload).
To copy your existing files, SSH into the server and run `/usr/local/bin/matrix-synapse-s3-storage-provider-shell`.
To copy your existing files, SSH into the server and run `/matrix/synapse/ext/s3-storage-provider/bin/shell`.
This launches a Synapse container, which has access to the local media store, Postgres database, S3 store and has some convenient environment variables configured for you to use (`MEDIA_PATH`, `BUCKET`, `ENDPOINT`, `UPDATE_DB_DAYS`, etc).
This launches a Synapse container, which has access to the local media store, Postgres database, S3 store and has some convenient environment variables configured for you to use (`MEDIA_PATH`, `BUCKET`, `ENDPOINT`, `UPDATE_DB_DAYS`, etc).
@ -76,12 +76,12 @@ Then use the following commands (`$` values come from environment variables - th
The `s3_media_upload upload` command may take a lot of time to complete.
The `s3_media_upload upload` command may take a lot of time to complete.
Instead of running the above commands manually in the shell, you can also run the `/usr/local/bin/matrix-synapse-s3-storage-provider-migrate` script which will run the same commands automatically. We demonstrate how to do it manually, because:
Instead of running the above commands manually in the shell, you can also run the `/matrix/synapse/ext/s3-storage-provider/bin/migrate` script which will run the same commands automatically. We demonstrate how to do it manually, because:
- it's what the upstream project demonstrates and it teaches you how to use the `s3_media_upload` tool
- it's what the upstream project demonstrates and it teaches you how to use the `s3_media_upload` tool
- allows you to check and verify the output of each command, to catch mistakes
- allows you to check and verify the output of each command, to catch mistakes
- includes progress bars and detailed output for each command
- includes progress bars and detailed output for each command
- allows you to easily interrupt slow-running commands, etc. (the `/usr/local/bin/matrix-synapse-s3-storage-provider-migrate` starts a container without interactive TTY support, so `Ctrl+C` may not work and you and require killing via `docker kill ..`)
- allows you to easily interrupt slow-running commands, etc. (the `/matrix/synapse/ext/s3-storage-provider/bin/migrate` starts a container without interactive TTY support, so `Ctrl+C` may not work and you and require killing via `docker kill ..`)
### Using another tool in combination with `s3_media_upload`
### Using another tool in combination with `s3_media_upload`
@ -119,7 +119,7 @@ As described in [How it works?](#how-it-works) above, when new media is uploaded
By default, we periodically ensure that all local files are uploaded to S3 and are then removed from the local filesystem. This is done automatically using:
By default, we periodically ensure that all local files are uploaded to S3 and are then removed from the local filesystem. This is done automatically using:
- the `/usr/local/bin/matrix-synapse-s3-storage-provider-migrate` script
- the `/matrix/synapse/ext/s3-storage-provider/bin/migrate` script
- .. invoked via the `matrix-synapse-s3-storage-provider-migrate.service` service
- .. invoked via the `matrix-synapse-s3-storage-provider-migrate.service` service
- .. triggered by the `matrix-synapse-s3-storage-provider-migrate.timer` timer, every day at 05:00
- .. triggered by the `matrix-synapse-s3-storage-provider-migrate.timer` timer, every day at 05:00
@ -37,7 +37,7 @@ If you'd like more customization power, you can start with one of the presets an
If you increase worker counts too much, you may need to increase the maximum number of Postgres connections too (example):
If you increase worker counts too much, you may need to increase the maximum number of Postgres connections too (example):
```yaml
```yaml
matrix_postgres_process_extra_arguments: [
devture_postgres_process_extra_arguments: [
"-c 'max_connections=200'"
"-c 'max_connections=200'"
]
]
```
```
@ -56,21 +56,27 @@ Certain Synapse administration tasks (managing users and rooms, etc.) can be per
If you'd like to use OpenID Connect authentication with Synapse, you'll need some additional reverse-proxy configuration (see [our nginx reverse-proxy doc page](configuring-playbook-nginx.md#synapse-openid-connect-for-single-sign-on)).
If you'd like to use OpenID Connect authentication with Synapse, you'll need some additional reverse-proxy configuration (see [our nginx reverse-proxy doc page](configuring-playbook-nginx.md#synapse-openid-connect-for-single-sign-on)).
This example configuration is for [keycloak](https://www.keycloak.org/), an opensource Identity Provider maintained by Red Hat.
For more detailed documentation on available options and how to setup keycloak, see the [Synapse documentation on OpenID Connect with keycloak](https://github.com/matrix-org/synapse/blob/develop/docs/openid.md#keycloak).
In case you encounter errors regarding the parsing of the variables, you can try to add `{% raw %}` and `{% endraw %}` blocks around them. For example ;
In case you encounter errors regarding the parsing of the variables, you can try to add `{% raw %}` and `{% endraw %}` blocks around them. For example ;
@ -12,15 +12,17 @@ growth of the Matrix community, and helps to make Matrix a success.
If you'd like to **help by enabling submission of general usage statistics** for your homeserver, add this to your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
If you'd like to **help by enabling submission of general usage statistics** for your homeserver, add this to your configuration file (`inventory/host_vars/matrix.<your-domain>/vars.yml`):
```yaml
```yaml
matrix_synapse_report_stats: true
matrix_synapse_report_stats: true # for synapse
matrix_dendrite_report_stats: true # for dendrite
```
```
## Usage statistics being submitted
## Usage statistics being submitted
When enabled, Synapse will regularly upload a few dozen statistics about your server.
When enabled, your homeserver will regularly upload a few dozen statistics about your server.
This data includes your homeserver's domain, the total number of users, the number of active
This data includes your homeserver's domain, the total number of users, the number of active
users, the total number of rooms, and the number of messages sent per day on your homeserver.
users, the total number of rooms, and the number of messages sent per day on your homeserver.
See [Synapse's documentation](https://github.com/matrix-org/synapse/blob/develop/docs/usage/administration/monitoring/reporting_homeserver_usage_statistics.md#available-statistics)
See [Synapse's documentation](https://github.com/matrix-org/synapse/blob/develop/docs/usage/administration/monitoring/reporting_homeserver_usage_statistics.md#available-statistics) or [Dendrite's documentation](https://github.com/matrix-org/dendrite/blob/main/docs/FAQ.md#what-is-being-reported-when-enabling-phone-home-statistics)
for the full list of statistics that are reported.
for the full list of statistics that are reported.
@ -47,6 +47,8 @@ When you're done with all the configuration you'd like to do, continue with [Ins
- [Configuring Conduit](configuring-playbook-conduit.md), if you've switched to the [Conduit](https://conduit.rs) homeserver implementation (optional)
- [Configuring Conduit](configuring-playbook-conduit.md), if you've switched to the [Conduit](https://conduit.rs) homeserver implementation (optional)
- [Configuring Dendrite](configuring-playbook-dendrite.md), if you've switched to the [Dendrite](https://matrix-org.github.io/dendrite) homeserver implementation (optional)
@ -13,17 +13,21 @@ The general command syntax is: `ansible-playbook -i inventory/hosts setup.yml --
Here are some playbook tags that you should be familiar with:
Here are some playbook tags that you should be familiar with:
- `setup-all` - runs all setup tasks for all components, but does not start/restart services
- `setup-all` - runs all setup tasks (installation and uninstallation) for all components, but does not start/restart services
- `install-all` - like `setup-all`, but skips uninstallation tasks. Useful for maintaining your setup quickly when its components remain unchanged. If you adjust your `vars.yml` to remove components, you'd need to run `setup-all` though, or these components will still remain installed
- `setup-SERVICE` (e.g. `setup-bot-postmoogle`) - runs the setup tasks only for a given role, but does not start/restart services. You can discover these additional tags in each role (`roles/*/main.yml`). Running per-component setup tasks is **not recommended**, as components sometimes depend on each other and running just the setup tasks for a given component may not be enough. For example, setting up the [mautrix-telegram bridge](configuring-playbook-bridge-mautrix-telegram.md), in addition to the `setup-mautrix-telegram` tag, requires database changes (the `setup-postgres` tag) as well as reverse-proxy changes (the `setup-nginx-proxy` tag).
- `setup-SERVICE` (e.g. `setup-bot-postmoogle`) - runs the setup tasks only for a given role, but does not start/restart services. You can discover these additional tags in each role (`roles/*/main.yml`). Running per-component setup tasks is **not recommended**, as components sometimes depend on each other and running just the setup tasks for a given component may not be enough. For example, setting up the [mautrix-telegram bridge](configuring-playbook-bridge-mautrix-telegram.md), in addition to the `setup-mautrix-telegram` tag, requires database changes (the `setup-postgres` tag) as well as reverse-proxy changes (the `setup-nginx-proxy` tag).
- `install-SERVICE` (e.g. `install-bot-postmoogle`) - like `setup-SERVICE`, but skips uninstallation tasks. See `install-all` above for additional information.
- `start` - starts all systemd services and makes them start automatically in the future
- `start` - starts all systemd services and makes them start automatically in the future
- `stop` - stops all systemd services
- `stop` - stops all systemd services
- `ensure-matrix-users-created` - a special tag which ensures that all special users needed by the playbook (for bots, etc.) are created
- `ensure-matrix-users-created` - a special tag which ensures that all special users needed by the playbook (for bots, etc.) are created
`setup-*` tags **do not start services** automatically, because you may wish to do things before starting services, such as importing a database dump, restoring data from another server, etc.
`setup-*` tags and `install-*` tags **do not start services** automatically, because you may wish to do things before starting services, such as importing a database dump, restoring data from another server, etc.
## 1. Installing Matrix
## 1. Installing Matrix
@ -40,7 +44,7 @@ There 2 ways to start the installation process - depending on whether you're [In
If this is **a brand new** Matrix server and you **won't be importing old data into it**, run all these tags:
If this is **a brand new** Matrix server and you **won't be importing old data into it**, run all these tags:
When this command completes, services won't be running yet.
When this command completes, services won't be running yet.
@ -82,6 +86,8 @@ Proceed to [Maintaining your setup in the future](#2-maintaining-your-setup-in-t
Feel free to **re-run the setup command any time** you think something is off with the server configuration. Ansible will take your configuration and update your server to match.
Feel free to **re-run the setup command any time** you think something is off with the server configuration. Ansible will take your configuration and update your server to match.
Note that if you remove components from `vars.yml`, or if we switch some component from being installed by default to not being installed by default anymore, you'd need to run the setup command with `--tags=setup-all` instead of `--tags=install-all`. See [Playbook tags introduction](#playbook-tags-introduction)
You can use the `/usr/local/bin/matrix-postgres-cli` tool to get interactive terminal access ([psql](https://www.postgresql.org/docs/11/app-psql.html)) to the PostgreSQL server.
You can use the `/matrix/postgres/bin/cli` tool to get interactive terminal access ([psql](https://www.postgresql.org/docs/11/app-psql.html)) to the PostgreSQL server.
If you are using an [external Postgres server](configuring-playbook-external-postgres.md), the above tool will not be available.
If you are using an [external Postgres server](configuring-playbook-external-postgres.md), the above tool will not be available.
@ -80,6 +80,8 @@ This playbook can upgrade your existing Postgres setup with the following comman
**Warning: If you're using Borg Backup keep in mind that there is no official Postgres 15 support yet.**
**The old Postgres data directory is backed up** automatically, by renaming it to `/matrix/postgres/data-auto-upgrade-backup`.
**The old Postgres data directory is backed up** automatically, by renaming it to `/matrix/postgres/data-auto-upgrade-backup`.
To rename to a different path, pass some extra flags to the command above, like this: `--extra-vars="postgres_auto_upgrade_backup_data_path=/another/disk/matrix-postgres-before-upgrade"`
To rename to a different path, pass some extra flags to the command above, like this: `--extra-vars="postgres_auto_upgrade_backup_data_path=/another/disk/matrix-postgres-before-upgrade"`
PostgreSQL can be tuned to make it run faster. This is done by passing extra arguments to Postgres with the `matrix_postgres_process_extra_arguments` variable. You should use a website like https://pgtune.leopard.in.ua/ or information from https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server to determine what Postgres settings you should change.
PostgreSQL can be tuned to make it run faster. This is done by passing extra arguments to Postgres with the `devture_postgres_process_extra_arguments` variable. You should use a website like https://pgtune.leopard.in.ua/ or information from https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server to determine what Postgres settings you should change.
**Note**: the configuration generator at https://pgtune.leopard.in.ua/ adds spaces around the `=` sign, which is invalid. You'll need to remove it manually (`max_connections = 300` -> `max_connections=300`)
**Note**: the configuration generator at https://pgtune.leopard.in.ua/ adds spaces around the `=` sign, which is invalid. You'll need to remove it manually (`max_connections = 300` -> `max_connections=300`)
@ -107,7 +109,7 @@ These are not recommended values and they may not work well for you. This is jus
Here is an example config for a small 2 core server with 4GB of RAM and SSD storage:
Here is an example config for a small 2 core server with 4GB of RAM and SSD storage:
Here is an example config for a 4 core server with 8GB of RAM on a Virtual Private Server (VPS); the paramters have been configured using https://pgtune.leopard.in.ua with the following setup: PostgreSQL version 12, OS Type: Linux, DB Type: Mixed type of application, Data Storage: SSD storage:
Here is an example config for a 4 core server with 8GB of RAM on a Virtual Private Server (VPS); the paramters have been configured using https://pgtune.leopard.in.ua with the following setup: PostgreSQL version 12, OS Type: Linux, DB Type: Mixed type of application, Data Storage: SSD storage:
- `ADMIN_VALUE` being either `0` (regular user) or `1` (admin)
- `USER` and `DOMAIN` pointing to a valid user on your server
If you're using the integrated Postgres server and not an [external Postgres server](configuring-playbook-external-postgres.md), you can launch a Postgres into the `synapse` database by:
- running `/matrix/postgres/bin/cli` - to launch [`psql`](https://www.postgresql.org/docs/current/app-psql.html)
- running `\c synapse` - to change to the `synapse` database
You can then proceed to run the query above.
**Note**: directly modifying the raw data of Synapse (or any other software) could cause the software to break. You've been warned!
@ -6,11 +6,11 @@ The playbook supports self-building of various components, which don't have a co
For other architectures (e.g. `arm32`, `arm64`), ready-made container images are used when available. If there's no ready-made image for a specific component and said component supports self-building, an image will be built on the host. Building images like this takes more time and resources (some build tools need to get installed by the playbook to assist building).
For other architectures (e.g. `arm32`, `arm64`), ready-made container images are used when available. If there's no ready-made image for a specific component and said component supports self-building, an image will be built on the host. Building images like this takes more time and resources (some build tools need to get installed by the playbook to assist building).
To make use of self-building, you don't need to do anything besides change your architecture variable (e.g. `matrix_architecture: arm64`). If a component has an image for the specified architecture, the playbook will use it directly. If not, it will build the image on the server itself.
To make use of self-building, you don't need to do anything. If a component has an image for the specified architecture, the playbook will use it directly. If not, it will build the image on the server itself.
Note that **not all components support self-building yet**.
Note that **not all components support self-building yet**.
List of roles where self-building the Docker image is currently possible:
Possibly outdated list of roles where self-building the Docker image is currently possible:
## Option 1 (if you are using the default matrix-postgres container):
## Option 1 (if you are using the integrated Postgres database):
You can reset a user's password via the Ansible playbook (make sure to edit the `<your-username>` and `<your-password>` part below):
You can reset a user's password via the Ansible playbook (make sure to edit the `<your-username>` and `<your-password>` part below):
@ -36,7 +36,7 @@ Use the Synapse User Admin API as described here: https://github.com/matrix-org/
This requires an [access token](obtaining-access-tokens.md) from a server admin account. *This method will also log the user out of all of their clients while the other options do not.*
This requires an [access token](obtaining-access-tokens.md) from a server admin account. *This method will also log the user out of all of their clients while the other options do not.*
If you didn't make your account a server admin when you created it, you can use the `/usr/local/bin/matrix-change-user-admin-status` script as described in [registering-users.md](registering-users.md).
If you didn't make your account a server admin when you created it, you can learn how to switch it now by reading about it in [Adding/Removing Administrator privileges to an existing Synapse user](registering-users.md#addingremoving-administrator-privileges-to-an-existing-synapse-user).
### Example:
### Example:
To set @user:domain.com's password to `correct_horse_battery_staple` you could use this curl command:
To set @user:domain.com's password to `correct_horse_battery_staple` you could use this curl command:
hosts:"{{ target if target is defined else 'matrix_servers' }}"
become:true
roles:
# Most of the roles below are not distributed with the playbook, but downloaded separately using `ansible-galaxy` via the `make roles` command (see `Makefile`).
- name:Fail with matrix_backup_borg_version advice if Postgres not enabled
- name:Fail with matrix_backup_borg_version advice if Postgres not enabled
ansible.builtin.fail:
ansible.builtin.fail:
msg:>-
msg:>-
You are not running a built-in Postgres server (`matrix_postgres_enabled:false`), so auto-detecting its version and setting `matrix_backup_borg_version` automatically based on that cannot happen.
You are not running a built-in Postgres server (`devture_postgres_enabled:false`), so auto-detecting its version and setting `matrix_backup_borg_version` automatically based on that cannot happen.
Consider setting `matrix_backup_borg_version` to your Postgres version manually.
Consider setting `matrix_backup_borg_version` to your Postgres version manually.
when:not matrix_postgres_enabled
when:not devture_postgres_enabled
- ansible.builtin.import_role:
- ansible.builtin.include_role:
name:custom/matrix-postgres
name:galaxy/com.devture.ansible.role.postgres
tasks_from:detect_existing_postgres_version
tasks_from:detect_existing_postgres_version
- name:Fail if detected Postgres version is unsupported
- name:Fail if detected Postgres version is unsupported
ansible.builtin.fail:
ansible.builtin.fail:
msg:"You cannot use borg backup with such an old version ({{ matrix_postgres_detected_version }}) of Postgres. Consider upgrading - link to docs for upgrading Postgres: docs/maintenance-postgres.md#upgrading-postgresql"
msg:"You cannot use borg backup with such an old version ({{ devture_postgres_detected_version }}) of Postgres. Consider upgrading - link to docs for upgrading Postgres: docs/maintenance-postgres.md#upgrading-postgresql"
when:"matrix_postgres_detected_version not in matrix_backup_borg_supported_postgres_versions"
when:"devture_postgres_detected_version not in matrix_backup_borg_supported_postgres_versions"
# If you change this to false you must install and update Docker manually. You also need to install the docker (https://pypi.org/project/docker/) Python package.
msg:"You need to set a valid homeserver implementation in `matrix_homeserver_implementation`"
msg:"You need to set a valid homeserver implementation in `matrix_homeserver_implementation`"
when:"matrix_homeserver_implementation not in ['synapse', 'dendrite', 'conduit']"
when:"matrix_homeserver_implementation not in ['synapse', 'dendrite', 'conduit']"
# We generally support Ansible 2.7.1 and above.
- name:Fail if running on Ansible < 2.7.1
ansible.builtin.fail:
msg:"You are running on Ansible {{ ansible_version.string }}, which is not supported. See our guide about Ansible: https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/docs/ansible.md"
when:
- "(ansible_version.major < 2) or (ansible_version.major == 2 and ansible_version.minor < 7) or (ansible_version.major == 2 and ansible_version.minor == 7 and ansible_version.revision < 1)"
# Though we do not support Ansible 2.9.6 which is buggy
- name:Fail if running on Ansible 2.9.6 on Ubuntu
ansible.builtin.fail:
msg:"You are running on Ansible {{ ansible_version.string }}, which is not supported. See our guide about Ansible: https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/docs/ansible.md"
when:
- ansible_distribution == 'Ubuntu'
- "ansible_version.major == 2 and ansible_version.minor == 9 and ansible_version.revision == 6"
- name:(Deprecation) Catch and report renamed settings
- name:(Deprecation) Catch and report renamed settings
- name:Fail if matrix_architecture is set incorrectly
ansible.builtin.fail:
msg:"Detected that variable matrix_architecture {{ matrix_architecture }} appears to be set incorrectly. See docs/alternative-architectures.md. Server appears to be {{ ansible_architecture }}."
when:matrix_architecture not in ['amd64', 'arm32', 'arm64']
- name:Fail if uppercase domain used
- name:Fail if uppercase domain used
ansible.builtin.fail:
ansible.builtin.fail:
msg:"Detected that you're using an uppercase domain name - `{{ item }}`. This will cause trouble. Please use all-lowercase!"
msg:"Detected that you're using an uppercase domain name - `{{ item }}`. This will cause trouble. Please use all-lowercase!"
@ -66,20 +60,6 @@
- "{{ matrix_server_fqn_element }}"
- "{{ matrix_server_fqn_element }}"
when:"item != item | lower"
when:"item != item | lower"
- name:Fail if using python2 on Archlinux
ansible.builtin.fail:
msg:"Detected that you're using python2 when installing onto Archlinux. Archlinux by default only supports python3."
when:
- ansible_distribution == 'Archlinux'
- ansible_python.version.major != 3
- name:Fail if architecture is set incorrectly
ansible.builtin.fail:
msg:"Detected that variable matrix_architecture {{ matrix_architecture }} appears to be set incorrectly. See docs/alternative-architectures.md. Server appears to be {{ ansible_architecture }}."
when:(ansible_architecture == "x86_64" and matrix_architecture != "amd64") or
(ansible_architecture == "aarch64" and matrix_architecture != "arm64") or
(ansible_architecture.startswith("armv") and matrix_architecture != "arm32")
- name:Fail if encountering usage of removed role (mx-puppet-skype)
- name:Fail if encountering usage of removed role (mx-puppet-skype)
Cody Wyatt Neiman
2 years ago