Interestingly, no one has reported this failure before #662 (Github
Issue).
It doesn't make sense to keep saying that we support such old Ansible
versions, when we're not even testing on anything close to those.
Time is also passing and such versions are getting more and more
ancient. It's time we bumped our requirements to something that is more
likely to work.
Change anonymized to more proper term as server vice this is not anonymized. Server name is the first parameter that is collected. And if server happens to be for individual use these statistics would be at personal level without any anonymising.
We recently had a report of the Postgres backup container's log file
growing the size of /var/lib/docker until it ran out of disk space.
Trying to prevent similar problems in the future.
If a Postgres dump contains ALTER TABLE ... OWNER_TO <username>
statements which set the owner to a username different from
'synapse' the post Postgres import task will fail complaining
about lack of role.
Changing the matrix_postgres_connection_username group var has no
effect. However, the ALTER TABLE statements (and accompanying comments)
can be rewritten to change the username to 'synapse', which permits the
import task to succeed.
From a sample of 1, having the owner set in this was causes no
discernable side effects on the homeserver.
One could also remove the two variables from the docs completely,
because they are set by the playbook automatically.
Error: javax.net.ssl.SSLPeerUnverifiedException: Certificate for
<matrix.<your-domain>> doesn't match any of the subject alternative
names: [<your-domain>]
Fixes#577 (Github Issue).
`matrix_container_images_self_build` was not really doing anything
anymore. It previously was influencing `matrix_*_self_build` variables,
but it's no longer the case since some time ago.
Individual `matrix_*_self_build` variables are still available.
People that would like to toggle self-building for a specific component
ought to use those.
These variables are also controlled automatically (via
`group_vars/matrix_servers`) depending on `matrix_architecture`.
In other words, self-building is being done automatically for
all components when they don't have a prebuilt image for the specified
architecture. Some components only support `amd64`, while others also
have images for other architectures.
This flag is necessary for Ansible to ask for the sudo password when using the non-root option as documented in the `hosts` file. Otherwise, Ansible errors out with `missing sudo password`.
We do this for 2 reasons:
- so we can control things which are not controllable using environment
variables (for example `stunServers` in jitsi/web, since we don't wish
to use the hardcoded Google STUN servers if our own Coturn is enabled)
- so playbook variable changes will properly rebuild the configuration.
When using Jitsi environment variables, the configuration is only built
once (the first time) and never rebuilt again. This is not the
consistent with the rest of the playbook and with how Ansible operates.
We're not perfect at it (yet), because we still let the Jitsi containers
generate some files on their own, but we are closer and it should be
good enough for most things.
Related to #415 (Github Pull Request).
I am new to synapse. Thanks so much for these playbooks!
I wasn't sure how you actual activate dimensions after setting these variables. Should you re run
ansible-playbook -i inventory/hosts setup.yml --tags=setup-all
ansible-playbook -i inventory/hosts setup.yml --tags=start
? If so perhaps you could tell the readers to do this somewhere in the help file
Thanks =]
This fixes an incorrect indentation in the database specification for
appservice-irc which caused matrix-appservice-irc to refuse to start
with the remarkably unhelpful error message:
```
ERROR:CLI Failed to run bridge.
```
This also updates doc links to the new matrixdotorg repo because the
tedomum repo contains out-of-date documentation.
Don't mention systemd-journald adjustment anymore, because
we've changed log levels to WARNING and Synapse is not chatty by default
anymore.
The "excessive log messages may get dropped on CentOS" issue no longer
applies to most users and we shouldn't bother them with it.
Continuation of #234 (Github Pull Request).
I had unintentionally updated the documentation for the feature,
saying the page is available at `https://matrix.DOMAIN/nginx_status`.
Looks like it wasn't the case, going against my expectations.
I'm correcting this with this patch.
The status page is being made available on both HTTP and HTTPS.
Serving over HTTP is likely necessary for services like
Longview
(https://www.linode.com/docs/platform/longview/longview-app-for-nginx/)
It's been pointed out that DEBUG logs could contain sensitive
information (access tokens, etc.), which makes them unsuitable
for sharing with others. INFO should be enough.
Also discussed previously in #213 (Github Pull Request).
shared-secret-auth and rest-auth logging is still at `INFO`
intentionally, as user login events seem more important to keep.
Those modules typically don't spam as much.
We do restart Synapse explicitly, but some other services
(bridges, matrix-corporal, ..) may not restart sometimes.
It's best to restart all services explicitly.
Somewhat related to #213 (Github Pull Request).
We've been moving in the opposite direction for quite a long time.
All services should just leave logging to systemd's journald.
Well, `config.yaml` has been playbook-managed for a long time.
It's now extended to match the default sample config of the Discord
bridge.
With this patch, we also make `registration.yaml` playbook-managed,
which leads us to consistency with all other bridges.
Along with that, we introduce `./config` and `./data` separation,
like we do for the other bridges.
With most people on Synapse v0.99+ and Synapse v1.0 now available,
we should no longer try to be backward compatible with Synapse 0.34,
because this just complicates the instructions for no good reason.
Changes to the original are:
- it tells people to stop and disable services, so that:
- services won't be running while you are copying files
- services won't accidentally start again later
- it does the file-copying in 1 step
- it does copying before running `--tags=setup-all`, so that existing files (SSL certificates, etc.) can be reused. Otherwise, the playbook starts from a blank slate, retrieves them anew, generates new signing keys anew, etc. Only to have those replaced by your own old backup later.
- it mentions DNS changes
- combines `--tags=setup-all,start` into a single step, thanks to the files being already copied
This doesn't replace all usage of `-v`, but it's a start.
People sometimes troubleshoot by deleting files (especially bridge
config files). Restarting Synapse with a missing registration.yaml file
for a given bridge, causes the `-v
/something/registration.yaml:/something/registration.yaml:ro` option
to force-create `/something/registration.yaml` as a directory.
When a path that's provided to the `-v` option is missing, Docker
auto-creates that path as a directory.
This causes more breakage and confusion later on.
We'd rather fail, instead of magically creating directories.
Using `--mount`, instead of `-v` is the solution to this.
From Docker's documentation:
> When you use --mount with type=bind, the host-path must refer to an existing path on the host.
> The path will not be created for you and the service will fail with an error if the path does not exist.
The bit about the matrix-make-user-admin script was messed up (it wasn't actually a code block so the "<username>" was hidden). For me at least it seems like the ``` syntax is much harder to accidentally mess up.
Fixes#129 (Github Issue).
Unfortunately, we rely on `service_facts`, which is only available
in Ansible >= 2.5.
There's little reason to stick to an old version such as Ansible 2.4:
- some time has passed since we've raised version requirements - it's
time to move into the future (a little bit)
- we've recently (in 82b4640072) improved the way one can run
Ansible in a Docker container
From now on, Ansible >= 2.5 is required.
Inspired by #128 (Github Issue), we've created a new Docker image
to replace https://hub.docker.com/r/qmxme/ansible
Adding dnspython or dig to `qmxme/ansible` doesn't seem like a good
idea (that might be accepted by them), given that it's specific to our
use case. That's why we'll be maintaining our own image from now on.
When using Let's Encrypt SSL certificates, a cronjob is set up to
automatically renew them. Though it does require a `cron`-compatible
program on the server.
This fixes the error that is caused by the `/etc/cron.d` directory
not existing and the `ansible-cron` module trying to write out a
file there -- without checking if the directory exists first.
It didn't mention `matrix_appservice_discord_client_id` and
`matrix_appservice_discord_bot_token`, which makes it hard for
beginners.
Related to #105 (Github Pull Request).
`matrix_synapse_no_tls` is now implicit, so we've gotten rid of it.
The `homeserver.yaml.j2` template has been synchronized with the
configuration generated by Synapse v0.99.1 (some new options
are present, etc.)
We had something like that on the Server Delegation how-to page,
but it's better if we have it on the SSL certificates page.
Relocated there and improved linking.
Fixes#94 (Github Issue)
This reverts commit 0dac5ea508.
Relying on pyOpenSSL is the Ansible way of doing things, but is
impractical and annoying for users.
`openssl` is easily available on most servers, even by default.
We'd better use that.
This makes all containers (except mautrix-telegram and
mautrix-whatsapp), start as a non-root user.
We do this, because we don't trust some of the images.
In any case, we'd rather not trust ALL images and avoid giving
`root` access at all. We can't be sure they would drop privileges
or what they might do before they do it.
Because Postfix doesn't support running as non-root,
it had to be replaced by an Exim mail server.
The matrix-nginx-proxy nginx container image is patched up
(by replacing its main configuration) so that it can work as non-root.
It seems like there's no other good image that we can use and that is up-to-date
(https://hub.docker.com/r/nginxinc/nginx-unprivileged is outdated).
Likewise for riot-web (https://hub.docker.com/r/bubuntux/riot-web/),
we patch it up ourselves when starting (replacing the main nginx
configuration).
Ideally, it would be fixed upstream so we can simplify.
With this change, the following roles are now only dependent
on the minimal `matrix-base` role:
- `matrix-corporal`
- `matrix-coturn`
- `matrix-mailer`
- `matrix-mxisd`
- `matrix-postgres`
- `matrix-riot-web`
- `matrix-synapse`
The `matrix-nginx-proxy` role still does too much and remains
dependent on the others.
Wiring up the various (now-independent) roles happens
via a glue variables file (`group_vars/matrix-servers`).
It's triggered for all hosts in the `matrix-servers` group.
According to Ansible's rules of priority, we have the following
chain of inclusion/overriding now:
- role defaults (mostly empty or good for independent usage)
- playbook glue variables (`group_vars/matrix-servers`)
- inventory host variables (`inventory/host_vars/matrix.<your-domain>`)
All roles default to enabling their main component
(e.g. `matrix_mxisd_enabled: true`, `matrix_riot_web_enabled: true`).
Reasoning: if a role is included in a playbook (especially separately,
in another playbook), it should "work" by default.
Our playbook disables some of those if they are not generally useful
(e.g. `matrix_corporal_enabled: false`).
As suggested in #63 (Github issue), splitting the
playbook's logic into multiple roles will be beneficial for
maintainability.
This patch realizes this split. Still, some components
affect others, so the roles are not really independent of one
another. For example:
- disabling mxisd (`matrix_mxisd_enabled: false`), causes Synapse
and riot-web to reconfigure themselves with other (public)
Identity servers.
- enabling matrix-corporal (`matrix_corporal_enabled: true`) affects
how reverse-proxying (by `matrix-nginx-proxy`) is done, in order to
put matrix-corporal's gateway server in front of Synapse
We may be able to move away from such dependencies in the future,
at the expense of a more complicated manual configuration, but
it's probably not worth sacrificing the convenience we have now.
As part of this work, the way we do "start components" has been
redone now to use a loop, as suggested in #65 (Github issue).
This should make restarting faster and more reliable.
This change is provoked by a few different things:
- #54 (Github Pull Request), which rightfully says that we need a
way to support ALL mxisd configuration options easily
- the upcoming mxisd 1.3.0 release, which drops support for
property-style configuration (dot-notation), forcing us to
redo the way we generate the configuration file
With this, mxisd is much more easily configurable now
and much more easily maintaneable by us in the future
(no need to introduce additional playbook variables and logic).
After having multiple people report issues with retrieving
SSL certificates, we've finally discovered the culprit to be
Ansible 2.5.1 (default and latest version on Ubuntu 18.04 LTS).
As silly as it is, certain distributions ("LTS" even) are 13 bugfix
versions of Ansible behind.
From now on, we try to auto-detect buggy Ansible versions and tell the
user. We also provide some tips for how to upgrade Ansible or
run it from inside a Docker container.
My testing shows that Ansible 2.4.0 and 2.4.6 are OK.
All other intermediate 2.4.x versions haven't been tested, but we
trust they're OK too.
From the 2.5.x releases, only 2.5.0 and 2.5.1 seem to be affected.
Ansible 2.5.2 corrects the problem with `include_tasks` + `with_items`.
This is a simplification and a way to make it consistent with
how we do Postgres imports (see 6d89319822), using
files coming from the server, not from the local machine.
By encouraging people NOT to use local files,
we potentially avoid problems such as #34 (Github issue),
where people would download `media_store` to their Mac's filesystem
and case-sensitivity issues will actually corrupt it.
By not encouraging local files usage, it's less likely that
people would copy (huge) directories to their local machine like that.
This is a simplification and a way to make it consistent with
how we do Postgres imports (see 6d89319822), using
files coming from the server, not from the local machine.
Adds support for managing certificates manually and for
having the playbook generate self-signed certificates for you.
With this, Let's Encrypt usage is no longer required.
Fixes Github issue #50.
Relay hostnames that have MX records are looked up by postfix
and the MX record's payload is used instead.
This special behavior may be undesirable, so we make sure to
point it out.
This is provoked by Github issue #46.
No client had made use of the well-known mechanism
so far, so the set up performed by this playbook was not tested
and turned out to be a little deficient.
Even though /.well-known/matrix/client is usually requested with a
simple request (no preflight), it's still considered cross-origin
and [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS)
applies. Thus, the file always needs to be served with the appropriate
`Access-Control-Allow-Origin` header.
Github issue #46 attempts to fix it at the "reverse-proxying" layer,
which may work, but would need to be done for every server.
It's better if it's done "upstream", so that all reverse-proxy
configurations can benefit.
We've had some people get confused into installing
Matrix Corporal and having pain with that.
With this documentation change, we try to make it clearer
that it's an advanced feature not to be touched unless
you know what you're doing.
On a similar note, we also make sure other things are properly
labeled as "(optional)" and/or "(advanced)".
Slavi Pantaleev
jens quade
Dominik Zajac
Aaron Raimist
Aki Salminen
merklaw
vaivars
Matthias
rebru
Rónán Duddy
bertiebaggio
Julian Strobl
Panagiotis Vasilopoulos
Justin Croonenberghs
Hugues Morisset
Tulir Asokan
jdreichmann
Pablo
Julian Foad
benkuly
kristbaum
Chris van Dijk
chagai95
Marcel Partap
Kevin Lanni
Christian Wolf
André Sterba
Mickaël Cornière
Rodrigo Belem
Horvath Gergely
Udo Rader
skoal
teutat3s
Sean O'Neil
PlanetSmasher
ayounggun
iLyas Bakouch
Sathis
Lee Verberne
prasket
Björn Marten
Hanno J. Gödecke
DreamsVoid
Logan Pulley
benkuly
fnoah
Noah Fleischmann
Paul Tötterman
Arehandoro
sudneo
Eduardo Beltrame
kingoftheconnors
p5t2vspoqqw
Esko
Michael Haak
Lyubomir Popov
Borjan Tchakaloff
Edgars Voroboks
Lionstiger
frlae
Plailect
Thomas vO
TheForcer
anadahz
Devr