From f6ebd4ce6214d7a11bbd2d692246526140854869 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Tue, 5 Feb 2019 11:07:08 +0200 Subject: [PATCH 01/16] Initial work on Synapse 0.99/1.0 preparation --- docs/configuring-dns.md | 48 +++++++--- docs/configuring-playbook.md | 2 +- docs/configuring-well-known.md | 59 ++++++++---- docs/installing.md | 6 +- docs/prerequisites.md | 8 +- group_vars/matrix-servers | 21 +++- roles/matrix-base/tasks/setup_well_known.yml | 9 +- .../static-files/well-known/matrix-server.j2 | 3 + .../matrix-corporal/tasks/setup_corporal.yml | 1 + roles/matrix-nginx-proxy/defaults/main.yml | 8 +- .../tasks/self_check_well_known.yml | 74 +++------------ .../tasks/self_check_well_known_file.yml | 65 +++++++++++++ .../nginx/conf.d/matrix-synapse.conf.j2 | 46 ++++++++- .../systemd/matrix-nginx-proxy.service.j2 | 3 + roles/matrix-synapse/defaults/main.yml | 20 +++- .../tasks/setup_synapse_main.yml | 55 +++++++++++ .../matrix-synapse/tasks/validate_config.yml | 12 ++- .../templates/synapse/homeserver.yaml.j2 | 95 ++++++++++++++++--- .../synapse/systemd/matrix-synapse.service.j2 | 9 +- 19 files changed, 415 insertions(+), 129 deletions(-) create mode 100644 roles/matrix-base/templates/static-files/well-known/matrix-server.j2 create mode 100644 roles/matrix-nginx-proxy/tasks/self_check_well_known_file.yml diff --git a/docs/configuring-dns.md b/docs/configuring-dns.md index 49b6a057..16bc4436 100644 --- a/docs/configuring-dns.md +++ b/docs/configuring-dns.md @@ -5,26 +5,50 @@ To set up Matrix on your domain, you'd need to do some DNS configuration. To use an identifier like `@:`, you don't actually need to install anything on the actual `` server. + +## General outline of DNS settings you need to do + +| Type | Host | Priority | Weight | Port | Target | +| ----- | ----------------------- | -------- | ------ | ---- | ---------------------- | +| A | `matrix` | - | - | - | `matrix-server-IP` | +| CNAME | `riot` | - | - | - | `matrix.` | +| SRV | `_matrix._tcp` | 10 | 0 | 8448 | `matrix.` | +| SRV | `_matrix-identity._tcp` | 10 | 0 | 443 | `matrix.` | + +The `_matrix._tcp` SRV record is a temporary measure and will not be necessary in the near future. +In fact, it will have to be removed at some point. To learn more about that, read below. + + +## Subdomains setup + +As the table above illustrates, you need to create 2 subdomains (`matrix.` and `riot.`) and point both of them to your new server's IP address (DNS `A` record or `CNAME` record is fine). + +The `riot.` subdomain is necessary, because this playbook installs the Riot web client for you. +If you'd rather instruct the playbook not to install Riot (`matrix_riot_web_enabled: false` when [Configuring the playbook](configuring-playbook.md) later), feel free to skip the `riot.` DNS record. + + +## `_matrix._tcp` SRV record setup (temporary requirement) + All services created by this playbook are meant to be installed on their own server (such as `matrix.`). -To accomplish such a "redirect", you need to instruct the Matrix network of this by setting up a DNS SRV record. +To use a Matrix user identifier like `@:` while hosting services on `matrix.`, we need to instruct the Matrix network of such a delegation/redirection by means of setting up a DNS SRV record. + The SRV record should look like this: - Name: `_matrix._tcp` (use this text as-is) - Content: `10 0 8448 matrix.` (replace `` with your own) +A [new file-based mechanism for Federation Server Discovery](configuring-well-known.md#introduction-to-federation-server-discovery) is superseding the `_matrix._tcp` SRV record. **During the transition phase, you'll need to set up both mechanisms**. We'll instruct you how to set up the file-based mechanism after the [installation phase](installing.md) for this playbook. + +Doing delegation/redirection of Matrix services using a DNS SRV record (`_matrix._tcp`) is a **temporary measure** that is only necessary before Synapse v1.0 is released. + +As more and more people upgrade to the Synapse v0.99 transitional release and just before the final Synapse v1.0 gets released, at some point in the near future **you will need to remove the `_matrix._tcp` SRV record** and leave only the [new file-based mechanism for Federation Server Discovery](configuring-well-known.md#introduction-to-federation-server-discovery) in place. + + +## `_matrix-identity._tcp` SRV record setup + To make the [mxisd](https://github.com/kamax-io/mxisd) Identity Server (which this playbook installs for you) be authoritative for your domain name, set up one more SRV record that looks like this: - Name: `_matrix-identity._tcp` (use this text as-is) - Content: `10 0 443 matrix.` (replace `` with your own) -Once you've set up these DNS SRV records, you should create 2 other domain names (`matrix.` and `riot.`) and point both of them to your new server's IP address (DNS `A` record or `CNAME` is fine). - -This playbook can then install all the services on that new server and you'll be able to join the Matrix network as `@:`. - -| Type | Host | Priority | Weight | Port | Target | -| ----- | ----------------------- | -------- | ------ | ---- | ---------------------- | -| SRV | `_matrix._tcp` | 10 | 0 | 8448 | `matrix.` | -| SRV | `_matrix-identity._tcp` | 10 | 0 | 443 | `matrix.` | -| A | `matrix` | - | - | - | `server-IP` | -| CNAME | `riot` | - | - | - | `matrix.` | -When ready to proceed, continue with [Configuring this Ansible playbook](configuring-playbook.md). \ No newline at end of file +When you're done with the DNS configuration and ready to proceed, continue with [Configuring this Ansible playbook](configuring-playbook.md). diff --git a/docs/configuring-playbook.md b/docs/configuring-playbook.md index 30d1b05f..9a61c153 100644 --- a/docs/configuring-playbook.md +++ b/docs/configuring-playbook.md @@ -1,6 +1,6 @@ # Configuring the Ansible playbook -Once you have your server and you have [configured your DNS records](configuring-dns.md#configuring-dns), you can proceed with configuring this playbook, so that it knows what to install and where. +Once you have your server and you have [configured your DNS records](configuring-dns.md), you can proceed with configuring this playbook, so that it knows what to install and where. You can follow these steps: diff --git a/docs/configuring-well-known.md b/docs/configuring-well-known.md index 4185a9d6..166ece1f 100644 --- a/docs/configuring-well-known.md +++ b/docs/configuring-well-known.md @@ -1,44 +1,66 @@ -# Configuring service discovery via .well-known +# Configuring Service Discovery via .well-known +Service discovery is a way for the Matrix network to discover where a Matrix server is. -## Introduction +There are 2 types of well-known service discovery that Matrix makes use of: -Service discovery lets various client programs which support it, to receive a full user id (e.g. `@username:example.com`) and determine where the Matrix server is automatically (e.g. `https://matrix.example.com`). +- (important) **Federation Server discovery** (`/.well-known/matrix/server`) -- assists other servers in the Matrix network with finding your server. Without a proper configuration, your server will effectively not be part of the Matrix network. Learn more in [Introduction to Federation Server Discovery](#introduction-to-federation-server-discovery) -This lets your users easily connect to your Matrix server without having to customize connection URLs. +- (not that important) **Client Server discovery** (`/.well-known/matrix/client`) -- assists programs that you use to connect to your server (e.g. Riot), so that they can make it more convenient for you by automatically configuring the "Homeserver URL" and "Identity Server URL" addresses. Learn more in [Introduction to Client Server Discovery](#introduction-to-client-server-discovery) -As [per the specification](https://matrix.org/docs/spec/client_server/r0.4.0.html#server-discovery) Matrix does service discovery using a `/.well-known/matrix/client` file hosted on the base domain (e.g. `example.com`). -However, this playbook installs your Matrix server on another domain (e.g. `matrix.example.com`) and not on the base domain (e.g. `example.com`), so it takes a little extra manual effort to set up the file. +## Introduction to Federation Server Discovery + +All services created by this playbook are meant to be installed on their own server (such as `matrix.`). + +As [per the Server-Server specification](https://matrix.org/docs/spec/server_server/r0.1.0.html#server-discovery), to use a Matrix user identifier like `@:` while hosting services on a subdomain like `matrix.`, we need to instruct the Matrix network of such a delegation/redirection by means of setting up a `/.well-known/matrix/server` file on the base domain (`). + +We have discussed this same thing already in the "`_matrix._tcp` SRV record setup (temporary requirement)" section of [Configuring DNS](configuring-dns.md). + +In short, you are required to set up both a `_matrix._tcp` DNS SRV record and the `/.well-known/matrix/server` file at the moment. + +As the Synapse server progresses towards v1.0, only the `/.well-known/matrix/server` file will be used. At that future moment, you would need to remove the `_matrix._tcp` SRV record because Synapse v1.0+ will do the wrong thing if a SRV record exists. During the transitional phase (before Synapse 1.0), we do need to have both a SRV record and a `/.well-known/matrix/server` file, in order to federate correctly with v0.99 and older Synapse versions. + +To learn how to set it up, read the Installing section below. -## Prerequisites +## Introduction to Client Server Discovery -To implement service discovery, your base domain's server (e.g. `example.com`) needs to support HTTPS. +Client Server Service discovery lets various client programs which support it, to receive a full user id (e.g. `@username:example.com`) and determine where the Matrix server is automatically (e.g. `https://matrix.example.com`). +This lets you (and your users) easily connect to your Matrix server without having to customize connection URLs. When using client programs that support it, you won't need to point them to `https://matrix.example.com` in Custom Server options manually anymore. The connection URL would be discovered automatically from your full username. -## Setting it up +As [per the Client-Server specification](https://matrix.org/docs/spec/client_server/r0.4.0.html#server-discovery) Matrix does Client Server service discovery using a `/.well-known/matrix/client` file hosted on the base domain (e.g. `example.com`). -To make things easy for you to set up, this playbook generates and hosts the well-known file on the Matrix domain's server (e.g. `https://matrix.example.com/.well-known/matrix/client`), even though this is the wrong place to host it. +However, this playbook installs your Matrix server on another domain (e.g. `matrix.example.com`) and not on the base domain (e.g. `example.com`), so it takes a little extra manual effort to set up the file. + +To learn how to set it up, read the Installing section below. + + +## Installing well-known files on the base domain's server -You have 2 options when it comes to installing the file on the base domain's server: +To implement the two service discovery mechanisms, your base domain's server (e.g. `example.com`) needs to support HTTPS. +To make things easy for you to set up, this playbook generates and hosts 2 well-known files on the Matrix domain's server (e.g. `https://matrix.example.com/.well-known/matrix/server` and `https://matrix.example.com/.well-known/matrix/client`), even though this is the wrong place to host them. -### (Option 1): **Copying the file manually** to your base domain's server +You have 2 options when it comes to installing the files on the base domain's server: + + +### (Option 1): **Copying the files manually** to your base domain's server **Hint**: Option 2 (below) is generally a better way to do this. Make sure to go with that one, if possible. All you need to do is: -- copy the `/.well-known/matrix/client` from the Matrix server (e.g. `matrix.example.com`) to your base domain's server (`example.com`). +- copy `/.well-known/matrix/server` and `/.well-known/matrix/client` from the Matrix server (e.g. `matrix.example.com`) to your base domain's server (`example.com`). -- set up the server at your base domain (e.g. `example.com`) so that it adds an extra HTTP header when serving the `/.well-known/matrix/client` file. [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS), the `Access-Control-Allow-Origin` header should be set with a value of `*`. If you don't do this step, web-based Matrix clients (like Riot) may fail to work. +- set up the server at your base domain (e.g. `example.com`) so that it adds an extra HTTP header when serving the `/.well-known/matrix/client` file. [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS), the `Access-Control-Allow-Origin` header should be set with a value of `*`. If you don't do this step, web-based Matrix clients (like Riot) may fail to work. Setting up headers for the `/.well-known/matrix/server` file is not necessary, as this file is only consumed by non-browsers, which don't care about CORS. This is relatively easy to do and possibly your only choice if you can only host static files from the base domain's server. -It is, however, **a little fragile**, as future updates performed by this playbook may regenerate the well-known file and you may need to notice that and copy it again. +It is, however, **a little fragile**, as future updates performed by this playbook may regenerate the well-known files and you may need to notice that and copy them over again. -### (Option 2): **Setting up reverse-proxying** of the well-known file from the base domain's server to the Matrix server +### (Option 2): **Setting up reverse-proxying** of the well-known files from the base domain's server to the Matrix server This option is less fragile and generally better. @@ -91,6 +113,9 @@ Make sure to: ## Confirming it works -No matter which method you've used to set up the well-known file, if you've done it correctly you should be able to see a JSON file at a URL like this: `https:///.well-known/matrix/client`. +No matter which method you've used to set up the well-known files, if you've done it correctly you should be able to see a JSON file at both of these URLs: + +- `https:///.well-known/matrix/server` +- `https:///.well-known/matrix/client` You can also check if everything is configured correctly, by [checking if services work](maintenance-checking-services.md). diff --git a/docs/installing.md b/docs/installing.md index 8ca36859..38536f5a 100644 --- a/docs/installing.md +++ b/docs/installing.md @@ -34,7 +34,7 @@ ansible-playbook -i inventory/hosts setup.yml --tags=start Now that the services are running, you might want to: -- [create your first user account](registering-users.md) -- or **finalize the installation process** by [Configuring service discovery via .well-known](configuring-well-known.md) -- or [Check if services work](maintenance-checking-services.md) +- **finalize the installation process** (required for federation to work!) by [Configuring Service Discovery via .well-known](configuring-well-known.md) +- or [create your first user account](registering-users.md) +- or [check if services work](maintenance-checking-services.md) - or learn how to [upgrade your services when new versions are released](maintenance-upgrading-services.md) diff --git a/docs/prerequisites.md b/docs/prerequisites.md index 13cf89a5..dc6b91f8 100644 --- a/docs/prerequisites.md +++ b/docs/prerequisites.md @@ -6,12 +6,10 @@ - the [Ansible](http://ansible.com/) program being installed on your own computer. It's used to run this playbook and configures your server for you. Take a look at [our guide about Ansible](ansible.md) for version requirements or alternative ways to run Ansible. -- properly configured DNS SRV record for `` (details in [Configuring DNS](configuring-dns.md#configuring-dns) below) +- an HTTPS-capable web server at the base domain name (``) which is capable of serving static files -- `matrix.` domain name pointing to your new server - this is where the Matrix Synapse server will live (details in [Configuring DNS](configuring-dns.md#configuring-dns) below) - -- `riot.` domain name pointing to your new server - this is where the Riot web UI will live (details in [Configuring DNS](configuring-dns.md#configuring-dns) below) +- properly configured DNS records for `` (details in [Configuring DNS](configuring-dns.md)) - some TCP/UDP ports open. This playbook configures the server's internal firewall for you. In most cases, you don't need to do anything special. But **if your server is running behind another firewall**, you'd need to open these ports: `80/tcp` (HTTP webserver), `443/tcp` (HTTPS webserver), `3478/tcp` (STUN over TCP), `3478/udp` (STUN over UDP), `8448/tcp` (Matrix federation HTTPS webserver), `49152-49172/udp` (TURN over UDP). -When ready to proceed, continue with [Configuring DNS](configuring-dns.md). \ No newline at end of file +When ready to proceed, continue with [Configuring DNS](configuring-dns.md). diff --git a/group_vars/matrix-servers b/group_vars/matrix-servers index 7b14b4e1..b79f5038 100644 --- a/group_vars/matrix-servers +++ b/group_vars/matrix-servers @@ -161,7 +161,7 @@ matrix_mxisd_systemd_wanted_services_list: | # ###################################################################### -# By default, this playbook sets up a reverse-proxy nginx proxy server on port 80/443. +# By default, this playbook sets up a reverse-proxy nginx proxy server on TCP ports 80, 443 and 8448. # This is fine if you're dedicating the whole server to Matrix. # If that's not the case, you may wish to disable this and take care of proxying yourself. matrix_nginx_proxy_enabled: true @@ -181,6 +181,11 @@ matrix_nginx_proxy_proxy_matrix_identity_api_enabled: "{{ matrix_mxisd_enabled } matrix_nginx_proxy_proxy_matrix_identity_api_addr_with_container: "matrix-mxisd:8090" matrix_nginx_proxy_proxy_matrix_identity_api_addr_sans_container: "127.0.0.1:8090" +# By default, we do TLS termination for the Matrix Federation API (port 8448) at matrix-nginx-proxy. +matrix_nginx_proxy_proxy_matrix_federation_api_enabled: true +matrix_nginx_proxy_proxy_matrix_federation_api_addr_with_container: "matrix-synapse:8048" +matrix_nginx_proxy_proxy_matrix_federation_api_addr_sans_container: "127.0.0.1:8048" + matrix_nginx_proxy_proxy_synapse_metrics: "{{ matrix_synapse_metrics_enabled }}" matrix_nginx_proxy_proxy_synapse_metrics_addr_with_container: "matrix-synapse:{{ matrix_synapse_metrics_port }}" matrix_nginx_proxy_proxy_synapse_metrics_addr_sans_container: "127.0.0.1:{{ matrix_synapse_metrics_port }}" @@ -272,9 +277,13 @@ matrix_riot_web_default_is_url: "{{ matrix_identity_server_url }}" matrix_synapse_trusted_third_party_id_servers: "{{ [hostname_matrix] if matrix_mxisd_enabled else matrix_synapse_id_servers_public }}" # Normally, matrix-nginx-proxy is enabled and nginx can reach Synapse over the container network. -# If matrix-nginx-proxy is not enabled, or you otherwise have a need for it, you can expose -# the Client/Server API's port to the local host (`127.0.0.1:8008`). -matrix_synapse_container_expose_client_server_api_port: "{{ not matrix_nginx_proxy_enabled }}" +# If matrix-nginx-proxy is not enabled, or you otherwise have a need for it, you can expose its ports +# to the local host. +# +# For exposing the Matrix Client API's port (plain HTTP) to the local host (`127.0.0.1:8008`). +matrix_synapse_container_expose_client_api_port: "{{ not matrix_nginx_proxy_enabled }}" +# For exposing the Matrix Federation API's port (plain HTTP) to the local host (`127.0.0.1:8048`). +matrix_synapse_container_expose_federation_api_port: "{{ not matrix_nginx_proxy_enabled }}" matrix_synapse_container_expose_metrics_port: "{{ not matrix_nginx_proxy_enabled }}" @@ -283,6 +292,10 @@ matrix_synapse_database_user: "{{ matrix_postgres_connection_username }}" matrix_synapse_database_password: "{{ matrix_postgres_connection_password }}" matrix_synapse_database_database: "{{ matrix_postgres_db_name }}" +# We do not enable TLS in Synapse by default. +# TLS is handled by the matrix-nginx-proxy, which proxies the requests to Synapse. +matrix_synapse_no_tls: true + matrix_synapse_email_enabled: "{{ matrix_mailer_enabled }}" matrix_synapse_email_smtp_host: "matrix-mailer" matrix_synapse_email_smtp_port: 8025 diff --git a/roles/matrix-base/tasks/setup_well_known.yml b/roles/matrix-base/tasks/setup_well_known.yml index 8a24bd2f..70b91f77 100644 --- a/roles/matrix-base/tasks/setup_well_known.yml +++ b/roles/matrix-base/tasks/setup_well_known.yml @@ -12,10 +12,13 @@ with_items: - "{{ matrix_static_files_base_path }}/.well-known/matrix" -- name: Ensure Matrix /.well-known/matrix/client configured +- name: Ensure Matrix /.well-known/matrix files configured template: - src: "{{ role_path }}/templates/static-files/well-known/matrix-client.j2" - dest: "{{ matrix_static_files_base_path }}/.well-known/matrix/client" + src: "{{ role_path }}/templates/static-files/well-known/matrix-{{ item }}.j2" + dest: "{{ matrix_static_files_base_path }}/.well-known/matrix/{{ item }}" mode: 0644 owner: "{{ matrix_user_username }}" group: "{{ matrix_user_username }}" + with_items: + - "client" + - "server" diff --git a/roles/matrix-base/templates/static-files/well-known/matrix-server.j2 b/roles/matrix-base/templates/static-files/well-known/matrix-server.j2 new file mode 100644 index 00000000..53ed8787 --- /dev/null +++ b/roles/matrix-base/templates/static-files/well-known/matrix-server.j2 @@ -0,0 +1,3 @@ +{ + "m.server": "{{ hostname_matrix }}:8448" +} diff --git a/roles/matrix-corporal/tasks/setup_corporal.yml b/roles/matrix-corporal/tasks/setup_corporal.yml index aba53c3f..f1fd2d9b 100644 --- a/roles/matrix-corporal/tasks/setup_corporal.yml +++ b/roles/matrix-corporal/tasks/setup_corporal.yml @@ -36,6 +36,7 @@ mode: 0644 when: "matrix_corporal_enabled" + # # Tasks related to getting rid of matrix-corporal (if it was previously enabled) # diff --git a/roles/matrix-nginx-proxy/defaults/main.yml b/roles/matrix-nginx-proxy/defaults/main.yml index af3489ca..f9b3d745 100644 --- a/roles/matrix-nginx-proxy/defaults/main.yml +++ b/roles/matrix-nginx-proxy/defaults/main.yml @@ -57,8 +57,14 @@ matrix_nginx_proxy_proxy_matrix_client_api_addr_sans_container: "127.0.0.1:8008" # This needs to be equal or higher than the maximum upload size accepted by Synapse. matrix_nginx_proxy_proxy_matrix_client_api_client_max_body_size_mb: 25 +# Controls whether proxying for the Matrix Federation API should be done. +matrix_nginx_proxy_proxy_matrix_federation_api_enabled: false +matrix_nginx_proxy_proxy_matrix_federation_api_addr_with_container: "matrix-synapse:8048" +matrix_nginx_proxy_proxy_matrix_federation_api_addr_sans_container: "localhost:8048" +matrix_nginx_proxy_proxy_matrix_federation_api_client_max_body_size_mb: "{{ matrix_nginx_proxy_proxy_matrix_client_api_client_max_body_size_mb * 3 }}" + # The tmpfs at /tmp needs to be large enough to handle multiple concurrent file uploads. -matrix_nginx_proxy_tmp_directory_size_mb: "{{ matrix_nginx_proxy_proxy_matrix_client_api_client_max_body_size_mb * 50 }}" +matrix_nginx_proxy_tmp_directory_size_mb: "{{ matrix_nginx_proxy_proxy_matrix_federation_api_client_max_body_size_mb * 50 }}" # A list of strings containing additional configuration blocks to add to the matrix domain's server configuration. matrix_nginx_proxy_proxy_matrix_additional_server_configuration_blocks: [] diff --git a/roles/matrix-nginx-proxy/tasks/self_check_well_known.yml b/roles/matrix-nginx-proxy/tasks/self_check_well_known.yml index 7e38f8b2..93ac88da 100644 --- a/roles/matrix-nginx-proxy/tasks/self_check_well_known.yml +++ b/roles/matrix-nginx-proxy/tasks/self_check_well_known.yml @@ -1,65 +1,13 @@ --- -- set_fact: - well_known_url_matrix: "https://{{ hostname_matrix }}/.well-known/matrix/client" - well_known_url_identity: "https://{{ hostname_identity }}/.well-known/matrix/client" - -# These well-known files may be served without a `Content-Type: application/json` header, -# so we can't rely on the uri module's automatic parsing of JSON. -- name: Check .well-known on the matrix hostname - uri: - url: "{{ well_known_url_matrix }}" - follow_redirects: false - return_content: true - register: result_well_known_matrix - ignore_errors: true - -- name: Fail if .well-known not working on the matrix hostname - fail: - msg: "Failed checking that well-known is configured at `{{ hostname_matrix }}` (checked endpoint: `{{ well_known_url_matrix }}`). Is port 443 open in your firewall? Full error: {{ result_well_known_matrix }}" - when: "result_well_known_matrix.failed" - -- name: Parse JSON for well-known payload at the matrix hostname - set_fact: - well_known_matrix_payload: "{{ result_well_known_matrix.content|from_json }}" - -- name: Fail if .well-known not CORS-aware on the matrix hostname - fail: - msg: "Well-known serving for `{{ hostname_matrix }}` (checked endpoint: `{{ well_known_url_matrix }}`) is not CORS-aware. The file needs to be served with an Access-Control-Allow-Origin header set." - when: "'access_control_allow_origin' not in result_well_known_matrix" - -- name: Report working .well-known on the matrix hostname - debug: - msg: "well-known is configured correctly for `{{ hostname_matrix }}` (checked endpoint: `{{ well_known_url_matrix }}`)" - -- name: Check .well-known on the identity hostname - uri: - url: "{{ well_known_url_identity }}" - follow_redirects: false - return_content: true - register: result_well_known_identity - ignore_errors: true - -- name: Fail if .well-known not working on the identity hostname - fail: - msg: "Failed checking that well-known is configured at `{{ hostname_identity }}` (checked endpoint: `{{ well_known_url_identity }}`). Is port 443 open in your firewall? Full error: {{ result_well_known_identity }}" - when: "result_well_known_identity.failed" - -- name: Parse JSON for well-known payload at the identity hostname - set_fact: - well_known_identity_payload: "{{ result_well_known_identity.content|from_json }}" - -- name: Fail if .well-known not CORS-aware on the identity hostname - fail: - msg: "Well-known serving for `{{ hostname_identity }}` (checked endpoint: `{{ well_known_url_identity }}`) is not CORS-aware. The file needs to be served with an Access-Control-Allow-Origin header set. See docs/configuring-well-known.md" - when: "'access_control_allow_origin' not in result_well_known_identity" - -# For people who manually copy the well-known file, try to detect if it's outdated -- name: Fail if well-known is different on matrix hostname and identity hostname - fail: - msg: "The well-known files at `{{ hostname_matrix }}` and `{{ hostname_identity }}` are different. Perhaps you copied the file manually before and now it's outdated?" - when: "well_known_matrix_payload != well_known_identity_payload" - -- name: Report working .well-known on the identity hostname - debug: - msg: "well-known is configured correctly for `{{ hostname_identity }}` (checked endpoint: `{{ well_known_url_identity }}`)" +- name: Perform well-known checks + include_tasks: "{{ role_path }}/tasks/self_check_well_known_file.yml" + with_items: + - path: /.well-known/matrix/server + purpose: Server Discovery + cors: false + - path: /.well-known/matrix/client + purpose: Client Discovery + cors: true + loop_control: + loop_var: well_known_file_check diff --git a/roles/matrix-nginx-proxy/tasks/self_check_well_known_file.yml b/roles/matrix-nginx-proxy/tasks/self_check_well_known_file.yml new file mode 100644 index 00000000..40161807 --- /dev/null +++ b/roles/matrix-nginx-proxy/tasks/self_check_well_known_file.yml @@ -0,0 +1,65 @@ +--- + +- set_fact: + well_known_url_matrix: "https://{{ hostname_matrix }}{{ well_known_file_check.path }}" + well_known_url_identity: "https://{{ hostname_identity }}{{ well_known_file_check.path }}" + +# These well-known files may be served without a `Content-Type: application/json` header, +# so we can't rely on the uri module's automatic parsing of JSON. +- name: Check .well-known on the matrix hostname + uri: + url: "{{ well_known_url_matrix }}" + follow_redirects: false + return_content: true + register: result_well_known_matrix + ignore_errors: true + +- name: Fail if .well-known not working on the matrix hostname + fail: + msg: "Failed checking that the well-known file for {{ well_known_file_check.purpose }} is configured at `{{ hostname_matrix }}` (checked endpoint: `{{ well_known_url_matrix }}`). Is port 443 open in your firewall? Full error: {{ result_well_known_matrix }}" + when: "result_well_known_matrix.failed" + +- name: Parse JSON for well-known payload at the matrix hostname + set_fact: + well_known_matrix_payload: "{{ result_well_known_matrix.content|from_json }}" + +- name: Fail if .well-known not CORS-aware on the matrix hostname + fail: + msg: "The well-known file for {{ well_known_file_check.purpose }} on `{{ hostname_matrix }}` (checked endpoint: `{{ well_known_url_matrix }}`) is not CORS-aware. The file needs to be served with an Access-Control-Allow-Origin header set." + when: "well_known_file_check.cors and 'access_control_allow_origin' not in result_well_known_matrix" + +- name: Report working .well-known on the matrix hostname + debug: + msg: "well-known for {{ well_known_file_check.purpose }} is configured correctly for `{{ hostname_matrix }}` (checked endpoint: `{{ well_known_url_matrix }}`)" + +- name: Check .well-known on the identity hostname + uri: + url: "{{ well_known_url_identity }}" + follow_redirects: false + return_content: true + register: result_well_known_identity + ignore_errors: true + +- name: Fail if .well-known not working on the identity hostname + fail: + msg: "Failed checking that the well-known file for {{ well_known_file_check.purpose }} is configured at `{{ hostname_identity }}` (checked endpoint: `{{ well_known_url_identity }}`). Is port 443 open in your firewall? Full error: {{ result_well_known_identity }}" + when: "result_well_known_identity.failed" + +- name: Parse JSON for well-known payload at the identity hostname + set_fact: + well_known_identity_payload: "{{ result_well_known_identity.content|from_json }}" + +- name: Fail if .well-known not CORS-aware on the identity hostname + fail: + msg: "The well-known file for {{ well_known_file_check.purpose }} on `{{ hostname_identity }}` (checked endpoint: `{{ well_known_url_identity }}`) is not CORS-aware. The file needs to be served with an Access-Control-Allow-Origin header set. See docs/configuring-well-known.md" + when: "well_known_file_check.cors and 'access_control_allow_origin' not in result_well_known_identity" + +# For people who manually copy the well-known file, try to detect if it's outdated +- name: Fail if well-known is different on matrix hostname and identity hostname + fail: + msg: "The well-known files for {{ well_known_file_check.purpose }} at `{{ hostname_matrix }}` and `{{ hostname_identity }}` are different. Perhaps you copied the file ({{ well_known_file_check.path }}) manually before and now it's outdated?" + when: "well_known_matrix_payload != well_known_identity_payload" + +- name: Report working .well-known on the identity hostname + debug: + msg: "well-known for {{ well_known_file_check.purpose }} ({{ well_known_file_check.path }}) is configured correctly for `{{ hostname_identity }}` (checked endpoint: `{{ well_known_url_identity }}`)" diff --git a/roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-synapse.conf.j2 b/roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-synapse.conf.j2 index a357e862..4786911a 100644 --- a/roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-synapse.conf.j2 +++ b/roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-synapse.conf.j2 @@ -39,7 +39,7 @@ server { ssl_prefer_server_ciphers on; ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"; - location /.well-known/matrix/client { + location /.well-known/matrix { root {{ matrix_static_files_base_path }}; expires 1m; default_type application/json; @@ -101,6 +101,10 @@ server { {{- configuration_block }} {% endfor %} + {# + This handles the Matrix Client API only. + The Matrix Federation API is handled by a separate vhost. + #} location /_matrix { {% if matrix_nginx_proxy_enabled %} {# Use the embedded DNS resolver in Docker containers to discover the service #} @@ -146,3 +150,43 @@ server { rewrite ^/$ /_matrix/static/ last; } } + +{% if matrix_nginx_proxy_proxy_matrix_federation_api_enabled %} +server { + listen 8448 ssl http2; + listen [::]:8448 ssl http2; + + server_name {{ matrix_nginx_proxy_proxy_matrix_hostname }}; + + server_tokens off; + root /dev/null; + + gzip on; + gzip_types text/plain application/json; + + ssl_certificate {{ matrix_ssl_config_dir_path }}/live/{{ matrix_nginx_proxy_proxy_matrix_hostname }}/fullchain.pem; + ssl_certificate_key {{ matrix_ssl_config_dir_path }}/live/{{ matrix_nginx_proxy_proxy_matrix_hostname }}/privkey.pem; + ssl_protocols {{ matrix_nginx_proxy_ssl_protocols }}; + ssl_prefer_server_ciphers on; + ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"; + + location / { + {% if matrix_nginx_proxy_enabled %} + {# Use the embedded DNS resolver in Docker containers to discover the service #} + resolver 127.0.0.11 valid=5s; + set $backend "{{ matrix_nginx_proxy_proxy_matrix_federation_api_addr_with_container }}"; + proxy_pass http://$backend; + {% else %} + {# Generic configuration for use outside of our container setup #} + proxy_pass http://{{ matrix_nginx_proxy_proxy_matrix_federation_api_addr_sans_container }}; + {% endif %} + + proxy_set_header Host $host; + proxy_set_header X-Forwarded-For $remote_addr; + + client_body_buffer_size 25M; + client_max_body_size {{ matrix_nginx_proxy_proxy_matrix_federation_api_client_max_body_size_mb }}M; + proxy_max_temp_file_size 0; + } +} +{% endif %} diff --git a/roles/matrix-nginx-proxy/templates/systemd/matrix-nginx-proxy.service.j2 b/roles/matrix-nginx-proxy/templates/systemd/matrix-nginx-proxy.service.j2 index e04b9ae8..48a6441d 100644 --- a/roles/matrix-nginx-proxy/templates/systemd/matrix-nginx-proxy.service.j2 +++ b/roles/matrix-nginx-proxy/templates/systemd/matrix-nginx-proxy.service.j2 @@ -22,6 +22,9 @@ ExecStart=/usr/bin/docker run --rm --name matrix-nginx-proxy \ --network={{ matrix_docker_network }} \ -p 80:8080 \ -p 443:8443 \ + {% if matrix_nginx_proxy_proxy_matrix_federation_api_enabled %} + -p 8448:8448 \ + {% endif %} -v {{ matrix_nginx_proxy_data_path }}/nginx.conf:/etc/nginx/nginx.conf:ro \ -v {{ matrix_nginx_proxy_confd_path }}:/etc/nginx/conf.d:ro \ -v {{ matrix_ssl_config_dir_path }}:{{ matrix_ssl_config_dir_path }}:ro \ diff --git a/roles/matrix-synapse/defaults/main.yml b/roles/matrix-synapse/defaults/main.yml index 13273f78..d72fc2ec 100644 --- a/roles/matrix-synapse/defaults/main.yml +++ b/roles/matrix-synapse/defaults/main.yml @@ -8,7 +8,12 @@ matrix_synapse_media_store_path: "{{ matrix_synapse_storage_path }}/media-store" matrix_synapse_ext_path: "{{ matrix_synapse_base_path }}/ext" # Controls whether the Synapse container exposes the Client/Server API port (tcp/8008). -matrix_synapse_container_expose_client_server_api_port: false +matrix_synapse_container_expose_client_api_port: false + +# Controls whether the Synapse container exposes the Server/Server (Federation) API port (tcp/8048). +# This is for the plain HTTP API. If you need Synapse to handle TLS encryption, +# that would be on another port (tcp/8448) controlled by `matrix_synapse_tls_federation_listener_enabled`. +matrix_synapse_container_expose_federation_api_port: false # Controls whether the Appservice IRC container exposes the Client/Server API port (tcp/9999). matrix_appservice_irc_container_expose_client_server_api_port: false @@ -60,6 +65,17 @@ matrix_synapse_root_log_level: "INFO" matrix_synapse_rc_messages_per_second: 0.2 matrix_synapse_rc_message_burst_count: 10.0 +# If you're serving Synapse behind an HTTPS-capable reverse-proxy, +# you can disable TLS completely (`matrix_synapse_no_tls: true`). +# Otherwise, you would need to provide certificate files to it. +matrix_synapse_no_tls: false +# Controls whether the TLS federation listener is enabled (tcp/8448). +# Note that federation may potentially be enabled on tcp/8008 as well. +# Only makes sense if federation is not disabled (`matrix_synapse_federation_enabled`). +matrix_synapse_tls_federation_listener_enabled: "{{ not matrix_synapse_no_tls }}" +matrix_synapse_tls_certificate_path: "/data/{{ hostname_matrix }}.tls.crt" +matrix_synapse_tls_private_key_path: "/data/{{ hostname_matrix }}.tls.key" + # Enable this to allow Synapse to report utilization statistics about your server to matrix.org # (things like number of users, number of messages sent, uptime, load, etc.) matrix_synapse_report_stats: false @@ -95,6 +111,8 @@ matrix_synapse_cache_factor: 0.5 # Controls whether Matrix Synapse will federate at all. # Disable this to completely isolate your server from the rest of the Matrix network. +# Also see: `matrix_synapse_tls_federation_listener_enabled` if you wish to keep federation enabled, +# but want to stop the TLS listener (port 8448). matrix_synapse_federation_enabled: true # A list of domain names that are allowed to federate with the given Matrix Synapse server. diff --git a/roles/matrix-synapse/tasks/setup_synapse_main.yml b/roles/matrix-synapse/tasks/setup_synapse_main.yml index 57a296bc..4d2598af 100644 --- a/roles/matrix-synapse/tasks/setup_synapse_main.yml +++ b/roles/matrix-synapse/tasks/setup_synapse_main.yml @@ -58,6 +58,61 @@ dest: "{{ matrix_synapse_config_dir_path }}/{{ hostname_matrix }}.log.config" mode: 0644 +# +# To make Synapse 0.99 happy, we need to generate a valid (self-signed is OK) certificate file that we provide to it. +# It won't be used for anything important, but it needs to be there. +# See https://github.com/matrix-org/synapse/issues/4554 +# +# Previously, Synapse would generate such certificate files and actually use them. +# So existing installations already have them. +# + +- name: Check if Synapse certificate exists + stat: + path: "{{ matrix_synapse_config_dir_path }}/{{ hostname_matrix }}.tls.crt" + register: matrix_synapse_certificate_stat + +- name: Ensure OpenSSL installed (RedHat) + yum: + name: + - openssl + state: present + update_cache: no + when: "not matrix_synapse_certificate_stat.stat.exists and ansible_os_family == 'RedHat'" + +- name: Ensure OpenSSL installed (Debian) + apt: + name: + - openssl + state: present + update_cache: no + when: "not matrix_synapse_certificate_stat.stat.exists and ansible_os_family == 'Debian'" + +# The proper way to do this is by using a sequence of +# `openssl_privatekey`, `openssl_csr` and `openssl_certificate`. +# +# Unfortunately, `openssl_csr` and `openssl_certificate` require `PyOpenSSL>=0.15` to work, +# which is not available on CentOS 7 (at least). +# +# We'll do it in a more manual way. +- name: Generate SSL certificate + command: | + openssl req -x509 \ + -sha256 \ + -newkey rsa:4096 \ + -nodes \ + -subj "/CN={{ hostname_matrix }}" \ + -keyout {{ matrix_synapse_config_dir_path }}/{{ hostname_matrix }}.tls.key \ + -out {{ matrix_synapse_config_dir_path }}/{{ hostname_matrix }}.tls.crt \ + -days 3650 + become: true + become_user: "{{ matrix_user_username }}" + when: "not matrix_synapse_certificate_stat.stat.exists" + +# +# End of tasks related to making Synapse 0.99 happy. +# + - name: Ensure matrix-synapse.service installed template: src: "{{ role_path }}/templates/synapse/systemd/matrix-synapse.service.j2" diff --git a/roles/matrix-synapse/tasks/validate_config.yml b/roles/matrix-synapse/tasks/validate_config.yml index e1454bfa..2f86e676 100644 --- a/roles/matrix-synapse/tasks/validate_config.yml +++ b/roles/matrix-synapse/tasks/validate_config.yml @@ -6,4 +6,14 @@ You need to define a required configuration setting (`{{ item }}`) for using Synapse. when: "vars[item] == ''" with_items: - - "matrix_synapse_macaroon_secret_key" \ No newline at end of file + - "matrix_synapse_macaroon_secret_key" + +- name: (Deprecation) Catch and report renamed settings + fail: + msg: > + Your configuration contains a variable, which now has a different name. + Please change your configuration to rename the variable (`{{ item.old }}` -> `{{ item.new }}`). + when: "item.old in vars" + with_items: + - {'old': 'matrix_synapse_container_expose_api_port', 'new': 'matrix_synapse_container_expose_client_api_port'} + diff --git a/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2 b/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2 index 2cca864e..8570c92f 100644 --- a/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2 +++ b/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2 @@ -1,19 +1,70 @@ # vim:ft=yaml -# PEM encoded X509 certificate for TLS. -# You can replace the self-signed certificate that synapse -# autogenerates on launch with your own SSL certificate + key pair -# if you like. Any required intermediary certificates can be -# appended after the primary certificate in hierarchical order. -tls_certificate_path: "/data/{{ hostname_matrix }}.tls.crt" +# PEM-encoded X509 certificate for TLS. +# This certificate, as of Synapse 1.0, will need to be a valid and verifiable +# certificate, signed by a recognised Certificate Authority. +# +# See 'ACME support' below to enable auto-provisioning this certificate via +# Let's Encrypt. +# +tls_certificate_path: "{{ matrix_synapse_tls_certificate_path }}" + +# PEM-encoded private key for TLS +tls_private_key_path: "{{ matrix_synapse_tls_private_key_path }}" + +# ACME support: This will configure Synapse to request a valid TLS certificate +# for your configured `server_name` via Let's Encrypt. +# +# Note that provisioning a certificate in this way requires port 80 to be +# routed to Synapse so that it can complete the http-01 ACME challenge. +# By default, if you enable ACME support, Synapse will attempt to listen on +# port 80 for incoming http-01 challenges - however, this will likely fail +# with 'Permission denied' or a similar error. +# +# There are a couple of potential solutions to this: +# +# * If you already have an Apache, Nginx, or similar listening on port 80, +# you can configure Synapse to use an alternate port, and have your web +# server forward the requests. For example, assuming you set 'port: 8009' +# below, on Apache, you would write: +# +# ProxyPass /.well-known/acme-challenge http://localhost:8009/.well-known/acme-challenge +# +# * Alternatively, you can use something like `authbind` to give Synapse +# permission to listen on port 80. +# +acme: + # ACME support is disabled by default. Uncomment the following line + # to enable it. + # + # enabled: true + + # Endpoint to use to request certificates. If you only want to test, + # use Let's Encrypt's staging url: + # https://acme-staging.api.letsencrypt.org/directory + # + # url: https://acme-v01.api.letsencrypt.org/directory -# PEM encoded private key for TLS -tls_private_key_path: "/data/{{ hostname_matrix }}.tls.key" + # Port number to listen on for the HTTP-01 challenge. Change this if + # you are forwarding connections through Apache/Nginx/etc. + # + # port: 80 -# PEM dh parameters for ephemeral keys -tls_dh_params_path: "/data/{{ hostname_matrix }}.tls.dh" + # Local addresses to listen on for incoming connections. + # Again, you may want to change this if you are forwarding connections + # through Apache/Nginx/etc. + # + # bind_addresses: ['::', '0.0.0.0'] -# Don't bind to the https port -no_tls: False + # How many days remaining on a certificate before it is renewed. + # + # reprovision_threshold: 30 + +# If your server runs behind a reverse-proxy which terminates TLS connections +# (for both client and federation connections), it may be useful to disable +# All TLS support for incoming connections. Setting no_tls to True will +# do so (and avoid the need to give synapse a TLS private key). +# +no_tls: {{ matrix_synapse_no_tls|to_json }} # List of allowed TLS fingerprints for this server to publish along # with the signing keys for this server. Other matrix servers that @@ -133,7 +184,8 @@ listeners: bind_addresses: - '0.0.0.0' {% endif %} -{% if matrix_synapse_federation_enabled %} + +{% if matrix_synapse_federation_enabled and matrix_synapse_tls_federation_listener_enabled %} # Main HTTPS listener # For when matrix traffic is sent directly to synapse. - @@ -168,7 +220,7 @@ listeners: # config: {} {% endif %} - # Unsecure HTTP listener, + # Unsecure HTTP listener for the Client API, # For when matrix traffic passes through loadbalancer that unwraps TLS. - port: 8008 tls: false @@ -181,6 +233,21 @@ listeners: - names: [client] compress: false +{% if matrix_synapse_federation_enabled %} + # Unsecure HTTP listener for the Federation API, + # For when matrix traffic passes through loadbalancer that unwraps TLS. + - port: 8048 + tls: false + bind_addresses: ['::'] + type: http + + x_forwarded: true + + resources: + - names: [federation] + compress: false +{% endif %} + # Turn on the twisted ssh manhole service on localhost on the given # port. # - port: 9000 diff --git a/roles/matrix-synapse/templates/synapse/systemd/matrix-synapse.service.j2 b/roles/matrix-synapse/templates/synapse/systemd/matrix-synapse.service.j2 index 9ba918d6..9d12d5f3 100644 --- a/roles/matrix-synapse/templates/synapse/systemd/matrix-synapse.service.j2 +++ b/roles/matrix-synapse/templates/synapse/systemd/matrix-synapse.service.j2 @@ -28,11 +28,14 @@ ExecStart=/usr/bin/docker run --rm --name matrix-synapse \ --tmpfs=/tmp:rw,noexec,nosuid,size={{ matrix_synapse_tmp_directory_size_mb }}m \ --network={{ matrix_docker_network }} \ -e SYNAPSE_CACHE_FACTOR={{ matrix_synapse_cache_factor }} \ - {% if matrix_synapse_federation_enabled %} + {% if matrix_synapse_container_expose_client_api_port %} + -p 127.0.0.1:8008:8008 \ + {% endif %} + {% if matrix_synapse_federation_enabled and matrix_synapse_tls_federation_listener_enabled %} -p 8448:8448 \ {% endif %} - {% if matrix_synapse_container_expose_client_server_api_port %} - -p 127.0.0.1:8008:8008 \ + {% if matrix_synapse_federation_enabled and matrix_synapse_container_expose_federation_api_port %} + -p 127.0.0.1:8048:8048 \ {% endif %} {% if matrix_synapse_container_expose_metrics_port %} -p 127.0.0.1:{{ matrix_synapse_metrics_port }}:{{ matrix_synapse_metrics_port }} \ From 74710427e583b868494128f68e7516541c5f68c1 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Tue, 5 Feb 2019 11:49:59 +0200 Subject: [PATCH 02/16] Allow for the federation port (tcp/8448)'s certificate to be changed If someone decides to not use `/.well-known/matrix/server` and only relies on SRV records, then they would need to serve tcp/8448 using a certificate for the base domain (not for the matrix) domain. Until now, they could do that by giving the certificate to Synapse and setting it terminate TLS. That makes swapping certificates more annoying (Synapse requires a restart to re-read certificates), so it's better if we can support it via matrix-nginx-proxy. Mounting certificates (or any other file) into the matrix-nginx-proxy container can be done with `matrix_nginx_proxy_container_additional_volumes`, introduced in 96afbbb5a. --- roles/matrix-nginx-proxy/defaults/main.yml | 2 ++ .../templates/nginx/conf.d/matrix-synapse.conf.j2 | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/roles/matrix-nginx-proxy/defaults/main.yml b/roles/matrix-nginx-proxy/defaults/main.yml index f9b3d745..30a41cb8 100644 --- a/roles/matrix-nginx-proxy/defaults/main.yml +++ b/roles/matrix-nginx-proxy/defaults/main.yml @@ -62,6 +62,8 @@ matrix_nginx_proxy_proxy_matrix_federation_api_enabled: false matrix_nginx_proxy_proxy_matrix_federation_api_addr_with_container: "matrix-synapse:8048" matrix_nginx_proxy_proxy_matrix_federation_api_addr_sans_container: "localhost:8048" matrix_nginx_proxy_proxy_matrix_federation_api_client_max_body_size_mb: "{{ matrix_nginx_proxy_proxy_matrix_client_api_client_max_body_size_mb * 3 }}" +matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate: "{{ matrix_ssl_config_dir_path }}/live/{{ matrix_nginx_proxy_proxy_matrix_hostname }}/fullchain.pem" +matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate_key: "{{ matrix_ssl_config_dir_path }}/live/{{ matrix_nginx_proxy_proxy_matrix_hostname }}/privkey.pem" # The tmpfs at /tmp needs to be large enough to handle multiple concurrent file uploads. matrix_nginx_proxy_tmp_directory_size_mb: "{{ matrix_nginx_proxy_proxy_matrix_federation_api_client_max_body_size_mb * 50 }}" diff --git a/roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-synapse.conf.j2 b/roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-synapse.conf.j2 index 4786911a..c618f789 100644 --- a/roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-synapse.conf.j2 +++ b/roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-synapse.conf.j2 @@ -164,8 +164,8 @@ server { gzip on; gzip_types text/plain application/json; - ssl_certificate {{ matrix_ssl_config_dir_path }}/live/{{ matrix_nginx_proxy_proxy_matrix_hostname }}/fullchain.pem; - ssl_certificate_key {{ matrix_ssl_config_dir_path }}/live/{{ matrix_nginx_proxy_proxy_matrix_hostname }}/privkey.pem; + ssl_certificate {{ matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate }}; + ssl_certificate_key {{ matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate_key }}; ssl_protocols {{ matrix_nginx_proxy_ssl_protocols }}; ssl_prefer_server_ciphers on; ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"; From 764a040a90f049a763210dd30a0c27990c77efd7 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Tue, 5 Feb 2019 12:08:00 +0200 Subject: [PATCH 03/16] Make /.well-known/matrix/server optional People who wish to rely on SRV records can prevent the `/.well-known/matrix/server` file from being generated (and thus, served.. which causes trouble). --- roles/matrix-base/defaults/main.yml | 8 ++++++ roles/matrix-base/tasks/setup_well_known.yml | 24 ++++++++++++----- .../tasks/self_check_well_known.yml | 27 ++++++++++++++----- 3 files changed, 46 insertions(+), 13 deletions(-) diff --git a/roles/matrix-base/defaults/main.yml b/roles/matrix-base/defaults/main.yml index 3f3d7f06..f86cf2f1 100644 --- a/roles/matrix-base/defaults/main.yml +++ b/roles/matrix-base/defaults/main.yml @@ -28,6 +28,14 @@ matrix_identity_server_url: ~ # The Docker network that all services would be put into matrix_docker_network: "matrix" +# Controls whether a `/.well-known/matrix/server` file is generated and used at all. +# +# If you wish to rely on DNS SRV records only, you can disable this. +# That implies that you'll be handling Matrix Federation API traffic (tcp/8448) +# using certificates for the base domain (`hostname_identity`) and not for the +# matrix domain (`hostname_matrix`). +matrix_well_known_matrix_server_enabled: true + # Variables to Control which parts of our roles run. run_setup: true run_import_postgres: true diff --git a/roles/matrix-base/tasks/setup_well_known.yml b/roles/matrix-base/tasks/setup_well_known.yml index 70b91f77..06e62617 100644 --- a/roles/matrix-base/tasks/setup_well_known.yml +++ b/roles/matrix-base/tasks/setup_well_known.yml @@ -12,13 +12,25 @@ with_items: - "{{ matrix_static_files_base_path }}/.well-known/matrix" -- name: Ensure Matrix /.well-known/matrix files configured +- name: Ensure Matrix /.well-known/matrix/client file configured template: - src: "{{ role_path }}/templates/static-files/well-known/matrix-{{ item }}.j2" - dest: "{{ matrix_static_files_base_path }}/.well-known/matrix/{{ item }}" + src: "{{ role_path }}/templates/static-files/well-known/matrix-client.j2" + dest: "{{ matrix_static_files_base_path }}/.well-known/matrix/client" mode: 0644 owner: "{{ matrix_user_username }}" group: "{{ matrix_user_username }}" - with_items: - - "client" - - "server" + +- name: Ensure Matrix /.well-known/matrix/server file configured + template: + src: "{{ role_path }}/templates/static-files/well-known/matrix-server.j2" + dest: "{{ matrix_static_files_base_path }}/.well-known/matrix/server" + mode: 0644 + owner: "{{ matrix_user_username }}" + group: "{{ matrix_user_username }}" + when: matrix_well_known_matrix_server_enabled + +- name: Ensure Matrix /.well-known/matrix/server file deleted + file: + path: "{{ matrix_static_files_base_path }}/.well-known/matrix/server" + state: absent + when: "not matrix_well_known_matrix_server_enabled" diff --git a/roles/matrix-nginx-proxy/tasks/self_check_well_known.yml b/roles/matrix-nginx-proxy/tasks/self_check_well_known.yml index 93ac88da..d12e3fd3 100644 --- a/roles/matrix-nginx-proxy/tasks/self_check_well_known.yml +++ b/roles/matrix-nginx-proxy/tasks/self_check_well_known.yml @@ -1,13 +1,26 @@ --- +- name: Determine well-known files to check (Matrix) + set_fact: + well_known_file_checks: + - path: /.well-known/matrix/client + purpose: Client Discovery + cors: true + +- block: + - set_fact: + well_known_file_check_matrix_server: + path: /.well-known/matrix/server + purpose: Server Discovery + cors: false + + - name: Determine domains that we require certificates for (mxisd) + set_fact: + well_known_file_checks: "{{ well_known_file_checks + [well_known_file_check_matrix_server] }}" + when: "matrix_well_known_matrix_server_enabled" + - name: Perform well-known checks include_tasks: "{{ role_path }}/tasks/self_check_well_known_file.yml" - with_items: - - path: /.well-known/matrix/server - purpose: Server Discovery - cors: false - - path: /.well-known/matrix/client - purpose: Client Discovery - cors: true + with_items: "{{ well_known_file_checks }}" loop_control: loop_var: well_known_file_check From b5404279742fbfcd44f2d027b58725f3a11384db Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Tue, 5 Feb 2019 12:54:27 +0200 Subject: [PATCH 04/16] Mention alternative ways to do Server Delegation --- docs/howto-server-delegation.md | 131 ++++++++++++++++++++++++++++++++ docs/prerequisites.md | 2 +- 2 files changed, 132 insertions(+), 1 deletion(-) create mode 100644 docs/howto-server-delegation.md diff --git a/docs/howto-server-delegation.md b/docs/howto-server-delegation.md new file mode 100644 index 00000000..29d57082 --- /dev/null +++ b/docs/howto-server-delegation.md @@ -0,0 +1,131 @@ +# Server Delegation + +To have a server on a subdomain (e.g. `matrix.`) handle Matrix federation traffic for the base domain (``), we need to instruct the Matrix network of such a delegation. + +By default, this playbook guides you into setting up [Server Delegation via a well-known file](#server-delegation-via-a-well-known-file). +However, that method may have some downsides that are not to your liking. Hence this guide about alternative ways to set up Server Delegation. + +It is a complicated matter, so unless you are affected by the [Downsides of well-known-based Server Delegation](#downsides-of-well-known-based-server-delegation), we suggest you stay on the simple/default path. + + +## Server Delegation via a well-known file + +Serving a `/.well-known/matrix/server` file from the base domain is the most straightforward way to set up server delegation, but it suffers from the following problems: + +As we already mention in [Configuring DNS](configuring-dns.md) and [Configuring Service Discovery via .well-known](configuring-well-known.md), +this playbook already properly guides you into setting up such delegation by means of a `/.well-known/matrix/server` file served from the base domain (``). + +If this is okay with you, feel free to not read ahead. + + +### Downsides of well-known-based Server Delegation + +Server Delegation by means of a `/.well-known/matrix/server` file is the most straightforward, but suffers from the following downsides: + +- you need to have a working HTTPS server for the base domain (``) + +- any downtime on the base domain (``) or network trouble between the matrix subdomain (`matrix.`) and the base `` may cause Matrix Federation outages. As the [Server-Server spec says](https://matrix.org/docs/spec/server_server/r0.1.0.html#server-discovery): + +> Errors are recommended to be cached for up to an hour, and servers are encouraged to exponentially back off for repeated failures. + +If this is not a concern for you, feel free to not read ahead. + +Otherwise, you can decide to go against the default for this playbook, and instead set up [Server Delegation via a DNS SRV record (advanced)](#server-delegation-via-a-dns-serv-record-advanced). + + +## Server Delegation via a DNS SRV record (advanced) + +**NOTE**: doing Server Delegation via a DNS SRV record is a more advanced way to do it and is not the default for this playbook. + +As per the [Server-Server spec](https://matrix.org/docs/spec/server_server/r0.1.0.html#server-discovery), it's possible to do Server Delegation using only a SRV record (without a `/.well-known/matrix/server` file). + +This prevents you from suffering the [Downsides of well-known-based Server Delegation](#downsides-of-well-known-based-server-delegation). + +To use DNS SRV record validation, you need to: + +- ensure that no `/.well-known/matrix/server` is served from the base domain, as that would interfere with DNS SRV record Server Delegation. To make the playbook **not** generate and serve the file, use the following configuration: `matrix_well_known_matrix_server_enabled: false`. + +- ensure that you have a `_matrix._tcp` DNS SRV record for your base domain (``) with a value of `10 0 8448 matrix.` + +- ensure that you are serving the Matrix Federation API (tcp/8448) with a certificate for `` (not `matrix.`!). See below. + + +### Obtaining certificates + +How you can obtain a valid certificate for `` on the `matrix.` server is up to you. + +If `` and `matrix.` is the same machine, you can let the playbook obtain the certificate for you by redefining the `matrix_ssl_domains_to_obtain_certificates_for` variable. Example: + +```yaml +matrix_ssl_domains_to_obtain_certificates_for: + - '{{ hostname_matrix }}' + - '{{ hostname_riot }}' + - '{{ hostname_identity }}' +``` + +This way, the playbook would obtain certificates for your base domain as well (referred to by the `hostname_identity` variable). +The certificate files would be available in `/matrix/ssl/config/live//...`. + +If `` and `matrix.` are not the same machine, you can copy over the certificate files manually. +If they get renewed automatically, you may also have to transfer them periodically. How often you do that is up to you, as long as the certificate files don't expire. + + +### Serving the Federation API with your certificates + +Regardless of which method for obtaining certificates you've used, once you've managed to get certificates for your base domain onto the `matrix.` machine you can put them to use. + + +### Serving the Federation API with your certificates and matrix-nginx-proxy + +**If you are using matrix-nginx-proxy**, a reverse-proxy webserver used by default in this playbook, you only need to override the certificates used for the Matrix Federation API. You can do that using: + +```yaml +# Adjust paths below to point to your certificate. +# +# NOTE: these are in-container paths. `/matrix/ssl` on the host is mounted into the container +# at the same path (`/matrix/ssl`) by default, so if that's the path you need, it would be seamless. +matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate: /matrix/ssl/config/live//fullchain.pem +matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate_key: /matrix/ssl/config/live//privkey.pem +``` + +If your files are not in `/matrix/ssl` but in some other location, you would need to mount them into the container: + +```yaml +matrix_nginx_proxy_container_additional_volumes: + - src: /some/path/on/the/host + dst: /some/path/inside/the/container + options: ro +``` + +You then refer to them (for `matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate` and `matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate_key`) by using `/some/path/inside/the/container`. + + +### Serving the Federation API with your certificates and another webserver + +**If you are NOT using matrix-nginx-proxy**, but rather some other webserver, you can set up reverse-proxying for the `tcp/8448` port by yourself. +Make sure to use the proper certificates for `` (not for `matrix.`) when serving the `tcp/8448` port. + +Proxying needs to happen to `127.0.0.1:8048` (unencrypted Synapse federation listener). + + +### Serving the Federation API with your certificates and Synapse handling Federation + +**Alternatively**, if you are **NOT using matrix-nginx-proxy** and **would rather not use your own webserver for Federation traffic**, you can let Synapse handle Federation by itself. + +To do that, make sure the certificate files are mounted into the Synapse container: + +```yaml +matrix_synapse_container_additional_volumes: + - src: /some/path/on/the/host + dst: /some/path/inside/the/container + options: ro +``` + +You can then tell Synapse to serve Federation traffic over TLS on `tcp/8448`: + +```yaml +matrix_synapse_no_tls: false +matrix_synapse_tls_federation_listener_enabled: true +matrix_synapse_tls_certificate_path: /some/path/inside/the/container/certificate.crt +matrix_synapse_tls_private_key_path: /some/path/inside/the/container/private.key +``` diff --git a/docs/prerequisites.md b/docs/prerequisites.md index dc6b91f8..4d8aa20c 100644 --- a/docs/prerequisites.md +++ b/docs/prerequisites.md @@ -6,7 +6,7 @@ - the [Ansible](http://ansible.com/) program being installed on your own computer. It's used to run this playbook and configures your server for you. Take a look at [our guide about Ansible](ansible.md) for version requirements or alternative ways to run Ansible. -- an HTTPS-capable web server at the base domain name (``) which is capable of serving static files +- an HTTPS-capable web server at the base domain name (``) which is capable of serving static files (unless you decide to use DNS SRV records for [Server Delegation](howto-server-delegation.md)) - properly configured DNS records for `` (details in [Configuring DNS](configuring-dns.md)) From 119016e8589c2e6477c1224a146318f76d7b368f Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Tue, 5 Feb 2019 13:06:17 +0200 Subject: [PATCH 05/16] Cache /.well-known/matrix files for longer --- .../templates/nginx/conf.d/matrix-synapse.conf.j2 | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-synapse.conf.j2 b/roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-synapse.conf.j2 index c618f789..17f67696 100644 --- a/roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-synapse.conf.j2 +++ b/roles/matrix-nginx-proxy/templates/nginx/conf.d/matrix-synapse.conf.j2 @@ -41,7 +41,12 @@ server { location /.well-known/matrix { root {{ matrix_static_files_base_path }}; - expires 1m; + {# + A somewhat long expires value is used to prevent outages + in case this is unreachable due to network failure or + due to the base domain's server completely dying. + #} + expires 4h; default_type application/json; add_header Access-Control-Allow-Origin *; } From 738c592c274ce662580160a45f9e793f8fd0eb0d Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Tue, 5 Feb 2019 13:33:39 +0200 Subject: [PATCH 06/16] Bump Synapse version (0.34.1.1 -> 0.99.0rc4) --- roles/matrix-synapse/defaults/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/matrix-synapse/defaults/main.yml b/roles/matrix-synapse/defaults/main.yml index d72fc2ec..af5fa68a 100644 --- a/roles/matrix-synapse/defaults/main.yml +++ b/roles/matrix-synapse/defaults/main.yml @@ -1,4 +1,4 @@ -matrix_synapse_docker_image: "matrixdotorg/synapse:v0.34.1.1-py3" +matrix_synapse_docker_image: "matrixdotorg/synapse:v0.99.0rc4-py3" matrix_synapse_base_path: "{{ matrix_base_data_path }}/synapse" matrix_synapse_config_dir_path: "{{ matrix_synapse_base_path }}/config" From 772154f3b907cd2ce0b9b660a29aa31b93ccbbf3 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Tue, 5 Feb 2019 13:38:20 +0200 Subject: [PATCH 07/16] Update Server Delegation docs a bit --- docs/howto-server-delegation.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/howto-server-delegation.md b/docs/howto-server-delegation.md index 29d57082..dae7c6cf 100644 --- a/docs/howto-server-delegation.md +++ b/docs/howto-server-delegation.md @@ -99,6 +99,9 @@ matrix_nginx_proxy_container_additional_volumes: You then refer to them (for `matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate` and `matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate_key`) by using `/some/path/inside/the/container`. +Make sure to reload matrix-nginx-proxy once in a while (`systemctl reload matrix-nginx-proxy`), so that newer certificates can kick in. +Reloading doesn't cause any downtime. + ### Serving the Federation API with your certificates and another webserver @@ -107,6 +110,8 @@ Make sure to use the proper certificates for `` (not for `matrix. Date: Tue, 5 Feb 2019 14:02:01 +0200 Subject: [PATCH 08/16] Remove some useless homeserver.yaml configuration --- .../templates/synapse/homeserver.yaml.j2 | 29 ------------------- 1 file changed, 29 deletions(-) diff --git a/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2 b/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2 index 8570c92f..d52ba1ab 100644 --- a/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2 +++ b/roles/matrix-synapse/templates/synapse/homeserver.yaml.j2 @@ -66,35 +66,6 @@ acme: # no_tls: {{ matrix_synapse_no_tls|to_json }} -# List of allowed TLS fingerprints for this server to publish along -# with the signing keys for this server. Other matrix servers that -# make HTTPS requests to this server will check that the TLS -# certificates returned by this server match one of the fingerprints. -# -# Synapse automatically adds the fingerprint of its own certificate -# to the list. So if federation traffic is handled directly by synapse -# then no modification to the list is required. -# -# If synapse is run behind a load balancer that handles the TLS then it -# will be necessary to add the fingerprints of the certificates used by -# the loadbalancers to this list if they are different to the one -# synapse is using. -# -# Homeservers are permitted to cache the list of TLS fingerprints -# returned in the key responses up to the "valid_until_ts" returned in -# key. It may be necessary to publish the fingerprints of a new -# certificate and wait until the "valid_until_ts" of the previous key -# responses have passed before deploying it. -# -# You can calculate a fingerprint from a given TLS listener via: -# openssl s_client -connect $host:$port < /dev/null 2> /dev/null | -# openssl x509 -outform DER | openssl sha256 -binary | base64 | tr -d '=' -# or by checking matrix.org/federationtester/api/report?server_name=$host -# -tls_fingerprints: [] -# tls_fingerprints: [{"sha256": ""}] - - ## Server ## # The domain name of the server, with optional explicit port. From 40f3793af702cba03e8aba10ad0672856978484f Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 6 Feb 2019 09:17:55 +0200 Subject: [PATCH 09/16] Upgrade Synapse to v0.99 and simplify dummy TLS cert logic --- group_vars/matrix-servers | 5 ++ roles/matrix-synapse/defaults/main.yml | 2 +- .../tasks/setup_synapse_main.yml | 55 ------------------- 3 files changed, 6 insertions(+), 56 deletions(-) diff --git a/group_vars/matrix-servers b/group_vars/matrix-servers index b79f5038..597a6a89 100644 --- a/group_vars/matrix-servers +++ b/group_vars/matrix-servers @@ -295,6 +295,11 @@ matrix_synapse_database_database: "{{ matrix_postgres_db_name }}" # We do not enable TLS in Synapse by default. # TLS is handled by the matrix-nginx-proxy, which proxies the requests to Synapse. matrix_synapse_no_tls: true +# Even though we don't do TLS at the Synapse side, Synapse v0.99 would still like to read +# some certificate file. The container contains a dummy certificate that could be used +# to prevent certificate file reading errors. It won't actually be used for anything else. +# See https://github.com/matrix-org/synapse/issues/4554 +matrix_synapse_tls_certificate_path: /conf/dummy.tls.crt matrix_synapse_email_enabled: "{{ matrix_mailer_enabled }}" matrix_synapse_email_smtp_host: "matrix-mailer" diff --git a/roles/matrix-synapse/defaults/main.yml b/roles/matrix-synapse/defaults/main.yml index af5fa68a..96825f94 100644 --- a/roles/matrix-synapse/defaults/main.yml +++ b/roles/matrix-synapse/defaults/main.yml @@ -1,4 +1,4 @@ -matrix_synapse_docker_image: "matrixdotorg/synapse:v0.99.0rc4-py3" +matrix_synapse_docker_image: "matrixdotorg/synapse:v0.99.0-py3" matrix_synapse_base_path: "{{ matrix_base_data_path }}/synapse" matrix_synapse_config_dir_path: "{{ matrix_synapse_base_path }}/config" diff --git a/roles/matrix-synapse/tasks/setup_synapse_main.yml b/roles/matrix-synapse/tasks/setup_synapse_main.yml index 4d2598af..57a296bc 100644 --- a/roles/matrix-synapse/tasks/setup_synapse_main.yml +++ b/roles/matrix-synapse/tasks/setup_synapse_main.yml @@ -58,61 +58,6 @@ dest: "{{ matrix_synapse_config_dir_path }}/{{ hostname_matrix }}.log.config" mode: 0644 -# -# To make Synapse 0.99 happy, we need to generate a valid (self-signed is OK) certificate file that we provide to it. -# It won't be used for anything important, but it needs to be there. -# See https://github.com/matrix-org/synapse/issues/4554 -# -# Previously, Synapse would generate such certificate files and actually use them. -# So existing installations already have them. -# - -- name: Check if Synapse certificate exists - stat: - path: "{{ matrix_synapse_config_dir_path }}/{{ hostname_matrix }}.tls.crt" - register: matrix_synapse_certificate_stat - -- name: Ensure OpenSSL installed (RedHat) - yum: - name: - - openssl - state: present - update_cache: no - when: "not matrix_synapse_certificate_stat.stat.exists and ansible_os_family == 'RedHat'" - -- name: Ensure OpenSSL installed (Debian) - apt: - name: - - openssl - state: present - update_cache: no - when: "not matrix_synapse_certificate_stat.stat.exists and ansible_os_family == 'Debian'" - -# The proper way to do this is by using a sequence of -# `openssl_privatekey`, `openssl_csr` and `openssl_certificate`. -# -# Unfortunately, `openssl_csr` and `openssl_certificate` require `PyOpenSSL>=0.15` to work, -# which is not available on CentOS 7 (at least). -# -# We'll do it in a more manual way. -- name: Generate SSL certificate - command: | - openssl req -x509 \ - -sha256 \ - -newkey rsa:4096 \ - -nodes \ - -subj "/CN={{ hostname_matrix }}" \ - -keyout {{ matrix_synapse_config_dir_path }}/{{ hostname_matrix }}.tls.key \ - -out {{ matrix_synapse_config_dir_path }}/{{ hostname_matrix }}.tls.crt \ - -days 3650 - become: true - become_user: "{{ matrix_user_username }}" - when: "not matrix_synapse_certificate_stat.stat.exists" - -# -# End of tasks related to making Synapse 0.99 happy. -# - - name: Ensure matrix-synapse.service installed template: src: "{{ role_path }}/templates/synapse/systemd/matrix-synapse.service.j2" From 91a757c5811c11552b6a466e47556d6673fed730 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 6 Feb 2019 09:25:13 +0200 Subject: [PATCH 10/16] Add support for reloading Synapse --- docs/howto-server-delegation.md | 3 ++- .../templates/synapse/systemd/matrix-synapse.service.j2 | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/howto-server-delegation.md b/docs/howto-server-delegation.md index dae7c6cf..fd151eb1 100644 --- a/docs/howto-server-delegation.md +++ b/docs/howto-server-delegation.md @@ -135,4 +135,5 @@ matrix_synapse_tls_certificate_path: /some/path/inside/the/container/certificate matrix_synapse_tls_private_key_path: /some/path/inside/the/container/private.key ``` -Every once in a while (before the certificates expire), you'll need to completely restart Synapse (unless [Synapse becomes capable of reloading certificates without restarting - issue #1180](https://github.com/matrix-org/synapse/issues/1180)). Restarting Synapse can be done like this: `systemctl restart matrix-synapse`. Restarting causes some downtime. +Make sure to reload Synapse once in a while (`systemctl reload matrix-synapse`), so that newer certificates can kick in. +Reloading doesn't cause any downtime. diff --git a/roles/matrix-synapse/templates/synapse/systemd/matrix-synapse.service.j2 b/roles/matrix-synapse/templates/synapse/systemd/matrix-synapse.service.j2 index 9d12d5f3..f11e99d4 100644 --- a/roles/matrix-synapse/templates/synapse/systemd/matrix-synapse.service.j2 +++ b/roles/matrix-synapse/templates/synapse/systemd/matrix-synapse.service.j2 @@ -51,6 +51,7 @@ ExecStart=/usr/bin/docker run --rm --name matrix-synapse \ ExecStop=-/usr/bin/docker kill matrix-synapse ExecStop=-/usr/bin/docker rm matrix-synapse +ExecReload=/usr/bin/docker exec matrix-synapse kill -HUP 1 Restart=always RestartSec=30 From 5148f8edf4c95986703d840c4b5676f2be84f49f Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 6 Feb 2019 09:36:03 +0200 Subject: [PATCH 11/16] Update docs --- docs/configuring-dns.md | 14 ++++++++++++++ docs/prerequisites.md | 2 +- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/docs/configuring-dns.md b/docs/configuring-dns.md index 16bc4436..5a49878d 100644 --- a/docs/configuring-dns.md +++ b/docs/configuring-dns.md @@ -5,6 +5,20 @@ To set up Matrix on your domain, you'd need to do some DNS configuration. To use an identifier like `@:`, you don't actually need to install anything on the actual `` server. +You do, however need to instruct the Matrix network that Matrix services for `` are delegated +over to `matrix.`. +As we discuss in [Server Delegation](howto-server-delegation.md), there are 2 different ways to set up such delegation: + +- using a `https:///.well-known/matrix/server` file (on the base domain!) +- using DNS SRV records + +This playbook mostly discusses the well-known file method, because it's easier to manage with regard to certificates. +If you decide to go with the alternative method ([Server Delegation via a DNS SRV record (advanced)](howto-server-delegation.md#server-delegation-via-a-dns-serv-record-advanced)), please be aware that the general flow that this playbook guides you through may not match what you need to do. + +To make matters worse, for backward compatibility until the Synapse server reaches v1.0, you need to set up a DNS SRV record anyway. +So don't be confused if the general flow of this playbook asks you for both DNS SRV records and for setting up a well-known file. +It's a temporary requirement during the Synapse v0.99/v1.0 transition. + ## General outline of DNS settings you need to do diff --git a/docs/prerequisites.md b/docs/prerequisites.md index 4d8aa20c..59e6c83d 100644 --- a/docs/prerequisites.md +++ b/docs/prerequisites.md @@ -10,6 +10,6 @@ - properly configured DNS records for `` (details in [Configuring DNS](configuring-dns.md)) -- some TCP/UDP ports open. This playbook configures the server's internal firewall for you. In most cases, you don't need to do anything special. But **if your server is running behind another firewall**, you'd need to open these ports: `80/tcp` (HTTP webserver), `443/tcp` (HTTPS webserver), `3478/tcp` (STUN over TCP), `3478/udp` (STUN over UDP), `8448/tcp` (Matrix federation HTTPS webserver), `49152-49172/udp` (TURN over UDP). +- some TCP/UDP ports open. This playbook configures the server's internal firewall for you. In most cases, you don't need to do anything special. But **if your server is running behind another firewall**, you'd need to open these ports: `80/tcp` (HTTP webserver), `443/tcp` (HTTPS webserver), `3478/tcp` (STUN over TCP), `3478/udp` (STUN over UDP), `8448/tcp` (Matrix Federation API HTTPS webserver), `49152-49172/udp` (TURN over UDP). When ready to proceed, continue with [Configuring DNS](configuring-dns.md). From 241a4f9ef906714c9a7d52968cc701680e70a195 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 6 Feb 2019 12:57:33 +0200 Subject: [PATCH 12/16] Add changelog entry for Synapse v0.99 --- CHANGELOG.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 72bd2c53..a0bb8762 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,21 @@ +# 2019-02-06 + +## Synapse v0.99 support and preparation for Synapse v1.0 + +Matrix is undergoing a lot of changes as it matures towards Synapse v1.0. +The first step is the Synapse v0.99 transitional release, which this playbook now supports. + +If you've been using this playbook successfully until now, you'd be aware that we've been doing [Server Delegation](docs/howto-server-delegation.md) using a `_matrix._tcp` DNS SRV record (as per [Configuring DNS](docs/configuring-dns.md)). + +Due to changes related to certificate file requirements that will affect us at Synapse v1.0, we'll have to stop using a **`_matrix._tcp` DNS SRV record in the future** (when Synapse goes to v1.0 - around 5th of March 2019). We **still need to keep the SRV record for now**, for backward compatibility with older Synapse versions (lower than v0.99). + +**What you need to do now** is make use of this transitional Synapse v0.99 release to **prepare your federation settings for the future**. You have 2 choices to prepare yourself for compatibility with the future Synapse v1.0: + +- (recommended) set up [Server Delegation via a well-known file](docs/howto-server-delegation.md#server-delegation-via-a-well-known-file), unless you are affected by the [Downsides of well-known-based Server Delegation](docs/howto-server-delegation.md#downsides-of-well-known-based-server-delegation). After some time, when most people have upgraded to Synapse v0.99 and older releases have disappeared, be prepared to drop your `_matrix._tcp` SRV record. + +- (more advanced) if the [Downsides of well-known-based Server Delegation](docs/howto-server-delegation.md#downsides-of-well-known-based-server-delegation) are not to your liking, **as an alternative**, you can set up [Server Delegation via a DNS SRV record](docs/howto-server-delegation.md#server-delegation-via-a-dns-serv-record-advanced). In such a case, you get to keep using your existing `_matrix._tcp` DNS SRV record forever and need to NOT set up a `/.well-known/matrix/server` file. Don't forget that you need to do certificate changes though. Follow the guide at [Server Delegation via a DNS SRV record](docs/howto-server-delegation.md#server-delegation-via-a-dns-serv-record-advanced). + + # 2019-02-01 ## TLS v1.3 support From 33726cdb0898d6dd5b8d004682a4e00382899efe Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 6 Feb 2019 13:02:17 +0200 Subject: [PATCH 13/16] Fix anchor --- CHANGELOG.md | 2 +- docs/configuring-dns.md | 2 +- docs/howto-server-delegation.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a0bb8762..210d18bd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,7 +13,7 @@ Due to changes related to certificate file requirements that will affect us at S - (recommended) set up [Server Delegation via a well-known file](docs/howto-server-delegation.md#server-delegation-via-a-well-known-file), unless you are affected by the [Downsides of well-known-based Server Delegation](docs/howto-server-delegation.md#downsides-of-well-known-based-server-delegation). After some time, when most people have upgraded to Synapse v0.99 and older releases have disappeared, be prepared to drop your `_matrix._tcp` SRV record. -- (more advanced) if the [Downsides of well-known-based Server Delegation](docs/howto-server-delegation.md#downsides-of-well-known-based-server-delegation) are not to your liking, **as an alternative**, you can set up [Server Delegation via a DNS SRV record](docs/howto-server-delegation.md#server-delegation-via-a-dns-serv-record-advanced). In such a case, you get to keep using your existing `_matrix._tcp` DNS SRV record forever and need to NOT set up a `/.well-known/matrix/server` file. Don't forget that you need to do certificate changes though. Follow the guide at [Server Delegation via a DNS SRV record](docs/howto-server-delegation.md#server-delegation-via-a-dns-serv-record-advanced). +- (more advanced) if the [Downsides of well-known-based Server Delegation](docs/howto-server-delegation.md#downsides-of-well-known-based-server-delegation) are not to your liking, **as an alternative**, you can set up [Server Delegation via a DNS SRV record](docs/howto-server-delegation.md#server-delegation-via-a-dns-srv-record-advanced). In such a case, you get to keep using your existing `_matrix._tcp` DNS SRV record forever and need to NOT set up a `/.well-known/matrix/server` file. Don't forget that you need to do certificate changes though. Follow the guide at [Server Delegation via a DNS SRV record](docs/howto-server-delegation.md#server-delegation-via-a-dns-srv-record-advanced). # 2019-02-01 diff --git a/docs/configuring-dns.md b/docs/configuring-dns.md index 5a49878d..cdb54090 100644 --- a/docs/configuring-dns.md +++ b/docs/configuring-dns.md @@ -13,7 +13,7 @@ As we discuss in [Server Delegation](howto-server-delegation.md), there are 2 di - using DNS SRV records This playbook mostly discusses the well-known file method, because it's easier to manage with regard to certificates. -If you decide to go with the alternative method ([Server Delegation via a DNS SRV record (advanced)](howto-server-delegation.md#server-delegation-via-a-dns-serv-record-advanced)), please be aware that the general flow that this playbook guides you through may not match what you need to do. +If you decide to go with the alternative method ([Server Delegation via a DNS SRV record (advanced)](howto-server-delegation.md#server-delegation-via-a-dns-srv-record-advanced)), please be aware that the general flow that this playbook guides you through may not match what you need to do. To make matters worse, for backward compatibility until the Synapse server reaches v1.0, you need to set up a DNS SRV record anyway. So don't be confused if the general flow of this playbook asks you for both DNS SRV records and for setting up a well-known file. diff --git a/docs/howto-server-delegation.md b/docs/howto-server-delegation.md index fd151eb1..1a338555 100644 --- a/docs/howto-server-delegation.md +++ b/docs/howto-server-delegation.md @@ -30,7 +30,7 @@ Server Delegation by means of a `/.well-known/matrix/server` file is the most st If this is not a concern for you, feel free to not read ahead. -Otherwise, you can decide to go against the default for this playbook, and instead set up [Server Delegation via a DNS SRV record (advanced)](#server-delegation-via-a-dns-serv-record-advanced). +Otherwise, you can decide to go against the default for this playbook, and instead set up [Server Delegation via a DNS SRV record (advanced)](#server-delegation-via-a-dns-srv-record-advanced). ## Server Delegation via a DNS SRV record (advanced) From 92aa5bfa2de6b18c702bba316050bc962befe73c Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 6 Feb 2019 13:03:23 +0200 Subject: [PATCH 14/16] Fix YAML indentation on documentation page --- docs/howto-server-delegation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/howto-server-delegation.md b/docs/howto-server-delegation.md index 1a338555..912fe6fa 100644 --- a/docs/howto-server-delegation.md +++ b/docs/howto-server-delegation.md @@ -94,7 +94,7 @@ If your files are not in `/matrix/ssl` but in some other location, you would nee matrix_nginx_proxy_container_additional_volumes: - src: /some/path/on/the/host dst: /some/path/inside/the/container - options: ro + options: ro ``` You then refer to them (for `matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate` and `matrix_nginx_proxy_proxy_matrix_federation_api_ssl_certificate_key`) by using `/some/path/inside/the/container`. From e9cfcb8429b323856cf0f14738ba06282e7dbb3b Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 6 Feb 2019 13:04:19 +0200 Subject: [PATCH 15/16] Fix another YAML indentation problem on documentation page --- docs/howto-server-delegation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/howto-server-delegation.md b/docs/howto-server-delegation.md index 912fe6fa..475ed0e3 100644 --- a/docs/howto-server-delegation.md +++ b/docs/howto-server-delegation.md @@ -123,7 +123,7 @@ To do that, make sure the certificate files are mounted into the Synapse contain matrix_synapse_container_additional_volumes: - src: /some/path/on/the/host dst: /some/path/inside/the/container - options: ro + options: ro ``` You can then tell Synapse to serve Federation traffic over TLS on `tcp/8448`: From ef903fe544398e04f4f247e7146457fae2a7d8ae Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Wed, 6 Feb 2019 13:30:24 +0200 Subject: [PATCH 16/16] Add some quick links --- docs/howto-server-delegation.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/howto-server-delegation.md b/docs/howto-server-delegation.md index 475ed0e3..aa9b0f95 100644 --- a/docs/howto-server-delegation.md +++ b/docs/howto-server-delegation.md @@ -74,6 +74,14 @@ If they get renewed automatically, you may also have to transfer them periodical Regardless of which method for obtaining certificates you've used, once you've managed to get certificates for your base domain onto the `matrix.` machine you can put them to use. +Based on your setup, you have different ways to go about it: + +- [Serving the Federation API with your certificates and matrix-nginx-proxy](#serving-the-federation-api-with-your-certificates-and-matrix-nginx-proxy) + +- [Serving the Federation API with your certificates and another webserver](#serving-the-federation-api-with-your-certificates-and-another-webserver) + +- [Serving the Federation API with your certificates and Synapse handling Federation](#serving-the-federation-api-with-your-certificates-and-synapse-handling-federation) + ### Serving the Federation API with your certificates and matrix-nginx-proxy