From 6962bfcc4238104922a1d334dcd579f28726df27 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Tue, 12 Sep 2017 12:41:44 +0300 Subject: [PATCH] Add support for not taking over a server (no matrix-nginx-proxy) and disabling Riot --- README.md | 62 +++++++++++++++++-- roles/matrix-server/defaults/main.yml | 11 ++++ .../matrix-server/tasks/setup_nginx_proxy.yml | 44 +++++++++++-- roles/matrix-server/tasks/setup_riot_web.yml | 41 +++++++++++- roles/matrix-server/tasks/setup_ssl.yml | 13 +++- roles/matrix-server/tasks/start.yml | 2 + .../cron.d/matrix-ssl-certificate-renewal.j2 | 5 +- .../nginx-conf.d/matrix-riot-web.conf.j2 | 8 +-- .../nginx-conf.d/matrix-synapse.conf.j2 | 8 +-- .../systemd/matrix-nginx-proxy.service.j2 | 2 +- .../systemd/matrix-riot-web.service.j2 | 3 + .../systemd/matrix-synapse.service.j2 | 3 + 12 files changed, 178 insertions(+), 24 deletions(-) diff --git a/README.md b/README.md index 31030241..862925dd 100644 --- a/README.md +++ b/README.md @@ -16,10 +16,12 @@ Using this playbook, you can get the following services configured on your serve - a [STUN/TURN server](https://github.com/coturn/coturn) for WebRTC audio/video calls -- a [Riot](https://riot.im/) web UI, which is configured to connect to your own Matrix Synapse server by default +- (optional default) a [Riot](https://riot.im/) web UI, which is configured to connect to your own Matrix Synapse server by default - free [Let's Encrypt](https://letsencrypt.org/) SSL certificate, which secures the connection to the Synapse server and the Riot web UI +- (optional default) an [nginx](http://nginx.org/) web server, listening on ports 80 and 443 - standing in front of all the other services. Using your own webserver [is possible](#using-your-own-webserver-instead-of-this-playbooks-nginx-proxy-optional) + Basically, this playbook aims to get you up-and-running with all the basic necessities around Matrix, without you having to do anything else. @@ -33,13 +35,17 @@ This is similar to the [EMnify/matrix-synapse-auto-deploy](https://github.com/EM - works on both **CentOS** (7.0+) and Debian-based distributions (**Debian** 9/Stretch+, **Ubuntu** 16.04+) +- this one keeps mostly everything in a single directory (`/matrix` by default) and **doesn't "contaminate" your server** with files all over the place + +- this one **doesn't necessarily take over** ports 80 and 443. By default, it sets up nginx for you there, but you can disable that and configure your own webserver (proxy) + - this one **runs everything in Docker containers** (like [silviof/docker-matrix](https://hub.docker.com/r/silviof/docker-matrix/) and [silviof/matrix-riot-docker](https://hub.docker.com/r/silviof/matrix-riot-docker/)), so it's likely more predictable - this one retrieves and automatically renews free [Let's Encrypt](https://letsencrypt.org/) **SSL certificates** for you - this one optionally can store the `media_store` content repository files on [Amazon S3](https://aws.amazon.com/s3/) (but defaults to storing files on the server's filesystem) -- this one optionally allows you to use an external PostgreSQL server for Matrix Synapse's database (but defaults to running one in a container) +- this one optionally **allows you to use an external PostgreSQL server** for Matrix Synapse's database (but defaults to running one in a container) Special thanks goes to: @@ -52,7 +58,7 @@ Special thanks goes to: ## Prerequisites -- **CentOS** (7.0+), **Debian** (9/Stretch+) or **Ubuntu** (16.04+) server with no services running on port 80/443 +- **CentOS** (7.0+), **Debian** (9/Stretch+) or **Ubuntu** (16.04+) server. This playbook can take over your whole server or co-exist with other services that you have there. - the [Ansible](http://ansible.com/) program, which is used to run this playbook and configures everything for you @@ -158,6 +164,33 @@ The database (as specified in `matrix_postgres_db_name`) must exist and be acces It must be empty or contain a valid Matrix Synapse database. If empty, Matrix Synapse would populate it the first time it runs. +## Using your own webserver, instead of this playbook's nginx proxy (optional) + +By default, this playbook installs its own nginx webserver (in a Docker container) which listens on ports 80 and 443. +If that's alright, you can skip ahead. + +If you don't want this playbook's nginx webserver to take over your server's 80/443 ports like that, +and you'd like to use your own webserver (be it nginx, Apache, Varnish Cache, etc.), you can. + +All it takes is editing your configuration file (`inventory/matrix./vars.yml`): + +``` +matrix_nginx_proxy_enabled: false +``` + +**Note**: even if you do this, in order [to install](#installing), this playbook still expects port 80 to be available. **Please manually stop your other webserver while installing**. You can start it back again afterwards. + +**If your own webserver is nginx**, you can most likely directly use the config files installed by this playbook at: `/matrix/nginx-proxy/conf.d`. Just include them in your `nginx.conf` like this: `include /matrix/nginx-proxy/conf.d/*.conf;` + +**If your own webserver is not nginx**, you can still take a look at the sample files in `/matrix/nginx-proxy/conf.d`, and: + +- ensure you set up (separate) vhosts that proxy for both Riot (`localhost:8765`) and Matrix Synapse (`localhost:8008`) + +- ensure that the `/.well-known/acme-challenge` location for each "port=80 vhost" is an alias to the `/matrix/ssl/run/acme-challenge` directory (for automated SSL renewal to work) + +- ensure that you restart/reload your webserver once in a while, so that renewed SSL certificates would take effect (once a month should be enough) + + ## Installing Once you have your server and you have [configured your DNS records](#configuring-dns), you can proceed with installing. @@ -226,12 +259,31 @@ You can do it via this Ansible playbook (make sure to edit the `` **Note**: `` is just a plain username (like `john`), not your full `@:` identifier. +## Uninstalling + +**Note**: If you have some trouble with your installation configuration, you can just re-run the playbook and it will try to set things up again. You don't need to uninstall and install fresh. + +However, if you've installed this on some server where you have other stuff you wish to preserve, and now want get rid of Matrix, it's enough to do these: + +- ensure all Matrix services are stopped (`systemctl stop 'matrix*'`) + +- delete the Matrix-related systemd .service files (`rm -f /etc/systemd/system/matrix*`) and reload systemd (`systemctl daemon-reload`) + +- delete all Matrix-related cronjobs (`rm -f /etc/cron.d/matrix*'`) + +- delete some helper scripts (`rm -f /usr/local/bin/matrix*`) + +- delete some cached Docker images (or just delete them all: `docker rmi $(docker images -aq)`) + +- uninstall Docker itself, if necessary + +- delete the `/matrix` directory (`rm -rf /matrix`) + + ## Deficiencies This Ansible playbook can be improved in the following ways: -- not expecting to run its own nginx server overtaking port 80/443, thus allowing own/custom proxying to be configured - - setting up automatic backups to one or more storage providers - [importing an old SQLite database](#Restoring-an-existing-SQLite=database-from-another-installation) likely works because of a patch, but may be fragile until [this](https://github.com/matrix-org/synapse/issues/2287) is fixed \ No newline at end of file diff --git a/roles/matrix-server/defaults/main.yml b/roles/matrix-server/defaults/main.yml index d61af287..9139b913 100644 --- a/roles/matrix-server/defaults/main.yml +++ b/roles/matrix-server/defaults/main.yml @@ -61,3 +61,14 @@ matrix_s3_media_store_enabled: false matrix_s3_media_store_bucket_name: "your-bucket-name" matrix_s3_media_store_aws_access_key: "your-aws-access-key" matrix_s3_media_store_aws_secret_key: "your-aws-secret-key" + +# By default, this playbook installs the Riot.IM web UI on the `hostname_riot` domain. +# If you wish to connect to your Matrix server by other means, +# you may wish to disable this. +matrix_riot_web_enabled: true + +# By default, this playbook sets up its own nginx proxy server on port 80/443. +# This is fine if you're dedicating the whole server to Matrix. +# But in case that's not the case, you may wish to prevent that +# and take care of proxying by yourself. +matrix_nginx_proxy_enabled: true diff --git a/roles/matrix-server/tasks/setup_nginx_proxy.yml b/roles/matrix-server/tasks/setup_nginx_proxy.yml index 1839684d..0363d062 100644 --- a/roles/matrix-server/tasks/setup_nginx_proxy.yml +++ b/roles/matrix-server/tasks/setup_nginx_proxy.yml @@ -1,5 +1,15 @@ --- +# +# Generic tasks that we always want to happen, regardless +# if the user wants matrix-nginx-proxy or not. +# +# If the user would set up their own nginx proxy server, +# the config files from matrix-nginx-proxy can be reused. +# +# It doesn't hurt to put them in place, even if they turn out +# to be unnecessary. +# - name: Ensure Matrix nginx-proxy paths exists file: path: "{{ item }}" @@ -11,10 +21,6 @@ - "{{ matrix_nginx_proxy_data_path }}" - "{{ matrix_nginx_proxy_confd_path }}" -- name: Ensure nginx Docker image is pulled - docker_image: - name: "{{ docker_nginx_image }}" - - name: Ensure Matrix Synapse proxy vhost configured template: src: "{{ role_path }}/templates/nginx-conf.d/{{ item }}.j2" @@ -24,6 +30,14 @@ - "matrix-synapse.conf" - "matrix-riot-web.conf" +# +# Tasks related to setting up matrix-nginx-proxy +# +- name: Ensure nginx Docker image is pulled + docker_image: + name: "{{ docker_nginx_image }}" + when: matrix_nginx_proxy_enabled + - name: Allow access to nginx proxy ports in firewalld firewalld: service: "{{ item }}" @@ -33,10 +47,30 @@ with_items: - "http" - "https" - when: ansible_os_family == 'RedHat' + when: "ansible_os_family == 'RedHat' and matrix_nginx_proxy_enabled" - name: Ensure matrix-nginx-proxy.service installed template: src: "{{ role_path }}/templates/systemd/matrix-nginx-proxy.service.j2" dest: "/etc/systemd/system/matrix-nginx-proxy.service" mode: 0644 + when: matrix_nginx_proxy_enabled + +# +# Tasks related to getting rid of matrix-nginx-proxy (if it was previously enabled) +# + +- name: Check existence of matrix-nginx-proxy service + stat: path="/etc/systemd/system/matrix-nginx-proxy.service" + register: matrix_nginx_proxy_service_stat + +- name: Ensure matrix-nginx-proxy is stopped + service: name=matrix-nginx-proxy state=stopped daemon_reload=yes + register: stopping_result + when: "not matrix_nginx_proxy_enabled and matrix_nginx_proxy_service_stat.stat.exists" + +- name: Ensure matrix-nginx-proxy.service doesn't exist + file: + path: "/etc/systemd/system/matrix-nginx-proxy.service" + state: absent + when: "not matrix_nginx_proxy_enabled and matrix_nginx_proxy_service_stat.stat.exists" diff --git a/roles/matrix-server/tasks/setup_riot_web.yml b/roles/matrix-server/tasks/setup_riot_web.yml index 9d11a037..fd4023ab 100644 --- a/roles/matrix-server/tasks/setup_riot_web.yml +++ b/roles/matrix-server/tasks/setup_riot_web.yml @@ -1,5 +1,9 @@ --- +# +# Tasks related to setting up riot-web +# + - name: Ensure Matrix riot-web paths exists file: path: "{{ matrix_nginx_riot_web_data_path }}" @@ -7,10 +11,12 @@ mode: 0750 owner: "{{ matrix_user_username }}" group: "{{ matrix_user_username }}" + when: matrix_riot_web_enabled - name: Ensure riot-web Docker image is pulled docker_image: name: "{{ docker_riot_image }}" + when: matrix_riot_web_enabled - name: Ensure Matrix riot-web configured template: @@ -22,9 +28,42 @@ with_items: - "riot.im.conf" - "config.json" + when: matrix_riot_web_enabled - name: Ensure matrix-riot-web.service installed template: src: "{{ role_path }}/templates/systemd/matrix-riot-web.service.j2" dest: "/etc/systemd/system/matrix-riot-web.service" - mode: 0644 \ No newline at end of file + mode: 0644 + when: matrix_riot_web_enabled + +# +# Tasks related to getting rid of riot-web (if it was previously enabled) +# + +- name: Check existence of matrix-riot-web service + stat: path="/etc/systemd/system/matrix-riot-web.service" + register: matrix_riot_web_service_stat + +- name: Ensure matrix-riot-web is stopped + service: name=matrix-riot-web state=stopped daemon_reload=yes + register: stopping_result + when: "not matrix_riot_web_enabled and matrix_riot_web_service_stat.stat.exists" + +- name: Ensure matrix-riot-web.service doesn't exist + file: + path: "/etc/systemd/system/matrix-riot-web.service" + state: absent + when: "not matrix_riot_web_enabled and matrix_riot_web_service_stat.stat.exists" + +- name: Ensure Matrix riot-web paths doesn't exist + file: + path: "{{ matrix_nginx_riot_web_data_path }}" + state: absent + when: "not matrix_riot_web_enabled" + +- name: Ensure riot-web Docker image doesn't exist + docker_image: + name: "{{ docker_riot_image }}" + state: absent + when: "not matrix_riot_web_enabled" diff --git a/roles/matrix-server/tasks/setup_ssl.yml b/roles/matrix-server/tasks/setup_ssl.yml index 21242392..d6c297ba 100644 --- a/roles/matrix-server/tasks/setup_ssl.yml +++ b/roles/matrix-server/tasks/setup_ssl.yml @@ -1,5 +1,14 @@ --- +- name: Determine domains to obtain certificates for (Matrix) + set_fact: + domains_to_obtain_certificate_for: "['{{ hostname_matrix }}']" + +- name: Determine domains to obtain certificates for (Riot) + set_fact: + domains_to_obtain_certificate_for: "{{ domains_to_obtain_certificate_for + [hostname_riot] }}" + when: matrix_riot_web_enabled + - name: Allow access to HTTP/HTTPS in firewalld firewalld: service: "{{ item }}" @@ -38,9 +47,7 @@ -e ACME_EMAIL={{ matrix_ssl_support_email }} willwill/acme-docker acmetool want {{ item }} --xlog.severity=debug - with_items: - - "{{ hostname_matrix }}" - - "{{ hostname_riot }}" + with_items: "{{ domains_to_obtain_certificate_for }}" - name: Ensure matrix-nginx-proxy is started (if previously installed & started) service: name=matrix-nginx-proxy state=started diff --git a/roles/matrix-server/tasks/start.yml b/roles/matrix-server/tasks/start.yml index 43072b40..3b1cafb0 100644 --- a/roles/matrix-server/tasks/start.yml +++ b/roles/matrix-server/tasks/start.yml @@ -13,6 +13,8 @@ - name: Ensure matrix-riot-web autoruns and is restarted service: name=matrix-riot-web enabled=yes state=restarted daemon_reload=yes + when: matrix_riot_web_enabled - name: Ensure matrix-nginx-proxy autoruns and is restarted service: name=matrix-nginx-proxy enabled=yes state=restarted daemon_reload=yes + when: matrix_nginx_proxy_enabled diff --git a/roles/matrix-server/templates/cron.d/matrix-ssl-certificate-renewal.j2 b/roles/matrix-server/templates/cron.d/matrix-ssl-certificate-renewal.j2 index 3769bf1a..47d2cbe9 100644 --- a/roles/matrix-server/templates/cron.d/matrix-ssl-certificate-renewal.j2 +++ b/roles/matrix-server/templates/cron.d/matrix-ssl-certificate-renewal.j2 @@ -11,11 +11,14 @@ MAILTO="{{ matrix_ssl_support_email }}" # # How renewal works? # -# acmetool will fail to bind to port :80 (because matrix-nginx-proxy is running there), +# acmetool will fail to bind to port :80 (because matrix-nginx-proxy or some other server is running there), # and will fall back to its "webroot" validation method. # # Thus, it would put validation files in `/var/run/acme/acme-challenge`. # These files can be retrieved via any vhost on port 80 of matrix-nginx-proxy, # because it aliases `/.well-known/acme-challenge` to that same directory. +# +# When a custom proxy server (not matrix-nginx-proxy provided by this playbook), +# you'd need to make sure you alias these files corretly or SSL renewal would not work. 15 4 */5 * * root /usr/bin/docker run --rm --name acmetool-host-grab --net=host -v {{ matrix_ssl_certs_path }}:/certs -v {{ matrix_ssl_certs_path }}/run:/var/run/acme -e ACME_EMAIL={{ matrix_ssl_support_email }} willwill/acme-docker acmetool --batch reconcile # --xlog.severity=debug diff --git a/roles/matrix-server/templates/nginx-conf.d/matrix-riot-web.conf.j2 b/roles/matrix-server/templates/nginx-conf.d/matrix-riot-web.conf.j2 index 779aad0a..b96fd5a6 100644 --- a/roles/matrix-server/templates/nginx-conf.d/matrix-riot-web.conf.j2 +++ b/roles/matrix-server/templates/nginx-conf.d/matrix-riot-web.conf.j2 @@ -6,7 +6,7 @@ server { location /.well-known/acme-challenge { default_type "text/plain"; - alias /acmetool-certs/run/acme-challenge; + alias {{ matrix_ssl_certs_path }}/run/acme-challenge; } location / { @@ -24,14 +24,14 @@ server { root /dev/null; ssl on; - ssl_certificate /acmetool-certs/live/{{ hostname_riot }}/fullchain; - ssl_certificate_key /acmetool-certs/live/{{ hostname_riot }}/privkey; + ssl_certificate {{ matrix_ssl_certs_path }}/live/{{ hostname_riot }}/fullchain; + ssl_certificate_key {{ matrix_ssl_certs_path }}/live/{{ hostname_riot }}/privkey; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_prefer_server_ciphers on; ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"; location / { - proxy_pass http://riot:8765; + proxy_pass http://{{ 'riot' if matrix_nginx_proxy_enabled else 'localhost' }}:8765; proxy_set_header X-Forwarded-For $remote_addr; } } diff --git a/roles/matrix-server/templates/nginx-conf.d/matrix-synapse.conf.j2 b/roles/matrix-server/templates/nginx-conf.d/matrix-synapse.conf.j2 index a11d11ac..a9e3b1ee 100644 --- a/roles/matrix-server/templates/nginx-conf.d/matrix-synapse.conf.j2 +++ b/roles/matrix-server/templates/nginx-conf.d/matrix-synapse.conf.j2 @@ -6,7 +6,7 @@ server { location /.well-known/acme-challenge { default_type "text/plain"; - alias /acmetool-certs/run/acme-challenge; + alias {{ matrix_ssl_certs_path }}/run/acme-challenge; } location / { @@ -24,14 +24,14 @@ server { root /dev/null; ssl on; - ssl_certificate /acmetool-certs/live/{{ hostname_matrix }}/fullchain; - ssl_certificate_key /acmetool-certs/live/{{ hostname_matrix }}/privkey; + ssl_certificate {{ matrix_ssl_certs_path }}/live/{{ hostname_matrix }}/fullchain; + ssl_certificate_key {{ matrix_ssl_certs_path }}/live/{{ hostname_matrix }}/privkey; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_prefer_server_ciphers on; ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"; location /_matrix { - proxy_pass http://synapse:8008; + proxy_pass http://{{ 'synapse' if matrix_nginx_proxy_enabled else 'localhost' }}:8008; proxy_set_header X-Forwarded-For $remote_addr; client_body_buffer_size 25M; diff --git a/roles/matrix-server/templates/systemd/matrix-nginx-proxy.service.j2 b/roles/matrix-server/templates/systemd/matrix-nginx-proxy.service.j2 index 5a47939b..c105c63b 100644 --- a/roles/matrix-server/templates/systemd/matrix-nginx-proxy.service.j2 +++ b/roles/matrix-server/templates/systemd/matrix-nginx-proxy.service.j2 @@ -17,7 +17,7 @@ ExecStart=/usr/bin/docker run --rm --name matrix-nginx-proxy \ --link matrix-synapse:synapse \ --link matrix-riot-web:riot \ -v {{ matrix_nginx_proxy_confd_path }}:/etc/nginx/conf.d \ - -v {{ matrix_ssl_certs_path }}:/acmetool-certs \ + -v {{ matrix_ssl_certs_path }}:{{ matrix_ssl_certs_path }} \ {{ docker_nginx_image }} ExecStop=-/usr/bin/docker kill matrix-nginx-proxy ExecStop=-/usr/bin/docker rm matrix-nginx-proxy diff --git a/roles/matrix-server/templates/systemd/matrix-riot-web.service.j2 b/roles/matrix-server/templates/systemd/matrix-riot-web.service.j2 index 2abcc7e0..510ba10d 100644 --- a/roles/matrix-server/templates/systemd/matrix-riot-web.service.j2 +++ b/roles/matrix-server/templates/systemd/matrix-riot-web.service.j2 @@ -9,6 +9,9 @@ ExecStartPre=-/usr/bin/docker kill matrix-riot-web ExecStartPre=-/usr/bin/docker rm matrix-riot-web ExecStart=/usr/bin/docker run --rm --name matrix-riot-web \ -v {{ matrix_nginx_riot_web_data_path }}:/data \ + {% if not matrix_nginx_proxy_enabled %} + -p 127.0.0.1:8765:8765 \ + {% endif %} {{ docker_riot_image }} ExecStop=-/usr/bin/docker kill matrix-riot-web ExecStop=-/usr/bin/docker rm matrix-riot-web diff --git a/roles/matrix-server/templates/systemd/matrix-synapse.service.j2 b/roles/matrix-server/templates/systemd/matrix-synapse.service.j2 index e14e8d23..a647db03 100644 --- a/roles/matrix-server/templates/systemd/matrix-synapse.service.j2 +++ b/roles/matrix-server/templates/systemd/matrix-synapse.service.j2 @@ -21,6 +21,9 @@ ExecStart=/usr/bin/docker run --rm --name matrix-synapse \ --link matrix-postgres:{{ matrix_postgres_connection_hostname }} \ {% endif %} -p 8448:8448 \ + {% if not matrix_nginx_proxy_enabled %} + -p 127.0.0.1:8008:8008 \ + {% endif %} -p 3478:3478 \ -p 3478:3478/udp \ -p {{ matrix_coturn_turn_udp_min_port }}-{{ matrix_coturn_turn_udp_max_port }}:{{ matrix_coturn_turn_udp_min_port }}-{{ matrix_coturn_turn_udp_max_port }}/udp \