From 4be35822ddb226f8c710a26a656180206050e8f0 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Mon, 5 Aug 2019 13:06:49 +0300 Subject: [PATCH 1/5] Add Email2Matrix support --- CHANGELOG.md | 11 +++ docs/configuring-playbook-email2matrix.md | 79 +++++++++++++++++ docs/configuring-playbook.md | 2 + group_vars/matrix_servers | 15 ++++ roles/matrix-email2matrix/defaults/main.yml | 43 +++++++++ roles/matrix-email2matrix/tasks/init.yml | 3 + roles/matrix-email2matrix/tasks/main.yml | 15 ++++ .../tasks/setup_email2matrix.yml | 88 +++++++++++++++++++ .../tasks/validate_config.yml | 7 ++ .../templates/config.json.j2 | 14 +++ .../systemd/matrix-email2matrix.service.j2 | 32 +++++++ setup.yml | 1 + 12 files changed, 310 insertions(+) create mode 100644 docs/configuring-playbook-email2matrix.md create mode 100644 roles/matrix-email2matrix/defaults/main.yml create mode 100644 roles/matrix-email2matrix/tasks/init.yml create mode 100644 roles/matrix-email2matrix/tasks/main.yml create mode 100644 roles/matrix-email2matrix/tasks/setup_email2matrix.yml create mode 100644 roles/matrix-email2matrix/tasks/validate_config.yml create mode 100644 roles/matrix-email2matrix/templates/config.json.j2 create mode 100644 roles/matrix-email2matrix/templates/systemd/matrix-email2matrix.service.j2 diff --git a/CHANGELOG.md b/CHANGELOG.md index cfcde2de..d73d19f6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,14 @@ +# 2019-08-05 + +## Email2Matrix support + +Support for [Email2Matrix](https://github.com/devture/email2matrix) has been added. + +It's an optional feature that you can enable via the playbook. + +To learn more, see the [playbook's documentation on Email2Matrix](./docs/configuring-playbook-email2matrix.md). + + # 2019-08-03 ## Synapse logging level has been reduced to WARNING diff --git a/docs/configuring-playbook-email2matrix.md b/docs/configuring-playbook-email2matrix.md new file mode 100644 index 00000000..cac1782d --- /dev/null +++ b/docs/configuring-playbook-email2matrix.md @@ -0,0 +1,79 @@ +# Setting up Email2Matrix (optional) + +The playbook can install and configure [email2matrix](https://github.com/devture/email2matrix) for you. + +See the project's [documentation](https://github.com/devture/email2matrix/blob/master/docs/README.md) to learn what it does and why it might be useful to you. + + +## Preparation + +### Port availability + +Ensure that port 25 is available on your Matrix server. +If you have `postfix` or some other email server software installed, you may need to manually remove it first (unless you need it, of course). + +If you really need to run an email server on the Matrix machine for other purposes, it may be possible to run Email2Matrix on another port (with a configuration like `matrix_email2matrix_smtp_host_bind_port: "127.0.0.01:2525"`) and have your other email server relay messages there. + +### Creating a user + +Before enabling Email2Matrix, you'd most likely wish to create a dedicated user (or more) that would be sending messages on the Matrix side. +Refer to [Registering users](registering-users.md) for ways to do that. A regular (non-admin) user works best. + +### Creating a shared room + +After creating a sender user, you should create one or more Matrix rooms that you share with that user. +It doesn't matter who creates and owns the rooms and who joins later (you or the sender user). + +What matters is that both you and the sender user are part of the same room and that the sender user has enough privileges in the room to be able to send messages there. +Inviting additional people to the room is okay too. + +Take note of each room's room id (different clients show the room id in a different place). +You'll need the room id when doing [Configuration](#configuration) below. + + +### Obtaining an access token for the sender user + +In order for the sender user created above to be able to send messages to the room, we'll need to obtain an access token for it. + +To do this, you can execute a command like this: + +``` +curl \ +--data '{"identifier": {"type": "m.id.user", "user": "email2matrix" }, "password": "MATRIX_PASSWORD_FOR_THE_USER", "type": "m.login.password", "device_id": "Email2Matrix", "initial_device_display_name": "Email2Matrix"}' \ +https://matrix.DOMAIN/_matrix/client/r0/login +``` + +Take note of the `access_token` value. You'll need the access token when doing [Configuration](#configuration) below. + + +## Configuration + +After doing the preparation steps above, adjust your `inventory/host_vars/matrix.DOMAIN/vars.yml` configuration like this: + +```yaml +matrix_email2matrix_enabled: true + +matrix_email2matrix_matrix_mappings: + - MailboxName: "my-mailbox" + MatrixRoomId: "!someRoom:DOMAIN" + MatrixHomeserverUrl: "https://matrix.DOMAIN" + MatrixUserId: "@email2matrix:DOMAIN" + MatrixAccessToken: "ACCESS_TOKEN_GOES_HERE" + IgnoreSubject: false + IgnoreBody: false + SkipMarkdown: false + + - MailboxName: "my-mailbox2" + MatrixRoomId: "!anotherRoom:DOMAIN" + MatrixHomeserverUrl: "https://matrix.DOMAIN" + MatrixUserId: "@email2matrix:DOMAIN" + MatrixAccessToken: "ACCESS_TOKEN_GOES_HERE" + IgnoreSubject: true + IgnoreBody: false + SkipMarkdown: true +``` + +You can also set `MatrixHomeserverUrl` to `http://matrix-synapse:8008`, instead of the public `https://matrix.DOMAIN`. +However, that's more likely to break in the future if you switch to another server implementation than Synapse. + +Re-run the playbook (`--tags=setup-email2matrix,start`) and try sending an email to `my-mailbox@matrix.DOMAIN`. diff --git a/docs/configuring-playbook.md b/docs/configuring-playbook.md index 4943415c..5c0e665a 100644 --- a/docs/configuring-playbook.md +++ b/docs/configuring-playbook.md @@ -80,3 +80,5 @@ When you're done with all the configuration you'd like to do, continue with [Ins - [Setting up Appservice IRC bridging](configuring-playbook-bridge-appservice-irc.md) (optional) - [Setting up Appservice Discord bridging](configuring-playbook-bridge-appservice-discord.md) (optional) + +- [Setting up Email2Matrix](configuring-playbook-email2matrix.md) (optional) diff --git a/group_vars/matrix_servers b/group_vars/matrix_servers index 503a45be..647d930c 100755 --- a/group_vars/matrix_servers +++ b/group_vars/matrix_servers @@ -263,6 +263,21 @@ matrix_dimension_container_http_host_bind_port: "{{ '' if matrix_nginx_proxy_ena ###################################################################### +###################################################################### +# +# matrix-email2matrix +# +###################################################################### + +matrix_email2matrix_enabled: false + +###################################################################### +# +# /matrix-email2matrix +# +###################################################################### + + ###################################################################### # diff --git a/roles/matrix-email2matrix/defaults/main.yml b/roles/matrix-email2matrix/defaults/main.yml new file mode 100644 index 00000000..2aece19d --- /dev/null +++ b/roles/matrix-email2matrix/defaults/main.yml @@ -0,0 +1,43 @@ +matrix_email2matrix_enabled: true + +matrix_email2matrix_base_path: "{{ matrix_base_data_path }}/email2matrix" +matrix_email2matrix_config_dir_path: "{{ matrix_email2matrix_base_path }}/config" + +matrix_email2matrix_docker_image: "devture/email2matrix:1.0" +matrix_email2matrix_docker_image_force_pull: "{{ matrix_email2matrix_docker_image.endswith(':latest') }}" + +# A list of extra arguments to pass to the container +matrix_email2matrix_container_extra_arguments: [] + +# List of systemd services that matrix-corporal.service depends on +matrix_email2matrix_systemd_required_services_list: ['docker.service'] + +# Controls where the matrix-email2matrix container exposes the SMTP (tcp/2525 in the container). +# +# Takes an ":" or "" value (e.g. "127.0.0.1:2525"). +# +# By default, we listen on port 25 on all of the host's network interfaces. +matrix_email2matrix_smtp_host_bind_port: "25" + +matrix_email2matrix_smtp_hostname: "{{ matrix_server_fqn_matrix }}" + +# A list of mailbox to Matrix mappings. +# +# Example: +# matrix_email2matrix_matrix_mappings: +# - MailboxName: "mailbox1" +# MatrixRoomId: "!bpcwlxIUxVvvgXcbjy:example.com" +# MatrixHomeserverUrl: "{{ matrix_homeserver_url }}" +# MatrixUserId": "@email2matrix:{{ matrix_domain }}" +# MatrixAccessToken": "TOKEN_HERE" +# IgnoreSubject: false +# +# - MailboxName: "mailbox2" +# MatrixRoomId: "!another:example.com" +# MatrixHomeserverUrl: "{{ matrix_homeserver_url }}" +# MatrixUserId": "@email2matrix:{{ matrix_domain }}" +# MatrixAccessToken": "TOKEN_HERE" +# IgnoreSubject: true +matrix_email2matrix_matrix_mappings: [] + +matrix_email2matrix_misc_debug: false diff --git a/roles/matrix-email2matrix/tasks/init.yml b/roles/matrix-email2matrix/tasks/init.yml new file mode 100644 index 00000000..007a5476 --- /dev/null +++ b/roles/matrix-email2matrix/tasks/init.yml @@ -0,0 +1,3 @@ +- set_fact: + matrix_systemd_services_list: "{{ matrix_systemd_services_list + ['matrix-email2matrix'] }}" + when: matrix_email2matrix_enabled|bool diff --git a/roles/matrix-email2matrix/tasks/main.yml b/roles/matrix-email2matrix/tasks/main.yml new file mode 100644 index 00000000..23114673 --- /dev/null +++ b/roles/matrix-email2matrix/tasks/main.yml @@ -0,0 +1,15 @@ +- import_tasks: "{{ role_path }}/tasks/init.yml" + tags: + - always + +- import_tasks: "{{ role_path }}/tasks/validate_config.yml" + when: "run_setup|bool and matrix_email2matrix_enabled|bool" + tags: + - setup-all + - setup-email2matrix + +- import_tasks: "{{ role_path }}/tasks/setup_email2matrix.yml" + when: run_setup|bool + tags: + - setup-all + - setup-email2matrix diff --git a/roles/matrix-email2matrix/tasks/setup_email2matrix.yml b/roles/matrix-email2matrix/tasks/setup_email2matrix.yml new file mode 100644 index 00000000..9d9109e3 --- /dev/null +++ b/roles/matrix-email2matrix/tasks/setup_email2matrix.yml @@ -0,0 +1,88 @@ +--- + +# +# Tasks related to setting up Email2Matrix +# + +- name: Ensure Email2Matrix paths exist + file: + path: "{{ item }}" + state: directory + mode: 0750 + owner: "{{ matrix_user_username }}" + group: "{{ matrix_user_username }}" + with_items: + - "{{ matrix_email2matrix_base_path }}" + - "{{ matrix_email2matrix_config_dir_path }}" + when: matrix_email2matrix_enabled|bool + +- name: Ensure Email2Matrix configuration file created + template: + src: "{{ role_path }}/templates/config.json.j2" + dest: "{{ matrix_email2matrix_config_dir_path }}/config.json" + owner: "{{ matrix_user_username }}" + group: "{{ matrix_user_username }}" + mode: 0640 + when: matrix_email2matrix_enabled|bool + +- name: Ensure Email2Matrix image is pulled + docker_image: + name: "{{ matrix_email2matrix_docker_image }}" + source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}" + force_source: "{{ matrix_email2matrix_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}" + force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_email2matrix_docker_image_force_pull }}" + when: matrix_email2matrix_enabled|bool + +- name: Ensure matrix-email2matrix.service installed + template: + src: "{{ role_path }}/templates/systemd/matrix-email2matrix.service.j2" + dest: "/etc/systemd/system/matrix-email2matrix.service" + mode: 0644 + register: matrix_email2matrix_systemd_service_result + when: matrix_email2matrix_enabled|bool + +- name: Ensure systemd reloaded after matrix-email2matrix.service installation + service: + daemon_reload: yes + when: "matrix_email2matrix_enabled|bool and matrix_email2matrix_systemd_service_result.changed" + +# +# Tasks related to getting rid of the Email2Matrix (if it was previously enabled) +# + +- name: Check existence of matrix-email2matrix service + stat: + path: "/etc/systemd/system/matrix-email2matrix.service" + register: matrix_email2matrix_service_stat + when: "not matrix_email2matrix_enabled|bool" + +- name: Ensure matrix-email2matrix is stopped + service: + name: matrix-email2matrix + state: stopped + daemon_reload: yes + register: stopping_result + when: "not matrix_email2matrix_enabled|bool and matrix_email2matrix_service_stat.stat.exists" + +- name: Ensure matrix-email2matrix.service doesn't exist + file: + path: "/etc/systemd/system/matrix-email2matrix.service" + state: absent + when: "not matrix_email2matrix_enabled|bool and matrix_email2matrix_service_stat.stat.exists" + +- name: Ensure systemd reloaded after matrix-email2matrix.service removal + service: + daemon_reload: yes + when: "not matrix_email2matrix_enabled|bool and matrix_email2matrix_service_stat.stat.exists" + +- name: Ensure Email2Matrix data path doesn't exist + file: + path: "{{ matrix_email2matrix_base_path }}" + state: absent + when: "not matrix_email2matrix_enabled|bool" + +- name: Ensure Email2Matrix Docker image doesn't exist + docker_image: + name: "{{ matrix_email2matrix_docker_image }}" + state: absent + when: "not matrix_email2matrix_enabled|bool" diff --git a/roles/matrix-email2matrix/tasks/validate_config.yml b/roles/matrix-email2matrix/tasks/validate_config.yml new file mode 100644 index 00000000..d8beecf4 --- /dev/null +++ b/roles/matrix-email2matrix/tasks/validate_config.yml @@ -0,0 +1,7 @@ +--- + +- name: Fail if no mappings + fail: + msg: > + You need to define at least one mapping in `matrix_email2matrix_matrix_mappings` for enabling Email2Matrix. + when: "matrix_email2matrix_matrix_mappings|length == 0" diff --git a/roles/matrix-email2matrix/templates/config.json.j2 b/roles/matrix-email2matrix/templates/config.json.j2 new file mode 100644 index 00000000..c1be97fd --- /dev/null +++ b/roles/matrix-email2matrix/templates/config.json.j2 @@ -0,0 +1,14 @@ +#jinja2: lstrip_blocks: "True" +{ + "Smtp": { + "ListenInterface": "0.0.0.0:2525", + "Hostname": {{ matrix_email2matrix_smtp_hostname|to_json }}, + "Workers": 10 + }, + "Matrix": { + "Mappings": {{ matrix_email2matrix_matrix_mappings|to_nice_json }} + }, + "Misc": { + "Debug": {{ matrix_email2matrix_misc_debug|to_json }} + } +} diff --git a/roles/matrix-email2matrix/templates/systemd/matrix-email2matrix.service.j2 b/roles/matrix-email2matrix/templates/systemd/matrix-email2matrix.service.j2 new file mode 100644 index 00000000..eb7e125b --- /dev/null +++ b/roles/matrix-email2matrix/templates/systemd/matrix-email2matrix.service.j2 @@ -0,0 +1,32 @@ +#jinja2: lstrip_blocks: "True" +[Unit] +Description=Email2Matrix +After=docker.service +Requires=docker.service + +[Service] +Type=simple +ExecStartPre=-/usr/bin/docker kill matrix-email2matrix +ExecStartPre=-/usr/bin/docker rm matrix-email2matrix + +ExecStart=/usr/bin/docker run --rm --name matrix-email2matrix \ + --log-driver=none \ + --user={{ matrix_user_uid }}:{{ matrix_user_gid }} \ + --cap-drop=ALL \ + --read-only \ + --network={{ matrix_docker_network }} \ + -p {{ matrix_email2matrix_smtp_host_bind_port }}:2525 \ + --mount type=bind,src={{ matrix_email2matrix_config_dir_path }}/config.json,dst=/config.json,ro \ + {% for arg in matrix_email2matrix_container_extra_arguments %} + {{ arg }} \ + {% endfor %} + {{ matrix_email2matrix_docker_image }} + +ExecStop=-/usr/bin/docker kill matrix-email2matrix +ExecStop=-/usr/bin/docker rm matrix-email2matrix +Restart=always +RestartSec=30 +SyslogIdentifier=matrix-email2matrix + +[Install] +WantedBy=multi-user.target diff --git a/setup.yml b/setup.yml index 341840a2..e190bae3 100755 --- a/setup.yml +++ b/setup.yml @@ -17,6 +17,7 @@ - matrix-riot-web - matrix-mxisd - matrix-dimension + - matrix-email2matrix - matrix-nginx-proxy - matrix-coturn - matrix-common-after From 847f887e1b8341bfa81d2657a50700fcbdb67f4e Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Mon, 5 Aug 2019 13:12:42 +0300 Subject: [PATCH 2/5] Update README --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index 3f35081d..e0c2506b 100644 --- a/README.md +++ b/README.md @@ -44,6 +44,8 @@ Using this playbook, you can get the following services configured on your serve - (optional) the [matrix-appservice-discord](https://github.com/Half-Shot/matrix-appservice-discord) bridge for bridging your Matrix server to [Discord](https://discordapp.com/) +- (optional) [Email2Matrix](https://github.com/devture/email2matrix) for relaying email messages to Matrix rooms + - (optional) [Dimension](https://github.com/turt2live/matrix-dimension), an open source integrations manager for matrix clients Basically, this playbook aims to get you up-and-running with all the basic necessities around Matrix, without you having to do anything else. @@ -108,6 +110,8 @@ This playbook sets up your server using the following Docker images: - [devture/exim-relay](https://hub.docker.com/r/devture/exim-relay/) - the [Exim](https://www.exim.org/) email server (optional) +- [devture/email2matrix](https://hub.docker.com/r/devture/email2matrix/) - the [Email2Matrix](https://github.com/devture/email2matrix) email server, which can relay email messages to Matrix rooms (optional) + - [devture/matrix-corporal](https://hub.docker.com/r/devture/matrix-corporal/) - [Matrix Corporal](https://github.com/devture/matrix-corporal): reconciliator and gateway for a managed Matrix server (optional) - [nginx](https://hub.docker.com/_/nginx/) - the [nginx](http://nginx.org/) web server (optional) From d2226401404617cf1de67689c1bedff110d1b787 Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Mon, 5 Aug 2019 14:10:24 +0300 Subject: [PATCH 3/5] Add firewall notice about email2matrix --- docs/configuring-playbook-email2matrix.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/configuring-playbook-email2matrix.md b/docs/configuring-playbook-email2matrix.md index cac1782d..b10d5c9e 100644 --- a/docs/configuring-playbook-email2matrix.md +++ b/docs/configuring-playbook-email2matrix.md @@ -9,7 +9,7 @@ See the project's [documentation](https://github.com/devture/email2matrix/blob/m ### Port availability -Ensure that port 25 is available on your Matrix server. +Ensure that port 25 is available on your Matrix server and open in your firewall. If you have `postfix` or some other email server software installed, you may need to manually remove it first (unless you need it, of course). If you really need to run an email server on the Matrix machine for other purposes, it may be possible to run Email2Matrix on another port (with a configuration like `matrix_email2matrix_smtp_host_bind_port: "127.0.0.01:2525"`) and have your other email server relay messages there. From bce165f24756b6129d809262157652aa17bf34db Mon Sep 17 00:00:00 2001 From: Slavi Pantaleev Date: Tue, 6 Aug 2019 07:14:52 +0300 Subject: [PATCH 4/5] Do not suggest DEBUG logging when submitting issues It's been pointed out that DEBUG logs could contain sensitive information (access tokens, etc.), which makes them unsuitable for sharing with others. INFO should be enough. --- docs/maintenance-and-troubleshooting.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/maintenance-and-troubleshooting.md b/docs/maintenance-and-troubleshooting.md index ff1f9e49..18bfdd83 100644 --- a/docs/maintenance-and-troubleshooting.md +++ b/docs/maintenance-and-troubleshooting.md @@ -21,14 +21,14 @@ sudo journalctl -fu matrix-synapse Because the [Synapse](https://github.com/matrix-org/synapse) Matrix server is originally very chatty when it comes to logging, we intentionally reduce its [logging level](https://docs.python.org/3/library/logging.html#logging-levels) from `INFO` to `WARNING`. -If you'd like to debug an issue or [report a Synapse bug](https://github.com/matrix-org/synapse/issues/new/choose) to the developers, it'd be better if you temporarily increasing the logging level to `INFO` or even `DEBUG`. +If you'd like to debug an issue or [report a Synapse bug](https://github.com/matrix-org/synapse/issues/new/choose) to the developers, it'd be better if you temporarily increasing the logging level to `INFO`. Example configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`): ```yaml -matrix_synapse_log_level: "DEBUG" -matrix_synapse_storage_sql_log_level: "DEBUG" -matrix_synapse_root_log_level: "DEBUG" +matrix_synapse_log_level: "INFO" +matrix_synapse_storage_sql_log_level: "INFO" +matrix_synapse_root_log_level: "INFO" ``` Re-run the playbook after making these configuration changes. From dc714571322b8f34c805f9e6422fdd9027d52fb1 Mon Sep 17 00:00:00 2001 From: Dan Arnfield Date: Tue, 6 Aug 2019 06:36:14 -0500 Subject: [PATCH 5/5] Update riot-web (1.3.0 -> 1.3.2) --- roles/matrix-riot-web/defaults/main.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/matrix-riot-web/defaults/main.yml b/roles/matrix-riot-web/defaults/main.yml index 4130c8a0..7a4421b7 100644 --- a/roles/matrix-riot-web/defaults/main.yml +++ b/roles/matrix-riot-web/defaults/main.yml @@ -1,6 +1,6 @@ matrix_riot_web_enabled: true -matrix_riot_web_docker_image: "bubuntux/riot-web:v1.3.0" +matrix_riot_web_docker_image: "bubuntux/riot-web:v1.3.2" matrix_riot_web_docker_image_force_pull: "{{ matrix_riot_web_docker_image.endswith(':latest') }}" matrix_riot_web_data_path: "{{ matrix_base_data_path }}/riot-web"