## (Potential Backward Compatibility Break) Major changes to Synapse workers
## (Potential Backward Compatibility Break) Major improvements to Synapse workers
People who are interested in running a Synapse worker setup should know that **our Synapse worker implementation is much more powerful now**:
- we've added support for [Stream writers](#stream-writers-support)
- we've added support for [multiple federation sender workers](#multiple-federation-sender-workers-support)
- we've added support for [multiple pusher workers](#multiple-pusher-workers-support)
- we've restored support for [`appservice` workers](#appservice-worker-support-is-back)
- we've restored support for [`user_dir` workers](#user-directory-worker-support-is-back)
- see the [Potential Backward Incompatibilities after these Synapse worker changes](#potential-backward-incompatibilities-after-these-synapse-worker-changes)
### Stream writers support
@ -38,6 +40,25 @@ From now on, you can have as many as you want to help with your federation traff
Until now, we only supported a single `pusher` worker (`matrix_synapse_workers_pusher_workers_count` could either be `0` or `1`).
From now on, you can have as many as you want to help with pushing notifications out.
### Appservice worker support is back
We previously had an `appservice` worker type, which [Synapse deprecated in v1.59.0](https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types). So did we, at the time.
The new way to implement such workers is by using a `generic_worker` and dedicating it to the task of talking to Application Services.
From now on, we have support for this.
With `matrix_synapse_workers_preset: one-of-each`, you'll get one `appserice` worker automatically.
You can also control the `appserice` workers count with `matrix_synapse_workers_appserice_workers_count`. Only `0` or `1` workers of this type are supported by Synapse.
### User Directory worker support is back
We previously had a `user_dir` worker type, which [Synapse deprecated in v1.59.0](https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types). So did we, at the time.
The new way to implement such workers is by using a `generic_worker` and dedicating it to the task of serving the user directory.
From now on, we have support for this.
With `matrix_synapse_workers_preset: one-of-each`, you'll get one `user_dir` worker automatically.
You can also control the `user_dir` workers count with `matrix_synapse_workers_user_dir_workers_count`. Only `0` or `1` workers of this type are supported by Synapse.
### Potential Backward Incompatibilities after these Synapse worker changes
- **Worker names** (container names, systemd services, worker configuration files) **have changed**. Workers are now labeled sequentially (e.g. `matrix-synapse-worker_generic_worker-18111` -> `matrix-synapse-worker-generic-0`). The playbook will handle these changes automatically.
- Due to increased worker types support above, people who use `matrix_synapse_workers_preset: one-of-each` should be aware that with these changes, **the playbook will deploy 8 additional workers** (6 stream writers, 1 `appservice` worker, 1 `user_dir` worker). This **may increase RAM/CPU usage**, etc. If you find your server struggling, consider disabling some workers with the appropriate `matrix_synapse_workers_*_workers_count` variables.
- **Metric endpoints have also changed** (`/metrics/synapse/worker/generic_worker-18111` -> `/metrics/synapse/worker/generic-worker-0`). If you're [collecting metrics to an external Prometheus server](docs/configuring-playbook-prometheus-grafana.md#collecting-metrics-to-an-external-prometheus-server), consider revisiting our [Collecting Synapse worker metrics to an external Prometheus server](docs/configuring-playbook-prometheus-grafana.md#collecting-synapse-worker-metrics-to-an-external-prometheus-server) docs and updating your Prometheus configuration. **If you're collecting metrics to the integrated Prometheus server** (not enabled by default), **your Prometheus configuration will be updated automatically**. Old data (from before this change) may stick around though.
- **the format of `matrix_synapse_workers_enabled_list` has changed**. You were never advised to use this variable for directly creating workers (we advise people to control workers using `matrix_synapse_workers_preset` or by tweaking `matrix_synapse_workers_*_workers_count` variables only), but some people may have started using the `matrix_synapse_workers_enabled_list` variable to gain more control over workers. If you're one of them, you'll need to adjust its value. See `roles/matrix-synapse/defaults/main.yml` for more information on the new format. The playbook will also do basic validation and complain if you got something wrong.
# matrix_synapse_workers_appservice_workers_count can only be 0 or 1. More instances are not supported.
# appservice workers were deprecated since Synapse v1.59 (see: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types).
# Our implementation uses generic worker services and assigns them to perform appservice work using the `notify_appservices_from_worker` Synapse option.
# matrix_synapse_notify_appservices_from_worker populates the `notify_appservices_from_worker` Synapse configuration used when Synapse workers are in use (`matrix_synapse_workers_enabled`).
# `notify_appservices_from_worker` is meant to point to a worker, which is dedicated to sending output traffic to Application Services.
matrix_synapse_notify_appservices_from_worker:"{{ (matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'appservice') | list)[0].name if (matrix_synapse_workers_enabled and matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'appservice') | list | length > 0) else '' }}"
# matrix_synapse_workers_user_dir_workers_count can only be 0 or 1. More instances are not supported.
# user_dir workers were deprecated since Synapse v1.59 (see: https://github.com/matrix-org/synapse/blob/v1.59.0/docs/upgrade.md#deprecation-of-the-synapseappappservice-and-synapseappuser_dir-worker-application-types).
# Our implementation uses generic worker services and assigns them to perform appservice work using the `update_user_directory_from_worker` Synapse option.
# matrix_synapse_update_user_directory_from_worker populates the `update_user_directory_from_worker` Synapse configuration used when Synapse workers are in use (`matrix_synapse_workers_enabled`).
# `update_user_directory_from_worker` is meant to point to a worker, which is dedicated to updating the user directory and servicing some user directory URL endpoints (`matrix_synapse_workers_user_dir_worker_client_server_endpoints`).
matrix_synapse_update_user_directory_from_worker:"{{ (matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'user_dir') | list)[0].name if (matrix_synapse_workers_enabled and matrix_synapse_workers_enabled_list | selectattr('type', 'equalto', 'user_dir') | list | length > 0) else '' }}"
# Default list of workers to spawn.
#
# Unless you populate this manually, this list is dynamically generated
# matrix_synapse_workers_user_dir_worker_client_server_endpoints contains the endpoints serviced by the `type = user_dir` (`app = generic_worker`) worker.
# matrix_synapse_known_worker_types contains the list of known worker types.
#
# A worker type is different than a worker app (e.g. `generic_worker`).
# For example, the `stream_writer` worker type is served by the `generic_worker` app, but is a separate type that we recognize.
#
# Some other types (`appservice` and `user_dir`) used to be Synapse worker apps, which got subsequently deprecated.
# We still allow these types of workers and map them to the `generic_worker` app,
# which is why we make sure they're part of the list below.
# We use the `unique` filter because they're part of `matrix_synapse_workers_avail_list` too (for now; scheduled for removal).
matrix_synapse_known_worker_types:|
{{
matrix_synapse_workers_avail_list
+
['stream_writer']
(
matrix_synapse_workers_avail_list
+
['stream_writer']
+
['appservice']
+
['user_dir']
) | unique
}}
# matrix_synapse_known_instance_map_eligible_worker_types contains the list of worker types that are to be injected into `matrix_synapse_instance_map`.
Slavi Pantaleev
2 years ago