Upgrade mautrix-facebook to the new Postgres-only version

I had intentionally held it back in 39ea3496a4
until:
- it received more testing (there were a few bugs during the
migration, but now it seems OK)
- this migration guide was written
development
Slavi Pantaleev 4 years ago
parent 9be9f43be7
commit 024a23ed17

@ -1,3 +1,63 @@
# 2021-01-20
## (Breaking Change) The mautrix-facebook bridge now requires a Postgres database
A new version of the [mautrix-facebook](https://github.com/tulir/mautrix-facebook) bridge has been released. It's a full rewrite of its backend and the bridge now requires Postgres. New versions of the bridge can no longer run on SQLite.
**TLDR**: if you're NOT using an [external Postgres server](docs/configuring-playbook-external-postgres.md) and have NOT forcefully kept the bridge on SQLite during [The big move to all-on-Postgres (potentially dangerous)](#the-big-move-to-all-on-postgres-potentially-dangerous), you will be automatically upgraded without manual intervention. All you need to do is send a `login` message to the Facebook bridge bot again.
Whether this change requires your intervention depends mostly on:
- whether you're using an [external Postgres server](docs/configuring-playbook-external-postgres.md). If yes, then [you need to do something](#upgrade-path-for-people-running-an-external-postgres-server).
- or whether you've force-changed the bridge's database engine to SQLite (`matrix_mautrix_facebook_database_engine: 'sqlite'` in your `vars.yml`) some time in the past (likely during [The big move to all-on-Postgres (potentially dangerous)](#the-big-move-to-all-on-postgres-potentially-dangerous)).
As already mentioned above, you most likely don't need to do anything. If you rerun the playbook and don't get an error, you've been automatically upgraded. Just send a `login` message to the Facebook bridge bot again. Otherwise, read below for a solution.
### Upgrade path for people NOT running an external Postgres server (default for the playbook)
If you're **not running an external Postgres server**, then this bridge either already works Postgres for you, or you've intentionally kept it back on SQLite with custom configuration (`matrix_mautrix_facebook_database_engine: 'sqlite'` in your `vars.yml`) .
Simply remove that custom configuration from your `vars.yml` file (if it's there) and re-run the playbook. It should upgrade you automatically.
You'll need to send a `login` message to the Facebook bridge bot again.
Alternatively, [you can stay on SQLite for a little longer](#staying-on-sqlite-for-a-little-longer-temporary-solution).
### Upgrade path for people running an external Postgres server
For people using the internal Postgres server (the default for the playbook):
- we automatically create an additional `matrix_mautrix_facebook` Postgres database and credentials to access it
- we automatically adjust the bridge's `matrix_mautrix_facebook_database_*` variables to point the bridge to that Postgres database
- we use [pgloader](https://pgloader.io/) to automatically import the existing SQLite data for the bridge into the `matrix_mautrix_facebook` Postgres database
If you are using an [external Postgres server](docs/configuring-playbook-external-postgres.md), unfortunately we currently can't do any of that for you.
You have 3 ways to proceed:
- contribute to the playbook to make this possible (difficult)
- or, do the above "steps" manually:
- stop the bridge (`systemctl stop matrix-mautrix-facebook`)
- create a new `matrix_mautrix_facebook` Postgres database for it
- run pgloader manually (we run it with default settings for this bridge)
- adjust the `matrix_mautrix_facebook_database_*` database variables (credentials, etc.)
- switch the bridge to use Postgres (`matrix_mautrix_facebook_database_engine: 'postgres'`)
- re-run the playbook (`--tags=setup-all,start`) and ensure the bridge works (`systemctl status matrix-mautrix-facebook` and `journalctl -fu matrix-mautrix-facebook`)
- send a `login` message to the Facebook bridge bot again
- or, [stay on SQLite for a little longer (temporary solution)](#staying-on-sqlite-for-a-little-longer-temporary-solution)
### Staying on SQLite for a little longer (temporary solution)
To keep using this bridge with SQLite for a little longer (**not recommended**), use the following configuration in your `vars.yml` file:
```yaml
# Force-change the database engine to SQLite.
matrix_mautrix_facebook_database_engine: 'sqlite'
# Force-downgrade to the last bridge version which supported SQLite.
matrix_mautrix_facebook_docker_image: "{{ matrix_mautrix_facebook_docker_image_name_prefix }}tulir/mautrix-facebook:da1b4ec596e334325a1589e70829dea46e73064b"
```
If you do this, keep in mind that **you can't run this forever**. This SQLite-supporting bridge version is not getting any updates and will break sooner or later. The playbook will also drop support for SQLite at some point in the future.
# 2021-01-17 # 2021-01-17
## matrix-corporal goes 2.0 ## matrix-corporal goes 2.0

@ -208,7 +208,8 @@ matrix_mautrix_facebook_login_shared_secret: "{{ matrix_synapse_ext_password_pro
matrix_mautrix_facebook_bridge_presence: "{{ matrix_synapse_use_presence if matrix_synapse_enabled else true }}" matrix_mautrix_facebook_bridge_presence: "{{ matrix_synapse_use_presence if matrix_synapse_enabled else true }}"
# Postgres is the default, except if not using `matrix_postgres` (internal postgres) # We'd like to force-set people with external Postgres to SQLite, so the bridge role can complain
# and point them to a migration path.
matrix_mautrix_facebook_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}" matrix_mautrix_facebook_database_engine: "{{ 'postgres' if matrix_postgres_enabled else 'sqlite' }}"
matrix_mautrix_facebook_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mau.fb.db') | to_uuid }}" matrix_mautrix_facebook_database_password: "{{ matrix_synapse_macaroon_secret_key | password_hash('sha512', 'mau.fb.db') | to_uuid }}"

@ -7,7 +7,7 @@ matrix_mautrix_facebook_container_image_self_build: false
matrix_mautrix_facebook_container_image_self_build_repo: "https://github.com/tulir/mautrix-facebook.git" matrix_mautrix_facebook_container_image_self_build_repo: "https://github.com/tulir/mautrix-facebook.git"
# See: https://mau.dev/tulir/mautrix-facebook/container_registry # See: https://mau.dev/tulir/mautrix-facebook/container_registry
matrix_mautrix_facebook_docker_image: "{{ matrix_mautrix_facebook_docker_image_name_prefix }}tulir/mautrix-facebook:da1b4ec596e334325a1589e70829dea46e73064b" matrix_mautrix_facebook_docker_image: "{{ matrix_mautrix_facebook_docker_image_name_prefix }}tulir/mautrix-facebook:latest"
matrix_mautrix_facebook_docker_image_name_prefix: "{{ 'localhost/' if matrix_mautrix_facebook_container_image_self_build else 'dock.mau.dev/' }}" matrix_mautrix_facebook_docker_image_name_prefix: "{{ 'localhost/' if matrix_mautrix_facebook_container_image_self_build else 'dock.mau.dev/' }}"
matrix_mautrix_facebook_docker_image_force_pull: "{{ matrix_mautrix_facebook_docker_image.endswith(':latest') }}" matrix_mautrix_facebook_docker_image_force_pull: "{{ matrix_mautrix_facebook_docker_image.endswith(':latest') }}"
@ -35,12 +35,15 @@ matrix_mautrix_facebook_homeserver_token: ''
# Database-related configuration fields. # Database-related configuration fields.
# #
# To use SQLite, stick to these defaults. # To use SQLite:
# - change the engine (`matrix_mautrix_facebook_database_engine: 'sqlite'`)
# - change to the last bridge version that supported SQLite:
# `matrix_mautrix_facebook_docker_image: "{{ matrix_mautrix_facebook_docker_image_name_prefix }}tulir/mautrix-facebook:da1b4ec596e334325a1589e70829dea46e73064b"`
# - plan your migration to Postgres, as this bridge does not support SQLite anymore (and neither will the playbook in the future).
# #
# To use Postgres: # To use Postgres:
# - change the engine (`matrix_mautrix_facebook_database_engine: 'postgres'`)
# - adjust your database credentials via the `matrix_mautrix_facebook_postgres_*` variables # - adjust your database credentials via the `matrix_mautrix_facebook_postgres_*` variables
matrix_mautrix_facebook_database_engine: 'sqlite' matrix_mautrix_facebook_database_engine: 'postgres'
matrix_mautrix_facebook_sqlite_database_path_local: "{{ matrix_mautrix_facebook_data_path }}/mautrix-facebook.db" matrix_mautrix_facebook_sqlite_database_path_local: "{{ matrix_mautrix_facebook_data_path }}/mautrix-facebook.db"
matrix_mautrix_facebook_sqlite_database_path_in_container: "/data/mautrix-facebook.db" matrix_mautrix_facebook_sqlite_database_path_in_container: "/data/mautrix-facebook.db"

@ -8,3 +8,24 @@
with_items: with_items:
- "matrix_mautrix_facebook_appservice_token" - "matrix_mautrix_facebook_appservice_token"
- "matrix_mautrix_facebook_homeserver_token" - "matrix_mautrix_facebook_homeserver_token"
- block:
- name: Fail if on SQLite, unless on the last version supporting SQLite
fail:
msg: >-
You're trying to use the mautrix-facebook bridge with an SQLite database.
Going forward, this bridge only supports Postgres.
To learn more about this, see our changelog: https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/CHANGELOG.md#breaking-change-the-mautrix-facebook-bridge-now-requires-a-postgres-database
when: "not matrix_mautrix_facebook_docker_image.endswith(':da1b4ec596e334325a1589e70829dea46e73064b')"
- name: Inject warning if still on SQLite
set_fact:
matrix_playbook_runtime_results: |
{{
matrix_playbook_runtime_results|default([])
+
[
"NOTE: Your mautrix-facebook bridge setup is still on SQLite. Your bridge is not getting any updates and will likely stop working at some point. To learn more about this, see our changelog: https://github.com/spantaleev/matrix-docker-ansible-deploy/blob/master/CHANGELOG.md#breaking-change-the-mautrix-facebook-bridge-now-requires-a-postgres-database"
]
}}
when: "matrix_mautrix_facebook_database_engine == 'sqlite'"

@ -8,6 +8,10 @@ homeserver:
# Whether or not to verify the SSL certificate of the homeserver. # Whether or not to verify the SSL certificate of the homeserver.
# Only applies if address starts with https:// # Only applies if address starts with https://
verify_ssl: true verify_ssl: true
# Whether or not the homeserver supports asmux-specific endpoints,
# such as /_matrix/client/unstable/net.maunium.asmux/dms for atomically
# updating m.direct.
asmux: false
# Application service host/registration related details # Application service host/registration related details
# Changing these values requires regeneration of the registration. # Changing these values requires regeneration of the registration.
@ -22,11 +26,7 @@ appservice:
# Usually 1 is enough, but on high-traffic bridges you might need to increase this to avoid 413s # Usually 1 is enough, but on high-traffic bridges you might need to increase this to avoid 413s
max_body_size: 1 max_body_size: 1
# The full URI to the database. SQLite and Postgres are fully supported. # The full URI to the database. Only Postgres is currently supported.
# Other DBMSes supported by SQLAlchemy may or may not work.
# Format examples:
# SQLite: sqlite:///filename.db
# Postgres: postgres://username:password@hostname/dbname
database: {{ matrix_mautrix_facebook_appservice_database|to_json }} database: {{ matrix_mautrix_facebook_appservice_database|to_json }}
# Public part of web server for out-of-Matrix interaction with the bridge. # Public part of web server for out-of-Matrix interaction with the bridge.
@ -38,6 +38,10 @@ appservice:
# The base URL where the public-facing endpoints are available. The prefix is not added # The base URL where the public-facing endpoints are available. The prefix is not added
# implicitly. # implicitly.
external: https://example.com/public external: https://example.com/public
# Shared secret for integration managers such as mautrix-manager.
# If set to "generate", a random string will be generated on the next startup.
# If null, integration manager access to the API will not be possible.
shared_secret: generate
# The unique ID of this appservice. # The unique ID of this appservice.
id: facebook id: facebook
@ -46,12 +50,17 @@ appservice:
# Display name and avatar for bot. Set to "remove" to remove display name/avatar, leave empty # Display name and avatar for bot. Set to "remove" to remove display name/avatar, leave empty
# to leave display name/avatar as-is. # to leave display name/avatar as-is.
bot_displayname: Facebook bridge bot bot_displayname: Facebook bridge bot
bot_avatar: mxc://maunium.net/ddtNPZSKMNqaUzqrHuWvUADv bot_avatar: mxc://maunium.net/ygtkteZsXnGJLJHRchUwYWak
# Authentication tokens for AS <-> HS communication. # Authentication tokens for AS <-> HS communication.
as_token: "{{ matrix_mautrix_facebook_appservice_token }}" as_token: "{{ matrix_mautrix_facebook_appservice_token }}"
hs_token: "{{ matrix_mautrix_facebook_homeserver_token }}" hs_token: "{{ matrix_mautrix_facebook_homeserver_token }}"
# Prometheus telemetry config. Requires prometheus-client to be installed.
metrics:
enabled: false
listen_port: 8000
# Bridge config # Bridge config
bridge: bridge:
# Localpart template of MXIDs for Facebook users. # Localpart template of MXIDs for Facebook users.
@ -76,6 +85,7 @@ bridge:
# "own_nickname" (user-specific!) # "own_nickname" (user-specific!)
displayname_preference: displayname_preference:
- name - name
- first_name
# The prefix for commands. Only required in non-management rooms. # The prefix for commands. Only required in non-management rooms.
command_prefix: "!fb" command_prefix: "!fb"
@ -120,6 +130,18 @@ bridge:
# Default to encryption, force-enable encryption in all portals the bridge creates # Default to encryption, force-enable encryption in all portals the bridge creates
# This will cause the bridge bot to be in private chats for the encryption to work properly. # This will cause the bridge bot to be in private chats for the encryption to work properly.
default: false default: false
# Options for automatic key sharing.
key_sharing:
# Enable key sharing? If enabled, key requests for rooms where users are in will be fulfilled.
# You must use a client that supports requesting keys from other users to use this feature.
allow: false
# Require the requesting device to have a valid cross-signing signature?
# This doesn't require that the bridge has verified the device, only that the user has verified it.
# Not yet implemented.
require_cross_signing: false
# Require devices to be verified by the bridge?
# Verification by the bridge is not yet implemented.
require_verification: true
# Whether or not the bridge should send a read receipt from the bridge bot when a message has # Whether or not the bridge should send a read receipt from the bridge bot when a message has
# been sent to Facebook. # been sent to Facebook.
delivery_receipts: false delivery_receipts: false
@ -161,6 +183,10 @@ bridge:
# Whether or not the bridge should try to "refresh" the connection if a normal reconnection # Whether or not the bridge should try to "refresh" the connection if a normal reconnection
# attempt fails. # attempt fails.
refresh_on_reconnection_fail: false refresh_on_reconnection_fail: false
# Set this to true to tell the bridge to re-send m.bridge events to all rooms on the next run.
# This field will automatically be changed back to false after it,
# except if the config file is not writable.
resend_bridge_info: false
# Permissions for using the bridge. # Permissions for using the bridge.
# Permitted values: # Permitted values:
@ -192,9 +218,7 @@ logging:
loggers: loggers:
mau: mau:
level: DEBUG level: DEBUG
fbchat: paho:
level: DEBUG
hbmqtt:
level: INFO level: INFO
aiohttp: aiohttp:
level: INFO level: INFO

Loading…
Cancel
Save