Replace custom/matrix-postgres-backup role with galaxy/com.devture.ansible.role.postgres_backup

This role is usable on its own and it's not tied to Matrix, so
extracting it out into an independent role that we install via
ansible-galaxy makes sense.

This also fixes the confusion from the other day, where
`matrix_postgres_*` had to be renamed to `devture_postgres_*`
(unless it was about `matrix_postgres_backup_*`).
We now can safely say that ALL `matrix_postgres_*` variables need to be
renamed.

Fixes https://github.com/spantaleev/matrix-docker-ansible-deploy/issues/2305
development
Slavi Pantaleev 2 years ago
parent a365e54f4d
commit 4eed49f931

@ -1,3 +1,12 @@
# 2022-11-30
## matrix-postgres-backup has been replaced by the com.devture.ansible.role.postgres_backup external role
Just like we've [replaced Postgres with an external role](#matrix-postgres-has-been-replaced-by-the-comdevtureansiblerolepostgres-external-role) on 2022-11-28, we're now replacing `matrix-postgres-backup` with an external role - [com.devture.ansible.role.postgres_backup](https://github.com/devture/com.devture.ansible.role.postgres_backup).
You'll need to rename your `matrix_postgres_backup`-prefixed variables such that they use a `devture_postgres_backup` prefix.
# 2022-11-28 # 2022-11-28
## matrix-postgres has been replaced by the com.devture.ansible.role.postgres external role ## matrix-postgres has been replaced by the com.devture.ansible.role.postgres external role

@ -1,6 +1,6 @@
# Setting up postgres backup (optional) # Setting up postgres backup (optional)
The playbook can install and configure [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) for you. The playbook can install and configure [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) for you via the [com.devture.ansible.role.postgres_backup](https://github.com/devture/com.devture.ansible.role.postgres_backup) Ansible role.
For a more complete backup solution (one that includes not only Postgres, but also other configuration/data files), you may wish to look into [borg backup](configuring-playbook-backup-borg.md) instead. For a more complete backup solution (one that includes not only Postgres, but also other configuration/data files), you may wish to look into [borg backup](configuring-playbook-backup-borg.md) instead.
@ -10,7 +10,7 @@ For a more complete backup solution (one that includes not only Postgres, but al
Minimal working configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`) to enable Postgres backup: Minimal working configuration (`inventory/host_vars/matrix.DOMAIN/vars.yml`) to enable Postgres backup:
```yaml ```yaml
matrix_postgres_backup_enabled: true devture_postgres_backup_enabled: true
``` ```
Refer to the table below for additional configuration variables and their default values. Refer to the table below for additional configuration variables and their default values.
@ -18,12 +18,13 @@ Refer to the table below for additional configuration variables and their defaul
| Name | Default value | Description | | Name | Default value | Description |
| :-------------------------------- | :--------------------------- | :--------------------------------------------------------------- | | :-------------------------------- | :--------------------------- | :--------------------------------------------------------------- |
|`matrix_postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups| |`devture_postgres_backup_enabled`|`false`|Set to true to use [docker-postgres-backup-local](https://github.com/prodrigestivill/docker-postgres-backup-local) to create automatic database backups|
|`matrix_postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.| |`devture_postgres_backup_schedule`| `'@daily'` |Cron-schedule specifying the interval between postgres backups.|
|`matrix_postgres_backup_keep_days`|`7`|Number of daily backups to keep| |`devture_postgres_backup_keep_days`|`7`|Number of daily backups to keep|
|`matrix_postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep| |`devture_postgres_backup_keep_weeks`|`4`|Number of weekly backups to keep|
|`matrix_postgres_backup_keep_months`|`12`|Number of monthly backups to keep| |`devture_postgres_backup_keep_months`|`12`|Number of monthly backups to keep|
|`matrix_postgres_backup_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Storagepath for the database backups| |`devture_postgres_base_path` | `"{{ matrix_base_data_path }}/postgres-backup"` | Base path for postgres-backup. Also see `devture_postgres_data_path` |
|`devture_postgres_data_path` | `"{{ devture_postgres_base_path }}/data"` | Storage path for postgres-backup database backups |
## Installing ## Installing

@ -289,7 +289,7 @@ devture_systemd_service_manager_services_list_auto: |
+ +
([{'name': (devture_postgres_identifier + '.service'), 'priority': 500, 'groups': ['matrix', 'postgres']}] if devture_postgres_enabled else []) ([{'name': (devture_postgres_identifier + '.service'), 'priority': 500, 'groups': ['matrix', 'postgres']}] if devture_postgres_enabled else [])
+ +
([{'name': 'matrix-postgres-backup.service', 'priority': 3000, 'groups': ['matrix', 'backup', 'postgres-backup']}] if matrix_postgres_backup_enabled else []) ([{'name': (devture_postgres_backup_identifier + '.service'), 'priority': 3000, 'groups': ['matrix', 'backup', 'postgres-backup']}] if devture_postgres_backup_enabled else [])
+ +
([{'name': 'matrix-prometheus.service', 'priority': 4000, 'groups': ['matrix', 'monitoring', 'prometheus', 'prometheus-core']}] if matrix_prometheus_enabled else []) ([{'name': 'matrix-prometheus.service', 'priority': 4000, 'groups': ['matrix', 'monitoring', 'prometheus', 'prometheus-core']}] if matrix_prometheus_enabled else [])
+ +
@ -2241,6 +2241,7 @@ matrix_ssl_pre_obtaining_required_service_name: "{{ 'matrix-dynamic-dns' if matr
# #
###################################################################### ######################################################################
######################################################################## ########################################################################
# # # #
# com.devture.ansible.role.postgres # # com.devture.ansible.role.postgres #
@ -2504,6 +2505,42 @@ devture_postgres_managed_databases_auto: |
# # # #
######################################################################## ########################################################################
########################################################################
# #
# com.devture.ansible.role.postgres_backup #
# #
########################################################################
devture_postgres_backup_enabled: false
devture_postgres_backup_identifier: matrix-postgres-backup
devture_postgres_backup_architecture: "{{ matrix_architecture }}"
devture_postgres_backup_base_path: "{{ matrix_base_data_path }}/postgres-backup"
devture_postgres_backup_container_network: "{{ matrix_docker_network }}"
devture_postgres_backup_uid: "{{ matrix_user_uid }}"
devture_postgres_backup_gid: "{{ matrix_user_gid }}"
devture_postgres_backup_connection_hostname: "{{ devture_postgres_connection_hostname if devture_postgres_enabled else '' }}"
devture_postgres_backup_connection_port: "{{ devture_postgres_connection_port if devture_postgres_enabled else 5432 }}"
devture_postgres_backup_connection_username: "{{ devture_postgres_connection_username if devture_postgres_enabled else '' }}"
devture_postgres_backup_connection_password: "{{ devture_postgres_connection_password if devture_postgres_enabled else '' }}"
devture_postgres_backup_postgres_data_path: "{{ devture_postgres_data_path if devture_postgres_enabled else '' }}"
devture_postgres_backup_databases: "{{ devture_postgres_managed_databases | map(attribute='name') if devture_postgres_enabled else [] }}"
########################################################################
# #
# /com.devture.ansible.role.postgres_backup #
# #
########################################################################
###################################################################### ######################################################################
# #
# matrix-sygnal # matrix-sygnal
@ -2965,26 +3002,6 @@ matrix_registration_database_password: "{{ '%s' | format(matrix_homeserver_gener
# #
###################################################################### ######################################################################
######################################################################
#
# matrix-postgres-backup
#
######################################################################
matrix_postgres_backup_connection_hostname: "{{ devture_postgres_connection_hostname if devture_postgres_enabled else '' }}"
matrix_postgres_backup_connection_port: "{{ devture_postgres_connection_port if devture_postgres_enabled else 5432 }}"
matrix_postgres_backup_connection_username: "{{ devture_postgres_connection_username if devture_postgres_enabled else '' }}"
matrix_postgres_backup_connection_password: "{{ devture_postgres_connection_password if devture_postgres_enabled else '' }}"
matrix_postgres_backup_postgres_data_path: "{{ devture_postgres_data_path if devture_postgres_enabled else '' }}"
matrix_postgres_backup_databases: "{{ devture_postgres_managed_databases | map(attribute='name') if devture_postgres_enabled else [] }}"
######################################################################
#
# /matrix-postgres-backup
#
######################################################################
###################################################################### ######################################################################
# #

@ -97,8 +97,11 @@
- custom/matrix-nginx-proxy - custom/matrix-nginx-proxy
- custom/matrix-coturn - custom/matrix-coturn
- custom/matrix-aux - custom/matrix-aux
- custom/matrix-postgres-backup
- role: galaxy/com.devture.ansible.role.postgres_backup
- custom/matrix-backup-borg - custom/matrix-backup-borg
- custom/matrix-user-creator - custom/matrix-user-creator
- custom/matrix-common-after - custom/matrix-common-after

@ -21,6 +21,9 @@
- src: git+https://github.com/devture/com.devture.ansible.role.postgres.git - src: git+https://github.com/devture/com.devture.ansible.role.postgres.git
version: 381feb7f671c60e8875298102a7c9b41242979ee version: 381feb7f671c60e8875298102a7c9b41242979ee
- src: git+https://github.com/devture/com.devture.ansible.role.postgres_backup.git
version: 77b1f9ae1aafa31c9078178c1036bf744c99d08b
- src: git+https://github.com/devture/com.devture.ansible.role.systemd_service_manager.git - src: git+https://github.com/devture/com.devture.ansible.role.systemd_service_manager.git
version: 6ccb88ac5fc27e1e70afcd48278ade4b564a9096 version: 6ccb88ac5fc27e1e70afcd48278ade4b564a9096

@ -1,45 +0,0 @@
---
# Project source code URL: https://github.com/prodrigestivill/docker-postgres-backup-local
matrix_postgres_backup_enabled: false
matrix_postgres_backup_connection_hostname: ''
matrix_postgres_backup_connection_port: 5432
matrix_postgres_backup_connection_username: "matrix"
matrix_postgres_backup_connection_password: ""
matrix_postgres_backup_extra_opts: "-Z9 --schema=public --blobs"
matrix_postgres_backup_schedule: "@daily"
matrix_postgres_backup_keep_days: 7
matrix_postgres_backup_keep_weeks: 4
matrix_postgres_backup_keep_months: 12
matrix_postgres_backup_healthcheck_port: "8080"
matrix_postgres_backup_databases: []
matrix_postgres_backup_path: "{{ matrix_base_data_path }}/postgres-backup"
# Specifies where the Postgres data is.
# We use this to autodetect the Postgres version during playbook runtime (by parsing the `PG_VERSION` file contained there).
# You can leave this empty to prevent auto-detection.
matrix_postgres_backup_postgres_data_path: ""
matrix_postgres_backup_architecture: amd64
# matrix_postgres_backup_docker_image_distro controls whether we use Alpine-based images (`-alpine`) or the normal Debian-based images.
# Alpine-based Postgres images are smaller and we usually prefer them, but they don't work on ARM32 (tested on a Raspberry Pi 3 running Raspbian 10.7).
# On ARM32, `-alpine` images fail with the following error:
# > LOG: startup process (PID 37) was terminated by signal 11: Segmentation fault
matrix_postgres_backup_docker_image_distro: "{{ 'alpine' if matrix_postgres_backup_architecture in ['amd64', 'arm64'] else 'debian' }}"
matrix_postgres_backup_docker_image_v9: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:9.6-{{ matrix_postgres_backup_docker_image_distro }}-2aa03d1"
matrix_postgres_backup_docker_image_v10: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:10-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
matrix_postgres_backup_docker_image_v11: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:11-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
matrix_postgres_backup_docker_image_v12: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:12-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
matrix_postgres_backup_docker_image_v13: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:13-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
matrix_postgres_backup_docker_image_v14: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:14-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
matrix_postgres_backup_docker_image_v15: "{{ matrix_container_global_registry_prefix }}prodrigestivill/postgres-backup-local:15-{{ matrix_postgres_backup_docker_image_distro }}-2cf00a5"
matrix_postgres_backup_docker_image_latest: "{{ matrix_postgres_backup_docker_image_v15 }}"
# This variable is assigned at runtime. Overriding its value has no effect.
matrix_postgres_backup_docker_image_to_use: '{{ matrix_postgres_backup_docker_image_latest }}'
matrix_postgres_backup_docker_image_force_pull: "{{ matrix_postgres_backup_docker_image_to_use.endswith(':latest') }}"

@ -1,20 +0,0 @@
---
- block:
- when: matrix_postgres_backup_enabled | bool
ansible.builtin.include_tasks: "{{ role_path }}/tasks/validate_config.yml"
- when: matrix_postgres_backup_enabled | bool
ansible.builtin.include_tasks: "{{ role_path }}/tasks/setup_install.yml"
tags:
- setup-all
- setup-postgres-backup
- install-all
- install-postgres-backup
- block:
- when: not matrix_postgres_backup_enabled | bool
ansible.builtin.include_tasks: "{{ role_path }}/tasks/setup_uninstall.yml"
tags:
- setup-all
- setup-postgres-backup

@ -1,51 +0,0 @@
---
- ansible.builtin.include_role:
name: galaxy/com.devture.ansible.role.postgres
tasks_from: detect_existing_postgres_version
when: 'matrix_postgres_backup_postgres_data_path != ""'
# If we have found an existing version (installed from before), we use its corresponding Docker image.
# If not, we install using the latest Postgres.
#
# Upgrading is supposed to be performed separately and explicitly (see `upgrade_postgres.yml`).
- ansible.builtin.set_fact:
matrix_postgres_backup_docker_image_to_use: "{{ matrix_postgres_backup_docker_image_latest if matrix_postgres_backup_detected_version_corresponding_docker_image | default('') == '' else matrix_postgres_backup_detected_version_corresponding_docker_image }}"
- name: Ensure postgres backup Docker image is pulled
community.docker.docker_image:
name: "{{ matrix_postgres_backup_docker_image_to_use }}"
source: "{{ 'pull' if ansible_version.major > 2 or ansible_version.minor > 7 else omit }}"
force_source: "{{ matrix_postgres_backup_docker_image_force_pull if ansible_version.major > 2 or ansible_version.minor >= 8 else omit }}"
force: "{{ omit if ansible_version.major > 2 or ansible_version.minor >= 8 else matrix_postgres_backup_docker_image_force_pull }}"
register: result
retries: "{{ devture_playbook_help_container_retries_count }}"
delay: "{{ devture_playbook_help_container_retries_delay }}"
until: result is not failed
- name: Ensure Postgres backup paths exist
ansible.builtin.file:
path: "{{ item }}"
state: directory
mode: 0700
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
with_items:
- "{{ matrix_postgres_backup_path }}"
- name: Ensure Postgres environment variables file created
ansible.builtin.template:
src: "{{ role_path }}/templates/{{ item }}.j2"
dest: "{{ matrix_postgres_backup_path }}/{{ item }}"
owner: "{{ matrix_user_username }}"
group: "{{ matrix_user_groupname }}"
mode: 0640
with_items:
- "env-postgres-backup"
- name: Ensure matrix-postgres-backup.service installed
ansible.builtin.template:
src: "{{ role_path }}/templates/systemd/matrix-postgres-backup.service.j2"
dest: "{{ devture_systemd_docker_base_systemd_path }}/matrix-postgres-backup.service"
mode: 0644

@ -1,39 +0,0 @@
---
- name: Check existence of matrix-postgres-backup service
ansible.builtin.stat:
path: "{{ devture_systemd_docker_base_systemd_path }}/matrix-postgres-backup.service"
register: matrix_postgres_backup_service_stat
- when: matrix_postgres_backup_service_stat.stat.exists | bool
block:
- name: Ensure matrix-postgres-backup is stopped
ansible.builtin.service:
name: matrix-postgres-backup
state: stopped
enabled: false
daemon_reload: true
- name: Ensure matrix-postgres-backup.service doesn't exist
ansible.builtin.file:
path: "{{ devture_systemd_docker_base_systemd_path }}/matrix-postgres-backup.service"
state: absent
- name: Check existence of matrix-postgres-backup backup path
ansible.builtin.stat:
path: "{{ matrix_postgres_backup_path }}"
register: matrix_postgres_backup_path_stat
# We just want to notify the user. Deleting data is too destructive.
- name: Inject warning if matrix-postgres backup data remains
ansible.builtin.set_fact:
devture_playbook_runtime_messages_list: |
{{
devture_playbook_runtime_messages_list | default([])
+
[
"NOTE: You are not using the local backup service to backup the PostgreSQL database, but some old data remains from before in `{{ matrix_postgres_backup_path }}`. Feel free to delete it."
]
}}
when: matrix_postgres_backup_path_stat.stat.exists | bool

@ -1,27 +0,0 @@
---
- name: Fail if required postgres-backup settings not defined
ansible.builtin.fail:
msg: >-
You need to define a required configuration setting (`{{ item }}`).
when: "vars[item] == ''"
with_items:
- "matrix_postgres_backup_connection_hostname"
- "matrix_postgres_backup_connection_username"
- "matrix_postgres_backup_connection_password"
- "matrix_postgres_backup_connection_port"
- "matrix_postgres_backup_schedule"
- "matrix_postgres_backup_keep_days"
- "matrix_postgres_backup_keep_weeks"
- "matrix_postgres_backup_keep_months"
- "matrix_postgres_backup_path"
- "matrix_postgres_backup_databases"
- name: (Deprecation) Catch and report renamed settings
ansible.builtin.fail:
msg: >-
Your configuration contains a variable, which now has a different name.
Please change your configuration to rename the variable (`{{ item.old }}` -> `{{ item.new }}`).
when: "item.old in vars"
with_items:
- {'old': 'matrix_postgres_backup_docker_image_suffix', 'new': 'matrix_postgres_backup_docker_image_distro'}

@ -1,12 +0,0 @@
#jinja2: lstrip_blocks: "True"
POSTGRES_USER={{ matrix_postgres_backup_connection_username }}
POSTGRES_PASSWORD={{ matrix_postgres_backup_connection_password }}
POSTGRES_HOST={{ matrix_postgres_backup_connection_hostname }}
POSTGRES_DB={{ matrix_postgres_backup_databases|join(', ') }}
POSTGRES_EXTRA_OPTS={{ matrix_postgres_backup_extra_opts }}
SCHEDULE={{ matrix_postgres_backup_schedule }}
BACKUP_KEEP_DAYS={{ matrix_postgres_backup_keep_days }}
BACKUP_KEEP_WEEKS={{ matrix_postgres_backup_keep_weeks }}
BACKUP_KEEP_MONTHS={{ matrix_postgres_backup_keep_months }}
HEALTHCHECK_PORT={{ matrix_postgres_backup_healthcheck_port }}
POSTGRES_PORT={{ matrix_postgres_backup_connection_port }}

@ -1,31 +0,0 @@
#jinja2: lstrip_blocks: "True"
[Unit]
Description=Automatic Backup of Matrix Postgres server
After=docker.service
Requires=docker.service
DefaultDependencies=no
[Service]
Type=simple
Environment="HOME={{ devture_systemd_docker_base_systemd_unit_home_path }}"
ExecStartPre=-{{ devture_systemd_docker_base_host_command_docker }} stop matrix-postgres-backup
ExecStartPre=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm matrix-postgres-backup 2>/dev/null || true'
ExecStart={{ devture_systemd_docker_base_host_command_docker }} run --rm --name matrix-postgres-backup \
--log-driver=none \
--user={{ matrix_user_uid }}:{{ matrix_user_gid }} \
--cap-drop=ALL \
--read-only \
--network={{ matrix_docker_network }} \
--env-file={{ matrix_postgres_backup_path }}/env-postgres-backup \
--mount type=bind,src={{ matrix_postgres_backup_path }},dst=/backups \
{{ matrix_postgres_backup_docker_image_to_use }}
ExecStop=-{{ devture_systemd_docker_base_host_command_docker }} stop matrix-postgres-backup
ExecStop=-{{ devture_systemd_docker_base_host_command_sh }} -c '{{ devture_systemd_docker_base_host_command_docker }} rm matrix-postgres-backup 2>/dev/null || true'
Restart=always
RestartSec=30
SyslogIdentifier=matrix-postgres-backup
[Install]
WantedBy=multi-user.target

@ -41,8 +41,10 @@
The matrix-postgres role in the playbook has been replaced with the com.devture.ansible.role.postgres role (https://github.com/devture/com.devture.ansible.role.postgres). The matrix-postgres role in the playbook has been replaced with the com.devture.ansible.role.postgres role (https://github.com/devture/com.devture.ansible.role.postgres).
The new role is pretty much the same, but uses differently named variables. The new role is pretty much the same, but uses differently named variables.
Please change your configuration (vars.yml) to rename all matrix-postgres variables (`matrix_postgres_*` -> `devture_postgres_*`). The matrix-postgres-backup role in the playbook has been replaced with the com.devture.ansible.role.postgres_backup role (https://github.com/devture/com.devture.ansible.role.postgres_backup).
Note that `matrix_postgres_backup_*` variables (used by the `matrix-postgres-backup` role) need to remain as they are for now. Do not rename those! The new role is pretty much the same, but uses differently named variables.
Please change your configuration (vars.yml) to rename all `matrix_postgres`-prefixed variables (`matrix_postgres_*` -> `devture_postgres_*`).
The following variables in your configuration need to be renamed: {{ vars | dict2items | rejectattr('key', 'match', 'matrix_postgres_backup_') | selectattr('key', 'match', 'matrix_postgres_.*') | map (attribute='key') | join(', ') }} The following variables in your configuration need to be renamed: {{ vars | dict2items | selectattr('key', 'match', 'matrix_postgres_.*') | map (attribute='key') | join(', ') }}
when: "vars | dict2items | rejectattr('key', 'match', 'matrix_postgres_backup_') | selectattr('key', 'match', 'matrix_postgres_.*') | list | items2dict" when: "vars | dict2items | selectattr('key', 'match', 'matrix_postgres_.*') | list | items2dict"

Loading…
Cancel
Save