#################
Stack Replication
#################

AWX Jobs (that create Kubernetes **CronJobs** and **Jobs**) are used to perform regular
Database and Media backups to an NFS volume to allow the material to be
replicated between stacks either on the Production or Development cluster.

- The Production cluster is home to the **Staging** and **Production** stacks
- The Development cluster is home to stacks controlled by individual developers
- An NFS provisioner takes care of PVC allocations for use within each cluster
- NFS volumes are created to allow files to be shared between the clusters

***********
Replication
***********

Replication of a stack relies on a database dump and copy of media files.
This is performed every day at 02:04 UTC.

- The **database** backup is driven by the ``postgres-replicator-backup`` **CronJob**
  in the ``production-stack`` namespace.
- The **media** backup is driven by the ``media-replicator`` **CronJob**
  in the ``production-stack`` namespace.

Details
=======

The **Production** PostgreSQL backup relies on a ``/backup`` volume used by
the corresponding ``postgres-back-hourly`` **CronJob**. The volume will
have an ``/hourly`` directory where you will find hourly backups (for the last
24 hours) in compressed backup files in the ``/hourly``directory.
A typical file wil be named ``/backup-2022-11-28T08:51:08Z-dumpall.sql.gz``.

The backup size is about 3.3GiB (Nov 2023).

Backups are created from within the CronJob using a ``pg_dump`` of the **frag**
database, which is written to a pre-provisioned volume on the NFS server
(``/nfs/kubernetes-db-replica``).

Database credentials can be found in the ``database`` **Secret**
in the ``production-stack`` **Namespace**. You'll find the ``root_password``
(for the built-in postgres user) and a ``user_password`` (for the the
fraglaysis user). There is a ``frag`` database with all privileges granted to
the ``fragalysis`` user.

Media content is backed up by the ``media-replicator`` **CronJob**. The files
are written to another pre-provisioned volume on the NFS server
(``/nfs/kubernetes-media-replica``).