The Future of PostgreSQL Backups: A Deep Dive into pg_hardstorage
Date Published
A PostgreSQL backup is only useful if the base backup is intact, the required WAL is available, and a restore has been tested. pg_hardstorage approaches that problem with a static Go binary that connects over PostgreSQL's replication protocol. It can run on a separate backup host without shell access or an agent on the database server.
Continuous WAL, with base backups as anchors
An always-on pg_hardstorage wal stream process receives WAL through a physical replication slot. Scheduled base backups provide consistent starting points for recovery; the stream carries changes forward. These are separate processes aimed at the same repository. Supervising the streamer, monitoring lag and retained WAL, and taking periodic base backups are operational requirements, not optional consequences of running init.
The repository stores chunks by content hash. FastCDC-based deduplication avoids storing identical content repeatedly across base backups, while each ordinary base backup remains independently restorable without an incremental chain. PostgreSQL 17+ incremental backups are also supported as an option, so “no backup chains” describes the default content-addressed full-backup model, not an absolute prohibition.
Install and take a first backup
The official installation page currently presents building from source as the available path and requires Go 1.26 or newer:
git clone https://github.com/cybertec-postgresql/pg_hardstorage
cd pg_hardstorage
make build
sudo make install
pg_hardstorage version
The project's README also advertises a curl installer and Homebrew tap, but the installation page says the tap is not packaged yet. Verify a release channel before using it in production; do not pipe an uninspected remote script into a shell.
A self-managed PostgreSQL 15–18 server needs a role with REPLICATION, wal_level set to replica or higher, and network access to its replication endpoint. The quickstart shows a non-interactive first backup:
pg_hardstorage init --yes \
--deployment prod \
--pg-connection "host=10.0.0.10 user=replicator dbname=postgres" \
--repo file:///srv/pg-backups/prod
For ongoing protection, run the WAL streamer under a supervisor and schedule base backups. For example:
pg_hardstorage wal stream prod \
--pg-connection "host=10.0.0.10 user=replicator dbname=postgres" \
--repo file:///srv/pg-backups/prod
pg_hardstorage backup prod \
--pg-connection "host=10.0.0.10 user=replicator dbname=postgres" \
--repo file:///srv/pg-backups/prod
These commands use placeholders. Configure authentication securely rather than placing a real password on a shell command line.
Verify, restore, and test the database
Repository verification checks signatures and chunk hashes. A restore drill then checks whether the bytes form a working database:
pg_hardstorage verify prod latest --repo file:///srv/pg-backups/prod
pg_hardstorage restore prod latest \
--repo file:///srv/pg-backups/prod \
--target /var/tmp/restore-drill
Start an isolated PostgreSQL instance on the restored directory and confirm the data you need. For point-in-time recovery, the architecture guide documents --to 'YYYY-MM-DD HH:MM:SS+00' alongside --target; choose a timestamp covered by your base backup and retained WAL. Do not assume a successful repository check alone proves that the database will start.
How it compares
pgBackRest: A mature choice with differential and incremental chains and extensive production use. Its typical host/SSH or archive integration differs from pg_hardstorage's remote replication-client model.
Barman: Supports an external backup server and PostgreSQL streaming options. It is not simply “SSH-only”; the deployment mode determines what host access is needed.
pg_hardstorage: Content-addressed ordinary full backups remove dependency on earlier backup manifests. Its replication-only model suits self-managed databases where the backup system has network access but no database-host shell access. It is newer, and the project's own comparison acknowledges pgBackRest's longer production track record.
Managed services such as RDS and Cloud SQL that do not expose the required physical replication and BASE_BACKUP interface are outside pg_hardstorage's supported scope. Patroni-aware failover mechanisms are documented, but gap-free recovery still depends on correct slot configuration and monitoring; prove it with a failover-and-restore drill.
A practical evaluation
Run a backup and a timed restore against a disposable self-managed PostgreSQL instance. Measure restore duration, confirm WAL coverage, exercise a streamer restart and a failover if applicable, then review repository permissions and retention. Those checks make the architectural promise testable in your own environment.
Plan your PostgreSQL recovery with Worlber
Worlber can help design a self-managed PostgreSQL backup and PITR strategy, configure continuous WAL protection, and run restore drills against your recovery objectives. Talk to Worlber about your PostgreSQL environment.