Databases Storage And Caching

ACID: Durability

Durability is the ACID property that concerns committed data surviving failures. After a database reports that a transaction committed, its changes should remain available after ordinary failures covered by the database's configured guarantee, such as a database process restart or host crash.

The word "configured" matters. Durability is implemented through storage engines, transaction logs, disk synchronization, replication policies, and infrastructure. A successful PDO::commit() call relies on that entire chain.

Commit Is A Contract Boundary

Before commit, the transaction may still be rolled back. After a successful commit response, the application normally treats the state change as accepted.

PHP example
<?php

declare(strict_types=1);

function recordPayment(PDO $pdo, string $paymentId, int $amountPence): void
{
    $pdo->beginTransaction();

    try {
        $statement = $pdo->prepare(
            'INSERT INTO payments (payment_id, amount_pence, status)
             VALUES (:payment_id, :amount, :status)'
        );
        $statement->execute([
            'payment_id' => $paymentId,
            'amount' => $amountPence,
            'status' => 'accepted',
        ]);

        $pdo->commit();
    } catch (Throwable $exception) {
        if ($pdo->inTransaction()) {
            $pdo->rollBack();
        }

        throw $exception;
    }
}

The PHP code establishes a transaction boundary. It does not itself force bytes onto durable storage. The database decides when it may acknowledge commit according to its engine and configuration.

Transaction Logs Support Recovery

Relational databases commonly record changes in a transaction log or write-ahead log before updated data pages are permanently written to their final locations. After a crash, recovery can replay committed log records and discard incomplete work.

This design avoids requiring every changed table page to be written synchronously during commit. The log supplies an ordered recovery record. Terminology differs: PostgreSQL commonly refers to WAL, while MySQL/InnoDB uses redo logs and other supporting logs.

Application developers do not normally manipulate these logs directly, but they should understand their purpose. Deleting, filling, corrupting, or misconfiguring log storage can affect availability and recovery.

Acknowledgement Depends On Flush Policy

Operating systems and storage devices use caches. A database may write data to an operating-system buffer without the device having persisted it to non-volatile media. Database flush and synchronization settings influence when commit is acknowledged relative to those layers.

Stronger synchronization can increase commit latency. Weaker settings can improve throughput while creating a window in which recently acknowledged transactions may be lost after a power or host failure.

The exact settings and names are database-specific. For example, MySQL/InnoDB and PostgreSQL expose different controls for log flushing and synchronous commit behavior. Do not copy performance tuning from a blog without understanding the failure window it introduces.

A development laptop, test container, and production database may intentionally use different settings. Production requirements should be documented rather than inferred from local behavior.

Durability Is Not Availability

Durable data may be temporarily unavailable. A database can retain every committed transaction but still be offline during recovery, failover, network repair, or storage replacement.

Availability asks whether the service can respond now. Durability asks whether acknowledged data survives covered failures. Replication and failover can improve availability, but they also introduce decisions about which replicas must acknowledge a commit.

A system can be durable but unavailable, or available while accepting a weaker durability mode. Use the correct term when discussing incidents and architecture.

Durability Is Not Backup

A backup protects against a different class of failure. If an administrator accidentally runs DELETE FROM customers, a durable database preserves that deletion. A synchronous replica may preserve it too.

Backups, snapshots, and point-in-time recovery can restore an earlier state. Their value depends on:

  • recovery point objective: how much recent data loss is acceptable;
  • recovery time objective: how long restoration may take;
  • retention period;
  • encryption and access controls;
  • off-site or independent storage;
  • successful restore testing.

A backup that has never been restored is an unverified hope. Durability, replication, and backups are complementary controls.

Replication Changes The Commit Question

With asynchronous replication, a primary can acknowledge a commit before a replica receives it. If the primary is then permanently lost, failover to a lagging replica can lose acknowledged transactions.

Synchronous replication can require one or more replicas to acknowledge receiving or persisting the transaction before the primary confirms commit. This can improve failure coverage but increases latency and may reduce availability when replicas are unreachable.

The phrase "we have replicas" does not specify durability. Ask:

  • Is replication synchronous or asynchronous?
  • What does a replica acknowledgement mean?
  • How many acknowledgements are required?
  • Can the system fall back to a weaker mode?
  • What happens during failover?
  • How is data divergence detected and repaired?

These are operational policies with application consequences.

Ambiguous Commit Outcomes

A difficult case occurs when the server commits but the connection drops before PHP receives the response. From the client perspective, commit() failed or timed out. The database may nevertheless contain the transaction.

Blindly retrying can duplicate a payment, order, or message. Use a stable business operation identifier protected by a unique constraint:

CREATE TABLE payments (
    id BIGINT PRIMARY KEY,
    operation_id VARCHAR(100) NOT NULL UNIQUE,
    amount_pence BIGINT NOT NULL CHECK (amount_pence >= 0),
    status VARCHAR(30) NOT NULL
);

On retry, the application uses the same operation_id. If the first attempt committed, the duplicate insert is rejected or the existing result can be returned. This makes the operation idempotent at the database boundary.

An auto-increment row ID created separately on each attempt is not an idempotency key. Use an identifier tied to the logical request.

Durable Messaging Requires Durable Intent

After committing an order, PHP may need to publish an event. A process crash can occur before the publish. The order is durable, but downstream systems never hear about it.

A transactional outbox stores the event intent in the same transaction as the business record. Because the outbox row is durable with the order, a worker can publish it later. Publication status and retry metadata should also survive worker restarts.

Queues have their own durability configuration. A message accepted only in memory may disappear when the broker restarts. Confirm acknowledgement, persistence, replication, and dead-letter behavior according to the importance of the work.

Filesystem Durability Has Similar Layers

Applications sometimes write uploaded files or generated documents alongside database rows. A successful PHP file-write call does not create one atomic durable transaction with SQL. Filesystem buffers, object storage semantics, and rename behavior have separate guarantees.

A robust workflow may upload to a temporary object key, verify the result, commit a database record referencing the final key, then promote or mark the object through a recoverable process. Another design stores pending status and uses cleanup jobs for abandoned objects.

Do not claim a database transaction covers a local file or object-store upload merely because both occurred in one try block.

Schema Changes And Durability

Database migrations need their own recovery plan. Some engines support transactional DDL for many operations; others implicitly commit around certain statements. Large online schema changes may involve shadow tables, replication, or long-running background work.

Before a risky migration, understand whether rollback is possible, how long locks may last, what happens after interruption, and whether backups or snapshots are current. Application-level rollBack() cannot reverse an engine operation that already committed implicitly.

Test Recovery, Not Just Insert And Read

A test that inserts a row, commits, and immediately selects it proves visibility on the running server. It does not prove crash recovery.

Durability verification belongs partly to operations. Depending on the environment, tests may include:

  1. Commit known records with stable identifiers.
  2. Stop the database process abruptly in a controlled non-production environment.
  3. Restart and allow recovery to complete.
  4. Verify committed records exist and uncommitted records do not.
  5. Inspect database recovery logs.
  6. Perform replica failover tests.
  7. Restore backups into an isolated environment and validate data.

Do not simulate power loss on production systems casually. Use documented disaster-recovery exercises and infrastructure designed for testing.

Monitor The Durability Chain

Useful operational signals include transaction-log disk capacity, checkpoint behavior, replication lag, replica health, backup completion, restore-test results, storage errors, and failover outcomes.

An application may need to reject writes when required durable dependencies are unavailable rather than silently falling back to a weaker guarantee. That choice belongs in an explicit availability-versus-durability policy.

Alerts should be actionable. A growing log volume may indicate a stuck replica, long-running transaction, failed archival process, or insufficient storage. Treat the cause, not only the disk threshold.

Choose Guarantees By Data Importance

Not every value needs identical durability. Losing a recomputable cache entry is different from losing a payment ledger record. Analytics counters may tolerate buffered or asynchronous writes; financial state may require stronger commit and replication policies.

Classify data and operations:

  • authoritative financial or legal records;
  • customer-created state;
  • operational workflow state;
  • rebuildable indexes and caches;
  • disposable telemetry.

Use that classification to choose synchronization, replication, backup, and recovery requirements. Do not weaken the entire database for speed because one low-value workload can tolerate loss. Separate workloads where practical.

Avoid False Confidence From Cloud Labels

Managed databases simplify operations but do not remove configuration choices. Multi-zone deployment, automated backups, point-in-time recovery, cross-region replicas, and high-availability labels each cover particular failures.

Read the provider's exact commitment and shared-responsibility model. Confirm retention, deletion protection, replica mode, maintenance behavior, restore procedure, and region-failure strategy. A service being "managed" does not decide the application's recovery objectives.

Review Checklist

For durable application state, ask:

  • What does the database promise before acknowledging commit?
  • Which flush or synchronization settings affect that promise?
  • Is replication synchronous, asynchronous, or able to degrade?
  • Can a connection failure leave the commit outcome unknown?
  • Does each retryable business operation have a stable idempotency key?
  • Are external messages represented by durable intent?
  • Are backups independent, retained, encrypted, and restorable?
  • Have crash recovery, failover, and restore procedures been tested?
  • Are log capacity and replication health monitored?
  • Are guarantees matched to the value of the data?

What You Should Be Able To Do

After this lesson, you should be able to explain how transaction logs and synchronization policies support durability, why replication acknowledgements affect failure coverage, and why a successful local read is not a crash-recovery test.

You should distinguish durability from availability, replication, and backup. You should also be able to design idempotent handling for ambiguous commits, use a transactional outbox for durable message intent, and ask operations teams precise questions about the guarantees behind a successful commit().

Practice

Classify Durability Controls

For each, state whether it primarily addresses commit durability, availability, disaster recovery, or routing. Explain why no single item replaces all the others.

Show solution
  • Transaction-log flushing supports local commit durability.
  • An asynchronous replica supports read capacity and recovery options but may lag acknowledged commits.
  • Synchronous acknowledgement can extend commit failure coverage to another node at a latency and availability cost.
  • A nightly backup supports recovery but may have a 24-hour recovery-point gap.
  • Point-in-time archives support restoration near a selected timestamp.
  • A load balancer health check routes around unavailable nodes but does not preserve data.

The controls address different failures and must be combined according to recovery objectives.

Handle An Ambiguous Commit

A checkout request times out while committing. Retrying with a new order ID might create a duplicate.

Design an idempotent database boundary using a client-visible operation ID and a unique constraint. Describe what the retry does when it finds the previous operation and what response is returned when the original outcome is still being reconciled.

Show solution

Generate one stable operation ID for the logical checkout and reuse it for every attempt. Store it in the orders table under a unique constraint. A retry first inserts or looks up that same ID.

If an order already exists, return its stored result instead of creating another. If processing is recorded but incomplete, return a pending response and reconcile through a worker rather than guessing. The operation ID, not an auto-increment row ID, identifies duplicate delivery of the same request.

Design Recovery Proof

Create a durability verification checklist for a payment database. It must cover controlled crash recovery, replica failover, backup restoration, recovery point and time objectives, monitoring, and evidence retention.

Separate checks an application test can perform from exercises that require operations infrastructure.

Show solution

Application tests verify stable operation IDs, committed versus rolled-back rows, outbox creation, and idempotent retries. Infrastructure exercises abruptly stop a test database after known commits, restart it, and verify recovery; promote a replica and measure acknowledged data loss; and restore an encrypted backup into isolation.

Record achieved recovery point and recovery time against targets. Monitor log storage, replication lag, backup jobs, and restore-test age. Retain dated reports, database versions, configurations, test identifiers, and discrepancies so the claim is auditable rather than assumed.