Data Freshness & Quality

Update Cadence

The OpenSafely data pipeline processes source data on a weekly basis (every Sunday). However, the consumer-facing models that you query are updated monthly — on the last Sunday of each calendar month.

This means:

Underlying data is continuously refreshed weekly, reducing processing load and failure risk.
The data you see in your queries updates once per month, providing a stable snapshot for analysis throughout the month.
After each monthly refresh, the transform_datetime column on each record indicates when that record was last made available with a relevant change.
The model_updated_datetime column indicates when the model itself last processed and updated the record.

Why monthly serve-layer updates?

Monthly snapshots provide a stable analytical baseline — queries run on the 1st of the month will return the same results as queries run on the 28th. This is particularly important for reproducible research and consistent reporting across teams.

Data Quality

Automated data quality checks run against all models as part of the monthly refresh cycle. These checks validate:

Completeness — key columns are not unexpectedly NULL
Uniqueness — primary key constraints are enforced (e.g. one row per patient per organisation)
Referential integrity — foreign key relationships between models are valid
Value ranges — numeric and categorical fields fall within expected bounds

Critical data quality failures are investigated and resolved before being considered for consumer visibility. Each model has a dedicated quality suite tailored to its specific constraints.

Data Scope

The data available through OpenSafely models is scoped to:

England GP practices only — organisations must be classified as GP practices operating in England.
Active sharing agreements — only organisations with an active data sharing agreement with the NHS England OpenSAFELY Secure Data Analytics Service are included.
Non-sensitive, non-confidential records — records flagged as sensitive or confidential at the source are excluded from clinical event models.

If a practice’s sharing agreement is activated or deactivated, the pipeline automatically includes or excludes that practice’s data at the next monthly refresh.

Pseudonymisation

All patient identifiers (patient_id, nhs_number) are pseudonymised before delivery. The pseudonymisation is:

Deterministic — the same source identifier always produces the same pseudonymised value for your user context.
User-scoped — different users receive different pseudonymised values for the same patient, preventing cross-user linkage outside the platform.
Stable — a patient’s nhs_number hash does not change if they transfer between practices (see Practice Transfers).