Table of Contents
- The Deletion Request That Cannot Be Fulfilled
- Data You Cannot Find Is Data You Cannot Protect
- Retention Is a Security Control
- Why Deletion Is Architecturally Hard
- Anonymisation Usually Is Not
- Purpose Limitation in Practice
- Building Privacy Into the Data Layer
- Common Pitfalls
- Conclusion
- Frequently Asked Questions
Key takeaway: Data minimisation is the only privacy control that cannot fail. Encryption can be broken, access controls misconfigured, and audit logs ignored — but data you never collected or already deleted cannot be exposed.
The Deletion Request That Cannot Be Fulfilled
A user asks you to delete their data. You have a documented process and a form. Someone runs a script against the primary database.
Their data remains in the analytics warehouse, in last month’s backups, in the search index, in the customer support system, in the email marketing platform, in the payment processor’s records, in application logs, in the data lake, in a spreadsheet an analyst exported in March, and in the machine learning training set built from a snapshot taken in January.
You reported the deletion as complete. It was not, and nobody involved was being dishonest — they simply did not know where the data had propagated.
This gap between policy and capability is the normal state. Privacy programmes commonly consist of documentation describing intentions about data whose location nobody has mapped. The documentation satisfies an audit. It does not describe what the system actually does.
Closing the gap is engineering work: knowing where data is, understanding how it flows, and building deletion that propagates. None of that is achievable by policy alone.
Data You Cannot Find Is Data You Cannot Protect
Every meaningful privacy obligation depends on inventory. You cannot delete, restrict, export, or report on data whose location is unknown.
Building a usable inventory means recording, for each system: what personal data it holds, which categories, where it came from, what it is used for, who can access it, how long it is retained, and what happens when a deletion request arrives.
Where data hides, consistently:
Analytics and warehouse copies. Frequently the largest concentration and the least governed, because it was built by a different team for a different purpose.
Backups. Every backup contains the data as it was. Deletion from production does not touch them.
Logs. Application logs accumulate identifiers, email addresses, and request bodies. Log retention is often longer than data retention, and log access is often broader.
Search indexes. Denormalised copies specifically designed for fast retrieval, updated by a pipeline that may not handle deletions.
Caches. Data persists past its source, sometimes indefinitely if invalidation is incomplete.
Third-party processors. Support tools, marketing platforms, payment providers, session recording services. Each holds a copy and each requires its own deletion path.
Exports and ad-hoc copies. Spreadsheets, notebooks, local development databases seeded from production. This category is the hardest to control and frequently the largest.
That last item deserves attention because the usual response — a policy prohibiting production data in development — fails without an alternative. Providing a realistic synthetic dataset is what actually stops the copying.
Retention Is a Security Control
Retention is usually framed as compliance. It is more usefully framed as security, because it directly reduces breach impact.
Data you deleted cannot be stolen. That is the only protection with no failure mode. Encryption depends on key management. Access control depends on correct configuration. Monitoring depends on someone reading it. Deletion depends on nothing.
The typical situation is that retention has never been decided, so the default is indefinite. Records from a customer who left six years ago sit in the same tables as active ones, with the same exposure and no business value.
Making retention real requires a few specific steps. Set a period per data category, driven by the purpose the data serves and any legal requirement to keep it. Implement automated expiry rather than intending to clean up periodically, because manual cleanup does not happen. Apply retention to derived data — the warehouse, indexes, and logs — not only to the source. And review the exceptions, because “we might need it” expands to cover everything if unchallenged.
The tension worth naming is that machine learning teams want maximal history, and that desire is legitimate. Resolving it usually means keeping aggregates and anonymised derivatives long-term while expiring identifiable records, which requires the anonymisation to be genuine.
Why Deletion Is Architecturally Hard
Deletion appears trivial and is not, for reasons rooted in how data systems are built.
Data is copied by design. Warehouses, replicas, indexes, and caches exist to duplicate data for performance. Each copy is a deletion target.
Append-only systems resist deletion. Event streams, audit logs, and immutable storage are deliberately unmodifiable. Deleting from them is either impossible or requires rewriting history.
Backups are snapshots. Deleting from a backup means restoring, modifying, and rewriting it, which is impractical at scale. The usual approach is documenting that backups expire on their own schedule and are only restored under controlled conditions.
Foreign keys and referential integrity. Deleting a user record can break orders, invoices, and audit trails that must be retained for other legal reasons.
Machine learning models. A model trained on personal data may retain information about individuals in its parameters. Removing one person’s contribution generally requires retraining.
Practical approaches that work within these constraints:
Crypto-shredding. Encrypt each subject’s data with a per-subject key and delete the key. The ciphertext remains and becomes unreadable, which addresses backups and append-only stores without modifying them. This is the most useful technique available for otherwise-undeletable systems.
Tombstone propagation. A deletion event published to every downstream consumer, which each handles for its own store. This requires the pipeline to exist and be complete.
Separating identity from records. Keeping personal identifiers in one store referenced by an opaque key from everywhere else means deletion touches one place and the remaining records become non-identifying.
That last pattern is the strongest architectural answer and it must be designed in early. Retrofitting it into a system where identifiers are denormalised everywhere is a substantial project.
Anonymisation Usually Is Not
Teams routinely claim anonymisation when they have performed pseudonymisation, and the distinction has legal consequences.
Pseudonymisation replaces direct identifiers with a token. The data remains personal data, because re-identification is possible with additional information. Obligations continue to apply.
Anonymisation makes re-identification impossible. Anonymised data falls outside most privacy regulation.
The difficulty is that genuine anonymisation is much harder than it appears, because combinations of non-identifying attributes are identifying. Postal code, birth date, and gender together identify a large share of individuals. Removing names accomplishes very little when the remaining fields are sufficiently specific.
Techniques that provide real protection: aggregation with a minimum group size, so no output describes fewer than some threshold of people; generalisation, replacing exact values with ranges; suppression of rare combinations; and differential privacy, which adds calibrated noise with a mathematical guarantee about what can be inferred.
Differential privacy is the only approach on that list with a formal guarantee, and it carries a genuine accuracy cost. For most analytical uses, aggregation with a minimum group size and suppression of outliers is proportionate.
The practical test: if you can trace a row back to a person using any data you or a third party holds, it is pseudonymised rather than anonymised, and it remains regulated.
Purpose Limitation in Practice
Personal data collected for one purpose should not be silently used for another. This principle is straightforward to state and consistently violated in practice, because data lakes are built precisely to enable unanticipated uses.
Where it breaks: support ticket data used to train a model, addresses collected for delivery used for marketing segmentation, security logs used for productivity monitoring, and health data collected for a benefit used in risk scoring. Each case involves data that was legitimately collected and then repurposed without a basis.
Implementing purpose limitation technically means tagging data with its collection purpose and permitted uses, enforcing those tags at query time so a marketing query cannot reach delivery-purpose fields, and requiring an explicit review when a new use is proposed.
This is more restrictive than most data platforms are designed to be, and partial implementation still helps — even tagging the highest-sensitivity categories and restricting them creates a meaningful boundary.
Building Privacy Into the Data Layer
Structural measures, ordered by effect:
| Measure | Effect | Effort |
|---|---|---|
| Do not collect unnecessary fields | Eliminates risk entirely | Low |
| Automated retention expiry | Bounds exposure over time | Moderate |
| Personal data in one referenced store | Makes deletion tractable | High if retrofitted |
| Per-subject encryption keys | Enables crypto-shredding | Moderate |
| Purpose tags enforced at query time | Prevents repurposing | High |
| Synthetic data for development | Stops production copies | Moderate |
| Field-level access control | Limits internal exposure | Moderate |
| Log scrubbing at write time | Prevents accumulation in logs | Low |
The first and last rows are the highest-return items and both are cheap. Not collecting a field is free and permanent. Scrubbing identifiers from logs at write time prevents a large, broadly-accessible accumulation that otherwise grows indefinitely.
The most common mistake in prioritisation is starting with the high-effort architectural items while continuing to collect unnecessary data and write identifiers into logs.
Common Pitfalls
Policy without inventory. Documented intentions about data nobody has located.
Deleting from production only. Warehouse, indexes, logs, caches, and processors all retain copies.
Calling pseudonymisation anonymisation. Tokenised data remains regulated.
Indefinite retention by default. Absent a decision, data accumulates forever.
Personal data in logs. Longer retention and broader access than the source system.
Production data in development. Provide synthetic data or the copying continues regardless of policy.
Ignoring third-party processors. Each holds copies requiring separate deletion paths.
Conclusion
Privacy engineering is mostly about knowing where data is and being able to remove it. Both are engineering capabilities rather than policy statements, and most programmes have the policy without the capability.
The highest-return work is also the least sophisticated. Stop collecting fields you do not need. Scrub identifiers from logs at write time. Set retention periods and automate expiry, including for derived data. Provide synthetic datasets so production data stops being copied into development.
Then address the architecture. Separating personal identifiers into one referenced store makes deletion tractable. Per-subject encryption keys enable crypto-shredding for backups and append-only systems where deletion is otherwise impossible. Purpose tags enforced at query time prevent the quiet repurposing that data lakes invite.
And be precise about anonymisation, because pseudonymised data carries the same obligations as the original. If a row can be traced to a person using any data anyone holds, it is still personal data.
Frequently Asked Questions
Do backups have to be modified to fulfil a deletion request? Generally not. The common accepted approach is documenting that backups expire on a defined schedule, are not used for ordinary processing, and that deletion is reapplied if a restore occurs. Crypto-shredding addresses it more thoroughly.
How long may data be retained? As long as necessary for the purpose it was collected for, plus any period a law requires. There is no universal number — it is per category and requires a documented justification.
Is hashing an identifier sufficient to anonymise it? No. A hash of an email address is a pseudonym: the same input always produces the same output, so the same person is linkable across records and the hash can be reversed by guessing candidate inputs.
What happens if a deletion request cannot be fully honoured? Document what was deleted, what was retained, and the legal basis for retention. Silent partial deletion reported as complete is the worse outcome.
Does a model trained on personal data need retraining after a deletion request? Possibly, depending on whether the model retains identifiable information and on your regulator’s position. Machine unlearning techniques exist and are immature. Avoiding identifiable data in training sets is the more practical answer.
Where should a privacy programme start? Inventory. Every subsequent obligation depends on knowing where data is. Begin with the highest-sensitivity categories rather than attempting comprehensive coverage immediately.
Is minimisation in tension with machine learning? Somewhat, and less than assumed. Aggregates and genuinely anonymised derivatives can be retained long-term while identifiable records expire. That requires the anonymisation to be real, which is where the effort belongs.
