Payroll data reconciliation

A worker could open the app and see one net-pay figure, then open the attached payslip and see another. Both appeared to describe the same payment, so from the worker’s point of view one of them had to be wrong.

I traced the problem through an external payroll data service, corrected the way revised records were handled, and built a reconciliation process for the historic archive. The work combined incident investigation, data modelling, document extraction, controlled remediation, and a wider assessment of how much the supplied data could be trusted.

Two representations of the same pay run

The service supplied both a human-readable PDF payslip and a structured payload for each payment. The application used the payload to display figures such as net pay, while the PDF remained the formal document available to the worker.

Those two outputs were expected to agree. In practice, they came through different paths and could diverge. The problem affected several payroll arrangements, including PEO, umbrella, CIS, and sole-trader records, so it could not be treated as a single-template parsing error.

Finding the hidden revision behaviour

The key discovery was that the provider could send a corrected payload using the same payslip ID as the earlier version. There was no explicit revision number, nor a new immutable ID linked back to the original record.

In a conventional integration, receiving the same identifier again indicates a duplicate. The application therefore ignored the later item instead of recognising it as a correction. That left the older payload in place even though a revised version existed.

A more dependable data contract would make that relationship explicit: either the same record ID with an incrementing version, or a new record ID with a link to the record it supersedes. Because the source did neither, I changed the ingestion and storage model to preserve successive versions and identify the current one without losing the history.

Reconciling the historic archive safely

Fixing the incoming flow did not repair records already held by the application. I built a repeatable process to work through the existing payslip archive, extract the relevant figures from the PDFs, and compare them with the corresponding payloads.

The scripts pulled values such as net pay and hours from several document layouts, then applied arithmetic and consistency checks to judge whether a correction could be made safely. Results were separated into clear cohorts:

• high-confidence mismatches that could be corrected automatically;
• ambiguous cases that needed review; and
• records where the available evidence was insufficient.

Only the high-confidence cohort was updated. That resolved roughly half of the affected records while keeping uncertain cases out of an automated change. The aim was not to maximise the number altered; it was to recover as much as possible without replacing one unreliable value with another.

The PDF and structured payload were compared, then separated into safe corrections, review cases, and records without enough evidence.

Testing the wider data supply

The investigation also exposed a broader product risk. The structured payload was not the same source data used by the payroll engine to produce the payslip. Some fields were missing, some were inconsistent, and at least one arrived as zero until a later product requirement made its importance clear.

I extended the analysis across the archive to identify which fields agreed with the formal payslip, which could be derived reliably, and which should not be treated as authoritative. This turned a vague lack of confidence into evidence that could guide later product work and conversations with the supplier.

AI-assisted development helped me create and adapt the extraction scripts quickly across the different layouts. The important decisions remained operational ones: defining the comparison rules, setting the confidence thresholds, preserving an audit trail, and deciding where automation had to stop.

The outcome

The work restored consistency for the records that could be corrected with confidence and prevented revised payloads from being silently discarded in future. It also produced a practical map of the data supply: what could be trusted, what needed validation, and where the PDF had to remain the reference point.

This is the kind of technical-operations problem I enjoy: following an issue across product, data, third-party services, and customer experience; finding the real failure mode; and building a controlled process that can operate safely at scale.

The organisation, supplier, worker records, source documents, and identifying implementation details have been withheld.