The Quality Collapse: The Reckoning

The Seven Pillars of Data Management, a novel in seven acts.

Chapter II, The Quality Collapse. Pillar II: Data Quality. Part 2 of 5.

The Investigation

Dr. Evelyn Chow received the call at 9:14 PM that same evening. She was at home in Westboro, reading a monograph on health informatics governance that she would later describe, with the dark humour of someone who has lived through institutional catastrophe, as “the most expensive piece of foreshadowing in my career.”

Evelyn was Lakeview Health Alliance’s Chief Medical Information Officer: a title that had been created for her three years earlier when the board recognized, belatedly, that an organization running $1.2 billion in annual healthcare operations on a unified electronic health record platform probably needed someone whose job was to ensure that the data in that platform was trustworthy. She was fifty-three, a former emergency physician who had transitioned into health informatics after fifteen years in the trauma bay at the Ottawa General, and she carried herself with the controlled calm of a woman who had told families that their loved ones were dead and had learned that the only way to deliver unbearable news was to be absolutely, unflinchingly present.

The caller was the Chief of Paediatrics, who told her that a four-year-old patient had nearly died from anaphylaxis triggered by an antibiotic that should never have been prescribed: and that the prescribing physician had checked the allergy record in CareConnect and been told, incorrectly, that the child had no allergies.

Evelyn set down her monograph. “Is the child stable?”

“For now. ICU. Intubated. They’re managing the reaction, but it was severe. Full systemic anaphylaxis. Another ten minutes and we’d be having a different conversation.”

“Pull the record. Lock it. Nobody touches it until I’ve seen it. And get me the data migration logs for that patient from 2019.”

She hung up. She sat in her reading chair for sixty seconds, counting her breaths the way she had learned in residency when the trauma bay was full and the ambulances kept coming and the only thing separating composure from collapse was the discipline of breathing. Then she stood, put on her coat, and drove to the hospital through streets that smelled of wet leaves and the first frost of autumn.

---

The Archaeology of a Record

What Evelyn found, over the next seventy-two hours of investigation, was not a single error. It was an ecosystem of errors: a cascading failure of data quality that had been building, silently, since the day the four hospital networks were merged into Lakeview Health Alliance.

The story began in 2014, with the amalgamation itself. Four hospitals. Four electronic health record systems. Four master patient indices. Four ways of recording a patient’s identity, history, allergies, medications, and diagnoses. The Ministry of Health had allocated $340 million for the integration, of which $86 million was designated for information technology. Of that $86 million, the project plan allocated $52 million for the CareConnect platform license and implementation, $18 million for hardware and infrastructure, $9 million for training, and $7 million for “data migration and validation.”

Seven million dollars. For the task of taking four separate databases, containing, in aggregate, 6.8 million patient records spanning decades of clinical care, and merging them into a single, accurate, trustworthy source of medical truth. Seven million dollars to ensure that every allergy, every diagnosis, every medication, every blood type, every surgical history was correctly mapped, validated, deduplicated, and reconciled.

It was not enough. Everyone involved in the project knew it was not enough. The Chief Information Officer at the time, a man named Gerald Foss who would retire two years later and move to a cottage near Parry Sound, had written a memo to the integration steering committee in March 2014 warning that the data migration budget was “insufficient to achieve acceptable quality thresholds” and recommending an additional $12 million for a comprehensive data validation program. The memo was acknowledged. The additional funding was not approved. Quarterly operating targets took priority. The migration proceeded.

The four databases were merged over a fourteen-month period. The migration team, twelve people, including four contract workers from a consulting firm in Mississauga, used a combination of automated matching algorithms and manual review to deduplicate patient records. The automated algorithm matched records based on a composite key of last name, date of birth, health card number, and postal code. Records that matched on three of four fields were automatically merged. Records that matched on two of four fields were flagged for manual review. Records that matched on fewer than two fields were treated as distinct patients.

The algorithm was reasonable. The thresholds were defensible. The problem was not the algorithm. The problem was what happened to the records that were flagged for manual review.

There were 347,000 of them.

Twelve people. Fourteen months. 347,000 records requiring human judgment about whether two records in two different systems represented the same human being or two different human beings who happened to share some identifying characteristics. The math was merciless: even at five minutes per record, a pace that any data quality professional would consider dangerously fast for medical records, the review would require 28,917 person-hours. That was seventeen full-time employees working for an entire year, assuming no vacation, no sick leave, no training, and no errors from fatigue.

The team had twelve people and fourteen months and a project manager who reported to a steering committee that reported to a board that had promised the Ministry of Health that the integration would be completed on time and on budget.

So they triaged. They reviewed the records they could. They made judgment calls under time pressure. And for the records they could not reach, an estimated 142,000 of them, they applied a default rule that the project manager documented in an email to Gerald Foss as “merge and monitor.” Merge the records based on the automated match. Monitor for discrepancies through the normal course of clinical care.

Monitor. As though the normal course of clinical care: the fifteen-minute family medicine appointment, the busy triage desk, the overworked pharmacist filling sixty prescriptions an hour, was an adequate substitute for a systematic data quality validation program. As though clinicians, who were already struggling with the cognitive burden of a new electronic health record system, would notice that a four-year-old boy’s allergy field had been silently corrupted during a migration they did not know had occurred.

They would not notice. They did not notice. For five years, nobody noticed.

Until Thomas Beaulieu could not breathe.

---

The Scale

Evelyn Chow did not stop at Thomas Beaulieu’s record. She could not. The logic of the failure demanded a larger investigation: if one record had been corrupted by the merge-and-monitor approach, how many others had been corrupted?

She assembled a data quality strike team: three health informaticists, two database analysts, a biostatistician from the research institute, and a clinical pharmacist who understood the downstream consequences of data errors in ways that the informaticists could not. She gave them a mandate and a deadline: within fourteen days, determine the scope of data quality failures in CareConnect’s master patient index and allergy registry.

The findings, presented to the Lakeview executive team on October 29th in a conference room on the seventh floor of the Carling Avenue administrative building, were the kind of findings that make an institution question its own identity.

Of the 6.8 million patient records in CareConnect, approximately 412,000 contained at least one data quality deficiency that the team classified as “clinically significant”, meaning the deficiency had the potential to affect a treatment decision. Of those 412,000 records, approximately 38,000 contained allergy data that was inaccurate, incomplete, or duplicated from another patient’s record. Of those 38,000, approximately 7,200 contained allergy data that was not merely incomplete but actively wrong: records where a known allergy had been overwritten, deleted, or attributed to the wrong patient during the migration.

7,200 patients. Each one a person who could walk into a Lakeview clinic, present with a sore throat or a urinary tract infection or a dental abscess, receive a prescription based on the electronic health record, and suffer a reaction that the record should have prevented. Each one a Thomas Beaulieu who had not yet encountered the wrong antibiotic at the wrong time.

Evelyn presented the findings without emotion, because emotion would have been a luxury that the room could not afford. She spoke in numbers and categories and risk classifications, and when she finished, the CEO of Lakeview Health Alliance, a woman named Dr. Constance Armitage who had been a cardiac surgeon before she was an executive and who understood, at a cellular level, what it meant when a system designed to protect patients failed, asked a single question.

“Have any of these patients been harmed?”

Evelyn paused. The pause lasted perhaps two seconds, but it was the kind of pause that fills a room like smoke.

“We don’t know. We have no mechanism to retrospectively identify adverse drug events that may have been caused by data quality failures. Our incident reporting system captures events that clinicians recognize and report. If a patient had a mild allergic reaction to a medication that the record said was safe, and the clinician attributed the reaction to something else, a food allergy, an environmental trigger, an idiopathic response, it would not appear in our incident data. The honest answer is: we may never know the full extent of the harm.”

The room was silent. Outside the window, the Ottawa River was visible through the trees along the parkway, grey and heavy with the coming winter, carrying water that had begun as snow in Algonquin and would end as current in the St. Lawrence, indifferent to the disaster unfolding in the building on its bank.

Dr. Armitage removed her reading glasses. She set them on the table with a precision that Evelyn recognized, the precision of a surgeon placing an instrument on a tray, a gesture of controlled discipline in a moment when discipline is the only thing preventing collapse.

“Notify the Ministry. Notify the College. Notify the patients.”

Three sentences. Each one an act of institutional courage that would cost Lakeview Health Alliance its reputation, its CEO’s career, and $189 million in settlements, fines, and remediation over the next three years.

← All writing