Via our eagle eyed correspondent Tash Whitaker comes this story from the UK health service:
Last month, the National Health Service took the unusual step of closing down a children’s heart surgery unit at a UK hospital, after data they had submitted showed that twice as many children and babies died in the unit than anywhere else in the UK. The UK media went went into a frenzy; people came out of the woodwork with stories about their treatment at the hospital, neglect and near death experiences in abundance.
Eleven days later and the unit is set to reopen. Turns out that there were not twice as many people dying after all, just a terminal case of data malaise. The data that the hospital submitted to the NHS was late and incomplete; in fact, 35% of the expected data was missing completely, with catastrophic results.
This particular hospital had obviously not stopped to think about the impact that bad quality data has on their business and on their customers. How many children and babies had heart surgery postponed as a result of the closure? How many may later die as a result of that postponement?
In a twist of fate, the unit was closed down only 24 hours after a High Court ruling that the hospital should keep its heart unit long term. I suspect that decision is now in jeopardy. How can the hospital’s reputation recover from something like this? Would you want your child to be operated on somewhere with a reputation for high death rates? A reputation that we know to be wrong but will no doubt stay with this hospital unit for many years to come.
The importance of data as a business asset is proclaimed regularly but we forget to mention that it can also be a liability. Most people don’t remember when good quality data helped them make decisions, helped them grow their business, or enabled them to beat the competition; but they sure as hell remember when it causes their business operations to cease, their reputation to be torn to ribbons and their status as a trusted entity to be shattered before their eyes.
(Sources: http://news.sky.com/story/1075720/leeds-hospitals-own-data-stopped-surgery
http://www.bbc.co.uk/news/health-22076206)
(Thanks to Tash for the alert and the excellent write up)
From Europe we learn of two stories with similar characteristics that tick all the boxes for classic Information Quality Trainwrecks.
From Germany we hear that due to errors in internal accounting in the recently nationalised Hypo Real Estate, the German National debt was overstated by €55 Billion (US$76 bn approx). This was doubly embarrassing for Germany as they had spent the last while criticising the accuracy of accounting by the Greek Government.
According to the Financial Post website:
In an era of austerity where their government has squabbled tirelessly for two years over a mooted €6-billion tax cut, Germans found it hard to fathom that their government was so suddenly and unexpectedly 55-billion euros better off.
The net effect of the error being found and fixed is that Germany’s Debt to GDP ratio will be 2.6% lower than previously thought.
The root cause appears to be a failure to standardise accounting practices between two banks who were being merged as part of a restructuring of the German banking system. This resulted in the missing billions being accounted for incorrectly on the balance sheet of the German government who owns the banks in question.
From Ireland we have a similar story of missing Billions. In this case a very simple accounting error resulted in monies that were loaned from one State agency (the National Treasury Management Agency) to another State Agency (the Housing Finance Agency) being accounted for by the Department of Finance in a way which resulted in €3.6billion being added to the Irish National Debt figures.
This (almost co-incidentally) resulted in a 2% misstatement of the Irish National debt. Also co-incidentally it is exactly the same figure as the Irish Government is seeking to reduce net expenditure by in its forthcoming budget.
The problem was first spotted by the NTMA in August of last year (2010) but, despite a number of emails and phone calls from the NTMA to the Department of Finance the error was not fixed until October 2011. For some reason there was a failure in the Department to recognise the error, understand the significance, or take action on it.
The Secretary General of the Department of Finance blames middle-management:
Secretary general of the department Kevin Cardiff said the error was made at “middle management” level and was never communicated up to a more senior level. He said the department was initiating an internal inquiry to examine the issue and would establish an external review to look at the systems and to put safeguards in place to ensure such mistakes were not repeated in the future.
Information Quality Professionals of course would consider looking at the SYSTEM, and part of that is the organisation culture which is in place in the Department which prevented a significant error in information from being acted upon.
Lessons to Learn:
There are a lot of lessons to learn from these stories. Among them:
- When bringing data together from different organisations, particularly when those organisations are being merged, it is important to ensure y0u review and standardise the “Information Product Specification” so that everyone knows what the standard terms, business rules, and meaning of data are in the NEW organisation and ACROSS organisational boundaries. Something as simple as knowing who has to put a value in the DEBIT column and where the corresponding CREDIT needs to be put should be clearly defined. Operational Definitions of critical concepts are essential.
- When errors are found, there needs to be clear and open channels of communication that allow the errors to be logged, assessed, and acted on where they have a material or significant effect. Organisational cultures where internal politics or historic arrogance lead managers to assume that the issue isn’t there or isn’t their problem ultimately result in the issue becoming bigger and more difficult to deal with.
- Don’t shoot the messenger. Don’t blame the knowledge worker. But ensure that there are mechanisms by which people can take accountability and responsibility. And that starts at the the top.
From Twitter we learn that CNN is reporting that NATO is combing Tripoli looking for Colonel Gadaffi (or any of the other variant spellings of that name).
Unfortunately, if CNN are to be believed, NATO has just invaded another country to find the errant Colonel (Libya is a little further to the left people…)
Of course, this is not the first time that CNN or other media outlets have made errors with geography. Here’s the CNN map showing the location of the Queensland flooding in Australia earlier this year:
And of course, the Gawker.com took great pleasure in reporting on how Google presented the Russian invasion of the former Soviet Republic of Georgia as being an attack on some good ol’ boys in the Deep South of the United States of America (at least according to their maps)
A great resource for information on mapping and cartographical errors is The Map Room Blog
Via The Miami Herald comes a story that highlights a number of impacts of poor quality information in key processes.
In Oklahoma, schools are placed on an “improvement list” if they fail to meet standards for two consecutive years. Once on the list the school show progress in improving standards for two years before they can be taken off the list. This can have implications for funding and access to resources as well. Some Oklahoma School districts are, it is reported, concerned that they don’t make the grade against Federal requirements.
Problems with the quality of demographic data in electronic testing performed by Pearson has affected the publication of the reports against which schools are graded. These will now be available a full month late, being released in September and not August as expected. This will affect the ability of School Boards to effectively respond to their report card.
Other problems reported on top of missed deadlines include errors in printing report cards to be sent to parents
Oklahoma’s Superintendent of Schools Janet Barresi has described the impacts of poor quality data in this process as a “ripple effect” that is “imposing an unacceptable burden on school districts” and has called for Pearson’s contract to be reviewed. Pearson are engaging an independent 3rd party to help verify the accuracy and validity of the scoring data (which they are confident in).
Oklahoma is not the first State where data issues have been a problem.
- In 2010 in Florida Pearson was penalised $14.7 million, and had to ramp up staffing levels and make changes to systems as a result of problems with information quality leading to delays. The problems here related to matching of student records.
- In 2010 in Wyoming, Pearson also had to pay penalties arising from problems with the testing, ranging from data going missing to other administrative problems such as improperly calibrated protractors.
This video from the Data Quality Campaign, a US Non-Profit working to improve standards of data quality in the US Education system, highlights the value of good quality and timely information in this important sector:
It was widely reported yesterday that students in Scotland who had signed up for SMS notification of their results had received them a day early, giving them a jump on their less technically minded compatriots and competitors and causing stress and distress to students, parents, educators, civil servants, and politicians.
The Scottish Education Authorities began a root cause investigation as the secrecy and security of the examination results system seemed to have been compromised.
Right now I suspect you are settling in for a tale of hackers and Jason Bourne like derring do. Well, here at IQtrainwrecks we never get that lucky. After all, this is a blog that looks at information and data quality problems.
According to The Register the root cause of this problem is good old data interchange and exchange across organisations.
- A template spreadsheet was used to perform the data interchange between the Scottish Education Authority and the company which provides the SMS gateway and related processing as a pro-bono to the Education Authority, AQL. A batch template is used rather than an on-line interface as the service is only used once a year.
- The template was populated and saved in a later version of Microsoft Excel
- The process of populating and saving the spreadsheet appended a white space to the end of each date stamp (the date that the sms was to be sent).
- The ETL process interpreted the “DATE” field as text (which it was thanks to that errant space) and rejected the field on the load. Luckily AQL had developed error handling for situations where a date field couldn’t be loaded and applied a default… the day of the file load (which was the day before the messages were to go out).
- As a result the SMS system read the file and sent the messages a full day early.
We spotted this on Gawker.com. From my experience using Google Maps, it rings true (I recently was sent 15 miles out of my way on a trip in rural Ireland).
It seems that Google Maps has plotted the location of a tourist attraction in New Jersey right at the end of a driveway to a private residence. So, on the 4th of July weekend, the owners of the property had to fend off increasingly irate visitors who were looking for the lake and wound up in a private driveway.
So, the data is inaccurate and of poor quality. Google have responded to their error and replotted the location of the tourist area at the lake? Not yet, according to the story on Gawker.
