Facts andDimensionsRequest a trial

Data quality

The errors automation alone won’t catch.

Public data often contains internal contradictions, broken formulae, hidden values and shifting definitions. Facts and Dimensions investigates these issues before the data reaches your analysis.

Data quality in practice

Source data can be published and still be wrong.

Automation is useful for collection, but accurate ingestion depends on judgement. The gallery below shows inconsistencies encountered in real public datasets. FAD handles the repair, clarification and publisher contact, leaving customers to work with the data.

39 documented examples · Select any image for a closer look.

A duplicated pair of values found in RTT.
These dates are inconsistent between the values in the rows, the title of the document, and the filename.
This spreadsheet contains a formula which fails to incorporate every data point. The sum should extend to cell F42380.
Dates in many datasets are transposed, or misinterpretted as different date formats.
The title states Q2, but the quarter column states Q1.
Here we have an example of a misspelt filename.
An integer value has been recorded with erroneous decimal places.
Year headings do not match the values in the table, or the sheet name.
These superscript numbers, in brackets, should not be misinterpreted as data.
When a CSV is prepared or modified in Excel, certain strings including numbers can be misinterpreted as floating point numbers.
This field is titled prevalence in other spreadsheets, but inconsistently titled calculation here.
These headings, from PROMs Key Data, are completely transposed.
Duplication of data within a spreadsheet can result in incorrect sum total values.
A spelling error in a heading, can lead to a spelling error in a field name.
Previous releases of this data contained a null value for the totals code field. This release featured a hyphen.
This file contains duplicated ODS codes alongside incorrect location name information.
Web scraping can introduce errors with misspellings such as this.
This spreadsheet has been formatted to only show the correct whole number, but the data contained in the cell, incorrectly has decimal places.
Between one data release and the next, the definition of this column has changed.
In this spreadsheet a the Number of Adult critical care beds, is reported as being 15.5.
Theses Diagnostic ID numbers have been set to 0.
There are true values, there are false values, and then there's this!
An integer value has been recorded with erroneous decimal places.
If a textual date has been used, and then misspelt, an automated ETL system will error.
This spreadsheet contains cells which have white text, effectively making the data invisible.
Another date mismatch between the filename and the file contents.
This spreadsheet has been created with a heading in column H, but the values for that column placed in column G.
When a CSV is prepared or modified in Excel, certain strings including numbers can be misinterpreted as floating point numbers.
The filename is for Quarter 1, but the sheet states Quarter 3.
The field headings here should state Org_code and Organisation_name.
The filename states April, the document Introduction sheet states May.
In an Excel spreadsheet, there can be hidden columns, causing sums of columns to be mismatched from the visible data.
CL is in the headings, where it should be CI.
This data contains a date and month separated by 2 spaces.
This CSV contains unquoted text, which contains commas, hence separating the columns incorrectly.
Some data sets take a great deal of effort to process into a usable format.
This CSV contains repeated headings in the middle of the data.
The filename suggests a date range covering July 2011 to 2012. The data shows just July 2011.
The total stated in the column (as text), disagrees with the total calculated by Excel.

See the data for yourself

Get every table in your browser, free for 30 days.

Request Azure accessStart on Snowflake