Unimpressed with the slow response time that I and others were getting to my query to the Office of National Statistics (to which I still never received a response) the month before last, Zoe O’Connell decided to send a Freedom of Information Act request demanding a response to a couple of similar questions. After some hassling (I suppose they’ve been busy, with the census and all), they finally responded. The original request and the full response is online now, as is Zoe’s blog post about the response. But here’s the short version of the response:
Polygamous marriages are not legally recognised in the UK and therefore any data received from a questionnaire that appeared to show polygamous relationship in the manner that you suggest would be read as an error. It is recognised that the majority of respondents recording themselves as being in a polygamous relationship in a UK census do so erroneously, for example, ticking the wrong box for one household member on the relationships question.
Therefore, the data to be used for statistical purposes would be adjusted by changing one or more of these relationships, so that each respondent is in a relationship with no more than one person. This is consistent with all previous UK censuses, and others around the world.
A copy of the original questionnaire would be retained as part of the historical record which would show such relationships as they were recorded. We do not attempt to amend the original record.
Any mismatches between the indicated sex and marital status of respondents will be resolved using a probabilistic statistical system which will not necessarily deal with each case in the same way. The system will look at other responses for each person, including those for the Household relationships, and will alter one or more variables to make the response consistent. In the example that you propose, it would either change the sex of one individual, or change the marital status to “Same-sex civil partnership”, depending on which is considered statistically more likely to be correct.
Honestly, I’m not particularly impressed. They’ve committed to maintaining a historical record of the original, “uncorrected” data, so that future statisticians can get a true picture of the answers given, but this is about the only positive point in this response. Treating unusual data as erroneous is akin to pretending that a societal change doesn’t exist, and that this approach is “consistent with previous censuses” neglects to entertain the possibility that this data has value that it might not have had previously.
Yes, there will be erroneous data: people who accidentally said that they had two husbands when they only have one, for example. And yes, this can probably (although they don’t state how they know to recognise this) be assumed to be more common that genuine cases where somebody meant to put that on their census (although there will also be an error rate amongst these people, too). But taking the broad brush approach of assuming that every case can be treated as an error reeks of the same narrow-mindedness as the (alleged; almost-certainly an urban legend) statement by Queen Victoria that lesbianism “didn’t exist.”
“Fixing” the data using probabilities just results in a regression towards the mean: “Hmm; this couple of men say they’re married: they could be civil partners, or it could be a mistake… but they’re in a county with statistically-few few gay people, so we’ll assume the latter.” Really: what?
I’m not impressed, ONS.
Update: a second FoI request now aims to determine how many “corrections” have been made on censuses, historically. One to watch.
“It is recognised that the majority of respondents recording themselves as being in a polygamous relationship in a UK census do so erroneously”
Either the general public are really bad at forms, or there are very few poly families. Or, both. Keep harassing them – the truest representation possible should be their goal.
I think you have to consider the differing and somewhat divergent points coming up here. I agree its unfortunate that they seem to be taking this approach to this (and one presumes other things of similar ilk) although its also worth accepting I feel that it *is* a somewhat small percentage of the whole UK population. Sadly, I can easily believe that a greater percentage of people make mistakes on the census than are in poly relationships. Given how many misunderstand “10 items or less” or “Diesel”…..
Anyway – that aside, I don’t think the ONS will have a massive anti-poly or indeed anti-anything axe. Likely as not they are simply civil servants trying to get the most useable data they can. Assuming all inputs you receive are perfect strikes me as a very silly idea, even if, unfortunately, that does sometimes means actually blurring your picture slightly.
It’s not about anyone having an “anti-” axe. You don’t need an axe to grind to be a bad statistician. You just have to be too lazy to record diverse response properly, and that’s enough to end up forcing a very conservative result.
It seems to me that it’d be simple enough to electronically record both the inputs as they were submitted (minus spoiled fields) and the sanitised probabilistic interpretation of the input, and make both available. Presumably if they’re being systematic about it, the data is being input as it is on the forms and then sanitised by computer rather than by some data input worker’s preference, and I’m pretty sure the storage space cost increase of holding both datasets would be negligible (compared with budget of project).
Although I agree with your sentiments I feel that I have to defend ONS and Statisticians in general here.
When we develop forms or databases we try to think of every eventuality that could possibly occur, but its not possible. For example, we have a database which accepts any postcode in the UK, has an option for missing postcode (homeless) or out of country home. We hadn’t considered Womens refuges, which often follow a completely different system.
Another example, we often use 999 or similiar to represent something being missing off the form. We hadn’t considered the fact people being born on the 9th September 2009 would be coded as “missing DOB” by mistake as a result of this.
Often things look a bit “funny” and I dig out the old data forms. Often I discover a perfectly legitmate explanation, which I simply can’t code, usually because the numbers of it occuring are so low it isn’t sensible too. However, I read the original forms and smile and their odd circumstance has been noted by me.
In the future that unusual circumstances may be mentioned in the set of a new database (as was the case with the DOB issue). ONS may not record what you want now, but that doesnt mean at some point they won’t change it.
I noticed that your original letter mentioned “multiple romantic relationships” and “three-way committed relationship”. The civil servants of the ONS interpreted this as “polygamous marriage”. That’s one way to interpret a V or a triad… I have a hunch that these are the most common forms of poly households. But think of the poor statisticians when they have to take N-shaped or even more complex relations into account. :-)
You’re right. At one point, I formed a corner of a square: I’m not even sure that the census forms would have supported stating that we were in this configuration, had we all lived together! We’d have had to draw them a diagram! (Although at least then there’d be no ambiguity that we hadn’t made a mistake!)