Data: 20 years of hurt

I went to the Institute for Government's 'data bites' event a couple of weeks ago - an evening of four short talks from folks working in public service with data. It was great, thank you to Gavin and the team.

I was particularly struck by the fourth talk, by Yvonne Gallagher from the National Audit Office (NAO). Yvonne was talking about the challenges of using data in government, in advance of an NAO report coming out later this month. You can watch the talk for yourself. It was eight minutes long with eight minutes for questions.

My main impression from Yvonne's talk was that 'data' has been a problem across government for over twenty years. I felt an overriding sensation of 'enough of this it's really time to sort it out now folks'. 

I heard Yvonne say that there had been multiple strategies launched over the last two decades to fix the problems, and I'm paraphrasing, but I heard that the time for launching strategies to fix the problem was over and it was time to actually do something. This made me uncomfortable, not because I don't do things but rather because I've been working on a data strategy [1].  

I also heard Yvonne say that thirty years ago, things had been better. There was an established practice of data modelling, data management, cataloguing and the like. I don't know if this is true, because I was busy watching 'Rude Dog and the Dweebs' and so on. Let's take it as truth. Yvonne was listing practices used back then that facilitated things working, and working together, through collaborative effort, documentation and widespread understanding. These practices are deeply uncool and unfashionable at the moment [2]. Forgotten, even.

I am increasingly convinced that it's time to be deeply uncool and unfashionable. To be boring [3].

Being boring

I am trying to work through an idea, or collection of ideas, where 'data' is a distinct practice that sits alongside 'digital' and 'technology', complementing them both. I've written about it already. Nobody has said "oh yeah Dan you are right take my money" yet, so clearly it needs further work.

I believe we [4] have problems with the language we use when we talk about 'data', and that it's too broad an umbrella to be meaningful. Leigh wrote a post about this (it's great). Leigh also proposed some data archetypes recently. Using Leigh's archetypes as a starting point, I think that the problems we face across government (or across public service, which I prefer) mostly come from:
  • The Register
  • The Database
  • The Description

In my opinion, these are the types of data that facilitate the absolute basics. If they aren't done well then impenetrable silos result.

Now, I believe that 'data' has this unwarranted mystique about it in general that people mistake for computer magic. At the root of the types of data I've listed above is a broad, human understanding and a set of human practices - like ownership for example. Not magic, just taking responsibility.

So, I suggest that over the past 20 years we outsourced too much of our collective responsibility to people who do (or claim to be able to do) computer magic. It's easy to point a finger at a Big IT company who've failed to deliver an outcome at great public expense, so lets do that.

*points finger at Big IT company who've failed to deliver an outcome at great public expense*

BUT WAIT I'm actually advocating for retaining a greater degree of ownership of the fundamentals of our organisations and how we describe them. So that a supplier doesn't imperfectly represent these things on our behalf when they don't have the expertise or long-term incentive to do so, and also have to do the heavy-lifting to work out what they are in the first place, probably from scratch.

The same applies when you have an in-house team, or are working closely with smaller agency partners. Hopefully in a multidisciplinary way. Who does the data modelling in a multidisciplinary team? Don't leave it to computer magic - it needs to have an explicit representation somewhere beyond a database only one person came up with [5]. How do the descriptions get done, beyond the content? Do you have a librarian [6]?

Librarians, yeah?

People joke about tech bros [7] coming up with ideas for things that already exist, like trains, and libraries. I think the libraries one often misses the point, because a library isn't just a quiet place where you can work that is free of charge (LOL). 

A library is an open, dynamic repository of knowledge that's been organised in a way that is collectively understood, with expert help on hand to aid discovery, and no technical barrier to entry.

The demise of libraries in our society is profoundly sad. And sorry but (as much as I love it) the internet / web hasn't satisfactorily replaced my definition above. There is a lack of balance and access for all, where 'access' means being able to contribute rather than just consume.

Make the thing

Everybody being able to contribute is central to my line of thought. I think that's where this 'data' practice will come from in a multidisciplinary delivery team. The opposite end of the pipe from developing a collective understanding of needs: developing and owning a collective understanding of your essential data. 

Will there be resistance? Probably. I could imagine people saying "hey man we're agile we don't do any work upfront that's waterfall UR so lame" or similar, but I don't think there's anything wrong with being prepared before diving in on the build. Design disciplines advocate for developing a rich understanding. The same should apply for data.

Beyond 'the thing', I do believe we gave away too much responsibility, when really we should have retained and maintained our corporate-level data assets which are more permanent than any individual system.

But, as Steve and Adam made me realise, you need to sweat these assets (so to speak) by putting them to use and managing them like any other product so that they are as good as they can possibly be - so that people want to use them rather than inventing their own.

Pull up your brain

There's work required here that isn't currently happening. It is unfashionable as I've said three times now. The benefits of doing it aren't immediately apparent, so it's a hard sell. We are also working in a time when "technology will save us" is considered a legitimate argument, and arguing the contrary will require some political courage and will [8]. 

Personally I think that's avoidance of responsibility and putting the effort in, and a case could definitely be made for financial savings over time through reduced friction and increased re-use.

I'll continue to work on this. Let me know your thoughts, I'd really appreciate them.


[1] Yvonne Gallagher didn't say this, that's me.

[2] I was reminded of the scene in 'Black Dynamite' where Black Dynamite declares war on the people who deal drugs in the community and the drug dealer says "but Black Dynamite, I deal drugs in the community!"

[3] As in dull. Not like Elon Musk making big holes. Have a link to a song by the Pet Shop Boys. Have a link to a song by Deftones.

[4] I'm using 'we' a shorthand for people working in public service. Let's say all over the world, apart from Estonia.

[5] This isn't sour grapes at not being able to do the Johnny Lee Miller myself. I do know a bit about developing a shared understanding of things.

[6] EDIT: here is a post from Silver about Librarians and the web that I saw just after I pressed 'publish'.

[7] Not a robot version of Bros, sadly.

[8] I work at the UK Parliament at the moment so 'political' here means organisational rather than party.

6 responses
Sometimes unfashionable is good. Joining your 'we' - this is not just important, but critical to the progress that so far has alluded us! But it is hard and probably thankless.
5 visitors upvoted this post.