I wrote about data conversations before. These kinds of forums evolve in my experience. Sometimes you decide they need a change or don't need to happen anymore. Sometimes you think they need to happen but others just aren't feeling it for whatever reason. Generally I like to stick at things for a while before deciding it's time to stop. Start the week with data has been going since May 2022. It was last year when it really came into its own. So yes, sticking at it.

Start the week with data is informed by the principles I set out in that data conversations blog post, especially "conversation not presentation" and "keep it frequent".

Every Monday morning we have a 30 minute video call with a speaker on something to do with data at Citizens Advice. Usually it's 15 minutes speaking to some slides and then the rest of the call for that all important conversation. I chair or maybe it's conduct the session. I sign off every session with "thank you for starting your week with data" which is neat and hasn't got old yet.

The invite list started as the Senior Leadership team but has grown organically to be wider than that. Every session is recorded and linked from a rolling log which is a Google doc. Slides are linked there too if they were used. All the presentations are good, but especially good ones are marked on the log with a fire emoji. Finally there's an email at the end of every week to the invite list telling them what's coming up on Monday and linking to the materials from that week's session.

The emails are especially repetitious. Also my inbox is mostly people declining Start the week with data. But every week numbers are pretty good - 25 to 40 people from all over this fairly big and diverse organisation.

The thing that made Start the week with data in 2023 so good was the variety in the programme. This forum isn't just the data experts talking about the expert data work they've done (although we have that too). We had 35 sessions with 24 different speakers from teams like

• Operations
  
• Finance
  
• Business Development
  
• Technology
  
• Policy and Advocacy
• Product and Delivery
• Evaluation
• Expert Advice and Content
• and more

The point is that everybody is working with data and can tell an interesting story about it that will be relevant to others.

One of my favourite sessions last year was our Finance team talking about the most successful systems migration I've ever seen. I'd never known a systems migration go well. And there was a strong piece of practical data improvement at the heart of the work, taking the opportunity to change data structures so that they could provide more meaningful insight.

We also get brand new insights at the session, like investigating access gaps for our clients across England and Wales. And we see things that absolutely everybody in the organisation should know, like fundamentals about our client base or how we measure our financial and social impact.

Another benefit of sticking with a forum like this for a while is you get the opportunity to revisit topics and see how they've developed. Or you hear about the next steps in a piece of data work that have been facilitated by something delivered earlier.

Managing to do 35 sessions isn't bad, especially accounting for holidays. It is a fair amount of effort to curate a programme like this, but the fact of doing that helps to build connections and visibility across the organisation for me. There's also something rewarding about encouraging people that they have an interesting story to tell. Every week I say please volunteer a topic if you want to and on the occasions people do come forward it makes me happy. But most of the effort is on me to keep that forward plan going. I think it's worth it though.

Recently a colleague said how much they appreciate the session because it's always relevant to them and their work and it gives them insight into what's happening at Citizens Advice in an effective way. I will take that as a win.

This is the work. It's fairly simple but it does require sustained effort to keep it going. If you're in a role where you're leading data improvement in a fairly large organisation I recommend giving something similar a try.

Thanks for reading.

We are blogging about the work we do. We intend to write a post each month with contributions from various members of the team. Here’s our previous post. 

We haven’t managed to post for a while because we’ve been especially busy. We’re working as a team to get around that in future. Dan’s going to try transcribing oral updates from the team so that we can all be contributing to writing in the open on the internet. 

As a team we believe in continuous improvement, building capability, and focusing on meeting data users’ needs. The opportunities we have for improving the way we work with data at Citizens Advice aren’t unique to our organisation. It’d be great if the practical approaches we take are useful to others.

Data Glossary

Josh (Data Architect)

Earlier this year we ran a survey for all staff to understand the experience of using data at Citizens Advice. This gave us valuable insights into opportunities for improvement, revealing that people want data to be quicker to find and easier to understand. 

One of the ways in which we’re trying to help is through the creation of our organisation wide Data Glossary. This is a hybrid of a traditional business glossary, data dictionary, and data catalog. It has a number of different components and aims to be a central hub to answer data related questions from across the organisation. We’ve created this in Google Sheets for now, as it was quick to create, change, and share widely across the organisation.

Here’s an outline of the sections we have so far and their purpose:

Terms

This section is what you’d expect to see from a business glossary but also includes some high level technical information. It lists our main data entities, describes what they mean in the context of the organisation, synonyms, which systems they can be found in, and the suggested way to uniquely identify them. 

Its main purpose is to encourage consistent terminology and language, as well as promoting data standards. It’s also been an important collaboration tool for our team to discuss with people from across the organisation on what their understanding of these terms are.

Data reports 

We created this section as a way to make it easier for people to find the reports they’re looking for. We’ve listed our main products, explaining what they can help with and which team can help answer any questions. It's particularly useful for any new members of staff as it gives them one place to look when trying to understand what data is widely used. It also provides a single place for anyone who’s trying to find a report that has been mentioned that they didn’t know about. 

Data sets 

This part is the start of our journey towards building a shared metadata framework. It’s a collection of our data assets, describing what they are, where they can be found, who has responsibility for maintaining and links to supporting sources, such as the data asset information document we’ve recently launched (more to come on this in a later post!)

We think this section has the biggest potential for making data more accessible across the organisation and has opportunities for automation with our data platform. 

The main purpose is for people to be able to search for a dataset they’re looking for and reduce the time spent searching for a link or the right team to speak to.

We also have sections with training videos, team-specific supporting documentation and links to external data sources. 

We’re continually revising these to improve their usefulness. 

What’s next?

We need to measure the current usefulness of the Data Glossary. For this we will start research with users on their experience using it. The data collected from this will influence any future changes we make. 

One proposed change we plan to make is to link out to data models for our main entities. The intent here is to raise the profile of these data models and provide a place to reference when creating existing data concepts in a new location.

We’ll also be monitoring the sustainability of keeping this data in Google Sheets, as well as exploring opportunities to automate the capturing and updating of metadata. 

Technology growth in this space has led to a number of emerging platforms offering great solutions for metadata management and we’ll explore whether we can make use of any of these in the future.

Though it’s early days for our Data Glossary, we’re seeing a consistent level of people using it and we think it’s making our data more accessible.

Learning from our regular open sessions

Since April Josh and I have been running regular sessions over Zoom about data architecture and data strategy. These sessions are open to all our colleagues at National Citizens Advice. After a couple of months we reflected on how we could improve them. We thought the main issue was that they weren’t inclusive or diverse enough.

We had been running the session at the same time and day each week. We changed this so that the schedule is a 5 week cycle of Tuesday / Wednesday / Thursday / Friday on subsequent weeks and then a week off. The sessions are all at different times of day to accommodate different working patterns, and they're not all at lunch time. 

We also wanted to bring in external speakers so that it wasn’t just Josh and I doing the talking. We had 8 different speakers from other organisations on a broad and fascinating range of data topics. Several of these sessions resulted in follow up calls to get into more detail on topics, like a session on data governance with Jenny.

We recognised that not everybody is comfortable asking questions on a call, so we allow for that when it’s a session with Q+A but we’ve also run more creative workshop sessions using Google Jamboard. We put more effort into communications and publicity too. Ahead of every session we email the invite list to tell them what we’re going to be covering so people can choose if they’re interested, and we include the recording and any materials from the previous session too.

Our open sessions are giving us a regular opportunity to showcase the wide range of data work that we do at Citizens Advice. It also provides an opportunity to get wider input - the sessions have been really helpful to me in developing a data strategy for example. We’ll keep reflecting on how they’re going and making improvements.

We want the barriers to entry for working with data to be as low as possible and work hard as a team to achieve that. As one of our guest speakers Adam put it we are normalising talking about data.

Interesting links

“Why data scientists shouldn’t need to know Kubernetes” via Jon

We wanted to start blogging about the work we do. We intend to write a post each month with contributions from various members of the team. Here’s our previous post from July.

As a team we believe in continuous improvement, building capability, and focusing on meeting data users’ needs. The opportunities we have for improving the way we work with data at Citizens Advice aren’t unique to our organisation. It’d be great if the practical approaches we take are useful to others.

Building better data products that meet users’ needs

Hamza (Senior Data Analyst)

The Citizens Advice service is made up of Citizens Advice - the national charity - and a network of over 260 local Citizens Advice members. Our network members are all independent charities delivering services to help people across England and Wales. At the national charity we have a responsibility to support each local charity to deliver their services in the best way possible.

There is a leadership self-assessment every year. This is where we ask each of the 260+ charities to rate themselves across organisational areas such as Governance, People Management, Equality, and Financial Management. At the same time, performance assessors from the national charity carry out the same assessment on each local charity. Where it’s evident that there are opportunities for improvements the national charity works closely with the local charity and suggests specific courses of action. 

The leadership self-assessment is vital because it embeds risk-based thinking, helping local charities to assure that their organisation is well run. It also accredits local charities to external quality standards that are recognised by funders.

In the national organisation we have a dedicated team that looks after all activities related to the leadership self-assessment. This team also includes a small number of performance assessors whose role is to assess each of our network members at least once a year. 

To help with monitoring these assessments, the team was using a tracker document in Google Sheets. The aim behind this tracker was to help plan resources, track how many assessments have been completed and for which charities, and analyse results from the assessments. 

As this tracker was also being used by several other teams, over time it became congested with multiple tabs, showing different pieces of information in different formats. In some cases, there were even multiple tabs that would show the same information but in different formats. It became evident that the tracker was losing focus and there was just too much information in different formats, not to mention that there were also broken formulas in some places too! It sounded like a job for the Data Science team... 

We collaborated extensively with the leadership self-assessment team and met with them to understand the problems they were facing and their data needs. Our task was to create a new, more consolidated and effective tracker tool that would enable its users to better plan resources and gain richer insights into the assessments. 

We discussed things such as

• which is the best platform to use (Google Sheets or Tableau)?

• which data is needed? 

• which format should the data be presented in? 

• how should the data be updated?

• who should have edit access to the data?

Having these initial discussions really helped us to build the right tool for their needs. For example, the stakeholders had a stronger preference for Google Sheets, therefore we built the new tool in Google Sheets and chose not to use Tableau. 

We adopted an agile approach to building the tool. We didn’t just go away and build a complete tool in one go for them and then think ‘job done’. Instead, we built it in steps. We built a version 1, presented this to the team, received feedback and then built version 2, presented this to the team, received feedback and so forth. In each version, we kept on refining the tool based on the feedback, making it easier to use. 

We focused more on building clear visualizations as opposed to just building tables of data. We also focused on making the tool interactive, for example building features that would allow users to extract specific data for their own needs. We try to encourage this kind of ‘self service’ as much as possible.

In the end, after several iterations and meetings with the stakeholders, we had built a tracker tool in Google Sheets that could better serve the needs of the leadership self-assessment team. The tool has been created in such a way that it involves little to no effort in terms of maintenance as all the data is updated automatically. The tracker tool is linked to a Google Form that is completed by the performance assessors for recording scores from each of their assessments. Therefore, as each new response comes in, all the visuals and data summaries in the tracker are updated automatically (this is one great benefit for using Google Forms for automatically analysing form data in any way you want). 

The new tracker tool we created has received great feedback from the end users. They feel they have something more focused, and that it helps them to answer the questions of the data that they need to ask.

Connecting data to give us new insights

Sarah (Channel Reporting Manager)

Josh (Data Architect)

Jon (Data Analyst)

How we identified the opportunity

There have been a few new hires recently in the Data Science team which has helped increase our bandwidth and allowed us to take on additional work, like solving long-standing problems. The formation of the Channel Reporting team has resulted in improved reporting, and we found that in order to take it to the next level and better meet the needs of users across the organisation we needed a greater level of detail in the data about advisors. 

Alongside this, the team also has data architecture skills now, which resulted in a detailed system context map being drawn for how data moves between products and teams at Citizens Advice. We call it ‘the data landscape’. This map allows us to hone in on improvement opportunities, for example we discovered a reliance on spreadsheets for managing advisor data.

What did we discover

We’ve historically had decentralised management of advisor data. This has worked fine for reporting and analysis, but we’re always looking to improve. We didn’t have a unique way to identify the same advisors across various systems and it has created inconsistencies in how the data is structured. Having this data would mean that our users could get a better and more joined up view of the performance of their service. It would help show activity data across channels, rather than viewing each channel in relative isolation.

What we did

We needed to change how we viewed the relationship between an advisor and the systems they use. For this we created a conceptual data model for advisors, which helped show the commonality across systems and what we could use as a common identifier. We found a number of different systems in which advisor profiles existed. The one where most of our advisors could be found was in our user authentication product. Advisors use this product in order to access various systems and it provides the best opportunity to have a common identifier for an advisor. 

The majority of our channel systems are connected to the user authentication product. This allowed us to quickly map the common ID to the advisors in those systems. In a system that isn’t currently connected we used our data skills to get to 80% of mapping identifiers to all advisors by matching google sheets via various VLOOKUPs.

This left us with just 1,100 advisors in that system that didn’t have a direct login or name match to an ID. We knew that many of these advisors probably did have a central user authentication ID - but weren’t matching due to inconsistencies in naming conventions and data entry errors - for example an advisor with local login “London Jon L” might have an existing ID, but registered against “SELondon Jon L”, or “London JJon L”, or “London Jonathan L”. [1]

Rather than match all of these manually (which would have taken us about a minute per advisor, based on the advisors we did match by hand), we wrote a quick script to help us look for these loose matches that may have been caused by human error. Our script used local advisor metadata such as office location to narrow its search criteria to a few hundred possible IDs out of thousands, and then identified close matches using a fuzzy matching algorithm. 

Fuzzy matching is a simple technique for matching text that’s approximately but not quite the same. This is super useful for identifying close matches caused by typos and nickname variations, which is a pretty common problem faced by volunteering charities! We used a standard fuzzy matching package for Python, but the algorithm can be found implemented in most languages, and is even implemented as a standard formula in Excel.  For example, with the package we used, the typo “Josh Tedgett” matched up against “Josh Tredgett” with a similarity score of 0.97, and less immediately obvious mismatches caused by nicknames like “Becky Harlow” and “Rebecca Harlow” still returned a high similarity score of 0.77. [2]

While this method returned a fair amount of false positives, it was much faster to work off these loose matches and either confirm or discard the script’s matches than it was to match all 1,100 advisors by hand. After writing the script (which took about an hour) and running it, it took us about an hour to match 600 of the remaining 1100 advisors, and discard the rest as having no likely matches. At the original rate of about a minute per advisor, it would have taken us about 18 hours to work through the same list manually.

Once we’d completed the matching process, we still had the challenge of maintaining this data set going forward. How were we going to make sure new advisors had the ID assigned to them? This is where colleagues in Technology have stepped in. They proposed an automated solution via our internal ticketing system. This will take the effort away from our Operations colleagues to maintain the data. Instead, an orchestration tool will pick up the new user requests and assign the ID in the consolidated dataset. It will also allow us to monitor the quality of the process as it can pick up any errors that have occurred.

How will it work and what this helps with

Once this work is complete, we will be able to drill down to a greater level of detail in our data products. We recently set up a new report that analyses performance for local Citizen Advice charities across all channels (phone, chat, and email). This report gives a more joined up view than before, and with our improved data the next iteration will give our network members deeper insight. 

Things to think about in the future

We think the new process will provide better insight for our users and more robust data management. There are still some aspects to work on. Part of the reason why we were able to progress on this work is that we weren’t aiming for perfection. We focused on making something better now and not trying to create anything that was difficult to upgrade or replace in the future.

Thanks for reading. Feel free to get in touch with any of us on Twitter or LinkedIn.

Footnotes

[1] This is not a real example

[2] This is not a real example

We wanted to start blogging about the work we do. We intend to write a post each month with contributions from various members of the team. 

As a team we believe in continuous improvement, building capability, and focusing on meeting data users’ needs. The opportunities we have for improving the way we work with data at Citizens Advice aren’t unique to our organisation. It’d be great if the practical approaches we take are useful to others.

Enabling self-service for users

The Citizens Advice service in England and Wales is made up of the National organisation and over 260 independent local Citizens Advice charities. Our work as the National charity involves helping to coordinate the efforts of the local charities so they can provide national level services as well as pooling expertise, tools, and insight. 

One example of this collaboration can be seen through our efforts to help manage Adviceline, our national phone service for England. The Data Science team currently provides an Adviceline Tableau dashboard to help local charities understand call volume and other performance and demand data. However, over 230 local Citizens Advice provide the service and they have a wide variety of reporting requirements. It’s impractical for the Channel Reporting team (currently 2 people) to provide over 230 unique daily reports. The Adviceline dashboard we provide is insufficient to meet the individual needs of local charities, even if we do our best to respond to user requests. 

So we decided to enable Tableau web editing, allowing all users to edit dashboards as they see fit. This will help to democratise data, putting additional power in local offices’ hands, whilst freeing up Data Science team capacity and allowing us to provide more advanced models and better quality data and reporting for other channels like web chat and email. 

Two potential barriers to successful implementation were the impact on load time for reports due to server capacity, and wanting the new features to be accessible to  staff in local charities. To tackle the first problem we sent surveys to help estimate demand for the feature from local charities and also estimate the maximum concurrent Tableau users. We then used this information to simulate the potential impact on server load time. To tackle data skills we will work with our Facilities team to deliver training via live sessions. These sessions will be recorded and later published on our internal learning platform to allow participants to revisit information. 

Tableau web editing gives us an opportunity to facilitate the local charities’ data needs like never before. There’s also the opportunity to build skills in the local Citizens Advice network so they can use our trusted data to provide high quality reporting that meets their individual needs. We’ll continue to monitor the implementation to measure impact and look for further ways to improve reporting across the network. 

Automating away the busywork - reducing a 1 - 3 hour daily process into 10 minutes of automation

Jon (Data Analyst)

I'm part of the Channel Reporting Team. Among other things, we provide data and reporting for remote help services across phone, webchat and email. By making this information readily available, the 260+ local offices in our network can access the data required for deploying their staff and volunteers as effectively as possible.

Sarah wrote about our Adviceline reporting, and we also provide reporting for the Welsh Advicelink service. There's always been a pressing need for this kind of daily reporting - especially with the pandemic causing a historic rise in people seeking help through remote services - so we can understand how well our organisation is meeting clients’ needs. 

We want to be as efficient as possible in this work, and make sure that our data is trusted. We recently made some improvements to how we provide this data.

Our reporting workflow was previously very complicated due to a number of factors:

• Access to even basic data from our telephony system requires manual effort.

• Operational adjustments were (and still are) made everyday to try and adapt to changing conditions - and our reporting needed to reflect that. The way in which the team had to deliver their reporting changed multiple times during the pandemic, making it difficult to ratify and streamline a single process for daily reporting.

• With so many different teams and offices having input on the reporting, our reports needed to pull in information maintained by many different stakeholders across several disparate sources. These sources include various SQL servers, a bunch of different google sheets, the aforementioned manual effort for data access, and daily call record CSVs sent to an email account.

As a result of these constantly shifting conditions and ad hoc adjustments, we ended up with a highly manual process that was difficult to run, and error prone if you didn’t have the requisite skills and knowledge.

When everything went right, the process would take about 45 minutes to complete every day. If there were complications the process would take anywhere from 1 to 3 hours to execute.

With almost a hundred different manual steps along the way the process was error prone. If you made a mistake at any point during the process, you had to catch it immediately or face restarting from scratch. If you didn’t catch an error it could take an entire day to roll back.

For a process producing a daily report with a noon deadline, this was clearly unacceptable. We knew we had to make it easier on ourselves, and reduce the possibility of errors along the way.

Using some simple Python scripting and Google Apps scripting we managed to distill the sequence into a largely automated process that takes ten minutes a day. Here's some of the things we did to achieve this:

• We built a Selenium Browser Automation script that crawled the telephone records we needed, and dumped them straight into our python scripts for ingest.

• We used Google Apps scripts to automate sourcing the data we needed from Google Sheets, and using pandas to perform the necessary lookups and data manipulation.

• We used Pyodbc to give our Python scripts access to the SQL servers they needed to both draw data from and update.

• We wrote a series of unit tests ensuring that the processes executed by the script all pass basic common-sense checks and don't contradict our knowledge about our telephone systems’ operational setup. In addition to these unit tests, the script still asks for a human to sign off on the changes it's about to make - while this means the script isn't fully automated, it means that we're aware of any recent changes made by the process, and can catch stuff that simple common-sense logic tests can't.

Pulling these changes into a Python prototype took about a day, but iterating and readying the script into a production-ready state took the better part of a week. The unit tests themselves were written iteratively over several weeks, and the scripts still get tweaks whenever we see room for improvement. The end result is that we no longer have to dread wrestling with our reporting processes in the morning, and we save hours of development time every week which we can spend on higher value work.

User feedback is invaluable thought not always easy to get - data survey

Josh (Data Architect)

Throughout this month the data team has been running a survey to understand what it’s like for staff to use data at Citizens Advice. We used Google Forms. 

The survey is a way to gain insight into how accessible our data is, the level of data capability across the organisation, and which data products are currently helping people with decision making. It can be a challenge to get people to participate in surveys. Thankfully we had support from executives and our internal communications team which gave us the response rate we aimed for.

We’ve now been able to calculate baseline metrics that give us an indication of how easy our data is to find, how confident people feel using data, and how often data is used in decision making. Of these three things, finding data seems to be the biggest pain point. Though this isn’t unique to Citizens Advice it is a challenge for us to solve. The survey also has a lot of insightful qualitative data that we’ll be digging into next month and following up with more focused research on specific areas.

Overall it’s been a great way to collect thoughts about our data from across the organisation. It’s also providing an indication of the areas we should be focusing which will make the experience of using data better for all. We will repeat the survey quarterly so that we can measure the impact of the improvements we make. To encourage the same level of participation as in this round of the survey we’ll be demonstrating how we’ve acted on the feedback to make a meaningful difference. We intend to extend the survey to the network of local Citizens Advice as well.

Building the team

Dan (Head of Data Science)

I love recruitment. Having the opportunity to build and lead a team is a privilege. This month we’ve had a new starter Rahul who is a Web Analyst working in the multidisciplinary Content Platform team. This is a model that I want to establish when it’s the best fit for the work. Having a dedicated data specialist in a product team is much better for meeting that product- and team’s data needs than being at arm’s length.

Rahul is being inducted remotely and I think that’s more challenging than being in person. On the other hand, I definitely prefer interviewing remotely. Maybe there’s something about it being a leveller, balancing out the dynamic and making it less intimidating than having to visit somebody’s office. Maybe a child will get locked out of somebody’s house in their pyjamas and it’ll be alright to rescue them midway through the call. 

Did I say somebody’s house? Ok I meant my house.

We had a successful interview campaign for a new Data Science Lead this month. I really enjoyed interviewing with Josh, and with Maz (Director of Content and Expert Advice). I have definitely learned from others  in the past year and improved my interviewing practice. There have been small things, like introducing yourself with your pronouns, or pasting the text of each question into the video chat function. There have been larger things, like putting the effort into producing a scoring guide for each question so the panel is working from a shared, explicit understanding. Writing the scoring guide takes me considerably longer than the questions. Finally there are things that mean you’re being explicit about your organisation’s values with candidates. I’ve included a question about equity, diversity and inclusion in interviews for a while but I’ve never been completely happy with how it was phrased. For this campaign Maz introduced a new formulation that was a big improvement.

Thanks for reading. Feel free to get in touch with any of us on Twitter or LinkedIn.

Interesting links 

• https://moderndatastack.substack.com/p/the-power-of-the-written-word 
• http://worrydream.com/LadderOfAbstraction/ - A demonstration of how better interactive visualizations make decision-making and design faster and easier.
• https://explosion.ai/blog/applied-nlp-thinking - An excellent article from Ines Montani about applied NLP - how do we make the jump from a business problem to a production-ready NLP solution? 
• https://data-collective.org.uk/2021/07/21/leadership-data/ - Why we need leadership on data in the voluntary sector, and how you can get involved

The title of this post is a question for me to answer and nobody else. This isn't a request for help or advice. I haven't been writing on the internet recently but something came up last week and I thought I could do some of that 'working out loud' like I used to. 

The session

About six weeks ago I started a weekly 'open session' about data architecture and data strategy with my colleague Josh (data architect). The idea was to start small and see if a regular 30 minute video call with an open invite would be a useful vehicle for sharing work and progress, and testing our work with colleagues. Josh and I were alternating each week on who would take the lead, so data architecture one week and data strategy the next. 

The problem

The working out loud

Footnotes

[1] If you've met me in real life you know that I can speak really slowly so watching a video where I'm talking at 1.5x speed works pretty well.

I work at Citizens Advice. The Covid-19 pandemic has had a dramatic impact across our services and seen an incredible response from staff in the organisation, for example

• Unprecedented demand for our website content
• Creation of new, trusted expert advice content at speed
  
• Stopping the provision of advice in person at the 250+ independent local Citizens Advice locations due to lockdown measures
  
• A resulting shift to providing advice through our other channels, such as telephone
• A pronounced change in the patterns of issues that our clients are coming to us with

I lead the Data team. I work closely with my colleague Tom, who leads the Impact team. Broadly speaking, my team is responsible for making data available, and Tom's team are responsible for asking questions of it.

On 19 March Tom and I were asked to draw together a wide variety of operational data into a single place, to help management and leadership navigate the crisis. It would include activity and demand data for various channels, and data on client demographics and breakdown by issue. It would also include a summary for a quick, headline view.

Citizens Advice colleagues have spoken about our data and what it is telling us on social media and in the news in the past couple of months. Rather than talking about our data myself, in this post I wanted to reflect on the process and experience of "making the thing", rather than what's in that thing or what it means.

It's been a really rewarding experience and I have learned lessons from it that I thought would be worth sharing.

Get something out there

It was crisis time, and we received an understandably loose brief as a result. We brought together a group from both of our teams, and came up with a first iteration in 4 days.

We made a spreadsheet. Spreadsheets are great.

It is a spreadsheet that's intended to be read by humans, rather than a spreadsheet that's a data source. We collectively agreed to making a spreadsheet, having been given a steer not to build something in the proprietary data analysis tool that's widely used at Citizens Advice. Initially we thought it could be a slide deck, and I had a strong view it should be in Excel, but the consensus was to go with Google sheets. G-Suite is what we use at Citizens Advice, and sheets has the advantage of being easy to share and less of a daily overhead to maintain than a slide deck.

Ideally we would have had a better understanding of user needs, and some clearer questions to be asked of the data. Regardless, our first version was well received, and put us in a position to improve it regularly. Me aside, the team we put together has a really good understanding of how Citizens Advice works, and I think this helped with our initial and continued success.

Ask for expert input

We have a wide range of expertise at Citizens Advice, including a really strong digital team in the form of Customer Journey. I was able to ask for input from a designer and content expert before we circulated the first version of the report. This really improved the product, helping it to be clearer and easier to navigate. Later, when we were trying to understand how the report was being used, I had input from a user researcher on survey questions.

Even if you aren't working in a 'model' multidisciplinary team, it doesn't mean you shouldn't take a multidisciplinary approach. And if you don't have this kind of expertise to hand, just asking somebody for an objective view and opening yourself up to constructive criticism is always good.

Work to understand your users

Keep iterating

The report has been constantly changed over the past couple of months, in response to new requests, changing circumstances, and a team attitude of continuous improvement. It's been good to be part of a group where it's understood from the outset that the thing will never be complete.

I think this has been supported by the team practice, which is curious and conversational. Speaking of which...

Have regular discussion

The first stage of the crisis called for daily updates. This is unsustainable over time, and developing the report helped us to understand the various states of the data we have. Some data is easy to automate, whereas some requires a large amount of manual intervention and also changes shape regularly, making it labour intensive to report on. This has been a helpful secondary outcome of the work, because it can help inform where we put effort to improve our underlying systems and practices.

Not everything we've done was useful, or used. So we've taken things out. In future, I will work to understand what's being used in a more methodical way. I missed an opportunity to ask a question in the first survey - "which tabs are you using?" and a pick list. We also tried to track usage using Google analytics, but it was unsatisfactory.

Due to the iterative nature of the work and the regular discussion of patterns, it also became clear to the team that the time periods for significant changes in the data were longer than daily. If we hadn't kept developing and discussing, we might have been saddled with a daily reporting burden for longer. This also gives us a sense of the various cadences of the different operational data we have at Citizens Advice. Not everything behaves like the instantaneous spike we'd expect to see on our website after a government announcement, for example. Our service is broad and varied, and to my mind that variety helps us to meet the complex needs of our clients.

Help people to understand the questions they need to ask

I think one of the most important areas of work for data maturity in public service is to help people to formulate the questions they want to ask of the data. It helps to focus the discussion and build of any data product. Generally speaking, "give me all the data" isn't the best starting point and there's no intrinsic good in having large amounts of data, hoping that some incredible truth will emerge from it.

In my experience over the past couple of months these questions have increasingly started coming from colleagues, which is great to see. A polite challenge of "what questions do you want to ask?" has been useful.

You don't need to serve everybody with one thing

I think we've resisted putting ever more measures in our report just because it's the first time so much of our data has been together in one place. The survey showed us that our users have different needs, and we recognised these might be better met by other products in future.

I said earlier about the thing never being complete, but that isn't the same as it not being finished - 'finished' as in set aside in order to move on to the next thing, taking what you learned with you.

Footnotes

Words are important. Have you heard this phrase 'data-driven'? I expect you have. I don't like it, but hey there are lots of things I don't like that are really popular nowadays.

One of the issues with buzzy management shorthand phrases that their use actively inhibits a shared understanding. One person's 'data-driven' might not be the same as another's, but they're all there in the boardroom [1] going 

"Franck, our strategical approach is data-driven and leverages machine learning cloud AI capabilities"

"Yes Wayne, 100% data-driven insightisation of our asset base"

"Totally Sandra, leveragise our data-driven operating model"

or similar.

For a while, I was working on a 'data-driven' website, where I think 'data-driven' was being used to mean "there is data from internal applications that automatically goes onto our public facing website". 

I always found that a bit strange, because (to my mind) that's just a legitimate way to approach making a website [2] and I don't understand why you would make part of what’s going on behind the pixels on the screen a thing of note. I don't think everybody involved understood that was what the phrase was being used to mean either. 'Data-driven' had become meaningless, and that meaning vacuum got filled with negative connotations and disdain, like a sad golem.

Say what you mean

I always preferred 'data-driven' meaning "we make decisions based on evidence". However, as I've worked on data strategy and (particularly) measurement in the past year, "we make decisions based on evidence" is also problematic.

Understanding intent and organisation's direction and focus is essential. If this is lacking in any way it can be disproportionately hard to develop goals and measures. Clear statements of intent really help to frame decisions.

So now to me 'data-driven' means "we know where we're going, and we make decisions based on evidence". But it's still not right.

What evidence?

So now to me 'data-driven' means "we know where we're going, and we make decisions based on sound evidence that is contextually appropriate".

Footnotes

This is a continuation of my line of thought from my previous two blog posts.

I speculate that there's a gap to be filled with ways of working with data that aren't happening at the moment as far as I'm aware [1].

Isn't it exciting when you notice that ideas have taken on a life of their own? Not my ideas, to be clear.

Those moments when you catch a hint that similar conversations are happening somewhere else and you're hearing part of an underground transmission. Like a story being passed around from person to person before anybody actually writes the book.

Blog posts. Talks. Snatches of something new. I reckon people are going to start to crack this 'talk about data' business fairly soon.

I read the 'Rewired State: 10 years on' blog post by James Darling and Richard Pope. Two paragraphs stuck out for me in particular (emphasis mine):

Legacy IT is still a major problem that the GDS movement hasn’t really addressed. The data layer is still unreformed. It remains an unanswered question if the UK’s ‘service-layer down’ approach, the ‘registry-layer up’ approach of Estonia, or some totally different model will ultimately prove to be the best route to a transformed government.

Both legacy and data lack a place in the new orthodoxy, and in user centred design more broadly. That’s probably because they are difficult and require political capital. It’s hard to justify investment in digital infrastructure when the benefits come in the next electoral cycle or to another department’s budget.

There's a scene in 'Velvet Goldmine' where Christian Bale's character (a young, awkward glam rock devotee at that point in the film) points at his hero on the television and says to his parents "that's me! that's me that is!". I think the data layer is still unreformed! I think data lacks a place in the new orthodoxy and in user centred design more broadly! [2]

A new new orthodoxy?

I met with Leigh from the Open Data Institute this week. We spoke about this broad topic, and the conversation helped me on with my thoughts. Claire joined us, and suggested that 'data' is a decade behind 'digital' in terms of developing and embedding a multidisciplinary working practice. This resonated with me, and I've heard others suggest similar things in recent months (the underground transmission!).

I swear there's something here. Something distinct from 'digital', but complimentary and with porous boundaries.

Technology is about computers. 'Digital' isn't about computers but lots of people still think it is. Most people think 'data' is 100% all about computers but actually it's even less about computers than 'Digital'.

In this multidisciplinary data practice I imagine being good with computers is a secondary skill - a means to an end. The engineering piece for services can be done collaboratively with others, and I'd expect increasingly over time it will become less bespoke [3]. If data lacks a place in that new orthodoxy maybe it's time to revisit some unfashionable roles, define new ones, and hire some librarians. Where I work we've got a couple of libraries. I'm a big fan of librarians.

So, maybe a practice featuring the full spectrum of data related roles, from the structural to the analytical.

A sort of infrastructure

In my second post I described the data I am most interested in, recognising that 'data' is a broad term which would benefit from more detailed definition:

• Reference data
• Master data
• Accurate representations of my organisations' most complex entities and rules that are derived through collaborative work with subject-matter experts, appropriately maintained, and reflected in the small number of systems we have that are necessarily specialist [1]
  
• Controlled metadata that enables discovery, aggregation, and re-use
• Any externally recognised structures or standards that increase the utility of my organisations' data on the web

Footnotes

I wrote a blog post yesterday and got a really thought-provoking response on Twitter so I thought I'd expand on it some.

It wasn't much of a post. I could reduce it down to "I think this is hard and I want to get better" with a little bit of a call to arms thrown in.

Being connected to an expert hive-mind on the internet is pretty great though. I don't know much, but I know I love data, so I'll try to reflect what I picked up from others since yesterday and hopefully frame where I'm at for future discussions and work in real life.

What are we talking about?

'Data' is too broad. Also labouring the point about a distinction between data and information probably isn't helpful. There's a huge amount of varied work across the 'data and information' spectrum. I think I need a richer and more specific vocabulary here to be able to work with the experts, let alone my desired broad audience. Credit to Michael for the spectrum description, Paul for describing the variety of work, and Sophie for calling out the pointless data vs. information debate.

What am I talking about?

At the time of writing I'm the Head of Data and Search at the UK Parliament. One of the things I'm responsible for is a data strategy for our organisations. I believe it's both possible and necessary to have an appropriately consistent set of principles and practices that apply to all of our data and the varied ways in which it is used.

Still, I have some particular areas of interest:

• Reference data
• Master data
• Accurate representations of my organisations' most complex entities and rules that are derived through collaborative work with subject-matter experts, appropriately maintained, and reflected in the small number of systems we have that are necessarily specialist [1]
  
• Controlled metadata that enables discovery, aggregation, and re-use
• Any externally recognised structures or standards that increase the utility of my organisations' data on the web

Services and infrastructure

However, a utilitarian approach to data can come with problems. I thought Sophie describes this particularly well in the context of building 'digital' services [2]:

...in “digital” they expect us to iterate and release quickly, and we’re like “now we need to go away and spend many many months/years restructuring all our data and information at vast expense before you can have the [service]”

No controversy or criticism of anybody intended here.

I expect the chances are going away to restructure all your data and information at vast expense isn't going to fly. So, I believe there can be a tension here between the 'digital' work and the 'data' work. Maybe lots of people have resolved this already, but hey I haven't [3].

Let's say it's ok, or better yet let's say it's desirable, to release your new 'digital' service when it's 60% complete, and then you iterate. In contrast, you don't want to release with your reference data being 60% correct. That would be bad.

To me, 'minimum viable data' is a small but irreducible core. You iterate outwards rather than throwing it all away.

^ note this is my only strong idea maybe I haven't expressed it as well as I could have done but I reckon it's good.

I advocate for working on fundamental data infrastructure and digital services at the same time. In an organisation that does more than one thing, there's going to be a need to share the kinds of data I've described across services. Starting from scratch for each service increases cost at the very least. Also, the amount of effort an organisation puts in to data integration is an indicator of how bad its data is because it should be as little as possible - high effort data integration is like data failure demand.

I have views [4] on what a cross-cutting data infrastructure team looks like, and I think having data expertise in multidisciplinary 'digital' teams is important as well. I don't believe that multidisciplinary teams focused on a particular service are best placed to lead on data infrastructure.

I suppose distance from users for infrastructure teams comes with risks, particularly because several people responded to my initial blog post to say that context was essential, and that rather than trying to talk about data in the abstract it's better to talk about it in terms of problems to solve or outcomes to achieve (thanks Franklin, Amanda, Ann, and Chris). I'm sure there's a way to balance this effectively.

Tips and tactics

I had several responses to my post about sticking with the metaphors, and hey maybe I need better metaphors. Not everybody likes baking. My favourite advice was from Beck, who said that passion for the subject helps. I'll try ramping it up.

I was also reassured to see (and be reminded of) lots of experienced, brilliant people who are either going through the same thing or who've already had success. I look forward to finding out more.

If you're interested in talking about data with me in real life please get in touch.

Footnotes