The episode preview:

Bad data doesn't suddenly appear when you start a transformation programme. It's usually been building for years!

• Missing information.

• Incorrect fields.

• Duplicates.

• Inconsistent data.

• Old records that nobody knows what to do with.

And when you start a migration, all of those problems suddenly become very visible.

But do you need to fix everything? No, you don't!

The first step is understanding your data and knowing what "good" looks like. From there, you can assess the quality, understand the scale of the problem and prioritise what actually needs attention.

In this episode, we talk about how to approach data quality properly, from profiling and prioritisation through to cleansing, enrichment and governance. Because at binary10, we're the data people. We help organisations make data-driven decisions they can trust. And trusted decisions start with trusted data! 💚

In this episode, we cover:

• How data quality is the leading cause for the delays in transformation programmes (1.20)

• Why data profiling should happen as early as possible (02:35)

• The AI innovations - but are you ready for those? (03:15)

• How to assess the scale of your data quality problems before deciding what to fix (05:30)

• Why prioritisation matters when you have thousands of data issues (06:55)

• The difference between data issues that stop a migration and those that can wait (07:30)

• Why you don't need to fix every single data quality issue before go-live (08:30)

• How data quality can support BAU as well as digital transformation (09:00)

• Different approaches to data cleansing and enrichment (10:16)

• When to fix data in the source system and when transformation rules can help (10:20)

• Why sometimes the best decision is not to fix the data at all (11:15)

• Why getting your data right before go-live isn't enough (13:36)

• The importance of data governance for maintaining quality after go-live (13:36)

• Why data quality is becoming increasingly important from a regulatory perspective (14:15)

• Why modernising older systems creates an opportunity to improve data quality (15:04)

• What's next for data quality tools and data visualisation (16:03)

Data quality is more than just "bad data":

When people hear data quality, they often think about fixing incorrect records. But it's much bigger than that.

• It's about understanding your data.

• Knowing what good looks like.

• Measuring where you are today.

• And then deciding what actually matters.

Because without that first step, you're just guessing. And when you're dealing with large datasets, guessing can get expensive very quickly.

You don't need to fix everything:

This is probably one of the biggest takeaways from the episode.

When you find data quality issues, the instinct can be: Let's fix all of it. But that's not always the right answer. Some issues might prevent data from loading into the new system. Some might cause problems with business processes. Others might have no real impact at all.

So the priority should be: What needs fixing before go-live? What can wait? And what doesn't actually need fixing?

That prioritisation can make a huge difference to the cost, timeline and risk of your transformation.

Start early. Understand your data.

One of the biggest mistakes is leaving data quality until halfway through a project. By then, you might discover that you've got far more work to do than expected and not enough time to do it. That's why data profiling and quality assessment should happen as early as possible.

It gives you a real picture of:

• What data you have

• What's wrong with it

• How much work is involved

• What needs to be prioritised

• Where the biggest risks are

You can't plan properly until you understand the problem.

Fix the data. Then keep it good.

There's another trap organisations can fall into.

You spend months cleansing, enriching and preparing your data. You go live. And then...The quality starts dropping again.

Without the right processes and governance in place, you can quickly end up with the same problems in your shiny new system. That's why transformation is also an opportunity to put the right data governance, processes and controls in place.

Don't just get your data into good shape. Keep it there.

Data quality isn't just a transformation problem:

Data quality often gets talked about in the context of migration and digital transformation. But it shouldn't only matter when you're replacing a system.

Poor-quality data can affect your organisation every day.

It can impact customers, suppliers, reporting, operations and decision-making. So data quality should become part of your normal way of working, not something you suddenly think about when a migration starts.

Final thoughts:

You can't make good decisions from data you don't trust. And you can't improve your data without first understanding it.

The organisations that get data quality right are the ones that start early, understand what good looks like, prioritise properly and put the right processes in place to keep their data healthy. Because data quality isn't about making every record perfect. It's about making sure your data is good enough, accurate enough and trusted enough to do what your organisation needs it to do.

🎧 Listen to the episode!

Need help improving your data quality?

Whether you're preparing for a data migration, starting a digital transformation programme or simply trying to get a better handle on the data you already have, we can help. From data quality assessment and profiling to cleansing, enrichment and governance, we'll help you understand your data and make a plan to improve it.

Get in touch with our team: CONTACT

EPISODE TRANSCRIPTION:

Disclaimer: This transcript was generated by an AI tool that did its best, but it's never met different British accents it could fully decode. Expect a few funny mistakes. Enjoy!

[00:35] - James B

Episode five.

hello and welcome again.

[00:37] - Steve S

Yeah, great to be back.

[00:39] - James B

Yeah, and really looking forward to today's podcast.

[00:42] - Steve S

It's going to be one of the best ones, I think. This one.

[00:44] - James B

I think so. It's it's a topic that's super important in in all walks of data and and activity. Yes, there's a huge link to data migration, but you know, and data integrations and all things really. But I think it's important that we talk about it to promote it more. Yes, because I think that you know we we still want to. You know, we're Ambitious. We want to make sure that, you know, hopefully within the next five years, every digital transformation program gets ahead on data. You know, far too often, are we still coming in halfway through, or you know, when things sort of start going wrong, and you have to

sort of save the day a bit at you know greater cost. And so, the topic we want to talk to everyone about today is data quality. Okay, because data quality is probably the leading cause of digital transformation programs having delays. And you know, becoming costly. But also, we want to make sure that we talk about data quality in its own right. You know, it isn't just about digital transformation programs. It's about your data strategy. It's about how you should look after your data. You know,

sort of constantly. And we'll also, you know, we'll talk about the key themes of data quality. You know, you've got your analysis. You know, your initial upfront. You know, how do you find what's wrong? You know, what does good look like? And then we'll talk a little bit about actual cleansing. You know, actually, well, how do you make that data? Better, and also you know specifically for digital transformation programs, you know what the the theme of enrichment means because it's linked to data quality. But what actually is it? So there's a lot to get through, and it's important that we cover it, and we'll try and do do it proud. But yeah, so Steve, you know what? What's why is it so

important to you know bring that data quality sort of aspect to a to a digital transformation program? If we start there, you know why is it so important to get ahead and look at that early?

[02:33] - Steve S

There's various different. I suppose themes around sort of data quality and and and areas why it's so important. I mean, one thing from a I suppose a data migration point of view, when we're trying to load data into a new system, if the if the the data values are incorrect, they don't align with the config, you know, or you've got missing data, it's not going to load. So what do you do in those situations? Now, you mentioned about enrichment; you might have to enrich the data because

legacy systems might not hold, you know, a field for You know something that is now commonplace, but wasn't 20 years ago. So, so there's yeah different aspects: missing data, incorrect data that needs to be changed, and so that's one aspect. I mean, another thing that I'm probably going a bit off tangent here, but we went to an exhibition last last year, and all of these vendors there, all these suppliers, were talking about these fantastic data tools they had that used AI to sort of sort of represent the data and do data reporting and data visualisation and everything. They were saying how wonderful these products were, and I'm sure they are, but not one of them touched on the point that if you know those products are only as good as the data they're looking at, and nobody spoke about the quality of data in the data lakes that people are putting these you know data reporting

tools on top of. So it's not just about getting the data into your system; it's maintaining it once it's in the system, and then it goes to your data lake. What's the quality of your data like in your data lake? The best way to to get your your data lake quality good is to fix it at source in your ERP system.

[04:05] - James B

Absolutely, and yeah, and and also we can think of data quality in terms of you know housekeeping. You know, a lot of people think of data quality and they think yes, bad values or poor characters in a field or, you know, but actually, if you think about you know P O S is a great example. You know,

where you've created a purchase order and you've received it almost you know almost to the full amount. You know, you might have. A pound left, ten pounds left, and that remains open, and no one closes it. Yeah. And so the amount of times we we start a migration, and a you know in a small to mid-sized company, we'll say yes, we've we've got two and a half thousand purchase orders to migrate, and straight away we say, hmm, you shouldn't really have two and a half thousand purchase orders. Let's double check that. And it's amazing the amount of times that you know what you'll find is actually, you know, eighty to ninety percent of those purchase orders are all old. You know, not going to be received anymore. So you know, we need. Need to work at closing those off because then you know what we might reduce the volume so much you don't even have a migration to do. Yes, and that's massive, you know, and and the amount of people that can miss that simple

thing, what an impact that can have, and so and and I think if we take a step back in terms of you know how do you begin this journey when we think of data quality, you know, I said at the start there it's so important to define what good looks like, you know, we all know the format of an email address, you know, and how many characters we're allowed on certain fields and all those sorts of things, but you know. Each organisation will have their own, you know, ways of working, their own

kind of, let's call them, sort of data standards, and I think it's so important that a company, as part of this exercise, can start defining some of those because the first step to data quality is actually the analysis. It's actually looking at what does good look like. What should the data values, you know, be, or you know, within a certain range, so that we can then actually actively, you know, analyse the data and confirm, yeah, this is this is the data that's. It's got the problems, you know, because so many times, you know, people just run and write reports, and you know, especially in digital transformation, you know, a classic one is someone starts jumping up and down saying, "Our

suppliers, we've we've checked all their VAT numbers, we've checked this, we've checked that, and we've got ten thousand suppliers that's got bad data." And then actually, if you look at it and say, "Well, hang on a minute, how many of those suppliers have got an open transaction? How many of them, you know, actually within maybe the last eighteen months that would be in the scope of needing to migrate them?" And it's like, "Oh, all right, yeah, I'll add that on as well." Oh, oh, actually, it's five hundred suppliers. Suppliers that have got these issues, so you know you got to be careful not to go running off thinking you're fixing data. You've got to do it, you know, efficiently. You've got

to do it in the right way. So you know, I think it's really important for everyone to understand that data quality begins with understanding what good looks like and then measuring against that. That will give you your true picture of you know of the challenge that you've got. And then you know, I

think Steve, I think the next step for me would then be priority. You know, we'll come on to cleansing, which is another world in itself. But why Why is priority important? You know, when it

comes to those, you know.

[07:00] - Steve S

Yeah, and I was going to say just on that before I get to the priorities is planning, and I think you need to plan your your data cleansing activity. The earlier you can start actually doing your data profiling and actually looking at the data quality assessment, the sooner the better, because it's only at that point you know the amount of work you've got to do, and if you do that halfway through your project, you might find well actually we haven't got time to do this now. So do it early, do it right at

the very start of the project as soon as you possibly can. Now then, you get onto you know the data profiling, and then you need to prioritise, you know, what fixes we need to do. Now it might be there's some you can actually think and say, okay, we've got data quality issues here, but we can actually leave them till after go live. They're not a priority. You need to work out what is essential to go before go live, and of those, what the high priorities. So we're looking at maybe if we're looking at

data migration, we look at what are the the fixes that are required in the data to be able to allow us. To load the data in the first place, so that is the number one. Because obviously, if we we can't load it, we've got a big problem there. But then the second priority will be okay. Well, we can load it. We can load this data. But then there might be a missing field or an incorrect field that will load okay, but it might cause an issue with the business process later down the line. So there, the second priority. So those two are essentials. After that, you're getting into nice to haves, and that's where yes, if it's something that quick fix, get it done before go live. If it's not You know, essential, but can be done in a sort of measured way. Afterwards, you do it after go live. So don't try and think that you have to fix every single issue before the go live. In an ideal world, yes, you do that, but it's not absolutely essential. And what you don't want to do is delay your go live by three months fixing issues that actually you could do afterwards.

[08:46] - James B

No, completely agree. And and it's like again, in that prioritisation, I mean, it could be. You know, we we've delivered projects where, fantastically, we have got ahead. And then people say, "Well, yes, we know these ones will be critical, you know, to you know prevent load failures and process

failures." But oh, actually, we're not starting for another three months. Well, then you know you could switch priority to support BAU. You know, you might have some data quality issues that are impacting you right now, that are fundamentally causing your suppliers, your customers' issues. You

know, let's prioritize them for the next couple of months, and then switch the priority if we're then kicking off a you know a digital transformation program. And I think you know we'll come on to talk about it. But again, data quality isn't just about. Digital transformation. Ironically, it comes up because it's you know, as we mentioned, you know, it's the biggest cause of failure and and delay and cost. But data quality should be a you know, it should be part of your your governance. It should

be part of your everyday life as you are managing that data. But we'll come on to talk about that because I think that you know, just the way of the way of the world naturally, people do start to

consider this part of digital transformation. And now we are pushing and promoting that the fact this is a great time to then. Get your target operating model right, so that once you are live, you've got those processes, that governance in place that will keep that data, you know, keep that quality. All that analysis we talked about, keep doing it, you know, keep doing it on your current systems, and then that can prove, you know, that you're ahead of the game. So, yeah, you know, fully behind all

those prioritization things you said. So, we move on to the actual task of cleansing and enrichment. We can talk about that. So, yeah, I mean, let's talk about some of the different ways that you can cleanse data.

[10:20] - Steve S

I suppose, yeah, I mean. We can we can do various different things. I mean, it depends. We can actually maybe apply business rules so they don't have to be fixed in legacy. Our normal recommendation was you you fix the issues in legacy where possible. Now, if we can apply business rules that where we're part of the transformation of the data from legacy to the target, we can apply that anyway. It you know it negates the need for someone to have to manually go into their legacy system and and start sorting you know old data out, but. Ideally, yes. Fix the data in the legacy system, unless we can apply defaults or sort of business rules to actually change it. That

way, so I suppose that would be one way. The other one is obviously enrichment. If there's no data at all for a particular field, that's maybe a more modern field that isn't in a twenty-year-old legacy system, yeah, we would normally compile that in spreadsheets or something like that, and we would

build those spreadsheets into our transformation process.

[11:15] - James B

Yeah. And I think, and just to add to that, I mean, you know, when you know, before I started my journey in data migration, when I thought of reconciliation and then finding sort of data quality issues and having to cleanse them, I'd always imagine just, you know, a team of people, you know, having to type on the keyboards and, you know, quickly go in and correct data and spend hours and hours. And and I've seen it, right? That that is one way that you can cleanse data, and it just depends. And that's why that analysis and then having those follow up kind of assessments of The best way to fix it, because as you said, you know, always the principle is that you should fix in source, you know, because you know if you're not doing a digital transformation, that's your world anyway, right? So

that's where you need to fix it. But likewise, with digital transformation, you know, the less that you can, you know, try and fix that as part of transformation, it just derisks the whole migration. But as we all know, some of these modern systems, some of the support costs, you know, the way that these relational databases are, you know, are fully secured and locked down, it can be quite difficult. To you know, the cost can be quite high to go and actually correct bulk loads of data in in your current system. So you have to explore different efficient ways of of correcting it. And and as part of a digital transformation, you get some great opportunities to do that, as you've talked about. You know, um, you know, as part of as part of transformations, you know. But but there are tools out

there, right? There are bespoke dedicated tools that can do bulk updates to data. You know, which is which is definitely an option. I think the other one that people never consider is don't do anything. You know, I mean, let's be honest. You know, as a bit like I said, with some of those historic suppliers or things that, you know, that it's not going to be needed. If that's what it is, you know, don't prioritize that. You know, don't you don't need to cleanse that data. You know, and I think I think that's really important as well to just have a you know a fair threshold where if it's not going to cause any downstream impact, you know, if there's if there's going to be no work on those particular

datasets, you could argue that actually yes, you know, cleansing isn't isn't needed. And like in Richmond, I think. You're absolutely right. Some people get carried away thinking that you have to enrich on the source system, but again, that can be costly, or there might not be fields or places where you can put it. So that is definitely an area that would more than likely only come up as part of a transformation, or you know, you might be bringing in additional processes or services to you know to a current system. But that's where again, getting the right tools that enable you to get the new data right, get it approved, get it good to go, test it, and then obviously put it live on the system.

[13:36] - Steve S

Yeah, and I mean, obviously, spoken a lot about getting data as part of the transformation, getting the data right to go into the system. The next stage is keeping it good in the new system. And I know I've banged on about this numerous times on previous podcasts. But you spend all this money doing

all this sort of data cleansing, data enrichment, and and and getting your data in a really good place to to migrate across to your new system. And then from day one, you haven't got any processes in place, and within weeks, you're getting rubbish data again in your new system. So, take the opportunity of of going through this digital transformation to put in place sort of data governance procedures to keep your data in a in a good shape in your new system after you've gone live.

[11:15] - James B

No, absolutely, and and you know what? This you know data quality is so important. Obviously, we've focused heavily on digital transformation programs and reducing the risk of, you know, failure and cost. But you know, as we as we progress forward, and as things like GDPR, DPA twenty eighteen, all these sort of risk based assessments, they're going to become more formal, and you know, organisations are going to have to meet certain data standards. So actually, I think you know

taking data quality seriously now, you know, let let's start doing it is going to save you in the long run because I can see penalties. You know, we we're already having data breaches. You know, people you know wanting to go and you know their right to be forgotten. All these sort of data requests that

are coming in now. It's essential that the quality of that data is is super accurate. Yeah.

[14:38] - Steve S

I mean, I think the I think the you know the GDPR laws are just the tip of the iceberg in what's coming down the line, aren't they? Really?

[15:04] - James B

No, absolutely. And I think and you know and another little sort of food for thought. If if you're on systems that are you know within the last five years. There's a good chance that the validations and the controls they put in place are keeping your data to a relatively good standard. Anyway, it's more obviously, you know, those organisations that have those 10 plus, 15 plus, 20 year old systems, you know, where the controls are, you know, a lot lighter. That that causes poor data quality. And

naturally, yes, if you're doing a digital transformation program to come onto a modern system, you know, that's where you absolutely need to get ahead on data. Just to finish, then I think because we're going to do a follow up episode on this, we're going to talk more about. You know, we've talked about doing the data cleansing, doing the data quality analysis. But what tools can help with that, and why are they so valuable? I know we're gonna we're gonna come on to to that in our next episode. But no, it's a subject that you know we're super passionate about, and we've got to keep promoting. And yeah, hopefully the the next session will sort of open that up a bit more and and give people a bit of a view of of what it actually looks and feels like.

[16:03] - Steve S

Yeah, I think I think it's it's key that you know, as part of identifying the data issues that you've got, you need to. Need to be able to sort of present that to people, and you're presenting it to people at different levels. So I think the data visualization is is quite key as well.

[16:18] - James B

Yeah, no, really looking forward to that one. Well, brilliant stuff, Steve. And yeah, I'll see you next time.

[16:23] - Steve S

See you next time.

FAQs:

1. What is data quality?

Data quality is about ensuring your organisation data is accurate, complete, consistent and fit for purpose, so organisations can make decisions they can trust.

2. How can you improve data quality?

Start by profiling your data, understanding what “good” looks like, identifying quality issues and prioritising what needs to be fixed before and after go-live. Listen to the podcast for more!

3. Why is data quality important for data migration?

Poor-quality data can cause migration failures, delays and additional costs. Improving data quality early helps reduce risk and ensures trusted data reaches the new system.