Common Data Management Issues

We established the importance of data management in our last post. In this post, we’re going to lay out some of the most common data issues we see, and in our next post we’ll lay out some solutions.

No Data Tracking at All

You may not think this would be a common scenario, but it happens more often than you’d expect. It’s almost impossible to track nothing in a modern business - most software tools track something off the shelf. But you’d be surprised how often businesses aren’t conscious of what they’re tracking, which is equivalent to tracking nothing.

This seems like a scenario that should be avoided, and it is. But it can be better than some alternative scenarios.

For instance, a business that is tracking the wrong things, yet believes they’re the right ones, will have more confidence in their decisions, yet be relying on incorrect data, which is even more dangerous.

Some businesses will also have habits and processes already set based on the tools they have and what they can track with those tools. If those tools aren’t suitable moving forward, or aren’t the best available, it can sometimes be more difficult to change habits than build new ones from scratch.

Generally, however, the situation of not tracking any data at all should be avoided. At the very least, think through the economics and key factors in your business. What would you track in an ideal world? What key statistics do you pay attention to qualitatively? You can then take that list, compare it with the tools you have, and begin to gather data where you can.

As Peter Drucker said, “What gets measured, gets managed”.

Disconnected Data Tracking

This is perhaps the situation that can be the most headache for a business, and also the most common.

Such companies have many different software tools, of varying age, and a large amount of data going back many years.

There’s a lot of data, but it’s inconsistent, there are no notes for context, and the systems make it difficult to export or import things, or share between platforms - it’s a nightmare.

The tendency in these scenarios is to try and make things work. Sometimes that’s possible.

Often, however, a lot of time gets wasted trying to make the data coherent, and then even if that effort is successful, the data isn’t clean or consistent enough to work with.

This requires careful consideration at the beginning of such an effort, to determine whether it’s worth trying to salvage this data, or whether analysis should be restricted to more recent or newly gathered data.

“Siloed” Data

With enterprise software, data sharing between platforms is often not a priority, and sometimes actively discouraged.

As a result, businesses often have a range of data, but each data set is limited to the software in which it was gathered, and there remains no good way to export the data.

When this data is essential, it means companies have to resort to things like manual data entry or complex data scraping systems.

The problem with these is that if everyone is not trained correctly, or someone is not directly responsible for this on an ongoing basis, there are lapses in data transfer which make it difficult to work with the data in future.

Not to mention that the whole process of transferring data is enormously inefficient and costly to the business.

Planning vs. Operational Data

Anyone that has worked in an operational capacity for a logistics or transportation business knows that what is planned is never what is operated.

The aim of planning is to get as close as possible to what will eventually be operated, but there are always unforeseen complications and disruptions.

Most businesses do not track both the planned and operated data, nor do they track the evolution of the planning data. This is a problem from a data management perspective.

When a company goes to use the data they have been tracking later, whether it be as they begin to implement some optimization tools, or they’re just trying to make some business decisions, they are basing their work off, at best, a planned solution that doesn’t mirror reality, and at worst, a planned solution that was massively disrupted when it came to operations.

You can easily see the issues with this. If something is planned on a consistent basis, canceled last minute every time, and this never gets corrected, it’s a big source of waste.

The other issue here is that there is never any visibility about the efficiency of the planning process. If things are consistently being changed between the planned schedule and the operated schedule, there is wasted effort on both ends to repeatedly correct this. Yet there is no way to identify that this is a consistent problem when both sets of data are not tracked.

Perhaps a particular flight is always booked for Friday, and but moved last-minute to Saturday 90% of the time. Or a particular person consistently cancels their weekly taxi pickup, and they live far from the main concentration of routes. Perhaps a particular flight is systematically delayed.

There is no way to identify and improve these inefficiencies if both planning and operational data is not tracked.

Context for Data

Context is the final piece of the puzzle when tracking planning and operational data. Even if planned and operated data is tracked, over time, we forget the reasoning for a particular change.

When we go to look at the data from the past year at the year-end review, how are we going to remember the reasoning for always delaying that Friday flight? How are we going to know why the dispatcher ignores a particular booking because they always cancel last minute?

In an ideal system, planning and operational data is tracked through major iterations, along with context for why changes were made. The reason for a flight being moved is noted. The reason behind a dispatcher’s decision to ignore a particular booking is noted.

This way, when data is being reviewed, inefficiencies can be easily picked out, and processes improved.

Missing Data

Data that is missing cannot be validated. If there were supposed to be 100 flights in a data set, and there is no base dataset to compare to, there is no way for a computer program to determine whether there are flights missing.

Making sure that data sets are complete is of the utmost importance. Errors in data entry, field format, and other things that affect how “clean” the data is, can usually be tracked. But if the data isn’t there, there’s no way to know.

Conclusion

Hopefully, with these common issues in mind, you can evaluate where your business is doing well, not doing so well, or not doing at all, and begin to think about how to improve.

In our next post, we’ll review the ideal data management scenario, and talk about how you might go about accomplishing it in your business.

Get our e-book Data Management in Transportation here.