Why AI projects in logistics stall on data, not on algorithms
A transport company wants to predict which shipments will be late. A WMS needs to pick more intelligently, automatically. An ERP needs to validate rates without manual intervention. The technology exists. But the moment you pull historical data from the TMS, the problem becomes immediately apparent: the same customer appears under four different names, postcodes are incorrect, weights are zero or estimated, and rate codes were created years ago and never reviewed since. A model trained on such data learns the errors just as well as the patterns. The result is a system that confidently points in the wrong direction.
Which data sources are most often the culprit?
Logistics operations typically run three core systems side by side: the TMS (Transport Management System), the WMS (Warehouse Management System), and the ERP. Each has its own data quality problems. The TMS holds operational shipment data, but customer records are rarely deduplicated: the same company appears as 'Bakker BV', 'Bakker B.V.', and 'Bakker Rotterdam'. Addresses are entered manually and not validated against a postcode table. The WMS holds weights and dimensions, but for items that have been in circulation for years, these are never updated after packaging changes. The ERP contains rates and customer groups, but historical migration data from a legacy system has filled gaps with default values that were never corrected. On top of that, these three systems rarely communicate cleanly with one another: a relation ID in the TMS does not always correspond to the debtor number in the ERP, and the WMS uses its own article coding. That lack of consistency is the first thing that needs to be addressed.
What are the concrete data cleaning steps?
Data cleaning in logistics consists of a handful of concrete steps. Deduplication: merging duplicate customer and supplier records based on name, address, and chamber of commerce number. This can be automated using fuzzy matching, but always requires a human check on the exceptions. Normalisation: standardising address fields, validating postcodes, converting country codes to ISO format. Aligning rate codes and article codes across systems so that the TMS, WMS, and ERP share the same keys. Validation rules: flagging missing weights and dimensions as unknown rather than zero, and setting up a workflow so that new records only become active once mandatory fields are filled in. Historical data audit: reviewing a sample of shipment data or inventory mutations from the past year to understand which fields are structurally missing or incorrect. That audit immediately tells you which data can be used for training and which must be excluded.
Cleaning before AI implementation or in parallel: when do you choose which?
This is the most frequently asked question, and the honest answer is: it depends on how severe the contamination is. If the core fields the model requires, such as customer ID, weight, route, and delivery time, are missing or demonstrably incorrect in more than twenty percent of records, it is wise to clean first before you start training. Otherwise you are investing time in a model you will need to retrain on clean data anyway. If the contamination is more limited and localised, you can run cleaning in parallel: the model starts on the reliable subset, and as more data is cleaned, it is expanded. One important caveat: running cleaning in parallel requires clear agreements about which data is already valid and which is not. That is a process and governance question, not a technical one. Operations must be on board: if planners continue to enter weights as zero because the system allows it, you will face the same problem again in a year.
The trade-off: cleaning takes time, but dirty data makes every model unreliable
Data cleaning is time-consuming and sometimes painstaking work. It requires collaboration between IT, operations, and the people who enter data day to day. But the calculation is straightforward: a model trained on data with structural errors produces outcomes that no one trusts, and rightly so. That means money spent on technology that ends up unused. Better to take three months longer on preparation and deliver a system that operations actually uses. Data cleaning in logistics AI is not a side task; it is the foundation. Cutting corners at the front end costs twice as much at the back end.