The Quality of the Telematics Data

Aug. 14, 2014

The AEM/AEMP telematics standard is a major step forward. Many people have worked long and hard to reach this point, and we should all be thankful for their efforts. The API that has been developed as part of this initiative and the data-aggregation software that transfers telematics data from mixed sources in a mixed fleet to a single end user storage location are also significant milestones. The journey that has brought telematics data collection technology close—but not quite—to mainstream business practice has been long and difficult. The question remains: How can we finally and completely bring telematics data into the main stream of our day-to-day business processes? What remains to be done?

The answer may lie in the fact that we have been seduced by the frequency and breadth of the data available and have lost sight of the need for reliability and integrity in the data. The frequency with which data are transmitted and the number of data fields or fault codes transmitted is of no consequence if there are gaps, errors and inconsistencies in the process. If there is no way to ensure the quality of the data, it’s simply garbage in/garbage out, but at the speed of light. Let’s not do with servers and sensors what we can do with pencil and paper.

Construction folk and particularly equipment folk are practical. We understand things in physical terms, and we like to see the way systems work. We need a physical, “feel, touch and see” confirmation of a situation. This plays heavily against systems that rely on electronic data or on data that flow from sensor to server without ever being touched or checked by human hands. Our experience has taught us to never rely on data that have not been verified by someone at some stage. We like the “touch it and check it” approach: It is our way of reducing human error and gives us confidence in the data we use in our business.

There is a real reluctance to abandon our current paper-based system with its quality checks and balances until such time as telematics data earns the confidence required to obsolete the old and rely on the new. Sure, our existing data-collection systems are expensive and cumbersome. They are certainly far from perfect, but we have, over many years, developed the checking, validating and auditing procedures needed to make us believe that the data we have are the best we can get.

Changing to a new technology involves substantial risk. Telematics data-collection systems will not gain wide acceptance in equipment-fleet management until the reliability and integrity of the data they provide give end users the confidence they need to obsolete existing data collection systems and do more with less. Adopting telematics data-collection systems on top of existing paper-based systems is simply a process of doing more with more. The question will always be asked, Is the more really necessary?

What should be done?

We are on the threshold of a breakthrough, and that great opportunity knocks for those who will perfect a solution to the next and necessary step. Let’s look at the diagram.

The top two rows of boxes represent the OEM and third-party telematics suppliers who provide the hardware and technology needed to collect sensor readings, transmit the zeroes and ones to their servers, and store them for either their own or their customer’s use. It appears, in today’s world, to be a relatively simple step, and we have been at it for a number of years. The technology and the systems are, however, neither mature nor stable and stories abound about reliability problems, transmission breaks, and constantly changing data protocols. Suffice to say that these two rows are far from perfect and that much work needs to be done to achieve the reliability, quality and confidence needed for a mainstream business system.

The downward arrows and the download layer in the diagram represent the AEM/AEMP telematics standard and accompanying API. Yes, we have developed (or most certainly can develop) the technology and the protocols needed to aggregate and transfer telematics data from mixed sources in a mixed fleet to a single data-storage location focused on the needs of a single equipment end user. Several companies are doing this successfully and are collecting the raw data as shown in the next layer of the diagram.

A number of very good and extremely promising focused applications have been developed using a combination of this raw data and other data available to the end user. We are, without doubt, seeing the potential of telematics, but most applications fall under the heading of doing more with more. Few if any instances are on record where telematics has rendered pencil and paper obsolete and where companies are doing more with less.

The key to success lies in the next layer of the diagram, data integrity, which represents the procedures and protocols needed to bridge the gap between the quality of the raw data downloaded into the system and the quality of the data required by mainstream business systems. Yes, some data-aggregation programs do some of this. But the challenge presented by a flow of messy and frequently inconsistent data is formidable. The capability of this layer holds the key to the confident use of telematics data in mainstream business systems and applications. If we are truly going to use telematics data to drive job costing, generate internal equipment charges, or produce PM work orders, then we must know that the data is reliable, verified and normalized.

Let’s list five minimum requirements for a competent data integrity layer.

1. Verify and report on the download process. At the most fundamental level, it is necessary to know that the download process is working and to have reports detailing when data was last synced between the telematics provider and the company. The frequency with which sensor data is sent from the machine to the telematics provider is of little value if this data is not frequently and regularly synced with the end user. This is where confidence building starts.

2. Scan for and report on missing units. There are endless accounts of machines that have gone missing and not reported for a given period. The integrity layer must scan for and provide actionable information on missing reports. If 30 percent of the telematics-connected units have not reported in the past 24 hours, is this because of reliability issues in the hardware or the communication system, or is it because the machine has indeed not been turned on. We must know, and we must be able to take action.

3. Screen for and report on suspect data values. Again, everyone has stories about questionable and erroneous values, and this is where competent software can play a major role. Clearly outlying values must be reported and suspect values flagged or replaced with values calculated from other data. There is a relationship between hours worked and fuel consumption. Software should constantly check this and use it to retain, replace or reject suspect values.

4. Normalize and standardize data. Variations in the form, format and definition of data will always occur regardless of efforts made to standardize data feeds. Software in the integrity layer can and should solve the problem and provide the routines to ensure that data conforms with back-office requirements.

5. Map data fields and push to business systems. Business systems and applications all have different data structures; there is no such thing as one size fits all. Software in the integrity layer can, should and (in many cases) does provide the mapping routines needed; and it ensures that reliable, verified and normalized data is pushed to business systems and applications as and when needed.

The data integrity layer appears to be the weak link in the chain that takes telematics data from machine to back office. A competent layer will identify reliability and suspect reporting issues in all the preceding layers and enable the action needed to resolve problems and improve performance. A competent layer will give confidence in the use of telematics data in all following layers. It is this confidence that will enable companies to obsolete existing data-collection systems and accept telematics data into the mainstream of their business processes. Managers responsible for the use of telematics data in their organization need to question telematics suppliers and/or data aggregators in detail about the capability of the data integrity layer included in their solution. The answer to the question “Can you download and aggregate data from mixed sources in a mixed fleet?” is often “Yes.” If you believe this, you need to devote substantial enquiry to the next question: “How can you help to ensure that the data will be reliable, verified and normalized?”

Think about the data integrity layer as the water treatment plant in your system. It is unwise to permit untreated water into our streams and rivers. It is unwise to permit untreated data into your business systems.

Mike would like to thank the many colleagues who helped with this article.