During the first e-commerce era, package location tracking arose as a showcase application. In 1994, FedEx.com’s success making such tracking information available online showed what the web could do, and it quickly got wide attention.

Today, GPS-oriented tracking systems such as Amazon and Uber have again redefined users’ expectations for efficient fleet systems. 

Internet of Things (IoT) technologies can meet the demand for location systems capable of real-time, complex event processing, but there are issues to overcome at each link of the chain. As GPS-based location system processing becomes more precise and immediate, there are a bewildering variety of new big data tools to process that data.

The pieces are plentiful, and freight and related businesses face difficult decisions when it comes to elements in real-time location systems, which represent a market expected to reach $7.5 billion by 2022, growing at 31.4% (CAGR) from 2016 to 2022, according to Allied Market Research. 

Tracking information flow

“Today, there is ‘Uberization’ and ‘Amazonization’ of the freight experience. It’s changed the way that people expect to interact with the freight provider,” said Ben Wiesen, CEO at Carrier Logistics, maker of FACTS transportation management software. 

Today, expectations are that people can see their freight orders as pegs moving on a digital map, delivery is second-day, and – often, as in the case of Amazon – is free, he added.

As a result, the trucking industries have made investments in information flows that collect data from IoT sensors in their network, said Wiesen, a 30-plus-year trucking management veteran. He said businesses will have to “skill-up”  to handle the incoming information not only as data but also as events that initiate steps in operations. 

Real-Time Data Collection

Front-end data collection for IoT has its challenges, and that doesn’t change even as, in more and more cases, the data acquired is ultimately fed to cloud computing centers. The front-end processing challenge is one of the drivers behind interest in edge computing.

That’s according to Ian Skerritt, marketing adviser at HiveMQ, a maker of MQTT message brokering and other software for IoT fleet management, logistics and related implementations. He emphasized that IoT fleet systems navigate rough environments where connections can be shaky. 

“The key challenge fleet systems have is that the vehicles are moving around,” Skerritt said. “Each vehicle typically connects over a cellular or satellite network.” But, as the vehicle moves around, connections between the vehicle and cloud can be lost. 

Lost connections create two issues for location tracking, he continued. One, real-time data may be ready to be sent but no network connection is available; and, two, systems must reestablish the vehicle-to-cloud connection as quickly as possible once the network is available again. 

HiveMQ’s MQTT software is designed to quickly reconnect communications sessions that have been interrupted and work “agnostically” with various types of data feeds, Skerrett said. Protocols competitive with MQTT include AMQP, HTTP and LwM2M as well as proprietary systems. 

Data Streams, Data Lakes

When IoT location data is collected, it is seldom as a steady stream. As IoT begins to fulfil the promise of thousands or even millions of devices in the field, incoming data spikes and overflows remain a concern. 

Skerrett said the industry has experienced an increase in integrations that use software like MQTT along with Apache Kafka, yet another message system. He said the two complement each other. 

Kafka is among a host of open source big data tool standards that have arisen in recent years to handle processing in real time. Along with associated streaming and query tooling, Kafka has become a nexus for new event processing methods for IoT location tracking and other purposes. 

Most enterprises today don’t use the data their machines and sensors generate continuously, because they cannot access and process it,” said Kai Waehner, who is a field engineer and architect at Confluent, a chief vendor driving Kafka software methods today. The data goes unused either because of technical limitations or complexity, which makes it difficult implement with legacy technologies, he said.

Waehner said inaccessible, unprocessed data hinders developers from being able to clearly communicate the business value of IoT data.

Microservices such as Kafka can be distributed on a large scale, he said. Thus, they can be “integrated with all required systems and allow processing of the data in real time.” The biggest underlying technical challenge for many is integrating with several different technologies, standards and interfaces, he continued. 

As with MQTT, there are plenty of alternatives to Kafka components, especially when used to create “data lakes” that hold fast arriving, undifferentiated IoT data before it is sorted and processed. Competitive tools to Kafka in all its varieties include Apache Spark, RabbitMQ and other systems.

Build vs. Buy

Advanced skills required for integrating different elements in today’s big data systems cause some in-house IoT teams to emphasize “buy” over “build.”  That can mean opting for more encompassing, end-to-end IoT platforms. 

Such commercial systems often include assorted open source big data components under the hood.  

“When you think about IoT scale, of the ingestion of streams of data into a data lake for real-time processing, it is a nontrivial task,” according to Steven Glapa, vice president of product management at Aeris Communications.

“Companies with a strong manufacturing bent, for example, may not put Kafka pieces together as deftly as companies that have already been doing it for a number of years,” Glapa said.

Today, the pressure on shippers for faster decision making renders traditional methods of integration unacceptable, and pre-integrated systems can be a quicker fit with back-end systems already in place, according to Luis Parajes, executive vice president of sales and marketing at collaborative logistics platform vendor Turvo Inc. 

“We organize every component of the logistics network so that our customers have a 360-degree view of their supply chain and how it’s performing,” he said. The goal is to give people real-time visibility into assets, items and orders in the supply chain along with the execution of transportation and settlement activities, he indicated. 

The Next Skill Set

Efforts to collect and process real-time location data and correlate it to weather, traffic, engine sensing and other data create large pools of information and set the stage for the next leap in IoT development. 

What’s next is complex event processing that automates decision making and actions very quickly, perhaps with a touch of AI, according to Carrier Logistics’ Wiesen. 

“The next skill set is knowing what to do with the data,” he said. As an example, he cited simple acts such as parking a truck at a loading dock, which now needs to kick off complex sequences of automated activity.

For businesses moving beyond basic reporting on fleet activity, small steps will include big leaps. But pragmatism is required. 

“Deciding on your technology roadmap means balancing the next best thing with what you can afford today, and what will be will be there tomorrow,” cautioned Wiesen, who sees a future narrowing of the IoT and big data standards as some IoT technologies become more mainstream.