Home • Resources • Discovering geo-data’s potential to optimise oceanic shipping using Exasol

Tech Blog

Discovering geo-data’s potential to optimise oceanic shipping using Exasol

Dr. Johannes Schildgen is a professor of databases at OTH Regensberg university and a former Exasolian. With a background in computer science and applied sciences, he has a special interest in examining big (geo) data. In 2020, he was asked to be the supervisor for former OTH student Mario Stelzer’s master thesis analysing global geo-data.

A project with big ambitions

Approached by a transport company, Stelzer was asked to understand how geo-data can help better optimise ocean shipping. For example, could he apply Loading...advanced analytics to Loading...big data to warn about transport delays, discover where ships were and whether they might collide?

Drawing upon on several gigabytes of historical and real-time transmitted ship positioning data from several ingested sources, he built an infrastructure to create a processing pipeline from data gathering to data storage.

The three main players in his team were Exasol, Apache Kafka and Apache Spark. Both Apache products are well known and widely used in the area of stream processing, which means data is processed immediately on ingestion.

While Exasol isn’t a stream processing platform, its open architecture and world-leading performance made it an ideal data storage and analysis companion. By using user defined functions (UDF), Exasol can handle positioning data in GeoJSON format.

100,000 ships per day

The platform was tasked with tracking and analysing data from 100,000 different ships per day. Updated in near real-time, the fastest ships send new location data every couple of seconds for maximum accuracy.
Timely ingestion of, and access to these datasets enable Stelzer to accurately pinpoint not just the transponder location but the physical shape and size of each vessel. With this level of detail, he can predict where any given ship can or, importantly, can’t go.

Geo-data in action

With an infrastructure in place, Stelzer put the data to work. Exasol’s ability to analyse high volumes of data in near real-time means it is possible for him to calculate how long it might take a ship to reach its destination. Pair this with other data sources, such as weather predictions, and there’s potential to avoid storms or save fuel based on wind speeds.

Along each route, he is able to predict collisions between vessels. With adequate warning, ships can slow down or change course. And, when they safely reach their destination, Stelzer uses Geofencing analysis to understand how many ships are in a specific zone.
For example, if a zoned harbour is at maximum capacity, then traffic delays are likely. Knowing this in advance allows a transporter to shuffle routes and avoid costly traffic jams.

Uncovering the potential of geo-data using Exasol

The real-world applications of this project are compelling. Combining Exasol, Apache Kafka, and Apache Spark has enabled Stelzer to take a deep dive into near real-time streaming geo-data, applying it to a complex problem yielding fascinating results.

mst94