A Big Data system architecture to support the monitoring of paved roads

Abstract

Today, everything is connected, including the exchange of data and the generation of new information. As a result, large amounts of data are being collected at an ever-increasing rate and in a variety of forms, a phenomenon now known as Big Data. Recent developments in information and communication technologies are driving the generation of significant amounts of data from multiple sources, namely sensors. In response to these technological advances and data challenges, this paper proposes a Big Data system architecture for paved road monitoring and implements part of this architecture on a section of road in Portugal as a case study. The challenge in the case study architecture is to collect and process sensor data in real time, at a rate of 500 records per second, producing 15 GBytes of data per day, using a real-time data stream for real-time monitoring and a batch data stream for deeper analysis. This allows users to obtain instant updates on road conditions such as the number of vehicles, loads, weather, and pavement temperatures on the road. They can monitor what is happening on the road in real time, receive alerts, and even gain insight into historical data, such as analysing the condition of structures or identifying traffic patterns.