Benchmarking Context-Aware Services for IoT-driven Transportation Systems

In the fast-growing realm of smart cities, integrating Internet of Things (IoT) devices into transportation systems is essential for improving efficiency and safety. Deploying these systems in real-world settings demands access to contextual data, and middleware systems to facilitate the exchange of both contextual and IoT data. Existing IoT-based data exchange systems such as Orion-LD, Stellio and Scorpio in the FIWARE space, support the modeling and representation of both context and IoT systems. This paper introduces a comprehensive testbed and a benchmarking platform designed to evaluate the performance of FIWARE context-aware brokers. The testbed incorporates real data from a real Bus Transportation Service in the city of Ioannina, Greece, as well as synthetic data enabling a realistic assessment of query and ingestion performance. The results show that microservices-based architectures like Stellio and Scorpio scale better than traditional designs like Orion-LD under high loads, but all brokers perform similarly at low loads. Furthermore, temporal queries present challenges for IoT applications due to their high cost across all evaluated brokers. However, write-optimized data stores offer an advantage by improving ingestion speed. The paper emphasizes the importance of understanding and addressing the operational inefficiencies of context-aware brokers to improve IoT system performance. Overall, this work introduces a novel benchmarking platform for smart transportation systems, featuring a realistic testbed with both real and synthetic IoT datasets, as well as detailed experimental results that identify key performance bottlenecks and offer potential optimization strategies.