Optimal Hadoop over ICN Placement Algorithm for Networking and Distributed Computing

Information-Centric Networking (ICN) is very promising for Hadoop-based distributed computing systems, where the data-centric approach is advantageous in reducing the data retrieval latency as well as the network traffic for Hadoop services. Moreover, the inherent in-network caching and computing features in ICN relaxes the end-to-end connectivity between consumers and producers (this helps networking, computation, and power efficiency as Hadoop nodes will use ICN services). Yet, building such a complex system needs new definitions and mappings on the architecture side. It needs also a flattening of the components and an optimization relative to data flow and computation performance. These issues are presented in this paper. Optimal optimization algorithms are then proposed, implemented and evaluated to improve the overall network performance. Experiments demonstrate that ICN support of Hadoop is a feasible architecture and show to improve the performance of Hadoop systems and reduce the end-to-end consumer delay.