On-in: An on-node and in-node based mechanism for big data collection in large-scale sensor networks

Nowadays, data are collected everywhere from searches on Google to posts on social media. Thus, the era of big data is started. Among many feasible sources, Wireless Sensor Network (WSN) becomes one of the vibrant big data sources where a huge volume of data is generated from various sensor nodes in large-scale networks. Compared to traditional networks, WSN faces serious challenges especially in data management and conserving sensor energies. In this work, we propose a novel two phases big data processing mechanism, called ON-IN: on-node and in-node (between nodes). In the first phase, we introduce the Newton's forward difference method to reduce the amount of data generated at each sensor node. Meanwhile, in the second phase we perform a clustering technique, i.e. PKmeans (Pattern-Kmeans) algorithm, and aim to reduce the redundancy among data generated by neighboring nodes. Through both simulations and experiments on real telosB motes, we evaluated the efficiency of our proposed mechanism in terms of reducing data transmission and conserving sensor energies, compared to other existing techniques.