DefinePK

Abstract

Data mining technology has become a boon for all the engineering fields in today s world. With the great outburst of data comes a greater responsibility (i.e., organizing and managing the data). Large amount of data is fragmented into smaller chunks of data and ran over hundreds or thousands of servers in parallel ways to extract useful data in big data analytics. With the boom in technology and the huge growth in the data, clustering has become a crucial part in identifying the similarities based on many parameters in a given data set that influences in decision making. To reinforce this clustering, we propose to use machine learning methods to influence over data redundancy and grouping i.e. Self Organized Mapping (SOM). Solving such problem involves in tackling the issues like clustering, visualization, abstraction and de-duplication. Here the work involves in the usage of Artificial Neural Network (ANN) for Self Organized Mapping that has a unique features, like construction of maps, self organization to form different clusters dynamically to support the volume, variety and variance of big data. In this paper, a novel technique is proposed that supports Hadoop in classifying or clustering of data using Self Organized Mapping. Map Reduce has always been used in combination with Hadoop, but in this implementation Map Reduce is used external to Hadoop job. Using Map Reduce externally is an advantage as Logical Block Address (LBA) and Hash Tag are obtained easily. From the results we found that this implementation has increased speed, data is structured and redundancy is achieved with improved efficiency.