Apache Hadoop is an open source software project based on JAVA. Basically, it is a framework that is used to run applications on large clustered hardware (servers). It is designed to scale from a single server to thousands of machines, with a very high degree of fault tolerance. Rather than relying on high-end hardware, the reliability of these clusters comes from the software’s ability to detect and handle faults on its own.
Credit for the creation of Hadoop goes to Doug Cutting and Michael J. Cafarella. Doug, a Yahoo employee, found it appropriate to change the name to his son’s toy elephant “Hadoop.” It was originally developed to support the Nutch search engine project distribution to rank a large number of indexes.
In simple terms, Hadoop is a way that applications can handle a large amount of data using a large number of servers. First, Google created Map-reduce to work on indexing big data and then Yahoo! created Hadoop to implement the Map Reduce feature for its own use.
Small map: The task tracking framework that understands and assigns work to the nodes in a cluster. The application has small divisions of work and each job can be assigned to different nodes in a cluster. It is designed in such a way that any flaw can be solved automatically by the framework itself.
HDFS– Hadoop distributed file system. It is a large-scale file system that spans all nodes in a Hadoop cluster for data storage. Link the file systems on many local nodes to make them one large file system. HDFS assumes that nodes will fail, so it achieves reliability by replicating data across multiple nodes.
As Big Data is the talk of the modern IT world, Hadoop shows the way to use Big Data. Makes analytics much easier considering terabytes of data. The Hadoop framework already has some great users to brag about like IBM, Google, Yahoo !, Facebook, Amazon, Foursquare, EBay, etc. for large applications. In fact, Facebook claims to have the largest Hadoop cluster at 21PB. Hadoop’s business purpose includes data analysis, web crawling, word processing, and image processing.
Most of the world’s data goes unused, and most companies don’t even try to use this data to their advantage. Imagine if you could afford to keep all the data generated by your business, and if you had a way to analyze that data. Hadoop will bring this power to a business.