Getting Started with Hadoop 2.0

Fri, 08 Aug 2014 13:30:34 +0000

Apache™ Hadoop® is an open source software project that enables the distributed processing of large data sets across clusters of commodity servers. It is designed to scale up from a single server to thousands of machines, with a very high degree of fault tolerance. Rather than relying on high-end hardware, the resiliency of these clusters comes from the software’s ability to detect and handle failures at the application layer.

Hadoop 1 popularized MapReduce programming for batch jobs and demonstrated the potential value of large scale, distributed processing. MapReduce, as implemented in Hadoop 1, can be I/O intensive, not suitable for interactive analysis, and constrained in support for graph, machine learning and on other memory intensive algorithms. Hadoop developers rewrote major components of the file system to produce Hadoop 2. To get started with the new version, it helps to understand the major differences between Hadoop 1 and 2.

Running Hadoop 1.1.2 on Ubuntu Linux (Single-Node Cluster)

Fri, 14 Jun 2013 13:15:21 +0000

In this tutorial I will describe the required steps for setting up a pseudo-distributed, single-node Hadoop cluster backed by the Hadoop Distributed File System, running on Ubuntu Linux.

Note: This walkthrough targets Hadoop 1.1.2 (2013) on Ubuntu 13.04, both long past end-of-life. Hadoop 1.x predates YARN, so the JobTracker/TaskTracker daemons, the conf/ layout, and config keys such as fs.default.name and mapred.job.tracker no longer apply to current releases (3.x uses etc/hadoop/, fs.defaultFS, and different web-UI ports). It is kept here as a historical reference; if you are setting up Hadoop today, follow the official Single Node Cluster guide instead.

Hadoop on Infinite Script

Getting Started with Hadoop 2.0

Running Hadoop 1.1.2 on Ubuntu Linux (Single-Node Cluster)