Steps To Setup Hadoop 2.4.0 (Single Node Cluster) on CentOS/RHEL

Apache Hadoop 2.4.0 significant improvements over the previous stable releases. This version has many improvements in HDFS and MapReduce. This how to guide will help you to install Hadoop 2.4.0 on CentOS 6.5 System. This article doesn’t includes overall configuration of hadoop, we have only basic configuration required to start working with hadoop.


Step 1. Install JAVA/JDK

Java is the primary requirement for running hadoop on any system, So make sure you have java installed on your system using following command.

# java -version 

java version "1.8.0_05"
Java(TM) SE Runtime Environment (build 1.8.0_05-b13)
Java HotSpot(TM) Client VM (build 25.5-b02, mixed mode)

If you don’t have java installed on your system, use one of following link to install it first.

Install JAVA/JDK 8 on CentOS and RHEL 6/5

Step 2. Setup Hadoop User

We recommend to create a normal (nor root) account for hadoop working. So create a system account using following command.

# useradd hadoop
# passwd hadoop

After creating account, it also required to set up key based ssh to its own account. To do this use execute following commands.

# su - hadoop
$ ssh-keygen -t rsa
$ cat ~/.ssh/ >> ~/.ssh/authorized_keys
$ chmod 0600 ~/.ssh/authorized_keys

Lets verify key based login. Below command should not ask for password but first time it will prompt for adding RSA to the list of known hosts.

$ ssh localhost
$ exit

Step 3. Download Hadoop 2.4.0

Now download hadoop 2.4.0 source archive file using below command. You can also select alternate download mirror for increasing download speed.

$ cd ~
$ wget
$ tar xzf hadoop-2.4.0.tar.gz

Step 4. Configure Hadoop 2.4.0

4.1. Setup Environment Variables

First we need to set environment variable uses by hadoop. Edit ~/.bashrc file and append following values at end of file.

export HADOOP_HOME=/home/hadoop/hadoop

Now apply the changes in current running environment

$ source ~/.bashrc

Now edit $HADOOP_HOME/etc/hadoop/ file and set JAVA_HOME environment variable

export JAVA_HOME=/opt/jdk1.8.0_05/

4.2. Edit Configuration Files

Hadoop has many of configuration files, which need to configure as per requirements of your hadoop infrastructure. Lets start with the configuration with basic hadoop single node cluster setup. first navigate to below location

$ cd $HADOOP_HOME/etc/hadoop

Edit core-site.xml


Edit hdfs-site.xml




Edit mapred-site.xml


Edit yarn-site.xml


4.3. Format Namenode

Now format the namenode using following command, make sure that Storage directory is

$ hdfs namenode -format

[Sample output]

14/05/04 21:30:55 INFO namenode.NameNode: STARTUP_MSG:
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host =
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 2.4.0
14/05/04 21:30:56 INFO common.Storage: Storage directory /home/hadoop/hadoopdata/hdfs/namenode has been successfully formatted.
14/05/04 21:30:56 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
14/05/04 21:30:56 INFO util.ExitUtil: Exiting with status 0
14/05/04 21:30:56 INFO namenode.NameNode: SHUTDOWN_MSG:
SHUTDOWN_MSG: Shutting down NameNode at

Step 5. Start Hadoop Cluster

Lets start your hadoop cluster using the scripts provides by hadoop. Just navigate to your hadoop sbin directory and execute scripts one by one.

$ cd $HADOOP_HOME/sbin/

Now run script.


[Sample output]
14/05/04 21:37:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [localhost]
localhost: starting namenode, logging to /home/hadoop/hadoop/logs/
localhost: starting datanode, logging to /home/hadoop/hadoop/logs/
Starting secondary namenodes [] starting secondarynamenode, logging to /home/hadoop/hadoop/logs/
14/05/04 21:38:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Now run script.


[Sample output]
starting yarn daemons
starting resourcemanager, logging to /home/hadoop/hadoop/logs/
localhost: starting nodemanager, logging to /home/hadoop/hadoop/logs/

Step 6. Access Hadoop Services in Browser

Hadoop NameNode started on port 50070 default. Access your server on port 50070 in your favorite web browser.


Now access port 8088 for getting the information about cluster and all applications


Access port 50090 for getting details about secondary namenode.


Access port 50075 to get details about DataNode


Step 7. Test Hadoop Setup

7.1 - Make the HDFS directories required using following commands.

$ bin/hdfs dfs -mkdir /user
$ bin/hdfs dfs -mkdir /user/hadoop

7.2 - Now copy all files from local file system /var/log/httpd to hadoop distributed file system using below command

$ bin/hdfs dfs -put /var/log/httpd logs

7.3 - Now browse hadoop distributed file system by opening below url in browser.


7.4 - Now copy logs directory for hadoop distributed file system to local file system.

$ bin/hdfs dfs -get logs /tmp/logs
$ ls -l /tmp/logs/


Founder of I love to work with Linux and other open source application. Also I like to share knowledge with my friends and other Linux users. Follow me on Twitter, Add me on Google+ or Like Facebook page.


  1. hi i run the some pi jps some error how to find
    [test@p2 hadoop-2.4.0]$ jps
    10476 SecondaryNameNode
    10296 DataNode
    10201 NameNode
    10762 NodeManager
    11086 Jps
    10661 ResourceManager

    [test@p2 hadoop-2.4.0]bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.0.jar pi 10 100
    Number of Maps = 10
    Samples per Map = 100
    14/05/22 13:18:02 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
    Wrote input for Map #0
    Wrote input for Map #1
    Wrote input for Map #2
    Wrote input for Map #3
    Wrote input for Map #4
    Wrote input for Map #5
    Wrote input for Map #6
    Wrote input for Map #7
    Wrote input for Map #8
    Wrote input for Map #9
    Starting Job
    14/05/22 13:18:03 INFO client.RMProxy: Connecting to ResourceManager at / Failed on local exception: Protocol message end-group tag did not match expected tag.; Host Details : local host is: “″; destination host is: “”:8032;
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(
    at $Proxy16.getNewApplication(Unknown Source)
    at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getNewApplication(
    at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
    at java.lang.reflect.Method.invoke(
    at $Proxy17.getNewApplication(Unknown Source)
    at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getNewApplication(
    at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.createApplication(
    at org.apache.hadoop.mapred.ResourceMgrDelegate.getNewJobID(
    at org.apache.hadoop.mapred.YARNRunner.getNewJobID(
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(
    at org.apache.hadoop.mapreduce.Job$
    at org.apache.hadoop.mapreduce.Job$
    at Method)
    at org.apache.hadoop.mapreduce.Job.submit(
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(
    at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(
    at org.apache.hadoop.examples.QuasiMonteCarlo.main(
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
    at java.lang.reflect.Method.invoke(
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(
    at org.apache.hadoop.examples.ExampleDriver.main(
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
    at java.lang.reflect.Method.invoke(
    at org.apache.hadoop.util.RunJar.main(
    Caused by: Protocol message end-group tag did not match expected tag.

  2. hi Rahul i follow is i run the pi jops some error came how to solue this problam pls
    test@p2 hadoop-2.4.0]$ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.0.jar pi 5 100
    Number of Maps = 5
    Samples per Map = 100
    14/05/22 19:24:01 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
    Wrote input for Map #0
    Wrote input for Map #1
    Wrote input for Map #2
    Wrote input for Map #3
    Wrote input for Map #4
    Starting Job
    14/05/22 19:24:03 INFO client.RMProxy: Connecting to ResourceManager at /
    14/05/22 19:24:04 INFO input.FileInputFormat: Total input paths to process : 5
    14/05/22 19:24:05 INFO mapreduce.JobSubmitter: number of splits:5
    14/05/22 19:24:05 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1400761600720_0003
    14/05/22 19:24:06 INFO impl.YarnClientImpl: Submitted application application_1400761600720_0003
    14/05/22 19:24:06 INFO mapreduce.Job: The url to track the job:
    14/05/22 19:24:06 INFO mapreduce.Job: Running job: job_1400761600720_0003

  3. When are you publishing next part of this article? I loved this and I am waiting to see how will you test your setup by running some example map reduce job.

  4. Steve Morin Reply to Steve

    Great article but here is a script that also install hbase, hdfs, and a number of other resources

  5. Dear Rahul,
    This article was very helpful for me. I hope that you will continue…..
    Thank you.

  6. Sasha Kacanski Reply to Sasha

    Very good article,
    Two issues,
    First exit; ssh localhost will not work for public/private key
    Should be ssh localhost; exit

    hadoop path shoudl be added for bin directory, thus
    export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
    Other than this, everything worked flawlessly and hadoop is up and running perfectly…
    Thanks much

  7. Dear Rahul,

    Thanks for all the steps. Please update the mapred-site.xml to mapred-site.xml.template.

    Also, please update the testing the setup.


  8. Dear Rahul,

    thanks for the tutorial, why

    “hadoop/etc/hadoop/” E212: Can’t open file for writing

    cant be written?


  9. a khettar Reply to a

    Hi Rahul
    Thank you for sharing this with us. I followed your steps in installing hadoop 2.4 on Centos Vritual Machine. I have an Hbase running on my mac machine, I get connection refused error when hbase tries to connect.
    Here is my setting in core-site.xml

    and on Hbase: hbase-site.xml


    I can telnet onto the port 54310 from the VM but not from a remote machine, i.e my local macbook which is running the virtual machine. Looks like the port is closed to remote client. I have disabled firewall but it didn’t help.

    Any idea?



  10. Hi Rahul,

    Thanks for taking the time to write up this guide, it was very helpful.

  11. Nice article. I Managed ot get as far as starting the scripts and ran into problems that required havking the scripts and then got a warnign that ssh could not regonise the host name but and eventually ran ok, but I could not access hadoop in the browser and wnhen trying to ls the filesystem I get an error
    hdfs dfs -ls
    Exception in thread “main” java.lang.RuntimeException: core-site.xml not found

    However this got me along way forther

  12. Hi Rahul,

    Thank for this article.

    For me everything worked as I followed through this article. But at the last I could not get details of my datanode when I browsed The browser says ‘Unable to connect’

    What could be the problem or what mistake I might have done? Please help.

  13. Hi, This one is a great article. I followed many other blogs for this problem. But none of them worked. This one simply worked with no error.

    But i have a little problem.
    I have installed hbase standalone mode.
    Now i want hbase to use hdfs. So in hbase-site.xml file i added this:



    But its giving error and not working. Any reason why its not working? I copied the same configuration of yours during my hadoop installation.


  14. Dear Sir
    after I start hadoop can not run please help advisor
    -used hadoop-2.5.1
    -java ‘ ok
    -but have message below
    /home/hadoop/jhadoop-2.5.1/bin/hdfs:line 262: /user/lib/jvm/java-1.7.0-openjdk- N o such file or directrory

  15. I used
    java -version “”
    OpenJDK Runtime Environment (rhel- u45-b15)
    OpenJDK Client VM (build 24.45-b08,mixed mode,sharing)
    after that I can not run hadoop how I get it to run

  16. Dear
    How I change command from
    -Old 64 bit
    -rhel- u51-b31
    -(build 24.51-b03,mixed mode)
    export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-
    What command change dorectory
    My VM ware java version
    -rhel- u45-b15
    -build 24.45-b08,mixed mode,sharing
    export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-
    please help advisor because I am beginner CENT OS and HODOOP
    Best regards

  17. supriya pati Reply to supriya

    We have followed the given steps for centos 6.5 and hadoop2.5.0. All the dameons get started but still the namenode is not visible on the browser using port no: 50070, error is displayed: “Connection to server failed”. Please can you suggest some solution!!!
    Thank You.

  18. Hi, I am trying to run wordcount example. But it is getting stuck at ACCEPTED state.
    It is not going into RUNNING state.
    Any help appreciated. I have followed the tutorial exact. But using 2.6.0 instead of 2.4.0

  19. export HADOOP_HOME=/home/hadoop/hadoop
    I didnt understand the above statement. When you create a user hadoop, a folder will be created in home directory. What the purpose of second hadoop?

    cd $HADOOP_HOME/etc/hadoop
    There is no hadoop folder in etc directory.
    I am confused because etc comes under the supervision of root user rather than hadoop user.

Leave a Reply