If you see WARN like
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
Then you have to re-compile hadoop native code manually and then put libhadoop-*.so in classpath
Prerequistes:
- $ sudo apt-get install build-essential
- $ sudo apt-get install g++ autoconf automake
And make sure that cmake is installed correctly.
$ cd $HADOOP_HOME/src
$ mvn package -Pdist,native -Dskiptests -Dtar
You should see the newly-built library in:
$ hadoop-dist/target/hadoop-2.3.0-cdh5.1.0/lib/native
Put following lines in hadoop-env.sh file
export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$HADOOP_HOME/lib/"
export HADOOP_COMMON_LIB_NATIVE_DIR="path/to/native"
OR simply,
$ cp $HADOOP_HOME/src/hadoop-dist/target/hadoop-2.3.0-cdh5.0.1/lib/native/* $HADOOP_HOME/lib/native/