Wednesday, June 5, 2013

Impala build steps on CentOS 6.3

  • Build boost-1.42.0
  • Before building impala
    • change be/CMakeLists.txt. I removed all boost RPMs and built boost libraries from sources. boost_date_time will be /usr/local/lib/libboost_date_time-mt.*. The build failed without this change. If you have boost RPMs 1.41 installed, you may not need to change this. But the build will fail with other issues.
      diff --git a/be/CMakeLists.txt b/be/CMakeLists.txt
      index c14bd31..cd5abac 100644
      --- a/be/CMakeLists.txt
      +++ b/be/CMakeLists.txt
      @@ -224,7 +224,7 @@ set (IMPALA_LINK_LIBS
      -  -lrt -lboost_date_time
      +  -lrt -lboost_date_time-mt
    • change to build release version and don't have to put -build_thirdparty in command line:
      diff --git a/ b/
      index 6ea491b..28b445a 100755
      --- a/
      +++ b/
      @@ -23,8 +23,8 @@ set -e
       # Exit on reference to uninitialized variable
       set -u
       for ARG in $*
  • After building, run shell/ This can generate a shell/build dir to have all files for impala-shell.
  • Prepare hadoop, hbase and hive config files, copy from /var/run/cloudera-scm-agent/process.
  • change bin/ like this
    for jar in `ls ${IMPALA_HOME}/fe/target/dependency/*.jar`; do
    export CLASSPATH
    Otherwise, you might see if you don't include impala-frontend.jar
    Exception in thread "main" java.lang.NoClassDefFoundError: com/cloudera/impala/common/JniUtil
    Caused by: java.lang.ClassNotFoundException: com.cloudera.impala.common.JniUtil
    Or this if you don't have hadoop-conf in the path
    E0605 09:32:23.236434  5272] Unsupported file system. Impala only supports DistributedFileSystem but the LocalFileSystem was found. fs.defaultFS(file:///) might be set incorrectly
    E0605 09:32:23.236655  5272] Impala is aborted due to improper configurations.
  • Impalad_flags
  • create a tarball of impala build because no such a open-source script.
    tar zcvf impala.tar.gz impala --exclude="*.class" --exclude="*.o" --exclude="impala/thirdparty" --exclude="impala/.git" --exclude="*.java" --exclude="*.cpp" --exclude="*.h" --exclude="expr-test"
  • start impalad
    cd impala_home
    export IMPALA_HOME=$PWD
    bin/ -build_type=release --flagfile=impalad_flags_path
  • start impala-shell
    cd impala_home
    export IMPALA_HOME=$PWD
    export IMPALA_SHELL_HOME=$PWD/shell/build/impala-shell-1.0.1
    $IMPALA_SHELL_HOME/impala-shell -i impalad-host:21001

No comments:

Post a Comment