Deploy LZO for YARN in CDH4

This page Using the LZO Parcel is only for MRv1, not for YARN. It took me a while to figure out how to set up LZO in YARN correctly.

You may experience different error messages if you do not configure YARN correctly:

• Class com.hadoop.compression.lzo.LzoCodec not found.
• Class com.hadoop.mapred.DeprecatedLzoTextInputFormat not found.
• No LZO codec found, cannot run.
• native-lzo library not available

Here are the steps to setup LZO correctly:

• You can follow the instruction in "Using the LZO Parcel" to install and activate the parcel
• Add LzoCodec and LzopCodec. In cloudera manager, find the field for core-site.xml: hdfs1->Configuration -> Service-Wide -> Advanced -> Cluster-wide Configuration Safety Valve for core-site.xml. and add this property:

  
      io.compression.codecs
   org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.BZip2Codec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec
  
  

• Add classpath and native library. In Cloudera manager, find this field in mapred-site.xml: yarn1->Configuration->Service-Wide->Advanced->YARN Service MapReduce Configuration Safety Valve, then add the following two properties:

    
      mapreduce.application.classpath
      $HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*,$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*,/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/*
    
    
      mapreduce.admin.user.env
      LD_LIBRARY_PATH=$HADOOP_COMMON_HOME/lib/native:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/native
    
  

• Restart YARN and put the configuration files to gateways.
• Don't forget to run "Deploy Client Configuration"

Make Eclipse Font Smaller in Gnome/Xfce

There are a lot of posts on the Internet. Here are the one working for me:

• To make the menu font smaller
  • create ~/.gtkrc-eclipse

    style "eclipse" {
      font_name = "Sans 9"
    }
    
    class "GtkWidget" style "eclipse"

  • If you want the buttons are more compact, please follow this page. I use like this, but have a problem the color of menu items on the menu bar is same as the background.

    style "eclipse" {
      font_name = "Sans 9"
    }
    
    class "GtkWidget" style "eclipse"
    
    style "compact-toolbar" {
        GtkToolbar::internal-padding = 0
        xthickness = 1
        ythickness = 1
    }
    
    style "compact-button" {
        xthickness = 0
        ythickness = 0
    }
    
    class "GtkToolbar" style "compact-toolbar"
    widget_class "*<GtkToolbar>*<GtkButton>" style "compact-button"
    

  • add this line into ~/.bashrc

    export GTK2_RC_FILES=/usr/share/themes/Clearlooks/gtk-2.0/gtkrc:$HOME/.gtkrc-eclipse 
    

• To make the tab font smaller, such as "Package Explorer", you need to modify the css file (version # could be different).

  $ vi $ECLIPSE_HOME/plugins/org.eclipse.platform_4.2.2.v201302041200/css/e4_default_gtk.css
  
  .MPartStack {
          font-size: 9;
          swt-simple: false;
          swt-mru-visible: false;
  }
  

  You may try to make the height of tabs smaller as suggested in this page, but I found it didn't work well because the buttons like 'minimum' or 'maximum' become too small and large when the mouse is hovering.

  .MPartStack {
    font-size: 9;
    font-family: Liberation Sans;
    swt-tab-renderer: null;
    swt-tab-height: 22px;
    swt-selected-tabs-background: #FFFFFF #ECE9D8 100%;
    swt-simple: false;
    swt-mru-visible: false;
  }