19.4. Configuring Hadoop Compatible Storage

This section describes how to configure Hadoop Compatible Storage in your storage environment and verify that it is functioning correctly.
To configure Hadoop compatible storage:
  1. Edit the core-site.xml file available at /usr/share/java/conf. The following is the sample core-site.xml file:
    <configuration>
      <property>
        <name>fs.glusterfs.impl</name>
    <value>org.apache.hadoop.fs.glusterfs.GlusterFileSystem</value>
    </property>
    
    <property>
       <name>fs.default.name</name>
       <value>glusterfs://192.168.1.36:9000</value>
    </property>
    
    <property>
       <name>fs.glusterfs.volname</name>
       <value>hadoopvol</value>
    </property>  
     
    <property>
       <name>fs.glusterfs.mount</name>
       <value>/mnt/glusterfs</value>
    </property>
    
    <property>
       <name>fs.glusterfs.server</name>
       <value>192.168.1.36</value>
    </property>
    
    <property>
       <name>quick.slave.io</name>
       <value>Off</value>
    </property>
    </configuration>
    
    The following table lists the fields of core-site.xml file that you can configure:

    Table 19.1. Configurable Fields

    Property Name Default Value Description
    fs.default.name glusterfs://192.168.1.36:9000 Any hostname in the trusted storage pool as the server and any port number.
    fs.glusterfs.volname volume-dist-rep Red Hat Storage volume to mount.
    fs.glusterfs.mount /mnt/glusterfs The directory used to fuse mount the volume.
    fs.glusterfs.server 192.168.1.36 Any hostname or IP address on the trusted storage pool.
    quick.slave.io Off Performance tunable option. If this option is set to On, the plugin will try to perform I/O directly from the disk file system (like ext3 or ext4) the file resides on. As a result, read performance improves and jobs run faster.

    Note

    This option is not tested widely.

  2. Copy glusterfs-0.20.2-0.1.jar and core-site.xml files to Hadoop’s lib/ and conf/ directory respectively using the following commands:
    # cp /usr/share/java/glusterfs-0.20.2-0.1.jar $HADOOP_HOME/lib/
    # cp /usr/share/java/conf/core-site.xml $HADOOP_HOME/conf/