Search This Blog

Thursday, September 18, 2014

HBase, Thrift2 and Erlang on Mac

There are not to many documents showing how Erlang can connect to HBase using Thrift protocol.  The very useful link which was published more than 5 years ago, and it discussed Thrift1.  Based on Lars George's post, Thrift2 will replace Thrift1 eventually.  The following will show what I have done to get Erlang to work with HBase using Thrift on a Mac machine.  It should be similar to get it work on a Linux machine.  Note that only Java is supported as first class citizen for HBase.  If you have the choice, I suggest using Java directly.
  1. Download thrift-0.9.0 and follow this tutorial to install it.
    cp -r thrift-0.9.0/lib/erl/*  /usr/local/opt/erlang/lib/erlang/lib/thrift-0.9.0 
    (homebrew installed erlang to /usr/local/opt/erlang.  Find out your own erlang directory.)
  2. Download latest stable hbase and decompress it.  I installed it to /usr/local/hbase-0.98.6.1-hadoop2, and will refer it as $HBASE_FOLDER.
  3. Download hbase.thrift and generate erlang bindings.
    thrift -gen erl hbase.thrift
    It generates erlang bindings in gen-erl folder.
  4. start hbase server:
    before starting hbase server, update conf/hbase-site.xml
    <configuration>
      <property>
        <name>hbase.rootdir</name>
        <value>file:///some-folder-hbase</value>
      </property>
      <property>
        <name>hbase.zookeeper.property.dataDir</name>
        <value>/some-folder-zookeeper</value>
      </property>
    </configuration>

    Then start hbase server:
    $HBASE_FOLDER/bin/start-hbase.sh
  5. start Thrift2:
    $HBASE_FOLDER/bin/base-daemon.sh start thrift2
  6. Create table 'ups' with column family 'ui' using HBase Shell (Thrift2 doesn't support create table yet):
    $HBASE_FOLDER/bin/hbase shell
    hbase(main):001:0>; create 'ups', 'ui'
  7. Download erlang client file and save it to gen-erl folder.
  8. Go to gen-erl folder, compile erlang files and start erlang shell:
    cd gen-erl
    erlc *.erl
    erl 
    1>hbase_thrift2_client:put("localhost", 9090).
    {{tclient,tHBaseService_thrift,
              {protocol,thrift_binary_protocol,
                        {binary_protocol,{transport,thrift_buffered_transport,
                                                    {buffered_transport,{transport,thrift_socket_transport,
                                                                                   {data,#Port<0 data-blogger-escaped-.716="">,infinity}},
                                                                        []}},
                                         true,true}},
              0},
     {ok,ok}}

    2>hbase_thrift2_client:get("localhost", 9090).
    {{tclient,tHBaseService_thrift,
              {protocol,thrift_binary_protocol,
                        {binary_protocol,{transport,thrift_buffered_transport,
                                                    {buffered_transport,{transport,thrift_socket_transport,
                                                                                   {data,#Port<0 data-blogger-escaped-.717="">,infinity}},
                                                                        []}},
                                         true,true}},
              0},
     {ok,{tResult,<<"zhentao-key1">>,
                  [{tColumnValue,<<"ui">>,<<"test">>,<<"xyz1">>,1411075681134,
                                 undefined},
                   {tColumnValue,<<"ui">>,<<"test1">>,<<"abc1">>,1411075681134,
                                 undefined}]}}}
The detailed erlang project can be found here

No comments:

Post a Comment