0

Javaクライアントだけを使用してHBaseを使用した単純なcrudアプリがあり、すべて問題ありませんでした。しかし、Spring を使用して同じアプリを実行しようとすると、zookeeper が接続を拒否したり閉じたりして、何が起こるかよくわかりません。ここにいくつかのコードがあります: xml config:

 <beans xmlns="http://www.springframework.org/schema/beans"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns:hdp="http://www.springframework.org/schema/hadoop"
   xsi:schemaLocation="
    http://www.springframework.org/schema/beans 
    http://www.springframework.org/schema/beans/spring-beans.xsd
    http://www.springframework.org/schema/hadoop 
    http://www.springframework.org/schema/hadoop/spring-hadoop.xsd">

    <hdp:configuration>
    </hdp:configuration> 
    <hdp:hbase-configuration delete-connection="true" />
    <bean id="hTemplate" class="org.springframework.data.hadoop.hbase.HbaseTemplate" >
        <property name="configuration" ref="hbaseConfiguration"></property>
    </bean>
    <bean id="usingHBase" class="com.mycompany.usinghbase.UsingHBase" />
 </beans>

これが私の簡単な方法です：

public class UsingHBase {

    @Autowired
    @Qualifier("hTemplate")
    private HbaseTemplate hBaseTemplate;

    public void addData(final String key, final String value) throws IOException {
        hBaseTemplate.execute("new_table", new TableCallback<Object>() {
            public Object doInTable(HTableInterface hti) throws Throwable {
              Put p = new Put(Bytes.toBytes("zz"));
              p.add(Bytes.toBytes("zz"), Bytes.toBytes(key), Bytes.toBytes(value));
                hti.put(p);
                return null;
            }
        });
    }

    public List<Object> findAll() {
        return hBaseTemplate.find("new_table", "zz", new RowMapper<Object>() {
            public Object mapRow(Result result, int rowNum) throws Exception {
                return result.toString();
            }
        });
    }

        public void DeleteData(final String key) throws IOException {
        hBaseTemplate.execute("new_table", new TableCallback<Object>() {
            public Object doInTable(HTableInterface hti) throws Throwable {
        Delete delete = new Delete(Bytes.toBytes("zz"));
        delete.deleteColumns(Bytes.toBytes("zz"), Bytes.toBytes(key));
        hti.delete(delete);
                return null;
            }
        });
    }
}

これが私のテストケースです：

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = "file:src/main/java/spring.xml")
public class UsingHBaseTest {

    @Autowired
    @Qualifier("hTemplate")
    private HbaseTemplate hBaseTemplate;

    @Autowired
    @Qualifier("usingHBase")
    private UsingHBase usingHBase;

    @Before
    public void setUp() {
    }

    @Test
    public void testAddData() throws Exception {
        System.out.println("addData");
        long startTime = System.currentTimeMillis();
    for (int i=1; i <= 1000; i++) {
            String column = "k" + i;
            String value = "v" + i;
            usingHBase.addData(column, value);
    }
    long EndTime = System.currentTimeMillis();
    System.out.println("Put value: " + (EndTime - startTime));
    }

    @Test
    public void testFindAll() {
        System.out.println("findData");
        long startTime = System.currentTimeMillis();
        usingHBase.findAll();
    long EndTime = System.currentTimeMillis();
    System.out.println("Put value: " + (EndTime - startTime));
    } 

    @Test
    public void testDeleteData() throws IOException {
        System.out.println("deleteData");
        long startTime = System.currentTimeMillis();
    for (int i=1; i <= 1000; i++) {
            String column = "k" + i;
            usingHBase.DeleteData(column);
    }
    long EndTime = System.currentTimeMillis();
    System.out.println("delete value: " + (EndTime - startTime));
    }
}

そしてスタックトレースが言うこと：

Running com.mycompany.usinghbase.UsingHBaseTest
13/03/25 11:28:57 INFO context.TestContextManager: @TestExecutionListeners is not present for class [class com.mycompany.usinghbase.UsingHBaseTest]: using defaults.
13/03/25 11:28:57 INFO xml.XmlBeanDefinitionReader: Loading XML bean definitions from URL [file:src/main/java/spring.xml]
13/03/25 11:28:59 INFO support.GenericApplicationContext: Refreshing org.springframework.context.support.GenericApplicationContext@26d0fffc: startup date [Mon Mar 25 11:28:59 VLAT 2013]; root of context hierarchy
13/03/25 11:28:59 INFO support.DefaultListableBeanFactory: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@4489140e: defining beans [hadoopConfiguration,hbaseConfiguration,hTemplate,usingHBase,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor,org.springframework.context.annotation.ConfigurationClassPostProcessor$ImportAwareBeanPostProcessor#0]; root of factory hierarchy
addData
2013-03-25 11:29:00.037 java[24078:1c03] Unable to load realm info from SCDynamicStore
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48 GMT
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:host.name=192.168.0.86
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:java.version=1.7.0_10
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:java.home=/Library/Java/JavaVirtualMachines/jdk1.7.0_10.jdk/Contents/Home/jre
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/Users/urijvoskresenskij/NetBeansProjects/UsingHBase/target/test-classes:/Users/urijvoskresenskij/NetBeansProjects/UsingHBase/target/classes:/Users/urijvoskresenskij/.m2/repository/org/apache/hbase/hbase/0.94.3/hbase-0.94.3.jar:/Users/urijvoskresenskij/.m2/repository/com/yammer/metrics/metrics-core/2.1.2/metrics-core-2.1.2.jar:/Users/urijvoskresenskij/.m2/repository/com/google/guava/guava/11.0.2/guava-11.0.2.jar:/Users/urijvoskresenskij/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/urijvoskresenskij/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/Users/urijvoskresenskij/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/Users/urijvoskresenskij/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:/Users/urijvoskresenskij/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/Users/urijvoskresenskij/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/Users/urijvoskresenskij/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar:/Users/urijvoskresenskij/.m2/repository/com/github/stephenc/high-scale-lib/high-scale-lib/1.1.1/high-scale-lib-1.1.1.jar:/Users/urijvoskresenskij/.m2/repository/commons-codec/commons-codec/1.4/commons-codec-1.4.jar:/Users/urijvoskresenskij/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/Users/urijvoskresenskij/.m2/repository/commons-io/commons-io/2.1/commons-io-2.1.jar:/Users/urijvoskresenskij/.m2/repository/commons-lang/commons-lang/2.5/commons-lang-2.5.jar:/Users/urijvoskresenskij/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:/Users/urijvoskresenskij/.m2/repository/log4j/log4j/1.2.16/log4j-1.2.16.jar:/Users/urijvoskresenskij/.m2/repository/org/apache/avro/avro/1.5.3/avro-1.5.3.jar:/Users/urijvoskresenskij/.m2/repository/org/xerial/snappy/snappy-java/1.0.3.2/snappy-java-1.0.3.2.jar:/Users/urijvoskresenskij/.m2/repository/org/apache/avro/avro-ipc/1.5.3/avro-ipc-1.5.3.jar:/Users/urijvoskresenskij/.m2/repository/org/jboss/netty/netty/3.2.4.Final/netty-3.2.4.Final.jar:/Users/urijvoskresenskij/.m2/repository/org/apache/velocity/velocity/1.7/velocity-1.7.jar:/Users/urijvoskresenskij/.m2/repository/org/apache/zookeeper/zookeeper/3.4.3/zookeeper-3.4.3.jar:/Users/urijvoskresenskij/.m2/repository/org/apache/thrift/libthrift/0.8.0/libthrift-0.8.0.jar:/Users/urijvoskresenskij/.m2/repository/org/apache/httpcomponents/httpclient/4.1.2/httpclient-4.1.2.jar:/Users/urijvoskresenskij/.m2/repository/org/apache/httpcomponents/httpcore/4.1.3/httpcore-4.1.3.jar:/Users/urijvoskresenskij/.m2/repository/org/jruby/jruby-complete/1.6.5/jruby-complete-1.6.5.jar:/Users/urijvoskresenskij/.m2/repository/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.jar:/Users/urijvoskresenskij/.m2/repository/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar:/Users/urijvoskresenskij/.m2/repository/org/mortbay/jetty/jsp-2.1/6.1.14/jsp-2.1-6.1.14.jar:/Users/urijvoskresenskij/.m2/repository/org/eclipse/jdt/core/3.1.1/core-3.1.1.jar:/Users/urijvoskresenskij/.m2/repository/org/mortbay/jetty/jsp-api-2.1/6.1.14/jsp-api-2.1-6.1.14.jar:/Users/urijvoskresenskij/.m2/repository/org/mortbay/jetty/servlet-api-2.5/6.1.14/servlet-api-2.5-6.1.14.jar:/Users/urijvoskresenskij/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar:/Users/urijvoskresenskij/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar:/Users/urijvoskresenskij/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.8.8/jackson-jaxrs-1.8.8.jar:/Users/urijvoskresenskij/.m2/repository/org/codehaus/jackson/jackson-xc/1.8.8/jackson-xc-1.8.8.jar:/Users/urijvoskresenskij/.m2/repository/org/slf4j/slf4j-api/1.4.3/slf4j-api-1.4.3.jar:/Users/urijvoskresenskij/.m2/repository/org/slf4j/slf4j-log4j12/1.4.3/slf4j-log4j12-1.4.3.jar:/Users/urijvoskresenskij/.m2/repository/tomcat/jasper-compiler/5.5.23/jasper-compiler-5.5.23.jar:/Users/urijvoskresenskij/.m2/repository/tomcat/jasper-runtime/5.5.23/jasper-runtime-5.5.23.jar:/Users/urijvoskresenskij/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar:/Users/urijvoskresenskij/.m2/repository/org/jamon/jamon-runtime/2.3.1/jamon-runtime-2.3.1.jar:/Users/urijvoskresenskij/.m2/repository/com/google/protobuf/protobuf-java/2.4.0a/protobuf-java-2.4.0a.jar:/Users/urijvoskresenskij/.m2/repository/com/sun/jersey/jersey-core/1.8/jersey-core-1.8.jar:/Users/urijvoskresenskij/.m2/repository/com/sun/jersey/jersey-json/1.8/jersey-json-1.8.jar:/Users/urijvoskresenskij/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/urijvoskresenskij/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar:/Users/urijvoskresenskij/.m2/repository/com/sun/jersey/jersey-server/1.8/jersey-server-1.8.jar:/Users/urijvoskresenskij/.m2/repository/asm/asm/3.1/asm-3.1.jar:/Users/urijvoskresenskij/.m2/repository/javax/xml/bind/jaxb-api/2.1/jaxb-api-2.1.jar:/Users/urijvoskresenskij/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/Users/urijvoskresenskij/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/urijvoskresenskij/.m2/repository/junit/junit/4.11/junit-4.11.jar:/Users/urijvoskresenskij/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/Users/urijvoskresenskij/.m2/repository/org/springframework/spring-aop/3.1.2.RELEASE/spring-aop-3.1.2.RELEASE.jar:/Users/urijvoskresenskij/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/Users/urijvoskresenskij/.m2/repository/org/springframework/spring-asm/3.1.2.RELEASE/spring-asm-3.1.2.RELEASE.jar:/Users/urijvoskresenskij/.m2/repository/org/springframework/spring-context-support/3.1.2.RELEASE/spring-context-support-3.1.2.RELEASE.jar:/Users/urijvoskresenskij/.m2/repository/javax/annotation/jsr250-api/1.0/jsr250-api-1.0.jar:/Users/urijvoskresenskij/.m2/repository/org/springframework/spring-test/3.1.2.RELEASE/spring-test-3.1.2.RELEASE.jar:/Users/urijvoskresenskij/.m2/repository/org/springframework/spring-beans/3.1.2.RELEASE/spring-beans-3.1.2.RELEASE.jar:/Users/urijvoskresenskij/.m2/repository/org/springframework/spring-context/3.1.2.RELEASE/spring-context-3.1.2.RELEASE.jar:/Users/urijvoskresenskij/.m2/repository/org/springframework/spring-expression/3.1.2.RELEASE/spring-expression-3.1.2.RELEASE.jar:/Users/urijvoskresenskij/.m2/repository/org/springframework/spring-core/3.1.2.RELEASE/spring-core-3.1.2.RELEASE.jar:/Users/urijvoskresenskij/.m2/repository/org/springframework/data/spring-data-hadoop/1.0.0.RELEASE/spring-data-hadoop-1.0.0.RELEASE.jar:/Users/urijvoskresenskij/.m2/repository/org/apache/hadoop/hadoop-streaming/1.0.4/hadoop-streaming-1.0.4.jar:/Users/urijvoskresenskij/.m2/repository/org/apache/hadoop/hadoop-core/1.0.4/hadoop-core-1.0.4.jar:/Users/urijvoskresenskij/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar:/Users/urijvoskresenskij/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/Users/urijvoskresenskij/.m2/repository/commons-net/commons-net/1.4.1/commons-net-1.4.1.jar:/Users/urijvoskresenskij/.m2/repository/net/java/dev/jets3t/jets3t/0.7.1/jets3t-0.7.1.jar:/Users/urijvoskresenskij/.m2/repository/net/sf/kosmosfs/kfs/0.3/kfs-0.3.jar:/Users/urijvoskresenskij/.m2/repository/hsqldb/hsqldb/1.8.0.10/hsqldb-1.8.0.10.jar:/Users/urijvoskresenskij/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar:/Users/urijvoskresenskij/.m2/repository/org/apache/hadoop/hadoop-tools/1.0.4/hadoop-tools-1.0.4.jar:/Users/urijvoskresenskij/.m2/repository/org/springframework/spring-tx/3.1.2.RELEASE/spring-tx-3.1.2.RELEASE.jar:
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/Users/urijvoskresenskij/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/var/folders/_q/z5blmlxs39d4pmmv1fj9mh500000gn/T/
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:os.name=Mac OS X
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:os.arch=x86_64
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:os.version=10.7.5
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:user.name=urijvoskresenskij
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:user.home=/Users/urijvoskresenskij
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Client environment:user.dir=/Users/urijvoskresenskij/NetBeansProjects/UsingHBase
13/03/25 11:29:01 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
13/03/25 11:29:02 INFO zookeeper.ClientCnxn: Opening socket connection to server /0:0:0:0:0:0:0:1:2181
13/03/25 11:29:02 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 24078@MacBook-Air-Urij.local
13/03/25 11:29:02 INFO client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
13/03/25 11:29:02 INFO zookeeper.ClientCnxn: Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
13/03/25 11:29:02 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x13d9ee87e610328, negotiated timeout = 40000
13/03/25 11:29:03 INFO client.HConnectionManager$HConnectionImplementation: Closed zookeeper sessionid=0x13d9ee87e610328
13/03/25 11:29:04 INFO zookeeper.ZooKeeper: Session: 0x13d9ee87e610328 closed
13/03/25 11:29:04 INFO zookeeper.ClientCnxn: EventThread shut down
13/03/25 11:29:04 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
13/03/25 11:29:04 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 24078@MacBook-Air-Urij.local
13/03/25 11:29:04 INFO zookeeper.ClientCnxn: Opening socket connection to server /127.0.0.1:2181
13/03/25 11:29:04 INFO client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
13/03/25 11:29:04 INFO zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:2181, initiating session
13/03/25 11:29:04 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x13d9ee87e61032c, negotiated timeout = 40000
13/03/25 11:29:04 INFO client.HConnectionManager$HConnectionImplementation: Closed zookeeper sessionid=0x13d9ee87e61032c
13/03/25 11:29:04 INFO zookeeper.ZooKeeper: Session: 0x13d9ee87e61032c closed
13/03/25 11:29:04 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
13/03/25 11:29:04 INFO zookeeper.ClientCnxn: EventThread shut down
13/03/25 11:29:04 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 24078@MacBook-Air-Urij.local
13/03/25 11:29:04 INFO zookeeper.ClientCnxn: Opening socket connection to server /0:0:0:0:0:0:0:1:2181
13/03/25 11:29:04 INFO client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
13/03/25 11:29:04 INFO zookeeper.ClientCnxn: Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
13/03/25 11:29:04 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x13d9ee87e61032d, negotiated timeout = 40000
13/03/25 11:29:04 INFO client.HConnectionManager$HConnectionImplementation: Closed zookeeper sessionid=0x13d9ee87e61032d
13/03/25 11:29:05 INFO zookeeper.ZooKeeper: Session: 0x13d9ee87e61032d closed
13/03/25 11:29:05 INFO zookeeper.ClientCnxn: EventThread shut down
13/03/25 11:29:05 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
13/03/25 11:29:05 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 24078@MacBook-Air-Urij.local
13/03/25 11:29:05 INFO zookeeper.ClientCnxn: Opening socket connection to server /fe80:0:0:0:0:0:0:1%1:2181
13/03/25 11:29:10 INFO client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.
13/03/25 11:29:10 INFO zookeeper.ClientCnxn: Socket connection established to fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181, initiating session
13/03/25 11:29:11 INFO zookeeper.ClientCnxn: Session establishment complete on server fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181, sessionid = 0x13d9ee87e610334, negotiated timeout = 40000

助けてください、人々。私には本当にあなたが必要だ！

4

2 に答える 2

1

xml で ZooKeeper クォーラムとポートを指定する必要があります

<hdp:configuration id="hadoopConfiguration">
    hbase.zookeeper.quorum=mynode
    hbase.zookeeper.property.clientPort=2181
</hdp:configuration> 
<hdp:hbase-configuration id="hbaseConfiguration" delete-connection="true" configuration="hadoopConfiguration" />
于 2014-01-16T19:05:22.237 に答える
0

Zookeeper、hbase regionserver、および master を起動してから、アプリケーションを実行してみてください。その場合、コードは機能するはずです。

于 2014-01-13T10:17:26.307 に答える