以下のコードを使用して、Oozie Java Action を介して Posgres Hive メタストアに接続しようとしています。
hive-site.xml をアクションに渡しているので、必要な情報がすべて含まれているはずです。
HiveMetaStoreClient client = new HiveMetaStoreClient(conf);
log.info("Successfully created the HiveMetaStoreClient");
try {
log.info(String.format("Loading the partitions for %s.%s", database, table));
List<Partition> partitions = client.listPartitions(database, table, (short) 200);
log.info(String.format("Processing %d partitions", partitions.size()));
for (Partition partition : partitions) {
StorageDescriptor sd = partition.getSd();
String location = sd.getLocation();
String newLocation = location.replace(from, to);
log.info(String.format("Moving from %s to %s", location, newLocation));
sd.setLocation(newLocation);
}
} catch (TException e) {
logExceptionStack(e);
}
ログは正確にはエラーではありませんが、DERBY バックエンドを持つ他のメタストアを調べていることを示唆しています。
HiveMetaStoreClientどこで問題を探し、正しいメタストアを指すように強制するか、私は困惑しています。
2016-02-08 16:48:05,972 INFO [uber-SubtaskRunner] com.xxxxxxx.PartitionMigrator.Program: Attempting to create the HiveMetaStoreClient
2016-02-08 16:48:06,123 INFO [uber-SubtaskRunner] com.xxxxxxx.PartitionMigrator.Program: hiveconf metastoreURI: null
2016-02-08 16:48:06,194 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
2016-02-08 16:48:06,222 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.ObjectStore: ObjectStore, initialize called
2016-02-08 16:48:06,385 INFO [uber-SubtaskRunner] DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
2016-02-08 16:48:06,385 INFO [uber-SubtaskRunner] DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
2016-02-08 16:48:06,506 WARN [uber-SubtaskRunner] DataNucleus.Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)
2016-02-08 16:48:06,840 WARN [uber-SubtaskRunner] DataNucleus.Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies)
2016-02-08 16:48:08,339 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
2016-02-08 16:48:09,286 INFO [uber-SubtaskRunner] DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
2016-02-08 16:48:09,286 INFO [uber-SubtaskRunner] DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
2016-02-08 16:48:10,400 INFO [uber-SubtaskRunner] DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table.
2016-02-08 16:48:10,400 INFO [uber-SubtaskRunner] DataNucleus.Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table.
2016-02-08 16:48:10,676 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY
2016-02-08 16:48:10,677 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.ObjectStore: Initialized ObjectStore
2016-02-08 16:48:10,798 WARN [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.1.0
2016-02-08 16:48:10,928 WARN [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.ObjectStore: Failed to get database default, returning NoSuchObjectException
2016-02-08 16:48:11,019 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.HiveMetaStore: Added admin role in metastore
2016-02-08 16:48:11,021 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.HiveMetaStore: Added public role in metastore
2016-02-08 16:48:11,097 INFO [uber-SubtaskRunner] org.apache.hadoop.hive.metastore.HiveMetaStore: No user is added in admin role, since config is empty
2016-02-08 16:48:11,193 INFO [uber-SubtaskRunner] com.xxxxxxx.PartitionMigrator.Program: Successfully created the HiveMetaStoreClient