6

私は単体テストでこれをやろうとしています:

val sConf = new SparkConf()
  .setAppName("RandomAppName")
  .setMaster("local")
val sc = new SparkContext(sConf)
val sqlContext = new TestHiveContext(sc)  // tried new HiveContext(sc) as well

しかし、私はこれを取得します:

[scalatest] Exception encountered when invoking run on a nested suite - java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient *** ABORTED ***
[scalatest]   java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
[scalatest]   at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346)
[scalatest]   at org.apache.spark.sql.hive.client.ClientWrapper.<init>(ClientWrapper.scala:120)
[scalatest]   at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveContext.scala:163)
[scalatest]   at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala:161)
[scalatest]   at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:168)
[scalatest]   at org.apache.spark.sql.hive.test.TestHiveContext.<init>(TestHive.scala:72)
[scalatest]   at mypackage.NewHiveTest.beforeAll(NewHiveTest.scala:48)
[scalatest]   at org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)
[scalatest]   at mypackage.NewHiveTest.beforeAll(NewHiveTest.scala:35)
[scalatest]   at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:253)
[scalatest]   at mypackage.NewHiveTest.run(NewHiveTest.scala:35)
[scalatest]   at org.scalatest.Suite$class.callExecuteOnSuite$1(Suite.scala:1491)

spark-submit を使用して実行すると、コードは完全に機能しますが、単体テストでは機能しません。単体テストでこれを修正するにはどうすればよいですか?

4

1 に答える 1

2

これは古い質問ですが、同様の問題に遭遇したため、最終的に spark-testing-baseを使用しました:

import com.holdenkarau.spark.testing.SharedSparkContext
import org.apache.spark.sql.hive.test.TestHiveContext
import org.scalatest.FunSuite

class RowToProtoMapper$Test extends FunSuite with SharedSparkContext {

    test("route mapping") {
        val hc = new TestHiveContext(sc)
        /* Some test */
    }
}
于 2016-07-07T16:01:00.843 に答える