2

DStream の各 RDD で SparkSQL を実行すると、ClassCastException が発生します。

java.lang.ClassCastException: org.apache.spark.rdd.ShuffledRDDPartition cannot be cast to org.apache.spark.rdd.ParallelCollectionPartition
    at org.apache.spark.rdd.ParallelCollectionRDD.compute(ParallelCollectionRDD.scala:102)
    at org.apache.spark.rdd.ParallelCollectionRDD.compute(ParallelCollectionRDD.scala:85)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:111)
    at org.apache.spark.scheduler.Task.run(Task.scala:51)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:187)
4

0 に答える 0