ピスパークリング2.1
次のコードを実行します。
hc = H2OContext.getOrCreate(spark)
h2o_frame = h2o.import_file('hdfs:path/to/my/file.csv')
spark_frame = hc.as_spark_frame(h2o_frame)
ドキュメントと同じように、問題なく動作します。
しかし、次のコードを試すと：
hc = H2OContext.getOrCreate(spark)
h2o_frame = h2o.H2OFrame(some_list)
spark_frame = hc.as_spark_frame(h2o_frame) #error at this line
次のエラーが表示されます。
File "/my_path/my_file.py", line 530, in _convert_and_append
spark_frame = hc.as_spark_frame(h2o_frame)
File "/my_path/.virtualenv/lib/python2.7/site-packages/pysparkling/context.py", line 196, in as_spark_frame
j_h2o_frame = h2o_frame.get_java_h2o_frame()
File "/my_path/.virtualenv/lib/python2.7/site-packages/pysparkling/context.py", line 38, in get_java_h2o_frame
self._java_frame = hc._jhc.asH2OFrame(self.frame_id)
File "/usr/hdp/current/spark2-client/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1133, in __call__
File "/usr/hdp/current/spark2-client/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/usr/hdp/current/spark2-client/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o67.asH2OFrame.
: java.lang.NullPointerException
at water.fvec.H2OFrame.<init>(H2OFrame.scala:38)
at water.fvec.H2OFrame.<init>(H2OFrame.scala:46)
at org.apache.spark.h2o.H2OContext.asH2OFrame(H2OContext.scala:234)
at org.apache.spark.h2o.JavaH2OContext.asH2OFrame(JavaH2OContext.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:280)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:745)
唯一の違いは、H2OFrame を初期化する方法です。この不一致の原因は何ですか? 足りないものはありますか？H2OFrame の作成方法が重要なのはなぜですか?
どんな助けでも大歓迎
編集：
some_list:
<type 'list'>
[(u'petal_length', 15800.0, 1.0, 0.42857966682031484), (u'petal_width', 14200.0, 0.8987341772151899, 0.3851791942309159), (u'sepal_length', 4808.2783203125, 0.30432141267800633, 0.13042596965182748), (u'sepal_width', 2057.6796875, 0.13023289161392404, 0.05581516929694175), (u'id', 0.0, 0.0, 0.0)]