apache-spark - Jupyter Notebook 内で Spark(scala) カーネルを実行したい。OSError の取得: [WinError 193] %1 は有効な Win32 アプリケーションではありません

Question

Traceback (most recent call last):
  File "c:\users\rdx\anaconda3\lib\runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\rdx\anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\RDX\Anaconda3\Scripts\ipython.exe\__main__.py", line 9, in <module>
  File "c:\users\rdx\anaconda3\lib\site-packages\IPython\__init__.py", line 119, in start_ipython
    return launch_new_instance(argv=argv, **kwargs)
  File "c:\users\rdx\anaconda3\lib\site-packages\traitlets\config\application.py", line 657, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-112>", line 2, in initialize
  File "c:\users\rdx\anaconda3\lib\site-packages\traitlets\config\application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "c:\users\rdx\anaconda3\lib\site-packages\IPython\terminal\ipapp.py", line 296, in initialize
    super(TerminalIPythonApp, self).initialize(argv)
  File "<decorator-gen-7>", line 2, in initialize
  File "c:\users\rdx\anaconda3\lib\site-packages\traitlets\config\application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "c:\users\rdx\anaconda3\lib\site-packages\IPython\core\application.py", line 398, in initialize
    self.parse_command_line(argv)
  File "c:\users\rdx\anaconda3\lib\site-packages\IPython\terminal\ipapp.py", line 291, in parse_command_line
    return super(TerminalIPythonApp, self).parse_command_line(argv)
  File "<decorator-gen-4>", line 2, in parse_command_line
  File "c:\users\rdx\anaconda3\lib\site-packages\traitlets\config\application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "c:\users\rdx\anaconda3\lib\site-packages\traitlets\config\application.py", line 514, in parse_command_line
    return self.initialize_subcommand(subc, subargv)
  File "c:\users\rdx\anaconda3\lib\site-packages\IPython\core\application.py", line 208, in initialize_subcommand
    return super(BaseIPythonApplication, self).initialize_subcommand(subc, argv)
  File "<decorator-gen-3>", line 2, in initialize_subcommand
  File "c:\users\rdx\anaconda3\lib\site-packages\traitlets\config\application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "c:\users\rdx\anaconda3\lib\site-packages\traitlets\config\application.py", line 452, in initialize_subcommand
    self.subapp.initialize(argv)
  File "<decorator-gen-116>", line 2, in initialize
  File "c:\users\rdx\anaconda3\lib\site-packages\traitlets\config\application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "c:\users\rdx\anaconda3\lib\site-packages\jupyter_console\app.py", line 138, in initialize
    self.init_shell()
  File "c:\users\rdx\anaconda3\lib\site-packages\jupyter_console\app.py", line 105, in init_shell
    JupyterConsoleApp.initialize(self)
  File "c:\users\rdx\anaconda3\lib\site-packages\jupyter_client\consoleapp.py", line 334, in initialize
    self.init_kernel_manager()
  File "c:\users\rdx\anaconda3\lib\site-packages\jupyter_client\consoleapp.py", line 288, in init_kernel_manager
    self.kernel_manager.start_kernel(**kwargs)
  File "c:\users\rdx\anaconda3\lib\site-packages\jupyter_client\manager.py", line 243, in start_kernel
    **kw)
  File "c:\users\rdx\anaconda3\lib\site-packages\jupyter_client\manager.py", line 189, in _launch_kernel
    return launch_kernel(kernel_cmd, **kw)
  File "c:\users\rdx\anaconda3\lib\site-packages\jupyter_client\launcher.py", line 123, in launch_kernel
    proc = Popen(cmd, **kwargs)
  File "c:\users\rdx\anaconda3\lib\subprocess.py", line 947, in __init__
    restore_signals, start_new_session)
  File "c:\users\rdx\anaconda3\lib\subprocess.py", line 1224, in _execute_child
    startupinfo)
OSError: [WinError 193] %1 is not a valid Win32 application

このリンクに記載されている手順に従いました: https://github.com/ibm-et/spark-kernel/wiki/Guide-to-Integrating-the-Spark-Kernel-with-Jupyter

Windows10で動かしたいです。

Anaconda と Spark をインストールしました。また、両方の環境パスが設定されています。ここに私のkernel.jsonファイルがあります：

{
    "display_name": "Spark 1.5.1 (Scala 2.10.4)",
    "language_info": { "name": "scala" },
    "argv": [
        "C:/Users/RDX/spark-kernel-master/dist/spark-kernel/bin/spark-kernel",
        "--profile",
        "{connection_file}"
    ],
    "codemirror_mode": "scala",
    "env": {
        "SPARK_OPTS": "--master=local[2] --driver-java-options=-Xms1024M --driver-java-options=-Xmx4096M --driver-java-options=-Dlog4j.logLevel=info",
        "MAX_INTERPRETER_THREADS": "16",
        "CAPTURE_STANDARD_OUT": "true",
        "CAPTURE_STANDARD_ERR": "true",
        "SEND_EMPTY_OUTPUT": "false",
        "SPARK_HOME": "X:\\Softwares\\BIG_Data_files\\spark-2.0.1-bin-hadoop2.7",
        "PYTHONPATH": "X:\\Softwares\\BIG_Data_files\\spark-2.0.1-bin-hadoop2.7/python:X:\\Softwares\\BIG_Data_files\\spark-2.0.1-bin-hadoop2.7/python/lib/py4j-0.10.3-src.zip"
     }
}

カーネルリストで利用できます。ただし、ノートブックから Spark カーネルを選択しているときにエラーが表示される

ipython kernelspec list 

利用可能なカーネル:

pyspark C:\Users\RDX.ipython\kernels\pyspark
python3 c:\users\rdx\anaconda3\lib\site-packages\ipykernel\resources
スパーク C:\ProgramData\jupyter\kernels\spark