Linux マシンでspark-job-serverを起動しようとしています。私は次のことをしました:
- cloudera ディストリビューション CDH(5.x) をインストールし、稼働させました
spark-job-server
上記のgithubからダウンロード- プロジェクトをいくつかのフォルダーに抽出しました
- sbt-0.13.8 をインストールし、パス変数を設定して sbt コマンドを実行できるようにする
- プロジェクト ディレクトリ (
spark-job-server/
) に移動し、sbt コマンドを実行しました。 - SBT シェルから、「re-start」と入力しました。これは、
spark-job-server
.
次に、この例外を受け取りました:
:: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[warn] :: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[warn] :: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[warn] :: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] org.apache.hadoop:hadoop-mapreduce-client-app:2.2.0
[warn] +- org.apache.hadoop:hadoop-client:2.2.0
[warn] +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
[warn] +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
[warn] org.apache.hadoop:hadoop-yarn-api:2.2.0
[warn] +- org.apache.hadoop:hadoop-client:2.2.0
[warn] +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
[warn] +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
[warn] org.apache.hadoop:hadoop-mapreduce-client-core:2.2.0
[warn] +- org.apache.hadoop:hadoop-client:2.2.0
[warn] +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
[warn] +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
[warn] org.apache.hadoop:hadoop-mapreduce-client-jobclient:2.2.0
[warn] +- org.apache.hadoop:hadoop-client:2.2.0
[warn] +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
[warn] +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
[warn] Credentials file /root/.bintray/.credentials does not exist
[warn] Credentials file /root/.bintray/.credentials does not exist
[warn] Credentials file /root/.bintray/.credentials does not exist
[trace] Stack trace suppressed: run last job-server-api/*:update for the full output.
[trace] Stack trace suppressed: run last akka-app/*:update for the full output.
[trace] Stack trace suppressed: run last job-server-api/*:update for the full output.
[trace] Stack trace suppressed: run last akka-app/*:update for the full output.
[error] (job-server-api/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[error] (akka-app/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[error] (job-server-api/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[error] (akka-app/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[error] Total time: 5 s, completed Aug 6, 2015 1:29:26 PM
いくつかの詳細情報:
- これらの依存関係を手動でダウンロードして配置しました
- 再起動コマンドを再実行しても、sbt は
.ivy2
フォルダーにコピーした同じ依存関係を見つけることができません
どうすればこれを修正できますか?