java - Cloudera Manager 4.0 ワードカウントの例

Question

Cloudera Manager をインストールし、2 つのノードをマスターノードに接続しました。このリンクで cloudera が提供するチュートリアルを使用して wordcount プログラムを実行しようとしています: https://ccp.cloudera.com/display/SUPPORT/Hadoop+Tutorial#HadoopTutorial-InputsandOutputs

Cloudera は、コマンドプロンプトで nano を使用して作成した WordCount.java ファイルをコンパイルするように指示します。そして、ディレクトリ Wordcount_Classes を作成し、そこに wordcount.java ファイルを置きました。Javaコードが機能するために、そのディレクトリをclouderaディレクトリの1つのどこかに移動する必要があるかどうかわかりませんか? または、私が何か他のことをしている場合。Cloudera Manager をインストールすると、すでにすべてのノードに ClouderaHadoop が自動的にインストールされています。私は自分の端末でこれを取得します：

mark@*****:~/wordcount_classes$ javac WordCount.java
WordCount.java:6: package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.Path;
                           ^
WordCount.java:7: package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.*;
^
WordCount.java:8: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.*;
^
WordCount.java:9: package org.apache.hadoop.mapred does not exist
import org.apache.hadoop.mapred.*;
^
WordCount.java:10: package org.apache.hadoop.util does not exist
import org.apache.hadoop.util.*;
^
WordCount.java:14: cannot find symbol
symbol  : class MapReduceBase
location: class org.myorg.WordCount
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                  ^
WordCount.java:14: cannot find symbol
symbol  : class Mapper
location: class org.myorg.WordCount
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                           ^
WordCount.java:14: cannot find symbol
symbol  : class LongWritable
location: class org.myorg.WordCount
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                  ^
WordCount.java:14: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                                ^
WordCount.java:14: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                                      ^
WordCount.java:14: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                                            ^
WordCount.java:14: interface expected here
  public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                 ^
WordCount.java:15: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount.Map
    private final static IntWritable one = new IntWritable(1);
                         ^
WordCount.java:16: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount.Map
    private Text word = new Text();
            ^
WordCount.java:18: cannot find symbol
symbol  : class LongWritable
location: class org.myorg.WordCount.Map
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                    ^
WordCount.java:18: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount.Map
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                      ^
WordCount.java:18: cannot find symbol
symbol  : class OutputCollector
location: class org.myorg.WordCount.Map
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                  ^
WordCount.java:18: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount.Map
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                  ^
WordCount.java:18: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount.Map
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                        ^
WordCount.java:18: cannot find symbol
symbol  : class Reporter
location: class org.myorg.WordCount.Map
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                             ^
WordCount.java:28: cannot find symbol
symbol  : class MapReduceBase
location: class org.myorg.WordCount
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                     ^
WordCount.java:28: cannot find symbol
symbol  : class Reducer
location: class org.myorg.WordCount
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                              ^
WordCount.java:28: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                      ^
WordCount.java:28: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                            ^
WordCount.java:28: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                                         ^
WordCount.java:28: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                                               ^
WordCount.java:28: interface expected here
  public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                     ^
WordCount.java:29: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount.Reduce
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                       ^
WordCount.java:29: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount.Reduce
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                          ^
WordCount.java:29: cannot find symbol
symbol  : class OutputCollector
location: class org.myorg.WordCount.Reduce
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                               ^
WordCount.java:29: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount.Reduce
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                               ^
WordCount.java:29: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount.Reduce
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                     ^
WordCount.java:29: cannot find symbol
symbol  : class Reporter
location: class org.myorg.WordCount.Reduce
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                                          ^
WordCount.java:15: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount.Map
    private final static IntWritable one = new IntWritable(1);
                                               ^
WordCount.java:16: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount.Map
    private Text word = new Text();
                            ^
WordCount.java:32: operator + cannot be applied to int,IntWritable.get
        sum += values.next().get();
            ^
WordCount.java:32: inconvertible types
found   : <nulltype>
required: int
        sum += values.next().get();
                                ^
WordCount.java:34: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount.Reduce
      output.collect(key, new IntWritable(sum));
                              ^
WordCount.java:39: cannot find symbol
symbol  : class JobConf
location: class org.myorg.WordCount
    JobConf conf = new JobConf(WordCount.class);
    ^
WordCount.java:39: cannot find symbol
symbol  : class JobConf
location: class org.myorg.WordCount
    JobConf conf = new JobConf(WordCount.class);
                       ^
WordCount.java:42: cannot find symbol
symbol  : class Text
location: class org.myorg.WordCount
    conf.setOutputKeyClass(Text.class);
                           ^
WordCount.java:43: cannot find symbol
symbol  : class IntWritable
location: class org.myorg.WordCount
    conf.setOutputValueClass(IntWritable.class);
                             ^
WordCount.java:49: cannot find symbol
symbol  : class TextInputFormat
location: class org.myorg.WordCount
    conf.setInputFormat(TextInputFormat.class);
                        ^
WordCount.java:50: cannot find symbol
symbol  : class TextOutputFormat
location: class org.myorg.WordCount
    conf.setOutputFormat(TextOutputFormat.class);
                         ^
WordCount.java:52: cannot find symbol
symbol  : class Path
location: class org.myorg.WordCount
    FileInputFormat.setInputPaths(conf, new Path(args[0]));
                                            ^
WordCount.java:52: cannot find symbol
symbol  : variable FileInputFormat
location: class org.myorg.WordCount
    FileInputFormat.setInputPaths(conf, new Path(args[0]));
    ^
WordCount.java:53: cannot find symbol
symbol  : class Path
location: class org.myorg.WordCount
    FileOutputFormat.setOutputPath(conf, new Path(args[1]));
                                             ^
WordCount.java:53: cannot find symbol
symbol  : variable FileOutputFormat
location: class org.myorg.WordCount
    FileOutputFormat.setOutputPath(conf, new Path(args[1]));
    ^
WordCount.java:55: cannot find symbol
symbol  : variable JobClient
location: class org.myorg.WordCount
    JobClient.runJob(conf);
    ^
49 errors

wordcount プログラムの実行を手伝ってくれる人、何をすべきかを教えてくれる人、または私が見つけられなかったオンラインの優れたチュートリアルを誰かが知っている人がいたら教えてくれませんか? ありがとう！

score 2 · Accepted Answer

javacのクラスパスに (インポートステートメントに対応する) 依存関係が必要です。

hadoop classpathHadoop では、ユーティリティを使用してこれを簡単に実行できます。

$ javac -classpath `hadoop classpath`:. WordCount.java

cannot find symbolクラスパスが適切に設定された後、この種のコンパイルの問題は発生しないはずです。

java - Cloudera Manager 4.0 ワードカウントの例

1 に答える 1

Related

Reference