Pythonコマンドを実行するPythonプログラムがあります
WARNING:root:executing - echo ~/Downloads/dw/hadoop_out/hadoop_out.2012-07-20 > /tmp/hadoop_load_file.tmp
WARNING:root:hadoop output file path written to /tmp/hadoop_load_file.tmp
WARNING:root:executing - /usr/local/hadoop/bin/hadoop dfs -cat ~/Downloads/dw/proposalLoader_out/part-00000 > ~/Downloads/dw/hadoop_out/hadoop_out.2012-07-20
WARNING:root:hadoop path exists, removing - /usr/local/hadoop/bin/hadoop dfs -test -d ~/Downloads/dw/proposalLoader_in
Deleted hdfs://master:54310/home/hduser/Downloads/dw/Loader_in
test: File does not exist: /home/hduser/Downloads/dw/Loader_out
WARNING:root:executing - /usr/local/hadoop/bin/hadoop dfs -mkdir ~/Downloads/dw/proposalLoader_in
WARNING:root:executing - /usr/local/hadoop/bin/hadoop dfs -copyFromLocal ~/Downloads/dw/load_input.txt ~/Downloads/dw/Loader_in
しかし、何が入っているかを見ると、HDFS何も得られません
$ hadoop dfs -ls ~/Downloads/dw/Loader_in
Found 1 items
-rw-r--r-- 3 hduser supergroup 0 2012-07-20 06:49 /home/hduser/Downloads/dw/Loader_in/load_input.txt
から同じコマンドをshell実行すると、実行されてデータが取得されますHDFS
$ /usr/local/hadoop/bin/hadoop dfs -copyFromLocal ~/Downloads/dw/load_input.txt ~/Downloads/dw/Loader_in
$ hadoop dfs -ls ~/Downloads/dw/Loader_in
Found 1 items
-rw-r--r-- 3 hduser supergroup 2311 2012-07-20 06:50 /home/hduser/Downloads/dw/Loader_in
この動作が見られる理由がわかりません。コマンドを実行するコードは次のとおりです。
def execute(command):
if not command:
return (-1, 'command can not be empty or null')
return subprocess.call(command, shell=True)