2

削減ステップ中にこのようなエラーが発生します。MapReduceの概要ページには、すべてのジョブが正常に完了したと報告されていますが、書き出そうとしたレコードの半分は書き込まれていません。このエラーの原因とその修正方法を教えてください。

    ApplicationError: 10 
Traceback (most recent call last):
  File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__
    rv = self.handle_exception(request, response, e)
  File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
    rv = self.router.dispatch(request, response)
  File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
    return handler.dispatch()
  File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/base_handler.py", line 65, in post
    self.handle()
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/handlers.py", line 181, in handle
    entity, input_reader, ctx, tstate)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/handlers.py", line 298, in process_data
    output_writer.write(output, ctx)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/output_writers.py", line 693, in write
    ctx.get_pool("records_pool").append(str(data))
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/output_writers.py", line 336, in append
    self.flush()
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/output_writers.py", line 371, in flush
    COUNTER_IO_WRITE_BYTES, len(str_buf))(self._ctx)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 297, in __exit__
    self.close()
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 291, in close
    self._make_rpc_call_with_retry('Close', request, response)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 427, in _make_rpc_call_with_retry
    _make_call(method, request, response)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 252, in _make_call
    _raise_app_error(e)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 195, in _raise_app_error
    raise FileNotOpenedError(e)
FileNotOpenedError: ApplicationError: 10

Traceback (most recent call last):
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/main.py", line 105, in main
    util.run_wsgi_app(APP)
  File "/python27_runtime/python27_lib/versions/1/google/appengine/ext/webapp/util.py", line 98, in run_wsgi_app
    run_bare_wsgi_app(add_wsgi_middleware(application))
  File "/python27_runtime/python27_lib/versions/1/google/appengine/ext/webapp/util.py", line 116, in run_bare_wsgi_app
    result = application(env, _start_response)
  File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1519, in __call__
    response = self._internal_error(e)
  File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__
    rv = self.handle_exception(request, response, e)
  File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
    rv = self.router.dispatch(request, response)
  File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
    return handler.dispatch()
  File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/base_handler.py", line 65, in post
    self.handle()
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/handlers.py", line 181, in handle
    entity, input_reader, ctx, tstate)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/handlers.py", line 298, in process_data
    output_writer.write(output, ctx)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/output_writers.py", line 693, in write
    ctx.get_pool("records_pool").append(str(data))
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/output_writers.py", line 336, in append
    self.flush()
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/output_writers.py", line 371, in flush
    COUNTER_IO_WRITE_BYTES, len(str_buf))(self._ctx)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 297, in __exit__
    self.close()
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 291, in close
    self._make_rpc_call_with_retry('Close', request, response)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 427, in _make_rpc_call_with_retry
    _make_call(method, request, response)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 252, in _make_call
    _raise_app_error(e)
  File "/base/data/home/apps/s~getaround3/dont-make-this-default-export-mr.366029363091232181/mapreduce/lib/files/file.py", line 195, in _raise_app_error
    raise FileNotOpenedError(e)
FileNotOpenedError: ApplicationError: 10
4

2 に答える 2

2

問題を解決したと思われるシャードの数を増やしました。シャードの数を増やすと、各スライスがファイルを開いたままにしておく時間が短縮されます。これが、私が抱えていた問題の原因であった可能性があります。

于 2013-03-27T00:24:43.703 に答える
0

ファイルを30秒以上開いたままにすると、このエラーも発生します。書き込みを複数の書き込みに分割してみてください。

詳細については、リンクを参照してくださいhttps://groups.google.com/forum/#!topic/google-appengine-java/5XBVaVDKvvs

于 2013-08-19T06:57:28.173 に答える