1

いくつかのパラメーター (一連の単語) に従って Web サイトを評価する Python 関数を作成しました。この関数は Python Mechanize を使用しており、ほとんどの場合問題なく動作します。

ただし、一部の Web サイトでは、端末で ctrl+c を押すまでハングします。これは何らかの JavaScript 関連の問題だと思いますが、タイムアウト機能を構築する方法はありますか?

これは私の機能です：

def rateSite(site_url,comparisonWords):
   #open the site
   localBrowser = mechanize.Browser()
   localBrowser.addheaders = [('User-agent', 'Mozilla/5.1 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/9.0.1')]
   localBrowser.set_handle_robots(False)
   site = localBrowser.open(site_url,timeout=5000)
   html = site.read()

   #rate the site
   for i in comparisonWords.split():
      #do some rating math

   return rating

これは、ctrl+c で取得したトレースバックです。

site=localBrowser.open(site_url,timeout=5000)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 209, in open
    return self._mech_open(url, data, timeout=timeout)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 236, in _mech_open
    response = UserAgentBase.open(self, request, data)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 202, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 612, in http_response
    "http", request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 219, in error
    result = apply(self._call_chain, args)
  File "/usr/lib/python2.7/urllib2.py", line 372, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 146, in http_error_302
    return self.parent.open(new)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 209, in open
    return self._mech_open(url, data, timeout=timeout)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 236, in _mech_open
    response = UserAgentBase.open(self, request, data)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 202, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 612, in http_response
    "http", request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 219, in error
    result = apply(self._call_chain, args)
  File "/usr/lib/python2.7/urllib2.py", line 372, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 146, in http_error_302
    return self.parent.open(new)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 209, in open
    return self._mech_open(url, data, timeout=timeout)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 236, in _mech_open
    response = UserAgentBase.open(self, request, data)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 202, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 612, in http_response
    "http", request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 219, in error
    result = apply(self._call_chain, args)
  File "/usr/lib/python2.7/urllib2.py", line 372, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 146, in http_error_302
    return self.parent.open(new)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 209, in open
    return self._mech_open(url, data, timeout=timeout)
  File "/usr/lib/python2.7/dist-packages/mechanize/_mechanize.py", line 236, in _mech_open
    response = UserAgentBase.open(self, request, data)
  File "/usr/lib/python2.7/dist-packages/mechanize/_opener.py", line 202, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/dist-packages/mechanize/_http.py", line 578, in http_response
    self._sleep(pause)
KeyboardInterrupt

これを解決する方法またはタイムアウトを作成する方法についてのヘルプは大歓迎です。

ありがとう！

4

1 に答える 1

1

timeout=50001時間以上です。あなたが意味するかもしれませんtimeout=5。

デフォルトmechanizeでは、あきらめる前に最大 10 回のリダイレクトに従います。 を参照してくださいHTTPRedirectHandler.max_redirections。

于 2012-03-13T15:51:55.053 に答える