1

送信するリクエストが最大 50 個しかない場合にうまく機能するコードを試していますが、現在はさらに多くのリクエスト (約 200 個) を試しています。サーバーが 60 個のリクエストの後に接続を閉じているように感じます。 「なし」の回答。

私が使用しているコードは次のとおりです。

urls = []
for kw in keywords:
    urls.append("https://play.google.com/store/search?q=" + kw + "&c=apps&hl=fr")

from lxml import html

rs = (grequests.get(u) for u in urls)
responses = grequests.map(rs)

for idx, r in enumerate(responses): # r = results page
    time.sleep(2)
    responses_tree = html.fromstring(r.content) # Readable HTML

反応:

Out[893]: 
[<Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 <Response [200]>,
 None,
 <Response [200]>,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None,
 None]

とトレースバック:

AttributeError                            Traceback (most recent call last)
C:\Users\Nathan\googlestore.py in <module>()
    249 for idx, r in enumerate(responses): # r = results page
    250     time.sleep(2)
--> 251     responses_tree = html.fromstring(r.content) # Readable HTML
    252     apps_links = responses_tree.xpath('//a[@class="title"]/@href') # Get all links
    253     if len(apps_links) > 10:

AttributeError: 'NoneType' object has no attribute 'content' 

どうすればこれを修正できますか? 50 のリクエストが送信されてから 30 秒ほど待ってから、さらに 50 を送信する必要がありますか? サーバーへのリクエストに関する Google ドキュメントへのリンクを持っている人はいますか?

ありがとう!

4

0 に答える 0