私はスクレイピーが初めてなので、この質問は単純すぎるように思えるかもしれません。http サイトからのダウンロードには問題ありませんでした。しかし、 this urlに対して同じことをしようとすると、次のエラーが発生します。
コード：
from scrapy.spider import BaseSpider
class Bundspider(BaseSpider):
name="bund"
allowed_domains=["www.bundesanzeiger.de"]
start_urls=[
"https://www.bundesanzeiger.de/ebanzwww/wexsservlet?page.navid=to_nlp_start"
]
def parse(self, response):
filename = response.url.split("/")[-2]
open(filename, 'wb').write(response.body)
エラー：
2013-03-20 01:20:54-0400 [scrapy] INFO: Scrapy 0.16.4 started (bot: tutorial)
2013-03-20 01:20:54-0400 [scrapy] DEBUG: Enabled extensions: LogStats, TelnetConsole, CloseSpider, WebService, CoreStats, SpiderState
2013-03-20 01:20:54-0400 [scrapy] DEBUG: Enabled downloader middlewares: HttpAuthMiddleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, DefaultHeadersMiddleware, RedirectMiddleware, CookiesMiddleware, HttpProxyMiddleware, HttpCompressionMiddleware, ChunkedTransferMiddleware, DownloaderStats
2013-03-20 01:20:54-0400 [scrapy] DEBUG: Enabled spider middlewares: HttpErrorMiddleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddleware
2013-03-20 01:20:54-0400 [scrapy] DEBUG: Enabled item pipelines:
2013-03-20 01:20:54-0400 [bund] INFO: Spider opened
2013-03-20 01:20:54-0400 [bund] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2013-03-20 01:20:54-0400 [scrapy] DEBUG: Telnet console listening on 0.0.0.0:6023
2013-03-20 01:20:54-0400 [scrapy] DEBUG: Web service listening on 0.0.0.0:6080
2013-03-20 01:20:54-0400 [bund] ERROR: Error downloading <GET https://www.bundesanzeiger.de/ebanzwww/wexsservlet?page.navid=to_nlp_start>: [('SSL routines', 'SSL23_GET_SERVER_HELLO', 'unknown protocol')]
2013-03-20 01:20:54-0400 [bund] INFO: Closing spider (finished)
2013-03-20 01:20:54-0400 [bund] INFO: Dumping Scrapy stats:
{'downloader/exception_count': 1,
'downloader/exception_type_count/OpenSSL.SSL.Error': 1,
'downloader/request_bytes': 271,
'downloader/request_count': 1,
'downloader/request_method_count/GET': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2013, 3, 20, 5, 20, 54, 814159),
'log_count/DEBUG': 6,
'log_count/ERROR': 1,
'log_count/INFO': 4,
'scheduler/dequeued': 1,
'scheduler/dequeued/memory': 1,
'scheduler/enqueued': 1,
'scheduler/enqueued/memory': 1,
'start_time': datetime.datetime(2013, 3, 20, 5, 20, 54, 796438)}
2013-03-20 01:20:54-0400 [bund] INFO: Spider closed (finished)