python - 基本認証で SSL を使用するように Elasticsearch を構成する方法

Question

Ramses ( http://ramses.tech ) を使用するアプリを IBM Bluemixにデプロイしようとしています。残念ながら、デプロイ プロセス中にアプリがクラッシュします。local.ini 構成ファイルで、次のように設定しました。

# ElasticSearch
elasticsearch.hosts = xxxx.dblayer.com:9999
elasticsearch.http_auth = user:secret
elasticsearch.http.use_ssl = true
elasticsearch.verify_certs = true
elasticsearch.sniff = false
elasticsearch.index_name = my_api
elasticsearch.index.disable = false
elasticsearch.enable_refresh_query = false
elasticsearch.enable_aggregations = false
elasticsearch.enable_polymorphic_query = false

ただし、アプリを ibm bluemix にデプロイしようとすると、次のエラーが発生します。

2016-10-16T13:40:52.226-0400[DEA/64]OUTStarting app instance (index 0) with guid 30875156-21f2-4e49-b115-882ec3efc41a
2016-10-16T13:41:01.121-0400[App/0]ERR2016-10-16 17:41:01,120 INFO [nefertari_mongodb.signals][MainThread] signals.setup_es_signals_for: setup_es_signals_for: <class 'nefertari_mongodb.documents.ESBaseDocument'>
2016-10-16T13:41:01.130-0400[App/0]ERR2016-10-16 17:41:01,129 INFO [nefertari.json_httpexceptions][MainThread] json_httpexceptions.includeme: Include json_httpexceptions
2016-10-16T13:41:01.130-0400[App/0]ERR2016-10-16 17:41:01,129 INFO [ramses][MainThread] __init__.includeme: Parsing RAML
2016-10-16T13:41:01.125-0400[App/0]ERR2016-10-16 17:41:01,124 INFO [nefertari][MainThread] __init__.includeme: nefertari 0.7.0
2016-10-16T13:41:01.186-0400[App/0]ERR2016-10-16 17:41:01,185 INFO [ramses.generators][MainThread] generators.generate_models: Configuring model for route `schools`
2016-10-16T13:41:01.191-0400[App/0]ERR2016-10-16 17:41:01,190 INFO [ramses.utils][MainThread] utils.resource_schema: Searching for model schema
2016-10-16T13:41:01.191-0400[App/0]ERR2016-10-16 17:41:01,190 INFO [ramses.models][MainThread] models.setup_data_model: Generating model class `School`
2016-10-16T13:41:01.184-0400[App/0]ERR2016-10-16 17:41:01,183 INFO [ramses][MainThread] __init__.includeme: Starting models generation
2016-10-16T13:41:01.198-0400[App/0]ERR2016-10-16 17:41:01,198 INFO [ramses.utils][MainThread] utils.resource_schema: Searching for model schema
2016-10-16T13:41:01.200-0400[App/0]ERR2016-10-16 17:41:01,199 INFO [nefertari_mongodb.signals][MainThread] signals.setup_es_signals_for: setup_es_signals_for: <class 'mongoengine.base.metaclasses.User'>
2016-10-16T13:41:01.199-0400[App/0]ERR2016-10-16 17:41:01,198 INFO [ramses.models][MainThread] models.setup_data_model: Generating model class `User`
2016-10-16T13:41:01.203-0400[App/0]ERR2016-10-16 17:41:01,203 INFO [nefertari_mongodb.signals][MainThread] signals.setup_es_signals_for: setup_es_signals_for: <class 'mongoengine.base.metaclasses.School'>
2016-10-16T13:41:01.206-0400[App/0]ERR2016-10-16 17:41:01,205 INFO [nefertari.elasticsearch][MainThread] elasticsearch.setup: Including Elasticsearch. {'chunk_size': 500, 'index.disable': 'false', 'verify_certs': 'true', 'host': 'https://xxxx.dblayer.com:9999', 'enable_refresh_query': 'false', 'enable_aggregations': 'false', 'port': '443', 'index_name': 'quartolio_api', 'enable_polymorphic_query': 'false', 'hosts': 'xxxx.dblayer.com:9999', 'sniff': 'false', 'http_auth': 'user:secret'}
2016-10-16T13:41:01.205-0400[App/0]ERR2016-10-16 17:41:01,204 INFO [ramses.generators][MainThread] generators.generate_models: Configuring model for route `users`
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 94, in perform_request
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/urllib3/connectionpool.py", line 384, in _make_request
2016-10-16T13:41:01.218-0400[App/0]ERR _stacktrace=sys.exc_info()[2])
2016-10-16T13:41:01.218-0400[App/0]ERR chunked=chunked)
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/httplib.py", line 1136, in getresponse
2016-10-16T13:41:01.218-0400[App/0]ERRTraceback (most recent call last):
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/httplib.py", line 453, in begin
2016-10-16T13:41:01.218-0400[App/0]ERR response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
2016-10-16T13:41:01.218-0400[App/0]ERR httplib_response = conn.getresponse(buffering=True)
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/urllib3/util/retry.py", line 251, in increment
2016-10-16T13:41:01.218-0400[App/0]ERR version, status, reason = self._read_status()
2016-10-16T13:41:01.218-0400[App/0]ERR2016-10-16 17:41:01,216 WARNI [elasticsearch][MainThread] base.log_request_fail: HEAD http://xxxx.dblayer.com:9999/my_api [status:N/A request:0.011s]
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/urllib3/connectionpool.py", line 594, in urlopen
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/httplib.py", line 417, in _read_status
2016-10-16T13:41:01.218-0400[App/0]ERR response.begin()
2016-10-16T13:41:01.218-0400[App/0]ERR raise six.reraise(type(error), error, _stacktrace)
2016-10-16T13:41:01.218-0400[App/0]ERR raise BadStatusLine(line)
2016-10-16T13:41:01.218-0400[App/0]ERRProtocolError: ('Connection aborted.', BadStatusLine("''",))
2016-10-16T13:41:01.221-0400[App/0]ERR return command.run()
2016-10-16T13:41:01.221-0400[App/0]ERR File "/home/vcap/app/.heroku/python/bin/pserve", line 11, in <module>
2016-10-16T13:41:01.221-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
2016-10-16T13:41:01.221-0400[App/0]ERR return loadapp(app_spec, name=name, relative_to=relative_to, **kw)
2016-10-16T13:41:01.221-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/urlmap.py", line 31, in urlmap_factory
2016-10-16T13:41:01.221-0400[App/0]ERR2016-10-16 17:41:01,221 ERROR [nefertari.json_httpexceptions][MainThread] json_httpexceptions.create_json_response: 400 BAD REQUEST: {"explanation": "('Connection aborted.', BadStatusLine(\"''\",))", "extra": {"data": "ConnectionError(('Connection aborted.', BadStatusLine(\"''\",))) caused by: ProtocolError(('Connection aborted.', BadStatusLine(\"''\",)))"}, "status_code": 400, "message": null, "timestamp": "2016-10-16T17:41:01Z", "title": "Bad Request"}
2016-10-16T13:41:01.222-0400[App/0]ERR val = callable(*args, **kw)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
2016-10-16T13:41:01.221-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 328, in run
2016-10-16T13:41:01.221-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 58, in main
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2016-10-16T13:41:01.222-0400[App/0]ERR File "/home/vcap/app/quartolio_api/quartolio_api/__init__.py", line 14, in main
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/config/__init__.py", line 755, in include
2016-10-16T13:41:01.222-0400[App/0]ERR **context.local_conf)
2016-10-16T13:41:01.221-0400[App/0]ERR global_conf=vars)
2016-10-16T13:41:01.221-0400[App/0]ERR sys.exit(main())
2016-10-16T13:41:01.221-0400[App/0]ERRSTACK BEGIN>>
2016-10-16T13:41:01.222-0400[App/0]ERR return fix_call(context.object, context.global_conf, **context.local_conf)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2016-10-16T13:41:01.222-0400[App/0]ERR return func(*args, params=params, **kwargs)
2016-10-16T13:41:01.221-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 363, in loadapp
2016-10-16T13:41:01.222-0400[App/0]ERR return self.object_type.invoke(self)
2016-10-16T13:41:01.221-0400[App/0]ERR return loadobj(APP, uri, name=name, **kw)
2016-10-16T13:41:01.222-0400[App/0]ERR params=params)
2016-10-16T13:41:01.222-0400[App/0]ERR config.include('nefertari.elasticsearch')
2016-10-16T13:41:01.222-0400[App/0]ERR app = loader.get_app(app_name, global_conf=global_conf)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2016-10-16T13:41:01.222-0400[App/0]ERR return context.create()
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/transport.py", line 327, in perform_request
2016-10-16T13:41:01.222-0400[App/0]ERR cls.api.indices.exists([index_name])
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/json_httpexceptions.py", line 78, in __init__
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/elasticsearch.py", line 213, in create_index
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
2016-10-16T13:41:01.223-0400[App/0]ERR return loadapp(app_spec, name=name, relative_to=relative_to, **kw)
2016-10-16T13:41:01.222-0400[App/0]ERR name=name, global_conf=global_conf).create()
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/json_httpexceptions.py", line 67, in exception_response
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/config/__init__.py", line 755, in include
2016-10-16T13:41:01.222-0400[App/0]ERR c(configurator)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/elasticsearch.py", line 69, in includeme
2016-10-16T13:41:01.222-0400[App/0]ERR return STATUS_MAP[status_code](**kw)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/json_httpexceptions.py", line 26, in add_stack
2016-10-16T13:41:01.222-0400[App/0]ERR val = callable(*args, **kw)
2016-10-16T13:41:01.222-0400[App/0]ERR c(configurator)
2016-10-16T13:41:01.222-0400[App/0]ERR ES.create_index()
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/ramses/__init__.py", line 56, in includeme
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/elasticsearch.py", line 60, in perform_request
2016-10-16T13:41:01.223-0400[App/0]ERR **context.local_conf)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/client/indices.py", line 225, in exists
2016-10-16T13:41:01.222-0400[App/0]ERR create_json_response(self, **kw)
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
2016-10-16T13:41:01.223-0400[App/0]ERR return loadobj(APP, uri, name=name, **kw)
2016-10-16T13:41:01.222-0400[App/0]ERR extra=dict(data=e))
2016-10-16T13:41:01.222-0400[App/0]ERR status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
2016-10-16T13:41:01.223-0400[App/0]ERR File "/home/vcap/app/.heroku/python/bin/pserve", line 11, in <module>
2016-10-16T13:41:01.222-0400[App/0]ERR msg += '\nSTACK BEGIN>>\n%s\nSTACK END<<' % add_stack()
2016-10-16T13:41:01.223-0400[App/0]ERRTraceback (most recent call last):
2016-10-16T13:41:01.224-0400[App/0]ERR c(configurator)
2016-10-16T13:41:01.222-0400[App/0]ERRSTACK END<<
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 328, in run
2016-10-16T13:41:01.223-0400[App/0]ERR global_conf=vars)
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/json_httpexceptions.py", line 58, in create_json_response
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2016-10-16T13:41:01.223-0400[App/0]ERR return command.run()
2016-10-16T13:41:01.223-0400[App/0]ERR val = callable(*args, **kw)
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 58, in main
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/scripts/pserve.py", line 363, in loadapp
2016-10-16T13:41:01.223-0400[App/0]ERR return self.object_type.invoke(self)
2016-10-16T13:41:01.223-0400[App/0]ERR return fix_call(context.object, context.global_conf, **context.local_conf)
2016-10-16T13:41:01.223-0400[App/0]ERR return context.create()
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
2016-10-16T13:41:01.224-0400[App/0]ERR extra=dict(data=e))
2016-10-16T13:41:01.223-0400[App/0]ERR return self.object_type.invoke(self)
2016-10-16T13:41:01.223-0400[App/0]ERR File "/home/vcap/app/quartolio_api/quartolio_api/__init__.py", line 14, in main
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/config/__init__.py", line 755, in include
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/ramses/__init__.py", line 56, in includeme
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/pyramid/config/__init__.py", line 755, in include
2016-10-16T13:41:01.224-0400[App/0]ERR config.include('nefertari.elasticsearch')
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
2016-10-16T13:41:01.224-0400[App/0]ERR ES.create_index()
2016-10-16T13:41:01.224-0400[App/0]ERR c(configurator)
2016-10-16T13:41:01.224-0400[App/0]ERRnefertari.json_httpexceptions.JHTTPBadRequest: ('Connection aborted.', BadStatusLine("''",))
2016-10-16T13:41:01.223-0400[App/0]ERR val = callable(*args, **kw)
2016-10-16T13:41:01.224-0400[App/0]ERR status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2016-10-16T13:41:01.223-0400[App/0]ERR config.include('ramses')
2016-10-16T13:41:01.223-0400[App/0]ERR app = loader.get_app(app_name, global_conf=global_conf)
2016-10-16T13:41:01.224-0400[App/0]ERR params=params)
2016-10-16T13:41:01.223-0400[App/0]ERR name=name, global_conf=global_conf).create()
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/elasticsearch.py", line 69, in includeme
2016-10-16T13:41:01.224-0400[App/0]ERR return func(*args, params=params, **kwargs)
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/elasticsearch/transport.py", line 327, in perform_request
2016-10-16T13:41:01.224-0400[App/0]ERR cls.api.indices.exists([index_name])
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/elasticsearch.py", line 60, in perform_request
2016-10-16T13:41:01.218-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/urllib3/connectionpool.py", line 643, in urlopen
2016-10-16T13:41:01.222-0400[App/0]ERR return self.object_type.invoke(self)
2016-10-16T13:41:01.223-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/urlmap.py", line 31, in urlmap_factory
2016-10-16T13:41:01.222-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
2016-10-16T13:41:01.218-0400[App/0]ERR2016-10-16 17:41:01,217 ERROR [nefertari.elasticsearch][MainThread] elasticsearch.perform_request: ('Connection aborted.', BadStatusLine("''",))
2016-10-16T13:41:01.222-0400[App/0]ERR config.include('ramses')
2016-10-16T13:41:01.223-0400[App/0]ERR sys.exit(main())
2016-10-16T13:41:01.224-0400[App/0]ERR File "/app/.heroku/python/lib/python2.7/site-packages/nefertari/elasticsearch.py", line 213, in create_index
2016-10-16T13:41:01.262-0400[App/0]OUT
2016-10-16T13:41:01.315-0400[DEA/64]ERRInstance (index 0) failed to start accepting connection

エラーに基づいて、https:// xxxx.dblayer.com:9999 ではなく、http ://xxxx.dblayer.com:9999 に接続しようとしているようです。

私は多くの異なる構成を試しました：

1. ネフェルタリからelasticsearch.hostandelasticsearch.port の代わりにelasticsearch.hostsand を設定してみました。Exception: Bad or missing settings for elasticsearch. 'hosts'

2. elasticsearch.hostshttps:// を前に付け て設定しようとしましたが、その結果、 ValueError: too many values to unpackfrom nefertari になります (コロンを区切り文字として使用して分割しようとしているようです)。

3. elasticsearch.hosts設定中に空白にするか、空白で囲まれたコロンだけに設定しようとしelasticsearch.host ましたがelasticsearch.port、ValueError: need more than 0 values to unpack

bluemix elasticsearch サービスに接続するように Elasticsearch を構成するにはどうすればよいですか?