メッセージ ハブにメッセージをプッシュするために kafka python クライアントを使用していますが、しばらくアプリを実行すると、メッセージ ハブへのメッセージの送信が停止することに気付きました。
その後、ログ ファイルで次のことに気付きました。
ConnectionError: socket disconnected
コードを更新して追加しましたretries=5：
from kafka import KafkaProducer
from kafka.errors import KafkaError
import ssl
sasl_mechanism = 'PLAIN'
security_protocol = 'SASL_SSL'
# Create a new context using system defaults, disable all but TLS1.2
context = ssl.create_default_context()
context.options &= ssl.OP_NO_TLSv1
context.options &= ssl.OP_NO_TLSv1_1
producer = KafkaProducer(bootstrap_servers = app.config['KAFKA_BROKERS_SASL'],
sasl_plain_username = app.config['KAFKA_USERNAME'],
sasl_plain_password = app.config['KAFKA_PASSWORD'],
security_protocol = security_protocol,
ssl_context = context,
sasl_mechanism = sasl_mechanism,
api_version = (0,10),
retries=5)
def send_message(message):
try:
producer.send(app.config['KAFKA_TOPIC'], message.encode('utf-8'))
# FIXME sending seems to be unreliable unless we flush
producer.flush()
except:
print("Unexpected error:", sys.exc_info()[0])
raise
私のコードは python フラスコ アプリから実行されます。着信する特定のタイプのすべてのリクエストがsend_message()メソッドを呼び出します。
以下は、Bluemix からの関連するログ行です。1 行か 2 行のコピーと貼り付けを見逃している可能性がありますが、何が起こっているのかを理解するのに十分であることを願っています。
APP/1 <BrokerConnection host=kafka01-prod01.messagehub.services.us-south.bluemix.net/23.246.202.51 port=9093>: Authenticated as xxxxx May 15, 2017 8:50:54 PM
APP/1 <BrokerConnection host=kafka01-prod01.messagehub.services.us-south.bluemix.net/23.246.202.51 port=9093>: socket disconnected May 15, 2017 8:50:54 PM
APP/1 Got error produce response on topic-partition TopicPartition(topic='movie_ratings', partition=0), retrying (4 attempts left). Error: ConnectionError: socket disconnectedMay 15, 2017 8:50:54 PM
APP/1 Node 0 connection failed -- refreshing metadata May 15, 2017 8:50:54 PM
APP/1 <BrokerConnection host=kafka01-prod01.messagehub.services.us-south.bluemix.net/23.246.202.51 port=9093>: Authenticated as xxxxxMay 15, 2017 8:50:54 PM
APP/1 <BrokerConnection host=kafka01-prod01.messagehub.services.us-south.bluemix.net/23.246.202.51 port=9093>: socket disconnected May 15, 2017 8:50:54 PM
APP/1 Node 0 connection failed -- refreshing metadata May 15, 2017 8:50:54 PM
APP/1 <BrokerConnection host=kafka01-prod01.messagehub.services.us-south.bluemix.net/23.246.202.51 port=9093>: Authenticated as xxxxx May 15, 2017 8:50:54 PM
APP/1 <BrokerConnection host=kafka01-prod01.messagehub.services.us-south.bluemix.net/23.246.202.51 port=9093>: socket disconnected May 15, 2017 8:50:55 PM
APP/1 Node 0 connection failed -- refreshing metadata May 15, 2017 8:50:55 PM
APP/1 Got error produce response on topic-partition TopicPartition(topic='movie_ratings', partition=0), retrying (2 attempts left). Error: ConnectionError: socket disconnected May 15, 2017 8:50:55 PM
APP/1 <BrokerConnection host=kafka01-prod01.messagehub.services.us-south.bluemix.net/23.246.202.51 port=9093>: Authenticated as xxxxx May 15, 2017 8:50:55 PM
APP/1 <BrokerConnection host=kafka01-prod01.messagehub.services.us-south.bluemix.net/23.246.202.51 port=9093>: socket disconnected May 15, 2017 8:50:55 PM
APP/1 Node 0 connection failed -- refreshing metadata May 15, 2017 8:50:55 PM
APP/1 Got error produce response on topic-partition TopicPartition(topic='movie_ratings', partition=0), retrying (1 attempts left). Error: ConnectionError: socket disconnected May 15, 2017 8:50:55 PM
APP/1 <BrokerConnection host=kafka01-prod01.messagehub.services.us-south.bluemix.net/23.246.202.51 port=9093>: Authenticated as xxxxx May 15, 2017 8:50:55 PM
APP/1 <BrokerConnection host=kafka01-prod01.messagehub.services.us-south.bluemix.net/23.246.202.51 port=9093>: socket disconnected May 15, 2017 8:50:55 PM
APP/1 Got error produce response on topic-partition TopicPartition(topic='movie_ratings', partition=0), retrying (0 attempts left). Error: ConnectionError: socket disconnected May 15, 2017 8:50:55 PM
APP/1 Node 0 connection failed -- refreshing metadata May 15, 2017 8:50:55 PM
APP/1 <BrokerConnection host=kafka01-prod01.messagehub.services.us-south.bluemix.net/23.246.202.51 port=9093>: Authenticated as xxxxx May 15, 2017 8:50:55 PM
APP/1 Node 0 connection failed -- refreshing metadata May 15, 2017 8:50:56 PM
APP/1 ConnectionError: socket disconnected May 15, 2017 8:50:56 PM
APP/1 Unable to import 'sasl'. Fallback to 'puresasl'. May 15, 2017 8:51:00 PM
APP/1 Closing active operation May 15, 2017 8:51:01 PM
私のトピックは存在します。私の完全なクライアント コードはこちら: https://github.com/snowch/movie-recommender-demo/blob/effc981cc9f799c41952719619f693172eebcd6a/web_app/app/messagehub_client.py
最も高く評価されているポインタ...