PBS スケジューリングを使用して、Linux クラスターで ipython 並列をセットアップしようとしています。
http://www.andreazonca.com/2013/04/ipython-parallell-setup-on-carver-at.htmlの指示に従っていました (公式の指示に従うのははるかに困難です)。ヘッド ノードでコマンドを実行しています。このコマンドは、ジョブを PBS を使用してスレーブ ノードに送信します (つまり、標準のクラスター構成)。
私の問題は、タイムアウトが発生することです。待機時間を 2 秒から 20 秒に増やしてみましたが、成功しませんでした。どんな助けでも大歓迎です。完全な出力は以下のとおりです。
実際には、最終的には、クラスター ヘッド ノードからではなく、ssh に接続されたラップトップから ipython コマンドを実行できるようにしたいと考えていますが、これは合理的な最初のステップだと思いました。
2013-08-11 13:56:07,380.380 [IPEngineApp] Config changed:
2013-08-11 13:56:07,381.381 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:07,381.381 [IPEngineApp] Config changed:
2013-08-11 13:56:07,382.382 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:07,381.381 [IPEngineApp] Config changed:
2013-08-11 13:56:07,381.381 [IPEngineApp] Config changed:
2013-08-11 13:56:07,383.383 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:07,382.382 [IPEngineApp] Config changed:
2013-08-11 13:56:07,382.382 [IPEngineApp] Config changed:
2013-08-11 13:56:07,383.383 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:07,382.382 [IPEngineApp] Config changed:
2013-08-11 13:56:07,383.383 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:07,383.383 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:07,383.383 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:07,382.382 [IPEngineApp] Config changed:
2013-08-11 13:56:07,383.383 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:07,387.387 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:07,387.387 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:07,388.388 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:07,388.388 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:07,388.388 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:07,389.389 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:07,389.389 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:07,389.389 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:07,389.389 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:07,389.389 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:07,389.389 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:07,389.389 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:07,389.389 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:07,389.389 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:07,389.389 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:07,389.389 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:07,389.389 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:07,389.389 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:07,389.389 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:07,389.389 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:07,389.389 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:07,389.389 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:07,390.390 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:07,390.390 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:07,390.390 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:07,390.390 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:07,390.390 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:07,390.390 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:07,390.390 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:07,390.390 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:07,390.390 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:07,390.390 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:07,390.390 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:07,390.390 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:07,390.390 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:07,390.390 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:07,390.390 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:07,390.390 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:07,391.391 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:07,391.391 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:07,391.391 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:07,391.391 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:07,391.391 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:07.391 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:07,391.391 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:07.391 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:07,391.391 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:07.391 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:07,391.391 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:07,392.392 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:07.392 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:07.392 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:07.392 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:07,392.392 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:07.392 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:07.393 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:07.402 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:07.402 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:07.402 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:07.402 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:07.403 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:07.403 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:07.403 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:07.403 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:01,273.273 [IPEngineApp] Config changed:
2013-08-11 13:56:01,273.273 [IPEngineApp] Config changed:
2013-08-11 13:56:01,274.274 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:01,274.274 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:01,272.272 [IPEngineApp] Config changed:
2013-08-11 13:56:01,274.274 [IPEngineApp] Config changed:
2013-08-11 13:56:01,275.275 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:01,275.275 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:01,273.273 [IPEngineApp] Config changed:
2013-08-11 13:56:01,276.276 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:01,276.276 [IPEngineApp] Config changed:
2013-08-11 13:56:01,276.276 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:01,278.278 [IPEngineApp] Config changed:
2013-08-11 13:56:01,278.278 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:01,279.279 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:01,279.279 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:01,279.279 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:01,279.279 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:01,279.279 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:01,279.279 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:01,279.279 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:01,280.280 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:01,280.280 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:01,280.280 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:01,280.280 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:01,280.280 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:01,280.280 [IPEngineApp] Config changed:
2013-08-11 13:56:01,280.280 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:01,280.280 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:01,280.280 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:01,280.280 [IPEngineApp] {'EngineFactory': {'timeout': 10}, 'IPEngineApp': {'log_level': 10}}
2013-08-11 13:56:01,280.280 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:01,280.280 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:01,280.280 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:01,280.280 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:01,280.280 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:01,281.281 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:01,281.281 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:01,281.281 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:01,281.281 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:01,281.281 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:01,281.281 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:01,281.281 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:01,281.281 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:01,281.281 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:01,281.281 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:01,281.281 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:01,281.281 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:01,282.282 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:01,282.282 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:01,282.282 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:01,282.282 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:01.282 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:01,282.282 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:01.282 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:01,282.282 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:01.282 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:01,283.283 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:01,283.283 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:01,283.283 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:01,284.284 [IPEngineApp] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-08-11 13:56:01,284.284 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:01,284.284 [IPEngineApp] Searching path [u'/home/username', u'/home/username/.ipython/profile_default'] for config files
2013-08-11 13:56:01,284.284 [IPEngineApp] Attempting to load config file: ipython_config.py
2013-08-11 13:56:01.284 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:01.284 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:01.284 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:01.284 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:01,284.284 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipython_config.py
2013-08-11 13:56:01,285.285 [IPEngineApp] Attempting to load config file: ipengine_config.py
2013-08-11 13:56:01,286.286 [IPEngineApp] Loaded config file: /home/username/.ipython/profile_default/ipengine_config.py
2013-08-11 13:56:01.286 [IPEngineApp] Loading url_file u'/home/username/.ipython/profile_default/security/ipcontroller-engine.json'
2013-08-11 13:56:01.295 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:01.295 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:01.296 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:01.296 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:01.296 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:01.296 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:01.298 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:01.299 [IPEngineApp] Registering with controller at tcp://127.0.0.1:53956
2013-08-11 13:56:17.411 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:17.412 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:17.413 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:17.412 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:17.412 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:17.413 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:17.413 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:17.413 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:11.304 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:11.304 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:11.305 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:11.306 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:11.306 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:11.306 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:11.307 [IPEngineApp] Registration timed out after 10.0 seconds
2013-08-11 13:56:11.309 [IPEngineApp] Registration timed out after 10.0 seconds
--------------------------------------------------------------------------
mpirun noticed that the job aborted, but has no info as to the process
that caused that situation.
--------------------------------------------------------------------------
Andrea Zonca からのアドバイスに従って更新
追加する
c.HubFactory.ip = '*'
なんらかの理由ですぐにではありませんが、役立つようです。
しかし、それでもうまくいきません:
私は一緒に走ります
./.ipython/profile_default/ipc 2 1
2013-09-09 12:19:51,884.884 [IPClusterStart] Using existing profile dir: u'/home/username/.ipython/profile_default'
2013-09-09 12:19:51.889 [IPClusterStart] Starting ipcluster with [daemon=False]
2013-09-09 12:19:51.890 [IPClusterStart] Creating pid file: /home/username/.ipython/profile_default/pid/ipcluster.pid
2013-09-09 12:19:51.890 [IPClusterStart] Starting Controller with LocalControllerLauncher
2013-09-09 12:19:52.890 [IPClusterStart] Starting 2 Engines with PBS
2013-09-09 12:19:52.904 [IPClusterStart] Job submitted with job id: '4783'
2013-09-09 12:20:22.904 [IPClusterStart] Engines appear to have started successfully
次に、ヘッド ノードで実行します。
>IPython.parallel import Client
>rc = Client()
>lview = rc.load_balanced_view()
しかし、からの出力
>rc.ids
は
[]
ということで走ってみました
ipcontroller --port=8888
次に nmap を実行します
$nmap -sT -O localhost
...
8888/tcp open sun-answerbook
...
これは、ポートが開いていることを示しており、十分な telnet がスレーブ ノードからの応答を返します。
しかし、上記の元のコマンドを実行すると、
./.ipython/profile_default/ipc 2 1
nmap は、開いているポートを表示しません。したがって、問題は、qsub ファイルで実行される ipengine が、コマンド ラインから実行される ipcontroller のようにポートを開いていないように思われます。
qsub ファイルは次のとおりです。
$cat /home/username/.ipython/profile_default/pbs.engine.template.ppn2
#!/bin/sh
#PBS -q longqueue
#PBS -l nodes={n/2}:ppn=2
cd $PBS_O_WORKDIR
which ipengine
mpirun -np {n} ipengine --timeout=20
ここに私の /home/username/.ipython/profile_default/ipcluster_config.py があります:
c = get_config()
c.IPClusterStart.controller_launcher_class = 'LocalControllerLauncher'
c.IPClusterStart.engine_launcher_class = 'PBS'
c.PBSLauncher.batch_template_file = u'/home/username/.ipython/profile_default/pbs.engine.template'