0

tf.keras で CNN をトレーニングしています。チェックポイントを保存した後、Keras は次のエポックを開始しませんでした

注: 1) セーバーとして tf.keras.callbacks.ModelCeckpoint を使用しました 2) トレーニングには fit_generator() を使用しました

def iterate_minibatches(inputs, targets, batchsize):
    assert len(inputs) == len(targets)
    indices = np.arange(len(inputs))
    np.random.shuffle(indices)

    for start_idx in np.arange(0, len(inputs) - batchsize + 1, batchsize):
        excerpt = indices[start_idx:start_idx + batchsize]
        yield load_images(inputs[excerpt], targets[excerpt])

#Model path
model_path = "C:/Users/Paperspace/Desktop/checkpoints/cp.ckpt"
#saver = tf.train.Saver(max_to_keep=3)
cp_callback = tf.keras.callbacks.ModelCheckpoint(model_path, 
                                                 verbose=1,
                                                 save_weights_only=True,
                                                period=2)

tb_callback =TensorBoard(log_dir="./Graph/{}".format(time()))

batch_size = 750
history = model.fit_generator(generator=iterate_minibatches(X_train, Y_train,batch_size),
                                  validation_data=iterate_minibatches(X_test, Y_test, batch_size),
                                  # validation_data=None,
                                  steps_per_epoch=len(X_train)//batch_size,
                                  validation_steps=len(X_test)//batch_size,
                                  verbose=1,
                                  epochs=30,
                                  callbacks=[cp_callback,tb_callback] 
                             )

実際の結果、問題なくトレーニングが停止します。次のエポックに進むと予想される結果。

**Log**

Epoch 1/30
53/53 [==============================] - 919s 17s/step - loss: 1.2445 - acc: 0.0718
426/426 [==============================] - 7058s 17s/step - loss: 1.7877 - acc: 0.0687 - val_loss: 1.2445 - val_acc: 0.0718
Epoch 2/30
WARNING:tensorflow:Your dataset iterator ran out of data.

Epoch 00002: saving model to C:/Users/Paperspace/Desktop/checkpoints/cp.ckpt
WARNING:tensorflow:This model was compiled with a Keras optimizer (<tensorflow.python.keras.optimizers.Adam object at 0x0000023A913DE470>) but is being saved in TensorFlow format with `save_weights`. The model's weights will be saved, but unlike with TensorFlow optimizers in the TensorFlow format the optimizer's state will not be saved.

Consider using a TensorFlow optimizer from `tf.train`.
WARNING:tensorflow:From C:\Users\Paperspace\Anaconda3\lib\site-packages\tensorflow\python\keras\engine\network.py:1436: update_checkpoint_state (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.train.CheckpointManager to manage checkpoints rather than manually editing the Checkpoint proto.
  0/426 [..............................] - ETA: 0s - loss: 0.0000e+00 - acc: 0.0687 - val_loss: 0.0000e+00 - val_acc: 0.0000e+00
4

2 に答える 2