Sharapolas' answer is right.
However, there is a better way than using a layer for building custom loss functions with complex interdependence of several outputs of a model.
The method I know is being used in practice is by never calling model.compile
but only model._make_predict_function()
. From there on, you can go on and build a custom optimizer method by calling model.output
in there. This will give you all outputs, [y2,y3] in your case. When doing your magic with it, get a keras.optimizer
and use it's get_update method using your model.trainable_weights and your loss. Finally, return a keras.function
with a list of the inputs required (in your case only model.input
) and the updates you just got from the optimizer.get_update call. This function now replaces model.fit.
The above is often used in PolicyGradient algorithms, like A3C or PPO. Here is an example of what I tried to explain:
https://github.com/Hyeokreal/Actor-Critic-Continuous-Keras/blob/master/a2c_continuous.py
Look at build_model and critic_optimizer methods and read kreas.backend.function documentation to understand what happens.
I found this way to have frequently problems with the session management and does not appear to work in tf-2.0 keras at all currently. Hence, if anyone knows a method, please let me know. I came here looking for one :)