A possible bug with tf 2.17.0

[tanwang2020]

Here are two versions of essentially the same code. The only difference is the lengths of the names of the keys. However, the first code generates error message while the second code does not. Here are the codes.
The first version:
import numpy as np
import tensorflow as tf
from tensorflow.keras.constraints import non_neg
from tensorflow.keras.layers import Input, Dense, concatenate
from tensorflow.keras.models import Model
print('TensorFlow Version: ',tf.version)

Example input data

X_train_dict = {
'green_fin_const': np.random.rand(558, 3),
'green_fin_inst': np.random.rand(558, 4),
'gov_sup': np.random.rand(558, 5),
'com_act': np.random.rand(558, 5),
'eco_city': np.random.rand(558, 4),
'type': np.random.rand(558, 1)
}
Y_train = np.random.rand(558, 2)

X_test_dict = {
'green_fin_const': np.random.rand(140, 3),
'green_fin_inst': np.random.rand(140, 4),
'gov_sup': np.random.rand(140, 5),
'com_act': np.random.rand(140, 5),
'eco_city': np.random.rand(140, 4),
'type': np.random.rand(140, 1)
}
Y_test = np.random.rand(140, 2)

Define input layers

inputs_1 = Input(shape=(3,), name='green_fin_const')
inputs_2 = Input(shape=(4,), name='green_fin_inst')
inputs_3 = Input(shape=(5,), name='gov_sup')
inputs_4 = Input(shape=(5,), name='com_act')
inputs_5 = Input(shape=(4,), name='eco_city')
inputs_6 = Input(shape=(1,), name='type')

Define dense layers

score_1 = Dense(1, activation='sigmoid', kernel_constraint=non_neg())(inputs_1)
score_2 = Dense(1, activation='sigmoid', kernel_constraint=non_neg())(inputs_2)
score_3 = Dense(1, activation='sigmoid', kernel_constraint=non_neg())(inputs_3)
score_4 = Dense(1, activation='sigmoid', kernel_constraint=non_neg())(inputs_4)
score_5 = Dense(1, activation='sigmoid', kernel_constraint=non_neg())(inputs_5)

Concatenate scores and type input

concatenated_scores = concatenate([score_1, score_2, score_3, score_4, score_5, inputs_6])

Define output layer

outputs = Dense(2, activation='softmax', kernel_constraint=non_neg())(concatenated_scores)

Create the model

model = Model(inputs=[inputs_1, inputs_2, inputs_3, inputs_4, inputs_5, inputs_6], outputs=outputs)

Compile model

model.compile(optimizer='nadam', loss='mse', metrics=['KLDivergence'])

Fit the model

model.fit(
x=X_train_dict,
y=Y_train,
validation_data=(X_test_dict, Y_test),
epochs=100,
batch_size=32,
verbose=0
)

Print input shapes to verify

for key, value in X_train_dict.items():
print(f'{key}: {value.shape}')
for key, value in X_test_dict.items():
print(f'{key}: {value.shape}')

model.summary()

The message from the system is:
TensorFlow Version: 2.17.0

ValueError Traceback (most recent call last)
Cell In[6], line 57
54 model.compile(optimizer='nadam', loss='mse', metrics=['KLDivergence'])
56 # Fit the model
---> 57 model.fit(
58 x=X_train_dict,
59 y=Y_train,
60 validation_data=(X_test_dict, Y_test),
61 epochs=100,
62 batch_size=32,
63 verbose=0
64 )
65 # Print input shapes to verify
66 for key, value in X_train_dict.items():

File ~\anaconda3\Lib\site-packages\keras\src\utils\traceback_utils.py:122, in filter_traceback..error_handler(*args, **kwargs)
119 filtered_tb = _process_traceback_frames(e.traceback)
120 # To get the full stack trace, call:
121 # keras.config.disable_traceback_filtering()
--> 122 raise e.with_traceback(filtered_tb) from None
123 finally:
124 del filtered_tb

File ~\anaconda3\Lib\site-packages\keras\src\layers\input_spec.py:227, in assert_input_compatibility(input_spec, inputs, layer_name)
222 for axis, value in spec.axes.items():
223 if value is not None and shape[axis] not in {
224 value,
225 None,
226 }:
--> 227 raise ValueError(
228 f'Input {input_index} of layer "{layer_name}" is '
229 f"incompatible with the layer: expected axis {axis} "
230 f"of input shape to have value {value}, "
231 "but received input with "
232 f"shape {shape}"
233 )
234 # Check shape.
235 if spec.shape is not None:

ValueError: Exception encountered when calling Functional.call().

Input 0 of layer "dense_27" is incompatible with the layer: expected axis -1 of input shape to have value 3, but received input with shape (None, 5)

Arguments received by Functional.call():
• inputs={'green_fin_const': 'tf.Tensor(shape=(None, 3), dtype=float32)', 'green_fin_inst': 'tf.Tensor(shape=(None, 4), dtype=float32)', 'gov_sup': 'tf.Tensor(shape=(None, 5), dtype=float32)', 'com_act': 'tf.Tensor(shape=(None, 5), dtype=float32)', 'eco_city': 'tf.Tensor(shape=(None, 4), dtype=float32)', 'type': 'tf.Tensor(shape=(None, 1), dtype=float32)'}
• training=True
• mask={'green_fin_const': 'None', 'green_fin_inst': 'None', 'gov_sup': 'None', 'com_act': 'None', 'eco_city': 'None', 'type': 'None'}

The second version is:
import numpy as np
import tensorflow as tf
from tensorflow.keras.constraints import non_neg
from tensorflow.keras.layers import Input, Dense, concatenate
from tensorflow.keras.models import Model
print('TensorFlow Version: ',tf.version)

Example input data

X_train_dict = {
'A': np.random.rand(558, 3),
'B': np.random.rand(558, 4),
'C': np.random.rand(558, 5),
'D': np.random.rand(558, 5),
'E': np.random.rand(558, 4),
'T': np.random.rand(558, 1)
}
Y_train = np.random.rand(558, 2)

X_test_dict = {
'A': np.random.rand(140, 3),
'B': np.random.rand(140, 4),
'C': np.random.rand(140, 5),
'D': np.random.rand(140, 5),
'E': np.random.rand(140, 4),
'T': np.random.rand(140, 1)
}
Y_test = np.random.rand(140, 2)

Define input layers

inputs_1 = Input(shape=(3,), name='A')
inputs_2 = Input(shape=(4,), name='B')
inputs_3 = Input(shape=(5,), name='C')
inputs_4 = Input(shape=(5,), name='D')
inputs_5 = Input(shape=(4,), name='E')
inputs_6 = Input(shape=(1,), name='T')

Define dense layers

score_1 = Dense(1, activation='sigmoid', kernel_constraint=non_neg())(inputs_1)
score_2 = Dense(1, activation='sigmoid', kernel_constraint=non_neg())(inputs_2)
score_3 = Dense(1, activation='sigmoid', kernel_constraint=non_neg())(inputs_3)
score_4 = Dense(1, activation='sigmoid', kernel_constraint=non_neg())(inputs_4)
score_5 = Dense(1, activation='sigmoid', kernel_constraint=non_neg())(inputs_5)

Concatenate scores and type input

concatenated_scores = concatenate([score_1, score_2, score_3, score_4, score_5, inputs_6])

Define output layer

outputs = Dense(2, activation='softmax', kernel_constraint=non_neg())(concatenated_scores)

Create the model

model = Model(inputs=[inputs_1, inputs_2, inputs_3, inputs_4, inputs_5, inputs_6], outputs=outputs)

Compile model

model.compile(optimizer='nadam', loss='mse', metrics=['KLDivergence'])

Fit the model

model.fit(
x=X_train_dict,
y=Y_train,
validation_data=(X_test_dict, Y_test),
epochs=100,
batch_size=32,
verbose=0
)

Print input shapes to verify

for key, value in X_train_dict.items():
print(f'{key}: {value.shape}')
for key, value in X_test_dict.items():
print(f'{key}: {value.shape}')

model.summary()

There is no error message from the system.

[Grvzard]

related issue #20036

[ghsanti]

What @tanwang2020 is seeing it's explainable with:

1. Keras expects a dictionary, not a list as inputs (it should actually test for dict/dict or list/list), so it is a bug in the sense that both should error.

Code works if you use this instead (click to open)

# define inputs dict, same as you will call the model in `.fit`
 inputs = {
 'green_fin_const': inputs_1,
 'green_fin_inst': inputs_2,
 'gov_sup': inputs_3,
 'com_act': inputs_4,
 'eco_city': inputs_5,
 'type': inputs_6
 }

# not the next line
# model = Model(inputs=[inputs_1, inputs_2, inputs_3, inputs_4, inputs_5, inputs_6], outputs=outputs)

# do this instead
model = Model(inputs=inputs, outputs=outputs)

Imports should be from `keras` (not `tf.keras` anymore)

import os
os.environ['KERAS_BACKEND'] = 'tensorflow'

import numpy as np

import tensorflow as tf

import keras
from keras.constraints import non_neg
from keras.layers import Input, Dense, concatenate
from keras.models import Model

print('TensorFlow Version: ',tf.__version__)
print('Keras Version', keras.__version__)

2. The second part works because dictionaries are iterable objects as well, and whereas in the first case your first key is com... which has length 5 and mismatches the original input with length3, the dictionary with 1 letter keys has the same order than one would have on a list.

In other words, the error you are seeing:

ValueError: Input 0 of layer "dense" is incompatible with the layer: expected axis -1 of input shape to have value 3, but received input with shape (None, 5)

Is probably using because this key is sorted first on iteration:

'com_act': np.random.rand(558, 5),

So the fact that the second approach works (explained above) is just coincidence, and should actually error.

---

I would still consider this a bug, but not for the reasons considered originally.

It seems this has not been fixed by other issues since !pip install git+https://github.com/keras-team/keras@master didn't fix the problem, but i could be wrong.

Here there is a gist in case others want to reuse it.

CC @Grvzard

[tanwang2020]

Thanks, ghsanti. Your suggestion worked.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No