python - Slow tensorflow code; can I batch evaluate and obtain multiple loss scores?

Question

I recently switched from using the 'keras' package in Python to using 'tensorflow.keras', since this seems to be preferred now. The latest version of Keras was also giving me issues that seemed like I'd have to modify the internal Keras code to fix, whereas tf.keras works fine. However, upon making this switch, some of my code was slowed down by a factor of 30-40. I've identified the following calls to "model.evaluate" as a bottleneck, though I'm not sure why it's so much slower than before. The code is structured something like this:

# 'model' is a tensorflow.keras.models.Sequential

n_scores = 10000

inputs = np.zeros((n_scores, 8, 10), dtype=np.bool)
outputs = np.zeros((n_scores, 10), dtype=np.bool)

# [populate inputs and outputs]

scores = []
for i in range(n_scores):
    score = model.evaluate(inputs[i,:,:], outputs[i,:])
    scores.append(score)

return scores

I'm thinking the major bottleneck is that I'm making a bunch of SMALL calls to tensorflow, rather than one LARGE call. Using a GPU actually makes it even slower, presumably due to all the loading/unloading.

I'd like to just make a call like

scores = model.evaluate(inputs, outputs)

but model.evaluate() seems to always output a single scalar, when I need the whole list of 10000 loss scores. I have been unable to find a solution in the documentation, is there a builtin way to do sort of a "batch evaluate" but get individual loss scores out for each sample?

标签： pythontensorflowkeras

Answer 1

根据 NONONONONONO 的评论，我的解决方案是

# 'model' is a tensorflow.keras.models.Sequential

n_scores = 10000

inputs = np.zeros((n_scores, 8, 10), dtype=np.bool)
outputs = np.zeros((n_scores, 10), dtype=np.bool)

# [populate inputs and outputs]

predictions = model.predict(inputs)

from tensorflow.keras.losses import CategoricalCrossentropy
cce = CategoricalCrossentropy()

scores = []
for i in range(n_scores):
    score = cce(predictions[i], outputs[i])
    scores.append(score)

return scores