我有一个关于LSTM模型评估的问题。我已经训练了一个LSTM模型,并将其存储在model.save(...)
中。现在我想要load_model
,并在验证集数据集上对其进行评估。由于神经网络是随机的,所以我多次运行它,并计算我感兴趣的不同指标的均值和方差。现在,我感到震惊的是,在第一次运行后,所有连续运行的每个指标都具有相同的性能。我不认为这是正确的,但我不知道错误发生在哪里。所以我的问题是:在设置我的模型验证时,我的错误是什么?我该如何解决这个问题呢?
下面是可以解释我正在做的事情的代码片段:
编译并拟合模型
def compile_and_fit( hparams,
MAX_EPOCHS,
model_path ):
window = WindowGenerator( input_width= hparams[HP_WINDOW_SIZE],
label_width=hparams[HP_WINDOW_SIZE], shift=1,
label_columns=['q_MARI'], batch_size = hparams[HP_BATCH_SIZE])
model = tf.keras.models.Sequential([
tf.keras.layers.LSTM(hparams[HP_NUM_UNITS], return_sequences=True, name="LSTM_1"),
tf.keras.layers.Dropout(hparams[HP_DROPOUT], name="Dropout_1"),
tf.keras.layers.LSTM(hparams[HP_NUM_UNITS], return_sequences=True, name="LSTM_2"),
tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(1))
])
learning_rate = hparams[HP_LEARNING_RATE]
model.compile(loss=tf.losses.MeanSquaredError(),
optimizer=tf.optimizers.Adam(learning_rate=learning_rate),
metrics=get_metrics())
history = model.fit(window.train,
epochs=MAX_EPOCHS,
validation_data=window.val,
callbacks= get_callbacks(model_path))
_, a,_,_,_,_ = model.evaluate(window.val)
return a, model, history
训练和保护它
a, model, history = compile_and_fit( hparams = hparams, MAX_EPOCHS = MAX_EPOCHS, model_path = run_path)
model.save(run_path)
加载并评估它
model = tf.keras.models.load_model(os.path.join(hparam_path, model_name),
custom_objects={"max_error": max_error, "median_absolute_error": median_absolute_error, "rev_metric": rev_metric, "nse_metric": nse_metric})
model.compile(loss=tf.losses.MeanSquaredError(), optimizer="adam", metrics=get_metrics())
metric_values = np.empty(shape = (nr_runs, len(metrics)), dtype=float)
for j in range(nr_runs):
window = WindowGenerator(input_width= hparam_vals[i], label_width=hparam_vals[i], shift=1,
label_columns=['q_MARI'])
metric_values[j]= np.array(model.evaluate(window.val))
means = metric_values.mean(axis=0)
varis = metric_values.var(axis=0)
print(f'means: {means}, varis: {varis}')
我得到的结果
为了设置培训,我遵循这两个指南:https://www.tensorflow.org/tutorials/structured_data/time_series https://www.tensorflow.org/tensorboard/hyperparameter_tuning_with_hparams
发布于 2021-01-28 20:48:56
LSTM不是随机的。对于相同的数据,评估结果应该是相同的。
https://stackoverflow.com/questions/65937266
复制相似问题