blocks|key|4382079|text|‘'roc_curve'，'auc’不是标准指标，你不能像那样将它们传递给metrics变量，这是不允许的。你可以传递一些像'fmeasure‘这样的东西，这是一个标准度量。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4382080|在这里查看可用的指标：https://keras.io/metrics/您可能还想看看如何制定自己的自定义指标：https://keras.io/metrics/#custom-metrics|offset|length|4382081|也可以看看这篇博文中提到的针对ROC的generate_results方法。https://vkolachalama.blogspot.in/2016/05/keras-implementation-of-mlp-neural.html|4382082|entityMap|0|LINK|mutability|MUTABLE|url|https://keras.io/metrics/|1|https://keras.io/metrics/#custom-metrics|2|https://vkolachalama.blogspot.in/2016/05/keras-implementation-of-mlp-neural.html^0|0|B|P|0|1K|14|1|0|12|28|2|0^^$0|@$1|2|3|4|5|6|7|T|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|U|8|@]|9|@$D|V|E|W|1|X]|$D|Y|E|Z|1|10]]|A|$]]|$1|F|3|G|5|6|7|11|8|@]|9|@$D|12|E|13|1|14]]|A|$]]|$1|H|3|-4|5|6|7|15|8|@]|9|@]|A|$]]]|I|$J|$5|K|L|M|A|$N|O]]|P|$5|K|L|M|A|$N|Q]]|R|$5|K|L|M|A|$N|S]]]]

'roc_curve','auc' are not standard metrics you can't pass them like that to metrics variable, this is not allowed. 
You can pass something like 'fmeasure' which is a standard metric.

Review the available metrics here: <a href="https://keras.io/metrics/" rel="noreferrer">https://keras.io/metrics/</a>
You may also want to have a look at making your own custom metric: <a href="https://keras.io/metrics/#custom-metrics" rel="noreferrer">https://keras.io/metrics/#custom-metrics</a>

Also have a look at generate_results method mentioned in this blog for ROC, AUC...
<a href="https://vkolachalama.blogspot.in/2016/05/keras-implementation-of-mlp-neural.html" rel="noreferrer">https://vkolachalama.blogspot.in/2016/05/keras-implementation-of-mlp-neural.html</a>

blocks|key|4382516|text|下面的解决方案对我很有效：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4382517|import+tensorflow+as+tf
from+keras+import+backend+as+K

def+auc(y_true,+y_pred):
++++auc+=+tf.metrics.auc(y_true,+y_pred)[1]
++++K.get_session().run(tf.local_variables_initializer())
++++return+auc

model.compile(loss="binary_crossentropy",+optimizer='adam',+metrics=[auc])|code-block|syntax|javascript|4382518|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

The following solution worked for me:

<pre><code>import tensorflow as tf
from keras import backend as K

def auc(y_true, y_pred):
 auc = tf.metrics.auc(y_true, y_pred)[1]
 K.get_session().run(tf.local_variables_initializer())
 return auc

model.compile(loss="binary_crossentropy", optimizer='adam', metrics=[auc])
</code></pre>

blocks|key|4382562|text|和你一样，我更喜欢使用scikit-learn的内置方法来评估AUROC。我发现在keras中实现这一点的最好、最简单的方法是创建一个自定义指标。如果tensorflow是您的后端，则只需几行代码即可实现：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4382563|import+tensorflow+as+tf
from+sklearn.metrics+import+roc_auc_score

def+auroc(y_true,+y_pred):
++++return+tf.py_func(roc_auc_score,+(y_true,+y_pred),+tf.double)

#+Build+Model...

model.compile(loss='categorical_crossentropy',+optimizer='adam',metrics=['accuracy',+auroc])|code-block|syntax|javascript|4382564|创建其他答案中提到的自定义回调不会对您的情况起作用，因为您的模型有多个输出，但这会起作用。此外，这种方法允许在训练和验证数据上评估指标，而keras回调不能访问训练数据，因此只能用于评估训练数据的性能。|4382565|entityMap^0|0|0|0^^$0|@$1|2|3|4|5|6|7|K|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|L|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|M|8|@]|9|@]|A|$]]|$1|I|3|-4|5|6|7|N|8|@]|9|@]|A|$]]]|J|$]]

Like you, I prefer using scikit-learn's built in methods to evaluate AUROC. I find that the best and easiest way to do this in keras is to create a custom metric. If tensorflow is your backend, implementing this can be done in very few lines of code:

<pre><code>import tensorflow as tf
from sklearn.metrics import roc_auc_score

def auroc(y_true, y_pred):
 return tf.py_func(roc_auc_score, (y_true, y_pred), tf.double)

# Build Model...

model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=['accuracy', auroc])
</code></pre>

Creating a custom Callback as mentioned in other answers will not work for your case since your model has multiple ouputs, but this will work. Additionally, this methods allows the metric to be evaluated on both training and validation data whereas a keras callback does not have access to the training data and can thus only be used to evaluate performance on the training data.

blocks|key|4382598|text|加上上面的答案，我得到了错误"ValueError:+bad+input+shape+..."，所以我指定了如下概率向量：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4382599|y_pred+=+model.predict_proba(x_test)[:,1]
auc+=+roc_auc_score(y_test,+y_pred)
print(auc)|code-block|syntax|javascript|4382600|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|I|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|J|8|@]|9|@]|A|$E|F]]|$1|G|3|-4|5|6|7|K|8|@]|9|@]|A|$]]]|H|$]]

Adding to above answers, I got the error "ValueError: bad input shape ...", so I specify the vector of probabilities as follows: 

<pre><code>y_pred = model.predict_proba(x_test)[:,1]
auc = roc_auc_score(y_test, y_pred)
print(auc)
</code></pre>

blocks|key|4382627|text|您可以通过以下方式提供指标来监控训练期间的auc：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|4382628|METRICS+=+[
++++++keras.metrics.TruePositives(name='tp'),
++++++keras.metrics.FalsePositives(name='fp'),
++++++keras.metrics.TrueNegatives(name='tn'),
++++++keras.metrics.FalseNegatives(name='fn'),+
++++++keras.metrics.BinaryAccuracy(name='accuracy'),
++++++keras.metrics.Precision(name='precision'),
++++++keras.metrics.Recall(name='recall'),
++++++keras.metrics.AUC(name='auc'),
]


model+=+keras.Sequential([
++++keras.layers.Dense(16,+activation='relu',+input_shape=(train_features.shape[-1],)),
++++keras.layers.Dense(1,+activation='sigmoid'),
++])

model.compile(
++++optimizer=keras.optimizers.Adam(lr=1e-3)
++++loss=keras.losses.BinaryCrossentropy(),
++++metrics=METRICS)|code-block|syntax|javascript|4382629|有关更详细的教程，请参阅：|4382630|https://www.tensorflow.org/tutorials/structured_data/imbalanced_data|offset|length|4382631|entityMap|0|LINK|mutability|MUTABLE|url^0|0|0|0|0|1W|0|0^^$0|@$1|2|3|4|5|6|7|T|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|U|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|V|8|@]|9|@]|A|$]]|$1|I|3|J|5|6|7|W|8|@]|9|@$K|X|L|Y|1|Z]]|A|$]]|$1|M|3|-4|5|6|7|10|8|@]|9|@]|A|$]]]|N|$O|$5|P|Q|R|A|$S|J]]]]

You can monitor auc during training by providing metrics the following way:

<pre><code>METRICS = [
 keras.metrics.TruePositives(name='tp'),
 keras.metrics.FalsePositives(name='fp'),
 keras.metrics.TrueNegatives(name='tn'),
 keras.metrics.FalseNegatives(name='fn'), 
 keras.metrics.BinaryAccuracy(name='accuracy'),
 keras.metrics.Precision(name='precision'),
 keras.metrics.Recall(name='recall'),
 keras.metrics.AUC(name='auc'),
]


model = keras.Sequential([
 keras.layers.Dense(16, activation='relu', input_shape=(train_features.shape[-1],)),
 keras.layers.Dense(1, activation='sigmoid'),
 ])

model.compile(
 optimizer=keras.optimizers.Adam(lr=1e-3)
 loss=keras.losses.BinaryCrossentropy(),
 metrics=METRICS)

</code></pre>

for a more detailed tutorial see: 
 <a href="https://www.tensorflow.org/tutorials/structured_data/imbalanced_data" rel="noreferrer">https://www.tensorflow.org/tutorials/structured_data/imbalanced_data</a>

blocks|key|4382663|text|使用tf.keras.metrics.AUC()设置模型架构:阅读Classification+metrics+based+on+True/False+positives+&+negatives上的Keras文档。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|4382664|def+model_architecture_ann(in_dim,lr=0.0001):
++++model+=+Sequential()
++++model.add(Dense(512,+input_dim=X_train_filtered.shape[1],+activation='relu'))
++++model.add(Dense(1,+activation='sigmoid'))
++++opt+=+keras.optimizers.SGD(learning_rate=0.001)
++++auc=tf.keras.metrics.AUC()
++++model.compile(loss='binary_crossentropy',+optimizer=opt,+metrics=[tf.keras.metrics.AUC(name='auc')])++++
++++model.summary()
++++return+model|code-block|syntax|javascript|4382665|entityMap|0|LINK|mutability|MUTABLE|url|https://keras.io/api/metrics/classification_metrics/#auc-class^0|X|1S|0|0|0^^$0|@$1|2|3|4|5|6|7|Q|8|@]|9|@$A|R|B|S|1|T]]|C|$]]|$1|D|3|E|5|F|7|U|8|@]|9|@]|C|$G|H]]|$1|I|3|-4|5|6|7|V|8|@]|9|@]|C|$]]]|J|$K|$5|L|M|N|C|$O|P]]]]

Set your model architecture with tf.keras.metrics.AUC():
Read the Keras documentation on <a href="https://keras.io/api/metrics/classification_metrics/#auc-class" rel="nofollow noreferrer">Classification metrics based on True/False positives &amp; negatives</a>.
<pre><code>def model_architecture_ann(in_dim,lr=0.0001):
 model = Sequential()
 model.add(Dense(512, input_dim=X_train_filtered.shape[1], activation='relu'))
 model.add(Dense(1, activation='sigmoid'))
 opt = keras.optimizers.SGD(learning_rate=0.001)
 auc=tf.keras.metrics.AUC()
 model.compile(loss='binary_crossentropy', optimizer=opt, metrics=[tf.keras.metrics.AUC(name='auc')]) 
 model.summary()
 return model
</code></pre>

I have a multi output(200) binary classification model which I wrote in keras.
In this model I want to add additional metrics such as ROC and AUC but to my knowledge keras dosen't have in-built ROC and AUC metric functions.
I tried to import ROC, AUC functions from scikit-learn
<pre><code>from sklearn.metrics import roc_curve, auc
from keras.models import Sequential
from keras.layers import Dense
.
.
.
model.add(Dense(200, activation='relu'))
model.add(Dense(300, activation='relu'))
model.add(Dense(400, activation='relu'))
model.add(Dense(300, activation='relu'))
model.add(Dense(200,init='normal', activation='softmax')) #outputlayer

model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=['accuracy','roc_curve','auc'])
</code></pre>
but it's giving this error:
<pre><code>Exception: Invalid metric: roc_curve
</code></pre>
How should I add ROC, AUC to keras?

How to compute Receiving Operating Characteristic (ROC) and AUC in keras?

我有一个多输出(200)二进制分类模型，它是我用keras编写的。在这个模型中，我想添加额外的指标，比如ROC和AUC，但在我的知识内核中没有内置的ROC和AUC指标函数。我试着从scikit-learn导入ROC，AUC函数from sklearn.metrics import roc_curve, aucfrom ...

问如何在keras中计算接收工作特性(ROC)和AUC？
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何在keras中计算接收工作特性(ROC)和AUC？EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何在keras中计算接收工作特性(ROC)和AUC？
EN