我试图推断一个tensorflow模型,我注意到'invoke‘方法似乎花费了0时间,这应该是不可能的。下面有相应的代码和我看到的输出。我不知道为什么1/10左右显示一个合理的推断时间,而其他9/10显示0。
注意:我尝试过使用100个不同的输入张量,对于一个测试文件(下面),我只是创建了一个700x100“映像”数据输入。
import os
import time
import cv2
import numpy as np
import tensorflow as tf
def load_model(model_path):
return (tf.lite.Interpreter(model_path=model_path))
def init_tflite(interpreter):
interpreter.allocate_tensors()
def predict_tflite(interpreter, x_test_):
input_details = interpreter.get_input_details()[0]
output_details = interpreter.get_output_details()[0]
# If required, quantize the input layer (from float to integer)
input_scale, input_zero_point = input_details["quantization"]
if (input_scale, input_zero_point) != (0.0, 0):
x_test_ = x_test_ / input_scale + input_zero_point
x_test_ = x_test_.astype(input_details["dtype"])
# Invoke the interpreter
interpreter.set_tensor(input_details["index"], [x_test_[0]])
t1 = time.time_ns()
interpreter.invoke()
t2 = time.time_ns()
y_pred = interpreter.get_tensor(output_details["index"])[0]
# If required, dequantized the output layer (from integer to float)
output_scale, output_zero_point = output_details["quantization"]
if (output_scale, output_zero_point) != (0.0, 0):
y_pred = y_pred.astype(np.float32)
y_pred = (y_pred - output_zero_point) * output_scale
return y_pred, t2-t1
def inference_model(frame, model):
nframe = cv2.resize(frame, (32,32))
nframe = np.expand_dims(nframe, axis=2)
image = np.expand_dims(nframe, axis=0)
image = np.array(image, dtype=np.float32)
pred, diff = predict_tflite(model, image)
print("Inference Time {a} ns".format(a=diff))
return np.argmax(pred) == 0
im = np.ones((700,100))
im = np.array(im, dtype=np.float32)
model = load_model('models/tflite_model_f16_no_opt.tflite')
init_tflite(model)
for i in range(0, 10):
pred = inference_model(im, model)
下面是10次循环迭代的输出:
PS C:\repos\image-based-cell-sorting\ibcs\cellsorter> python .\tflite_test.py
2022-09-28 14:30:34.605393: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2022-09-28 14:30:34.605835: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Inference Time 0 ns
Inference Time 0 ns
Inference Time 0 ns
Inference Time 0 ns
Inference Time 0 ns
Inference Time 0 ns
Inference Time 0 ns
Inference Time 447900 ns
Inference Time 0 ns
Inference Time 0 ns
发布于 2022-09-28 19:40:45
我想出了问题,这和TF无关。问题是时间库的更新速度不够快,无法满足我打电话给它的频率。我以为我使用的是_ns,因为它会提供ns的精度,但似乎不能以这种方式使用这个库。
我可以通过循环前后的时间除以n来获得平均时间,但是,我之所以花时间在中间,是因为我想知道实时的推断时间,因为在我的应用程序中正在抓取图片。我得另找个办法。
https://stackoverflow.com/questions/73886335
复制相似问题