珊瑚AI TPU的SSD MobileNet V2 FPNLite 320x320转换问题
珊瑚AI TPU的SSD MobileNet V2 FPNLite 320x320转换问题
提问于 2021-06-20 11:29:15
我试着在Coral TPU的Rasperry PI上运行Tensorflow模型lite。模型为SSDMobileNET-2，在PC机上完全量化或浮点I/O转换后工作良好，但在Coral上运行时却得到了很多错误的结果。通常是假阳性类0(映射到person)。有人能帮我吗，我想不出怎么解决这个问题？
Tensorflow版本: 2.5.0
Tensorflow Lite版本: 2.5.0
我采取了以下步骤：
下载模型：http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.tar.gz
- I将输入调整为320x320，但结果与原来的300x300相同。
- i将保存的模型转换为tf友好格式：
。
python3 object_detection/export_tflite_graph_tf2.py --pipeline_config_path /home/pawel/proj/net_models/ssd_mobilenet_v2_320x320_coco17_tpu-8-init/pipeline.config --trained_checkpoint_dir /home/pawel/proj/net_models/ssd_mobilenet_v2_320x320_coco17_tpu-8-init/checkpoint --output_directory /home/pawel/proj/net_models/ssd_mobilenet_v2_320x320_coco17_tpu-8-fixed-input- 模型转换为TF格式，model_path指向上一步输出，我尝试量化真假，并注释了下面的代码：
converter = tf.lite.TFLiteConverter.from_saved_model(model_path)
if quantize:
# converter.optimizations = [tf.lite.Optimize.DEFAULT]
# converter.representative_dataset = representative_data_gen
# converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
# converter.inference_input_type = tf.uint8
# converter.inference_output_type = tf.uint8
converter.representative_dataset = representative_data_gen
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8, tf.lite.OpsSet.SELECT_TF_OPS]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
converter.allow_custom_ops = True
print(converter.experimental_new_quantizer) # outputs True
print(converter.experimental_new_converter) # outputs True
else:
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
converter.optimizations = []
tflite_model = converter.convert()
with open(lite_model_path, 'wb') as f:
f.write(tflite_model)数据提供程序，使用步骤5中的代码：
def representative_data_gen():
from cocodataset import CocoDataSet
coco = CocoDataSet(input_size, input_size)
images = coco.get_calibration_dataset(500)
for img in images:
yield [img]- Repressentative数据集-Coco2017估值数据-500个样本。
class CocoDataSet:
...
def get_calibration_dataset(self, limit: int):
with open(self.annotation_file, 'r') as f:
annotations = json.load(f)
image_info = annotations['images']
random.shuffle(image_info)
image_info = image_info[:limit]
image_paths = []
for img in image_info:
image_path = self.image_dir + img['file_name']
image_paths.append(image_path)
print(f"{limit} images will be returned")
images = []
fl = True
for i, path in enumerate(image_paths):
print(f"Loading {i}/{len(image_paths)}:" + path)
image = cv.imread(path)
image = cv.cvtColor(image, cv.COLOR_BGR2RGB)
tensor = np.zeros((self.input_height, self.input_width, 3), dtype=np.uint8)
_, _, channel = tensor.shape
h, w, _ = image.shape
scale = min(self.input_width / w, self.input_height / h)
w, h = int(w * scale), int(h * scale)
image = cv.resize(image.copy(), (w, h), interpolation=cv.INTER_LINEAR)
reshaped = image
margin_x = (self.input_width - w) // 2
margin_y = (self.input_height - h) // 2
tensor[margin_y:h + margin_y, margin_x:w + margin_x] = reshaped
tensor = np.expand_dims(tensor, axis=0)
tensor = tensor.astype(np.float32) - 127.5
tensor = tensor * 0.007843
images.append(tensor)
return imagescompilation:
- Coral AI
edgetpu_compiler ssd_mobilenet_v2_coral.tflite基于Coral的
- 推理。它与Coral .
中提供的移动网运行良好。
x, y, scale = self.set_image_input(self.interpreter, region)
self.interpreter.invoke()
detection_boxes = self.get_output_tensor(self.interpreter, 0)
detection_classes = self.get_output_tensor(self.interpreter, 1, np.int)
detection_scores = self.get_output_tensor(self.interpreter, 2)
count = self.get_output_tensor(self.interpreter, 3, np.int)8.按比例和居中的输入图像：
def set_image_input(self, interpreter: tflite.Interpreter, image: np.ndarray) -> (int, int, float):
self.did = self.did + 1
width, height = (self.input_height, self.input_width)
stretch = False
if stretch:
h, w, _ = (self.input_height, self.input_width, 1)
else:
h, w, _ = image.shape
cv.imwrite(f"{self.logs_dir}/image{self.did}.png", image)
scale = min(width / w, height / h)
w, h = int(w * scale), int(h * scale)
tensor = self.input_tensor(interpreter)
tensor.fill(0)
_, _, channel = tensor.shape
image = cv.resize(image.copy(), (w, h), interpolation=cv.INTER_LINEAR)
reshaped = image
if tensor.dtype == np.float32:
reshaped = reshaped * (1.0/255) - 1
margin_x = (self.input_width - w) // 2
margin_y = (self.input_height - h) // 2
tensor[margin_y:h + margin_y, margin_x:w + margin_x] = reshaped
return margin_x, margin_y, scale- 获取输出张量：
def get_output_tensor(self, interpreter: tflite.Interpreter, index: int, result_type=np.float):
output_details = interpreter.get_output_details()[index]
quantization = output_details['quantization']
dtype = output_details['dtype']
tf_index = output_details['index']
tensor = np.squeeze(interpreter.get_tensor(tf_index))
if quantization != (0, 0):
input_scale, input_zero_point = quantization
tensor = (tensor.astype(np.float32) - input_zero_point) * input_scale
if tensor.dtype != result_type:
tensor = tensor.astype(result_type)
return tensor我注意到，当我运行几次转换时，结果略有不同--代表性的数据是从集合中随机获得的。在我在PC上运行的Coral编译模型上，这种差异更加明显。
回答 2
Stack Overflow用户
发布于 2021-06-21 01:35:24
精度量化的SSD模型比浮子模型稍低，您可能可以尝试有效的-det模型：https://www.tensorflow.org/lite/api_docs/python/tflite_model_maker/object_detector。
Stack Overflow用户
发布于 2021-08-11 18:00:06
最新更新的Coral编译器和RaspberryPI运行时解决了这个问题。
更新后的RPI运行时: 2.5.0.post1
更新后的Edge CPU编译器: 16.0.384591198
一些示例检测(等级、分数)：
珊瑚：
[(0, 0), (1, 0.7210485935211182), (0, 0), (1, 0.6919153332710266), (1, 0.7428985834121704), (1, 0.8485066890716553), (24, 0.6919153332710266)]TF Lite - CPU：
[(0, 0), (1, 0.7210485935211182), (0, 0), (1, 0.6919153332710266), (1, 0.7356152534484863), (1, 0.8412233591079712), (24, 0.7028403282165527)]TF CPU：
[(0, 0), (1, 0.75359964), (0, 0), (1, 0.7409908), (1, 0.7797077), (1, 0.8114069), (24, 0.750371)]页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/68055368
复制相关文章
相似问题
腾讯云开发者
