实战 | 红酒瓶标签曲面展平+文字识别(附源码)

Color Space

发布于 2022-09-26 10:38:21

1.2K0

发布于 2022-09-26 10:38:21

文章被收录于专栏：OpenCV与AI深度学习OpenCV与AI深度学习

导读

本文将详细介绍如何将红酒瓶上的曲面标签展平并做文字识别。（公众号：OpenCV与AI深度学习）

背景介绍本文的目标是让计算机从一张简单的照片中读取一瓶红酒上标签文字的内容。因为酒瓶标签上的文本在圆柱体上是扭曲的，我们无法直接提取并识别字符，所以一般都会将曲面标签展平之后再做识别，以提升准确率。

第一部分：传统方法提取标签以上图为例，先尝试使用传统图像处理方法提取标签轮廓。【1】转为灰度图 + 自适应二值化

【2】高斯滤波平滑 + 固定阈值二值化

【3】轮廓提取排序，查找最大面积轮廓

【4】批量测试，检测算法稳定性

批量测试后发现在其他图片上并不能很好的提取标签轮廓，所以我们需要考虑其他方法来解决。

第二部分：使用深度学习图像分割网络(U-Net)提取标签【1】准备数据集(图像 + mask标签)

【2】训练U-Net网络模型 U-Net网络代码(TensorFlow实现)：


def build_model(self, Config):
        
  inputs = tf.keras.layers.Input((256,256,3))
  s = tf.keras.layers.Lambda(lambda x: x / 255)(inputs)

  #Contraction path
  c1 = tf.keras.layers.Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(s)
  c1 = tf.keras.layers.Dropout(Config['contraction_1_dropout'])(c1)
  c1 = tf.keras.layers.Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c1)
  p1 = tf.keras.layers.MaxPooling2D((2, 2))(c1)

  c2 = tf.keras.layers.Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(p1)
  c2 = tf.keras.layers.Dropout(Config['contraction_2_dropout'])(c2)
  c2 = tf.keras.layers.Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c2)
  p2 = tf.keras.layers.MaxPooling2D((2, 2))(c2)

  c3 = tf.keras.layers.Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(p2)
  c3 = tf.keras.layers.Dropout(Config['contraction_3_dropout'])(c3)
  c3 = tf.keras.layers.Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c3)
  p3 = tf.keras.layers.MaxPooling2D((2, 2))(c3)

  c4 = tf.keras.layers.Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(p3)
  c4 = tf.keras.layers.Dropout(Config['contraction_4_dropout'])(c4)
  c4 = tf.keras.layers.Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c4)
  p4 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(c4)

  c5 = tf.keras.layers.Conv2D(256, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(p4)
  c5 = tf.keras.layers.Dropout(Config['contraction_5_dropout'])(c5)
  c5 = tf.keras.layers.Conv2D(256, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c5)

  #Expansive path 
  u6 = tf.keras.layers.Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same')(c5)
  u6 = tf.keras.layers.concatenate([u6, c4])
  c6 = tf.keras.layers.Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(u6)
  c6 = tf.keras.layers.Dropout(Config['expansive_1_dropout'])(c6)
  c6 = tf.keras.layers.Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c6)

  u7 = tf.keras.layers.Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same')(c6)
  u7 = tf.keras.layers.concatenate([u7, c3])
  c7 = tf.keras.layers.Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(u7)
  c7 = tf.keras.layers.Dropout(Config['expansive_2_dropout'])(c7)
  c7 = tf.keras.layers.Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c7)

  u8 = tf.keras.layers.Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same')(c7)
  u8 = tf.keras.layers.concatenate([u8, c2])
  c8 = tf.keras.layers.Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(u8)
  c8 = tf.keras.layers.Dropout(Config['expansive_3_dropout'])(c8)
  c8 = tf.keras.layers.Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c8)

  u9 = tf.keras.layers.Conv2DTranspose(16, (2, 2), strides=(2, 2), padding='same')(c8)
  u9 = tf.keras.layers.concatenate([u9, c1], axis=3)
  c9 = tf.keras.layers.Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(u9)
  c9 = tf.keras.layers.Dropout(Config['expansive_4_dropout'])(c9)
  c9 = tf.keras.layers.Conv2D(16, (3, 3), activation='relu', kernel_initializer='he_normal', padding='same')(c9)

  outputs = tf.keras.layers.Conv2D(1, (1, 1), activation='sigmoid')(c9)

  model = tf.keras.Model(inputs=[inputs], outputs=[outputs])
  return model

【3】图像推理验证

个别因干扰而分割失败的情况(暂时忽略)：

第三部分：曲面标签展平与文字识别【1】根据分割结果提取6个特征点

调整图像大小、二值化、对齐U-Net预测： # mask is the U-net output image # src is the source image # self is the parent class labelVision mask = cv2.cvtColor(mask,cv2.COLOR_GRAY2RGB) mask=cv2.resize(mask,(src.shape[1],src.shape[0])) mask = np.round(mask) #binary transform r_src, r_mask = self.align_vertically(src, mask)