今天将介绍使用小波变换来对多模态医学图像进行融合。
1、基于小波变换的图像融合回顾
小波变换融合算法基本思想:首先对源图像进行小波变换,然后按照一定规则对变换系数进行合并;最后对合并后的系数进行小波逆变换得到融合图像。
1.1、小波分解原理简介
LL:水平低频,垂直低频
LH:水平低频,垂直高频
HL:水平高频,垂直低频
HH:水平高频,垂直高频
其中,L表示低频,H表示高频,下标1、2表示一级或二级分解。在每一分解层上,图像均被分解为LL,LH,HH和HL四个频带,下一层的分解仅对低频分量LL进行分解。这四个子图像中的每一个都是由原图与一个小波基函数的内积后,再经过在x和y方向都进行2倍的间隔采样而生成的,这是正变换,也就是图像的分解;逆变换,也就是图像的重建,是通过图像的增频采样和卷积来实现的。
1.2、融合规则
规则一:系数绝对值较大法
该融合规则适合高频成分比较丰富,亮度、对比度比较高的源图像,否则在融合图像中只保留一幅源图像的特征,其他的特征被覆盖。小波变换的实际作用是对信号解相关,并将信号的全部信息集中到一部分具有大幅值的小波系数中。这些大的小波系数含有的能量远比小系数含有的能量大,从而在信号的重构中,大的系数比小的系数更重要。
规则二:加权平均法
权重系数可调,适用范围广,可消除部分噪声,源图像信息损失较少,但会造成图像对比度的下降,需要增强图像灰度。
2、基于小波变换的多模态医学图像融合代码实现
我将分享python版本代码来融合多模态MR图像,融合策略是低频图像采用平均值法,高频图像采用最大值法。python版本中需要用到PyWavelets库,可以使用下面命令来安装,具体可以见原文链接。
pip install PyWavelets
python版本代码:
import pywt
import numpy as np
import SimpleITK as sitk
# This function does the coefficient fusing according to the fusion method
def fuseCoeff(cooef1, cooef2, method):
if (method == 'mean'):
cooef = (cooef1 + cooef2) / 2
elif (method == 'min'):
cooef = np.minimum(cooef1, cooef2)
elif (method == 'max'):
cooef = np.maximum(cooef1, cooef2)
return cooef
# Params
FUSION_METHOD1 = 'mean' # Can be 'min' || 'max || anything you choose according theory
FUSION_METHOD2 = 'max'
# Read the two image
I2_itk = sitk.ReadImage("Brats18_2013_1_1_flair.nii.gz", sitk.sitkInt16)
I1_itk = sitk.ReadImage("Brats18_2013_1_1_t1ce.nii.gz", sitk.sitkInt16)
I1 = sitk.GetArrayFromImage(I1_itk)
I2 = sitk.GetArrayFromImage(I2_itk)
# First: Do wavelet transform on each image
wavelet = 'db2'
"""
haar family: haar
db family: db1, db2, db3, db4, db5, db6, db7, db8, db9, db10, db11, db12, db13, db14, db15, db16, db17, db18, db19, db20, db21, db22, db23, db24, db25, db26, db27, db28, db29, db30, db31, db32, db33, db34, db35, db36, db37, db38
sym family: sym2, sym3, sym4, sym5, sym6, sym7, sym8, sym9, sym10, sym11, sym12, sym13, sym14, sym15, sym16, sym17, sym18, sym19, sym20
coif family: coif1, coif2, coif3, coif4, coif5, coif6, coif7, coif8, coif9, coif10, coif11, coif12, coif13, coif14, coif15, coif16, coif17
bior family: bior1.1, bior1.3, bior1.5, bior2.2, bior2.4, bior2.6, bior2.8, bior3.1, bior3.3, bior3.5, bior3.7, bior3.9, bior4.4, bior5.5, bior6.8
rbio family: rbio1.1, rbio1.3, rbio1.5, rbio2.2, rbio2.4, rbio2.6, rbio2.8, rbio3.1, rbio3.3, rbio3.5, rbio3.7, rbio3.9, rbio4.4, rbio5.5, rbio6.8
dmey family: dmey
gaus family: gaus1, gaus2, gaus3, gaus4, gaus5, gaus6, gaus7, gaus8
mexh family: mexh
morl family: morl
cgau family: cgau1, cgau2, cgau3, cgau4, cgau5, cgau6, cgau7, cgau8
shan family: shan
fbsp family: fbsp
cmor family: cmor
"""
cooef1 = pywt.wavedecn(I1[:, :], wavelet)
cooef2 = pywt.wavedecn(I2[:, :], wavelet)
# Second: for each level in both image do the fusion according to the desire option
fusedCooef = []
for i in range(len(cooef1)):
# The first values in each decomposition is the apprximation values of the top level
if i == 0:
fusedCooef.append(fuseCoeff(cooef1[0], cooef2[0], FUSION_METHOD1))
else:
c1 = fuseCoeff(cooef1[i]['aad'], cooef2[i]['aad'], FUSION_METHOD2)
c2 = fuseCoeff(cooef1[i]['ada'], cooef2[i]['ada'], FUSION_METHOD2)
c3 = fuseCoeff(cooef1[i]['add'], cooef2[i]['add'], FUSION_METHOD2)
c4 = fuseCoeff(cooef1[i]['daa'], cooef2[i]['daa'], FUSION_METHOD2)
c5 = fuseCoeff(cooef1[i]['dad'], cooef2[i]['dad'], FUSION_METHOD2)
c6 = fuseCoeff(cooef1[i]['dda'], cooef2[i]['dda'], FUSION_METHOD2)
c7 = fuseCoeff(cooef1[i]['ddd'], cooef2[i]['ddd'], FUSION_METHOD2)
dictobj = {'aad': c1, 'ada': c2, 'add': c3, 'daa': c4, 'dad': c5, 'dda': c6, 'ddd': c7}
fusedCooef.append(dictobj)
# Third: After we fused the cooefficent we nned to transfor back to get the image
fusedImage = pywt.waverecn(fusedCooef, wavelet)
fusedImage = fusedImage.astype(np.int)
fused_itk = sitk.GetImageFromArray(fusedImage)
fused_itk.SetOrigin(I2_itk.GetOrigin())
fused_itk.SetSpacing(I2_itk.GetSpacing())
fused_itk.SetDirection(I2_itk.GetDirection())
sitk.WriteImage(fused_itk, 'fused_itk.mha')
3、融合结果
下图是T1增强图像和FLAIR图像。
融合结果
如果碰到任何问题,随时留言,我会尽量去回答的。