EM算法简介

expectation–maximization (EM) algorithm is an iterative method to find maximum likelihood（MLE） or maximum a posteriori (MAP) estimates of parameters in statistical models, where the model depends on unobserved latent variables.

(1) 起因: 给定一系列样本，求解含有隐变量的极大似然估计(MLE)

(2) 求解: 算法推导

`Qi`表示隐变量`z`的分布，需要满足条件:

，比如要将班上学生聚类，假设隐藏变量z是身高，那么`Qi`就是连续的高斯分布，如果按照隐藏变量是男女，那么就是伯努利分布.

(3) 结论: 算法总结

EM算法的应用

GMM

GMM(Gaussian Mixture Model)就是指对样本的概率密度（density estimation）分布进行估计，而估计采用的模型是多个高斯模型的加权和，其中的每个高斯模型就代表了一个类(Cluster). 实际分布中可以把模型定义为任何分布的mixture model，为何是高斯混合模型呢? 原因如下两点:

• 计算比较方便
• 理论任意多的高斯分布可以近似任意概率分布

Wji表示第i个数据点属于第j个cluster的概率. 具体的Wji的计算可以使用贝叶斯公式:

sklearn中的GMM

API地址在这：GuassianMixture_API 官方的guide是这么介绍的:

The GaussianMixture object implements the expectation-maximization (EM) algorithm for fitting mixture-of-Gaussian models.

```import matplotlib as mpl
import matplotlib.pyplot as plt

import numpy as np

from sklearn import datasets
from sklearn.mixture import GaussianMixture
from sklearn.model_selection import StratifiedKFold

print(__doc__)

colors = ['navy', 'turquoise', 'darkorange']

def make_ellipses(gmm, ax):
for n, color in enumerate(colors):
if gmm.covariance_type == 'full':
covariances = gmm.covariances_[n][:2, :2]
elif gmm.covariance_type == 'tied':
covariances = gmm.covariances_[:2, :2]
elif gmm.covariance_type == 'diag':
covariances = np.diag(gmm.covariances_[n][:2])
elif gmm.covariance_type == 'spherical':
covariances = np.eye(gmm.means_.shape[1]) * gmm.covariances_[n]
v, w = np.linalg.eigh(covariances)
u = w[0] / np.linalg.norm(w[0])
angle = np.arctan2(u[1], u[0])
angle = 180 * angle / np.pi  # convert to degrees
v = 2. * np.sqrt(2.) * np.sqrt(v)
ell = mpl.patches.Ellipse(gmm.means_[n, :2], v[0], v[1],
180 + angle, color=color)
ell.set_clip_box(ax.bbox)
ell.set_alpha(0.5)

# Break up the dataset into non-overlapping training (75%) and testing
# (25%) sets.
skf = StratifiedKFold(n_splits=4)
# Only take the first fold.
train_index, test_index = next(iter(skf.split(iris.data, iris.target)))

X_train = iris.data[train_index]
y_train = iris.target[train_index]
X_test = iris.data[test_index]
y_test = iris.target[test_index]

n_classes = len(np.unique(y_train))

# Try GMMs using different types of covariances.
estimators = dict((cov_type, GaussianMixture(n_components=n_classes,
covariance_type=cov_type, max_iter=20, random_state=0))
for cov_type in ['spherical', 'diag', 'tied', 'full'])

n_estimators = len(estimators)

plt.figure(figsize=(3 * n_estimators // 2, 6))
left=.01, right=.99)

for index, (name, estimator) in enumerate(estimators.items()):
# Since we have class labels for the training data, we can
# initialize the GMM parameters in a supervised manner.
estimator.means_init = np.array([X_train[y_train == i].mean(axis=0)
for i in range(n_classes)])

# Train the other parameters using the EM algorithm.
estimator.fit(X_train)

h = plt.subplot(2, n_estimators // 2, index + 1)
make_ellipses(estimator, h)

for n, color in enumerate(colors):
data = iris.data[iris.target == n]
plt.scatter(data[:, 0], data[:, 1], s=0.8, color=color,
label=iris.target_names[n])
# Plot the test data with crosses
for n, color in enumerate(colors):
data = X_test[y_test == n]
plt.scatter(data[:, 0], data[:, 1], marker='x', color=color)

y_train_pred = estimator.predict(X_train)
train_accuracy = np.mean(y_train_pred.ravel() == y_train.ravel()) * 100
plt.text(0.05, 0.9, 'Train accuracy: %.1f' % train_accuracy,
transform=h.transAxes)

y_test_pred = estimator.predict(X_test)
test_accuracy = np.mean(y_test_pred.ravel() == y_test.ravel()) * 100
plt.text(0.05, 0.8, 'Test accuracy: %.1f' % test_accuracy,
transform=h.transAxes)

plt.xticks(())
plt.yticks(())
plt.title(name)

plt.legend(scatterpoints=1, loc='lower right', prop=dict(size=12))

plt.show()```

EM还有用在DGM(Bayesian network)中的，这些就比较高深了，暂时还没做了解，以后再补.

`参考` 1. EM算法在wiki上的解释 2. Jerry Lead的博客 3. zouxy09的博客 4. 一个EM算法的总结 5. GMM模型 6. sina博客介绍的GMM 7. scikit-learn中的GMM

89 篇文章41 人订阅

0 条评论

相关文章

2948

3986

1032

3498

47014

25510

1154

3516

1406

K-NN算法与K-Means算法的原理与区别（附带源码示例）

KNN算法 K-Means算法 目标  确定某个元素所属的分类 将已存在的一系列元素分类 算法类别 监督的分类算法 无监督的聚类算法 ...

2877