# 深入机器学习系列14-FactorAnalysis

1. Introduction

An extension ofprincipal component analysis(PCA)in the sense of approximating covariance matrix.

Goal

To describe the covariance relationships among many variables in terms of a few underlying unobservable random variables, called factors.

To reduce dimensions and solve the problem with n

2. Orthogonal Factor Model（正交因子模型）

A Factor Analysis Example

We have a training data. Here is its scatter plot.

Generate a k dimension variable

There exists a transformation matrixwhich maps F into n dimension space:

For real instance has errors, add error

Factor Analysis Model

Suppose

The factor model postulates thatis linearly related to a few unobservable random variables, calledcommon factors（共同因子）, through

Assume:

If, it becomes oblique factor model（斜交因子模型）

Define thecommunity（变量共同度，或公因子方差）:

Define thespecific variance（特殊因子方差）:

Ambiguity of L

Let T be any m × m orthogonal matrix. Then, we can express

where,

After rotation, communitydoesn’t change.

3. Estimation

3.1 Principal Component Method

1) Get correlation matrix

2) Spectral Decompositions

3) Determine

Rule of thumb: choose

4) Estimation

The contribution to the total sample variance tr(S) from the first common factor is then（公共因子的方差贡献）

In general, the proportion of total sample variance(after standardization) due to thefactor=

3.2 Maximum Likelihood Method

1) Joint distribution:

2) Marginal distribution:

3) Conditional distribution:

4) Log likelihood:

EM estimation

E Step:

M Step:

Parameter Iteration:

Get more detail on【机器学习-斯坦福】因子分析（Factor Analysis）

4. Factor Rotation

An orthogonal matrix, and let.

Goal:to rotatesuch that a ‘simple’ structure is achieved.

Kaiser (1958)’svarimaxcriterion（方差最大旋转） :

1) define

2) chooses.t.

5. Factor Scores

Weighted Least Squares Method

Suppose that,, andare known.

Then

Regression Method

From the mean of the conditional distribution ofis

• 发表于:
• 原文链接：http://kuaibao.qq.com/s/20180108G0EPLX00?refer=cp_1026
• 腾讯「云+社区」是腾讯内容开放平台帐号（企鹅号）传播渠道之一，根据《腾讯内容开放平台服务协议》转载发布内容。

2019-03-24

2019-03-24

2019-03-24

2019-03-24

2019-03-24

2018-06-11

2018-06-01

2019-03-24

2019-03-24

2019-03-24

2019-03-24