Machine learning related math

Jensen’s Inequality

\[\phi (E[X]) \le E[\phi(X)]\]

if \(\phi\) is a convex function, a simple example: X~U(0,1) and \(\phi(x) = x^{2}\).

constant-volume transformation in probability?

SVM

used in i-revnet.

Exponential dispersion model

Natural exponential family

Exponential family

Generalized linear model

Introduction to Generalized Linear Models

Conjugate prior

A conjugate prior is an algebraic convenience, giving a closed-form expression for the posterior; otherwise numerical integration may be necessary. Further, conjugate priors may give intuition, by more transparently showing how a likelihood function updates a prior distribution.All members of the exponential family have conjugate priors.

Stationary process

For many applications strict-sense stationarity is too restrictive. Other forms of stationarity such as wide-sense stationarity or N-th order stationarity are then employed.

Gaussian Process

zh https://zhuanlan.zhihu.com/p/31203558

Independent component analysis

mixing function f, mapping latent(source) variable to observed data.

check https://arxiv.org/pdf/1805.08651.pdf

Proving the identifiability of linear ICA (Comon, 1994) was a great advance on the classical theory of factor analysis, where an orthogonal factor rotation could not be identified.

Sylvester’s determinantal identity

Maximum Mean Discrepancy

ppt application

NAT

Empirical risk minimization

in mixup.

Principles of Risk Minimization for Learning Theory,NIPS

Rejection Sampling

https://arxiv.org/pdf/1808.04730.pdf

https://hci.iwr.uni-heidelberg.de/vislearn/HTML/people/jakob_kruse/publications/innf19/innf19kruse.pdf

[approximate Bayesian computation(ABC)]

used for obtain the true posterior in https://arxiv.org/pdf/1808.04730.pdf.