<![CDATA[
This is note I wrote as I was take following lecture
http://www.kocw.net/home/search/kemView.do?kemId=1189957
================================================================================
* Example
* Each vector is 3 dimensions
* Data has 2 classes $$$\omega_1$$$, $$$\omega_2$$$, and each class follows Gaussian distribution
And you know parameters $$$\mu$$$, $$$\Sigma$$$ of each Gaussian distribution
]]>
<![CDATA[
* Parameters are given as follow
- Mean of $$$\omega_1$$$: $$$\mu_1 = [0 \;\; 0 \;\; 0]^T$$$
- Mean of $$$\omega_2$$$: $$$\mu_2 = [1 \;\; 1 \;\; 1]^T$$$
- Covariance matrix: $$$\Sigma_1=\Sigma_2 = \begin{bmatrix} \frac{1}{4}&&0&&0\\0&&\frac{1}{4}&&0\\0&&0&&\frac{1}{4} \end{bmatrix}$$$
- Prior probaiblity: $$$p(\omega_2) = 2p(\omega_1)$$$
for example, if $$$p(\omega_1)= \frac{1}{3}$$$, then, $$$p(\omega_2)= \frac{2}{3}$$$
* Question:
- Find linear decision function
- Classify $$$x=[0.1 \;\; 0.8 \;\; 0.8]^T$$$ from either $$$\omega_1$$$ or $$$\omega_2$$$
================================================================================
* Find linear decision function $$$g_i(x)$$$
* If you want to get $$$g_i(x)$$$ by using MAP
$$$g_i(x)
= \ln{[p(\omega_i|x)]} \\
= \ln{[\dfrac{P[x|\omega_i]P[\omega_i]}{P[x]}]}$$$
$$$g_i(x)
=-\frac{1}{2} (x-\mu_i)^T \Sigma_i^{-1} (x-\mu_i) + \ln{P(\omega_i)} \\
=-\frac{1}{2} \begin{bmatrix} x_1-\mu_1\\x_2-\mu_2\\x_3-\mu_3 \end{bmatrix}^T
\begin{bmatrix} 4&&0&&0\\0&&4&&0\\0&&0&&4 \end{bmatrix}
\begin{bmatrix} x_1-\mu_1\\x_2-\mu_2\\x_3-\mu_3 \end{bmatrix} + \ln{P(\omega_i)}$$$
================================================================================
* Let's put class into i
* i=1: $$$\omega_1$$$
$$$g_1(x) \\
=-\frac{1}{2} \begin{bmatrix} x_1-0\\x_2-0\\x_3-0 \end{bmatrix}^T
\begin{bmatrix} 4&&0&&0\\0&&4&&0\\0&&0&&4 \end{bmatrix}
\begin{bmatrix} x_1-0\\x_2-0\\x_3-0 \end{bmatrix} + \ln{\dfrac{1}{3}} \\
= -2(x_1^2+x_2^2+x_3^2)+\ln{\dfrac{1}{3}}$$$
* i=2: $$$\omega_2$$$
$$$g_2(x) \\
=-\frac{1}{2} \begin{bmatrix} x_1-1\\x_2-1\\x_3-1 \end{bmatrix}^T
\begin{bmatrix} 4&&0&&0\\0&&4&&0\\0&&0&&4 \end{bmatrix}
\begin{bmatrix} x_1-1\\x_2-1\\x_3-1 \end{bmatrix} + \ln{\dfrac{2}{3}} \\
= -2((x_1-1)^2+(x_2-1)^2+(x_3-1)^2)+\ln{\dfrac{2}{3}}$$$
Compare: $$$g_1(x)$$$ and $$$g_2(x)$$$
* If $$$g_1(x)>g_2(x)$$$, then, classify new data into $$$\omega_1$$$
* If $$$g_1(x)<g_2(x)$$$, then, classify new data into $$$\omega_2$$$
Let's put values
Compare following:
Class 1: $$$-2(x_1^2+x_2^2+x_3^2)+\ln{\dfrac{1}{3}}$$$
Class 2: $$$-2((x_1-1)^2+(x_2-1)^2+(x_3-1)^2)+\ln{\dfrac{2}{3}}$$$
Class 1: $$$x_1+x_2+x_3$$$
Class 2: $$$\dfrac{6-\ln{2}}{4}=1.32$$$
Class 1: $$$0.1+0.7+0.8=1.6$$$
Class 2: $$$1.32$$$
New data should be in $$$\omega_1$$$
================================================================================
]]>