================================================================================ https://datascienceschool.net/view-notebook/5e2ee247518f47a6ba2d0894b33c73a3/ ================================================================================ result=Bernoulli_trial(trial) set_of_result={val1,val2} ================================================================================ Bernoulli random variable $$$\subset$$$ Discrete random variable value=Bernoulli_random_variable(result_from_trial) set_of_value={0,1} ================================================================================ Probability density function $$$Bern(x;\mu)$$$ of Bernoulli random variable $$$\text{Bern}(x;\mu) = \begin{cases} \mu & \text{if }x=1, \\ 1-\mu & \text{if }x=0 \end{cases}$$$ * $$$\mu$$$ : parameter, probability of 1 occuring * $$$x$$$: variable * Code value=Bernoulli_RN(result_from_trial) print("value",value) # 0 or 1 prob_val=PDF_of_Bernoulli_RV(value) print("prob_val",prob_val) # mu or 1-mu ================================================================================ PDF of PDF of Bernoulli RV $$$\text{Bern}(x;\mu) = \mu^x(1-\mu)^{(1-x)}$$$ ================================================================================ If a random variable X occurs by driving force of PDF of Bernoulli RV, you can say RV X follows Bernoulli distribution $$$X \sim Bern(x;\mu)$$$ ================================================================================ PMF of Bernoulli random variable $$$P(x=0)=0.4$$$ $$$P(x=1)=0.6$$$ ================================================================================ Simulation: is to generate samples which will be into random variable ================================================================================ val1=Bernoulli_random_variable(result1_from_trial1) val2=Bernoulli_random_variable(result2_from_trial2) # ... val100=Bernoulli_random_variable(result100_from_trial100) ================================================================================ Moments of Bernoulli distribution * $$$E[X]=\mu$$$ val_1=Bernoulli_RN(result_1_from_trial_1) ... val_n=Bernoulli_RN(result_n_from_trial_n) mu=mean(val_1,...,val_n) ================================================================================ $$$Var[X]=\mu(1-\mu)$$$ ================================================================================ Parameter estimation * You get sample from population * You analyze sample * You estimate parameters (mean, var, std) of population ================================================================================ $$$\hat{\mu} = \dfrac{\sum_{i=1}^N x_i}{N}= \dfrac{N_1}{N}$$$ N=number_of_sample N_1=number_of_1_occured pred_mean_of_pop=mean(N_1,N) ================================================================================ How to use Bernoulli distribution 1. Frequentist usage - Input: categorical values like 0 or 1, True or False - Bernoulli distribution decribes ratio of event occuring 2. Bayesian usage - Output data: categorical values like spam or ham - Which output value has more large likelihood? ================================================================================ * You get 10 emails * It turned out 6 emails are spam * Likelihood of coming email is spam is 60% * Above situation can be described by Bernoulli distribution which has $$$\mu=0.6$$$ $$$P_Y=Bern(y;\mu=0.6)$$$ * $$$Y$$$: random variable, 0: ham, 1: spam ================================================================================ * Spam email will more likely have some words * Suppose 4 words which spam email likes to contain * Suppose email x, x contains 1st and 3rd word $$$x = \begin{bmatrix} 1 \\ 0 \\ 1 \\ 0 \end{bmatrix}$$$ * If you have multiple emails, you can write $$$X = \begin{bmatrix} 1 & 0 & 1 & 0 \\ 1 & 1 & 1 & 0 \\ 1 & 1 & 0 & 1 \\ 0 & 0 & 1 & 1 \\ 1 & 1 & 0 & 0 \\ 1 & 1 & 0 & 1 \\ \end{bmatrix}$$$ ================================================================================ Pattern of spam email $$$(X_1,X_2,,X_3,X_4)$$$ $$$X$$$: Bernoulli random variable $$$X_1 \sim Bern(x_1;\mu_1)$$$: probability of spam containing word $$$x_1$$$ $$$X_2 \sim Bern(x_2;\mu_2)$$$: probability of spam containing word $$$x_2$$$ $$$X_3 \sim Bern(x_3;\mu_3)$$$: probability of spam containing word $$$x_3$$$ $$$X_4 \sim Bern(x_4;\mu_4)$$$: probability of spam containing word $$$x_4$$$ ================================================================================ You can calculate parameter by using above feature vectors $$$\hat{\mu}_1 = \frac{5}{6}$$$ $$$\hat{\mu}_2 = \frac{4}{6}$$$ $$$\hat{\mu}_3 = \frac{3}{6}$$$ $$$\hat{\mu}_4 = \frac{3}{6}$$$