Table of contents

What is Expectation Maximization?

Expectation Maximization (EM) are set of rules in statistics, which are an iterative approach for obtaining maximum a posteriori (MAP) or maximum likelihood approx. of considerations for statistical models, as this model are based on unnoticeable hidden variables. The expectation maximization iteration swings between creating a functional log for the expectation likelihood, carrying out an expectation (E) step & the factors which computes maximizing the expected likelihood which were round in E step, maximization (M) step.

These parameters and factors are later used to decide the distribution of the hidden factors while executing E step next time. In simpler form, Expectation step uses available or present parameters to re-build hidden structure, whereas Maximization step deals with hidden observation to re-estimate the parameters.

The conditional expectation is considered as a probability theory, in which an unplanned variable is other variable which is also random is matched to the average of the previous each probable ‘condition’. As mentioned, when the unplanned variable can be defined over a separate likelihood space, the ‘condition’ are a sections of this likelihood spaces. This characterisitic can later be generalized to any likelihood space using measure theory.

For better understanding, we will illustrate Conditional Expectation example.

Example of Conditional Expectation

Let us assume that everyday rainfall measures (mm of rain per day) obtained by weather department every day continuously for 10 years i.e. from 1st Jan, 2000 to 31st Dec, 2009. The conditional expectation of regular rainfall being aware of the specific month of that year respectively is the middling of regular rainfall over the days of the 10 year period that arrives in a specific month.

These statistics can later may be used as a function per day (so for example the data for 3rd March 1992, might be the addition of regular rainfalls in the every month of March during the period of 10 years, distributed by the number of these days, which is 310) or as a function of just the month (so for this case, the count for the month of March would be same as the count of the earlier case).

In Conditional Expectation, if the H event is probability a positive, then the case is particularly for Y a distinct unplanned variable and for y in the collection of Y if the event H is Y = y. Assuming (Ω, Ƒ, Ƥ) be a likelihood space, X a random variable on that likelihood space, and HЄƑ an event with only positive probability Ƥ(H) > 0. Then, for the H event, provided the conditional expectation of X is

Here x is any value from the range of X &

Is the conditional probability of A knowing H.

Conditional Expectation on the basis of sigma algebra

In this example of Conditional Expectation, the livelihood space (Ω, Ƒ, Ƥ) is the [0, 1] break with the Lebesgue measure.

Expectation (E) steps are considered inaccurate. This is because the result calculated at initial stage are done by considering fixed values, parameters which are dependent on data of the function Q. Once the value of Q is found, the next step is to maximized (M) step & determine fully by EM algorithm.

Even if an EMiteration upsurge the practical data which is marginal, possibility function there is no assurance that the structure come together to a maximum likelihood estimator. For several different mode distributions, this is understood as that an EM algorithm may meet to a local maximum of the practical data likelihood function, depending on initial values. There are a variety of metaheuristic or heuristic approaches for avoidance a local maximum such as random restart or put on virtual annealing methods.

EM is mainly beneficial when the likelihood is a growing family: the E step turn out to be the addition of expectations of adequate data, and the M step includes maximizing a linear function.

In order to achieve Expectation Maximization properties proof better result, we works on (Q(θ|θ(t))

Instead of direct betterment of

log p(X|θ)

For any value of Z with non-zero livelihood P(Z | X, θ, we can arrange as:

The suggestion in likelihood theory is known as the law of iterated expectations, the law of total expectation, the smoothing theorem, Adam’s Law and the Tower Rule among other names, define that if X is an integrable changeable variable (which means a random changing variable satisfying E( | X | < ∞) and Y is any unplanned variable, it is not compulsory to be integrable, on the same likelihood space, then

E(X) = E (E(X | Y)),

in which the value which is expected of the conditional expected value of X as long as Y is the same as the expected of X.

The conditional expected value E(X | Y) is an unplanned variable in its own right, the value be determined by on the value of Y. cannot be avoided that the conditional expected value of X assumed the event Y = y is a function of y. If we put pen to paper E(X | Y = y) = g(y) then the unplanned variable E(X | Y ) is just g(Y).

For conditional expectation as a random variable such as E [X | Y = 1] or E [X | Y = 4] are any numbers. Considering E [X | Y = y], it is that number which is related to y. So that it is considered as a function of y.

Let us explain you with the example of the Uniform Distribution by solving a problem related to conditional expectation. Assuming that the X & Y as an independent geometric random values with the p as parameter. Suppose n is a positive integer & has the value which is greater than 2. Thus, we have to find the conditional mass of X which is represented as X + Y = n & the conditional expectation of X provided is X + Y = n.

Conditional Variance

In livelihood statistics & theory, a conditional variance is the variance or an unplanned variable given the value(s) of 1 or more other variables. Conditional variance plays a significant role in ARCH (Autoregressive Conditional Heteroskedasticity) model.

The conditional variance of any randomly chosen variable Y provided another random variable X is expressed as

Var(Y | X) = E((Y – E(Y – E(Y | X))2 | X)

The conditional variance states the total amount of variance which are not considered if we use E(X|Y) to ‘predict’ the value of Y. As expected, E(Y|X) stands for expectation of Y over provided value of X, which can be recall, is an unplanned variable in itself. So the result, Var(Y|X) in its own is an unexpected variable & it is a function of X.

A tourist lost his way & arrived to a junction with 3 roads. The 1st road will bring the tourist to the same starting point after he walks for an hour. The 2nd road will bring the tourist to the same point where he is standing now but only after walking for 6 hours. After walking for 2hours, the last road will end up in city. But there is no direction mentioned on the road side,

Let us consider that the tourist picks a road similarly possible at all times.

We will solve this problem using conditional expectation

Considering that the time T is consumed time to reach city, & Di is the duration which tourist select his I door for i = 1, 2, 3. To calculate E[T] we will state on the door scalper picks at the 1st time

E[T] = E[T|D1]P(D1) + E[T|D2]P(D2) + E[T|D3]P(D3)

= (E [T | D1] + E [T | D2] + E [T | D3])

= (1 + E [T] + 6 + E [T] + 2)

Where the fact is that even after returning to the point, the tourist does not keep remember of its previous selections.

After arranging the terms form available equation, we find E[T] = 9

The day before exam, every student goes to the GSI’s office for asking a question that will be asked in the exam whose probability is denoted as p. The total no. of students going in working hours that day is Poisson distributed with mean λ.

The main idea in this problem is again conditioning.

Let N be the total count of students who enter the GSI’s office on that day, & let A be the event that the GSI does not have to answer an exam question.