Analysis of variance

ANOVA

ANOVA in statistics is a powerful tool for determining the influence of different groups of observations among themselves. The analysis of variance was introduced by Fisher, an English scientist who made a huge contribution to the development of science. ANOVA is an acronym for ANalysis Of VAriance.

Example

Suppose you want to conduct an empirical study of gasoline quality, for this you fill up the tank at one gas station and drive n kilometers, repeat such an experiment, say, five times, then conduct the same experiment, only at a different gas station. You have two sets of data - refueling A and refueling B. Certainly, the figures are scattered, but there is still some dependence, so that would determine whether refueling affects gasoline consumption (or the data are not related) You are using variance analysis.

The analysis of variance allows you to determine which of the factors affects more, intra-group or intergroup. In the example above, you will be able to determine how much the choice of gas station affects gasoline consumption. This is the essence of the dispersion analysis: to find out whether the selected factor is significant for the selected observations.

In a sense, the analysis of variance is similar to regression and correlation analyses, because it allows determine the influence of variables on each other.

Analysis

In theory, a simple model is built to analyze the variance, similar to the one studied in time series analysis.

Model

The model of the analysis of variance includes the average value, the effect of the experiment and a random error:

y = μ + τ + ε
τ - experiment effect, ε - random error

Single-factor

One-factor analysis of variance considers the influence of one criterion, it is done this way: we conduct two experiments, in one of them we include an additional factor and analyze whether this factor has made changes. As initial data, consider the results of a number of experiments:

μ = (44 + 49.4 + 99.8 + 36.4) / 4 = 57.4
The square of errors within groups (Square Sum within group):
SS_w = Σ_iΣ_j(y_ij - μ_i)² = 4161.2
The square of errors between groups (Square Sum between group):
SS_b = Σ_i(μ_i - μ)² = 2482.32
Given the degrees of freedom, the expected average is:
MS_w = SS_w / a(n-1) = 277.41
MS_b = SS_b / a-1 = 620.58
Value of F_crit :
F₀ = MS_b/MS_w = 2.237

Fischer's test: if the value of F₀ turns out to be greater than the value of F _λ,4,15, then the factor has an impact.

For n = 20 and a = 5, F_λ,n-a,a-1 = F_λ,15,4= 5.86
Since F₀ = 2.237 < 5.86, then we assume that the introduced factor did not have an effecton the results of the experiment.

Two-factor

In two- factor analysis , three hypotheses are put forward for verification:

• Factors A and B do not affect the result
• Factor A does not affect the result
• Factor B does not affect the result

To carry out a two-factor analysis, it is necessary to make groups of results: several measurements for all values of each of the factors, i.e.:

Next, the average value for each factor value is calculated, i.e. the average for A1, the average for B1, etc. Then it is calculated the total average for all results. Let's set the number of criteria: k = 2 (the number of criteria A) and m = 2 (the number of criteria B).

T = ΣΣΣx_ijk
The sum of elements under the influence of factor A:
T_Ai = Σx_i·k
The sum of elements under the influence of factor B:
T_Bj = Σx_·jk
The sum of elements under the influence of factor AB:
T_AiBj = Σx_ij·
SST = Σx²_ijk - T²/N
SSA = ΣT²_Ai/n·m - T²/N
SSB = ΣT²_Bj/n·k - T²/N
SSAB = ΣΣT²_AiBj/n - SSA - SSB - T²/N
SSE = ΣΣΣx²_ijk - ΣΣT²_AiBj/n

SST = SSA + SSB + SSAB + SSE

MSE = SSE/(n-1)·m·k
MSA = SSA/k-1
MSB = SSB/m-1
MSAB = SSAB/(m-1)·(k-1)
Test "Criterion A does notaffect the result", ν₁= k-1:
F_A = MS_A/MS_E
Test "Criterion B does notaffect the result", ν₁= m-1:
F_B = MS_B/MS_E
Test "Criteria A and B do notaffect the result", ν₁ = (k-1)(m-1):
F_int = MS_AB/MS_E

For each F, if F > F _α,ν1,ν2, then the hypothesis is rejected. ν₂ = N-mk

Multifactorial

Multivariate analysis is similar to two-factor analysis - the same operations are performed, but the criteria are grouped and the influence of each of the factors is found iteratively.

With repeated measurements

The analysis of variance with repeated measurements indicates that several tests were performed for each criterion measurements of a random variable to obtain a more accurate result (since ANOVA) uses the intra-group sum of squares.

Application

Dispersion analysis is used in a wide variety of branches of science and production when it is necessary to study the dependence of the criteria on the difference in average values, while comparing not the average value, but the spread the results are around the mean, i.e. the variance.

Solving problems

As an example, let's give a problem from metrology. The plant houses five machines that produce shafts. It is necessary to determine whether the choice of a machine tool or the training of an employee affects the result of production. For analysis measurements are made for each machine and employee, the result is a table:

Let's use the method of two-factor analysis, factor A is the operator, factor B is the machine. Calculate the sums of squares, to do this, you need to calculate the average value for each of the groups:

SSA = 0.256
SSB = 0.794
SSAB = 0.333
SSE = 0.408

MSA = 0.256
MSB = 0.199
MSAB = 0.083
MSE = 0.102

F_A = 2.51
F_B = 1.951
F_AB = 0.814

Critical values for the Fischer test:
F_{crit A} = F_{0.1, 1, 90} = 2.77
F_{crit B} = F_{0.1, 4, 90} = 2.01
F_{crit AB} = F_{0.1, 4, 90} = 2.01

Results table:

In excel/Open Calc

To solve the variance analysis in a spreadsheet, you will need the following formulas:

Download table in the formats ods and xls.