Mathematical Models

I. Deterministic Models

 

1) Statistical Models

 

Answer the following question

 

What methods do we use to estimate the parameters of the underlying relationship?

 

Notations

 

1)   Dependent variable (y)

 

2)   Independent variable (x)

 

Equation of a straight Line =

 

y = bo + b1 x

 

 

bo = y-intercept

 

b1 = slope

 

(no error in reading y for a given value of x we can predict y exactly using the above equation)

 

Simple to use unrealistic

 

 

 

 

 

 

 

What to do

 

1)   Plot a scatter diagram

 

A model that allows for the possibility that the observations do not all lie on a straight line is

 

y = bo + b1 x + e

 

 

where e is a random error

 

the average value of e for a given value of x = 0

 

Thus, the expected value of (y) (average value) for a fixed value of x is the line

 

E(y) = bo + b1 x

 

The observed values of y deviate above or below the line by a random error e

 

To estimate the values of the constants bo and b1

 

 

The Method of Least Squares

 

Let y` denote the predicted value of y for a given value of x

 

Then the error of prediction (residual) is y-y` (the difference between the actual value of y and what we predict it to be)

 

 

 

 

 

 

 

The least squares method chooses the prediction line

 

y` = b`o + b`1 x that minimizes the sum of the squared errors of prediction

 

å (y-y`)2 for all sample points

 

 

 

Thus, the method of least squares consists of finding those estimates b`o and b`1 that minimize å (y-y`)2

 

The estimates:

 

b`1 = Sxy / Sxx and b`o = Y - b`1 X

 

 

Where,

 

Sxx = å (x X )2 = å x2 (åx)2/n

 

Sxy = å (x X ) ( y Y ) = å xy ( ( åx ) ( å y ) ) /n

 

 


 

Example

 

 

x

y

4.2

2.8

3.8

2.5

4.8

3.1

3.4

2.1

4.5

2.9

4.6

2.8

4.3

2.6

3.7

2.4

3.9

2.5

 

 

 

Calculate

 

Sxx and Sxy

 

 

X

 

x2

y

y2

xy

4.2

17.64

2.8

7.84

11.76

3.8

14.44

2.5

6.25

9.50

4.8

23.04

3.1

9.61

14.88

3.4

11.56

2.1

4.41

7.14

4.5

20.25

2.9

8.41

13.05

4.6

21.16

2.8

7.84

12.88

4.3

18.49

2.6

6.76

11.18

3.7

13.69

2.4

5.76

8.88

3.9

15.21

2.5

6.25

9.75

 

37.2

 

155.48

 

23.7

 

63.13

 

99.02

 

 

 

 

Correlation coefficient

 

A measure of the strength of the relationship between two variables x and y

 

r = Sxy / sqr ( SxxSyy)

 

where

 

Syy = å y2 - (åy)2 / n

 

Or

 

r = b`1 sqr ( Sxx / Syy )

 

 

Properties of r

 

1)   r lies between 1 and +1. r > 0 indicates a positive linear relationship and r < 0 a negative linear relationship between x and y. r = 0 indicates no linear relationship between x and y

 

2)   r2 gives the proportion of the total variability of the y values that can be accounted for by the independent variable x

 

r2 = å( y` - Y )2 / Syy = S2xy / SxxSyy

 

 

1       - r2 = å ( y - y` )2 / Syy

 

 

 

 

 

 

 

Example

 

Y

X

1.4

1

2.3

2

3.1

3

4.2

4

5.1

5

5.8

6

6.8

7

7.6

8

8.7

9

9.5

10

 

 

Calculate Syy , Sxx , and Sxy

 

 

Calculate r

 

 

Lack of fit in linear regression

 

 

1)   scatter plot

 

2)   plot of the residuals ( y y`) versus y` may give an indication of the need for higher order terms in the model

 

Linear = no apparent pattern

 

Higher order model = apparent pattern

 

 

 

 

 

Inferences about bo and b1

 

The expected value of bo = mbo and the expected value of

b1 = m b1

 

 

The standard error of bo = s bo = s e sqr ( å x2/nSxx )

 

 

The standard error of b1 = s b1 = s e / sqr ( Sxx )

 

 

Estimates of s b1 and s bo

 

 

S2e = å(y- Y)2 / n-2

 

 

Se = (S2e )

 

 

a) 100( 1- a )% confidence intervals

 

 

bo ta/2 Se sqr ( å x2 / nSxx )

 

 

b1 ta/2 Se / sqr ( Sxx )

 

 

Note : d.f. = n 2

 

 

 

 

b) Statistical tests

 

 

Ho: bo = 0

 

Ha : bo > 0

b0 < 0

bo 0

 

 

T.S.: t = bo` / Se sqr ( å x2 / nSxx )

 

R.R.: for a given value of a and d.f. = n 2

 

1.    Reject Ho if t > ta

 

2.    Reject Ho if t < - ta

 

3.    Reject Ho if | t | > ta/2

 

Ho: b1 = 0

 

Ha: b1 > 0

b1 < 0

b1 0

 

T.S. : t = b1` / ( Se / sqr( Sxx) )

 

 

R.R.: for a given value of a and d.f. = n 2

 

Same as for bo