Assignment 3 - 2011

Due date(s):	27 January 2011
(PDF)	Assignment questions (updated: 25 Jan, 16:30)

<rst> <rst-options: 'toc' = False/> <rst-options: 'reset-figures' = False/> .. rubric:: Assignment objectives

- Interpreting and using confidence intervals. - Using tests of differences between two population samples. - Simulating (creating) data and verifying whether theory matches the simulation.

Question 1 [1]

==

Your manager is asking for the average viscosity of a product that you produce in a batch process. Recorded below are the 12 most recent values, taken from consecutive batches. State any assumptions, and clearly show the calculations which are required to estimate a 95% confidence interval for the mean. Interpret that confidence interval for your manager, who is not sure what a confidence interval is.

.. math:: \text{Raw data:} &\qquad [13.7,\, 14.9,\, 15.7,\, 16.1,\, 14.7,\, 15.2,\, 13.9,\, 13.9,\, 15.0,\, 13.0,\, 16.7,\, 13.2] \\ \text{Mean:} &\qquad 14.67\\ \text{Standard deviation:} & \qquad 1.16

You should the `course statistical tables <Tables_of_the_normal_and_t-distribution>`_, rather than computer software, to calculate any limits.

.. answer

The confidence interval for a mean requires the assumption that the individual numbers are taken from a normal distribution, and they are sampled independently (no sample has an effect on the others). Under these assumptions we can calculate a |z|-value for the sampled mean, :math:`\overline{x}`, and construct upper and lower bounds reflecting the probability of sampling that |z|-value.

.. math:: \begin{array}{rcccl} -c_n &\leq& \dfrac{\overline{x} - \mu}{\sigma/\sqrt{n}} &\leq& c_n \\ \end{array}

Since we don't know the value of :math:`\sigma`, we use the sampled value, :math:`s=1.16`. But this means our |z|-value is no longer normally distributed, rather it is :math:`t`-distributed. The limits, :math:`\pm c_t` that contain 95% of the area under the |t|-distribution, with 11 degrees of freedom, are 2.20 (or any close approximation from the tables provided). From this we get the confidence interval:

.. math:: \begin{array}{rcccl} -c_t &\leq& \dfrac{\overline{x} - \mu}{s / \sqrt{n}} &\leq& c_t \\ 14.67 - \dfrac{2.20 \times 1.16}{\sqrt{12}} &\leq& \mu &\leq& 14.67 + \dfrac{2.20 \times 1.16}{\sqrt{12}} \\ 13.93 &\leq& \mu &\leq& 15.41 \end{array}

This confidence interval means that we have 95% confidence that the true average viscosity lies within these bounds. If we took 100 groups of 12 samples, then the limits calculated from 95 of those groups are expected to contain the true mean. It is **incorrect** to say that there is 95% probability the true mean lies within these bounds; the true mean is fixed, there is no probability associated with it.

Question 2 [1.5]

=====

1. . At the 95% confidence level, for a sample size of 7, compare and comment on the upper and lower bounds of the confidence interval that you would calculate if:

a) you know the population standard deviation b) you have to estimate it for the sample.

Assume that the calculated standard deviation from the sample, :math:`s` matches the population :math:`\sigma = 4.19`.

1. . As a follow up, overlay the probability distribution curves for the normal and :math:`t`-distribution that you would use for a sample of data of size :math:`n=7`.

1. . Repeat part of this question, using larger sample sizes. At which point does the difference between the :math:`t`- and normal distributions become *practically* indistinguishable?

1. . What is the implication of this?

.. answer

This question aims for you to prove to yourself that the :math:`t`-distribution is **wider (more broad)** than the normal distribution, and as a result, the confidence interval is wider as well. This is because we are less certain of the data's spread when using the estimate.

The confidence intervals are:

.. math::

\begin{array}{rcccl} - c_n &\leq& \displaystyle \frac{\overline{x} - \mu}{\sigma/\sqrt{n}} &\leq & c_n\\ \\ - c_t &\leq& \displaystyle \frac{\overline{x} - \mu}{s/\sqrt{n}} &\leq & c_t \end{array}

The 95% region spanned by the :math:`t`-distribution with 6 degrees of freedom has upper and lower limits at :math:`c_t = \pm` ``qt((1-0.95)/2, df=6)``, i.e. from **-2.45** to **2.45**. The equivalent 95% region spanned by the normal distribution is :math:`c_n = \pm` ``qnorm((1-0.95)/2)``, spanning from **z=-1.96** to **z=1.96**. Everything else in the center of the 2 inequalities is the same, so we only need to compare :math:`c_t` and :math:`c_n`.

TODO: plots of t- and normal distribution

Question 3 [1]

===

You plan to run a series of 22 experiments to measure the economic advantage, if any, of switching to a corn-based raw material, rather than using your current sugar-based material. You can only run one experiment per day, and there is a high cost to change between raw material dispensing systems. Describe two important precautions you would implement when running these experiments, so you can be certain your results will be accurate.

.. answer

Two important precautions one has to take are:

#. Keep all disturbance factors as constant as possible. #. Randomize the order of the experiments, despite the cost, to obtain independent experimental measurements. Randomization is the insurance we pay to ensure the results are not confounded by unmeasured disturbances.

Question 4 [1.5]

=====

We have emphasized several times in class this week that engineering data often violate the assumption of independence. In this question you will create sequences of autocorrelated data, i.e. data that lack independence. The simplest form of autocorrelation is what is called lag-1 autocorrelation:

.. math::

x_k = \phi x_{k-1} + a_k

For this question let :math:`a_k \sim \mathcal{N}\left(\mu=0, \sigma^2 = 25.0 \right)` and consider these 3 cases:

A: :math:`\qquad \phi = +0.7`

B: :math:`\qquad \phi = 0.0`

C: :math:`\qquad \phi = -0.6`

For each case above perform the following analysis (if you normally submit code with your assignment, then only provide the code for one of the above cases):

1. Simulate the following :math:`i = 1, 2, \ldots, 1000` times:

* Create a vector of 100 autocorrelated :math:`x_k` values using the above formula, using the current level of :math:`\phi` * Calculate the mean of these 100 values, call it :math:`\overline{x}_i` and store it in a vector

2. Use this vector of 1000 means and answer:

* Assuming independence, which is obviously not correct for 2 of the 3 cases, nevertheless, from which population should :math:`\overline{x}` be from, and what are the 2 parameters of that population? * Now, using your 1000 simulated means, estimate those two population parameters. * Compare your estimates to the theoretical values.

Comment on the results, and the implication of this regarding tests of significance (i.e. statistical tests to see if a significant change occurred or not).

Question 5 [2]

=====

We emphasized in class that the best method of testing for a significant difference is to use an external reference data set. The data I used for the example in class are available on `the dataset website <http://openmv.net/info/batch-yields>`_, including the 10 new data points from feedback system B.

1. . Use these data and repeat for yourself (in R, MATLAB, or Python) the calculations described in class. Reproduce the dot plot, but particularly, the risk value of 11%, from the above data. Note the last 10 values in the set of 300 values are the same as "group A" used in the course slides. The 10 yields from group B are: :math:`[83.5, 78.9, 82.7, 93.2, 86.3, 74.7, 81.6, 92.4, 83.6, 72.4]`.

1. . The risk factor of 11% seemed too high to reliably recommend system B to your manager. The vendor of the new feedback has given you an opportunity to run 5 more tests, and now you have 15 values in group B:

.. math::

[83.5, 78.9, 82.7, 93.2, 86.3, 74.7, 81.6, 92.4, 83.6, 72.4, \bf 79.1, 84.6, 86.9, 78.6, 77.1]

Recalculate the average difference between 2 groups of 15 samples, redraw the dot plot and calculate the new risk factor. Comment on these values and *make a recommendation to your manager*. Use bullet points to describe the factors you take into account in your recommendation.

.. note:: You can construct a dot plot by installing the ``BHH2`` package in R and using its ``dotPlot`` function. The ``BHH2`` name comes from Box, Hunter and Hunter, 2nd edition, and you can read about their case study with the dot plot on page 68 to 72 of their book. The case study in class was based on their example.

.. raw:: latex

\vspace{0.5cm} \hrule %\begin{center}END\end{center} </rst>