9 min reading time

Definitive screening designs (DSD): screen factors and detect curvature in one experiment

5.15. Definitive screening designs (DSD): screen factors and detect curvature in one experiment

The previous section let a computer build a design to order, by optimizing a criterion. This section turns to a structured, ready-made family that meets many of the same needs without an optimization step: the definitive screening design.

The traditional way to run an investigation with many factors is in two phases. First you screen: a two-level fractional factorial or a Plackett-Burman design with as few runs as possible, to find out which of the many factors actually matter. Then, on the handful that survive, you run a separate response surface experiment, a central composite or Box-Behnken design, to model the curvature and find an optimum. The logic is sound, but it has real friction. It takes two rounds of experimentation, with the delay and overhead of stopping, analysing, and restarting. The screening design is at two levels, so it cannot see curvature at all; you only discover whether a factor bends the response in the second phase. And a low-resolution screening design aliases main effects with two-factor interactions, so a large “main effect” may really be an interaction in disguise.

A definitive screening design (DSD), introduced by Jones and Nachtsheim in 2011, collapses the two phases into one. It is a three-level design, economical (about \(2k + 1\) runs for \(k\) factors), that screens the main effects and lets you detect curvature, all from a single set of runs.

5.15.1. The foldover construction and its consequences

The construction is a foldover of a conference matrix with a centre run added, a route to the DSD due to Xiao, Lin, and Bai (2012). A conference matrix \(\mathbf{C}\) of order \(m\) is an \(m \times m\) matrix with zeros on the diagonal and \(\pm 1\) off it, whose columns are orthogonal: \(\mathbf{C}^T\mathbf{C} = (m-1)\mathbf{I}\). The design stacks \(\mathbf{C}\), its mirror image \(-\mathbf{C}\), and a row of zeros:

\[\begin{split}\mathbf{D} = \begin{bmatrix} \mathbf{C} \\ -\mathbf{C} \\ \mathbf{0} \end{bmatrix}\end{split}\]

so that every factor is run at all three levels \(-1, 0, +1\).

The foldover is what gives the design its remarkable properties, and the reason is worth understanding because it returns when we meet OMARS designs. Consider what happens when we negate a run, flipping the sign of every factor at once, which is exactly what the \(-\mathbf{C}\) block does. A main effect, being linear in a single factor, reverses sign. The intercept, a quadratic \(x_i^2\), and a two-factor interaction \(x_i x_j\) (where both factors flip together, so the product is unchanged) all stay exactly as they were. So when a run is added to its negated mirror, every product between a main-effect column and a second-order column cancels between the two halves, while the second-order columns reproduce themselves identically. The consequences are:

  • main effects are orthogonal to one another, and estimated as cleanly as in a two-level design;

  • main effects are orthogonal to (unaliased with) every two-factor interaction and every quadratic effect, so a real interaction can never masquerade as a main effect;

  • all \(k\) quadratic effects are estimable, so curvature can be detected in every factor; and

  • under effect sparsity (the common assumption that only a few of the many factors actually drive the response), a sufficiently large design has a further useful property. A three-factor full quadratic has ten parameters, so the design needs at least ten runs, a count a DSD of five or more factors comfortably provides (the run count is \(2k+1\) for even \(k\) and \(2k+3\) for odd \(k\), so a five-factor DSD already has thirteen runs). Meeting the run count is necessary but not sufficient: Jones and Nachtsheim (2011) prove the stronger result that from six factors upward, restricted to any three of the factors, the runs form a design able to fit the full quadratic model in just those three factors. So if only a handful of factors turn out to matter, the same single set of runs supports a complete response-surface model in them. (A small DSD cannot do this: the nine-run, four-factor design used as the running example in Judging and comparing designs has fewer runs than that ten-parameter model requires.)

There is one limitation, and it comes from the very same mechanism. Folding cancels the cross-products between the main effects and the second-order terms, but it does nothing to separate the second-order terms from one another: a two-factor interaction column is identical in \(\mathbf{C}\) and in \(-\mathbf{C}\). So the two-factor interactions remain correlated among themselves (and partially with the quadratics), and you cannot estimate all of them cleanly at once. A DSD is therefore at its best when only a few factors turn out to be active, so that only a few second-order terms compete. That residual entanglement among the second-order effects is precisely what the next section, on OMARS designs, sets out to manage.

5.15.2. A worked DSD: generating and reading the design

It helps to see a DSD as a concrete table of runs. The process_improve package builds one from a list of factors: ask for the "dsd" design type and you get back the coded run matrix. Here we take six continuous factors, \(A\) through \(F\). Six factors give \(2k + 1 = 13\) runs; the centre run is part of the construction, so no extra centre points are requested.

Python, 25 lines
import numpy as np
import plotly.graph_objects as go
from process_improve.experiments import Factor, generate_design

factors = [Factor(name=c, low=-1, high=1) for c in "ABCDEF"]   # six factors
dsd = generate_design(factors, design_type="dsd", random_seed=42)
print(dsd.n_runs, dsd.metadata["construction"])               # 13 paley_q=5

# generate_design returns the runs in randomized execution order. Reorder them into
# the construction pattern [C; -C; 0] so the foldover structure is visible: the rows
# whose first non-zero entry is +1 form C, their sign-flipped mirrors form -C, and the
# all-zero row is the centre run.
levels = dsd.design[dsd.factor_names].to_numpy(dtype=float)
top = [i for i, r in enumerate(levels) if r.any() and r[r != 0][0] > 0]
mirror = [next(j for j, s in enumerate(levels) if np.array_equal(s, -levels[i]))
          for i in top]
centre = [i for i, r in enumerate(levels) if not r.any()]
matrix = levels[top + mirror + centre]

fig = go.Figure(go.Heatmap(
    z=matrix, x=dsd.factor_names, zmid=0, showscale=False,
    colorscale=[[0, "#0072B2"], [0.5, "#EDEDED"], [1, "#D55E00"]]))
fig.update_layout(xaxis_title="Factor", yaxis_title="Run (construction order)",
                  yaxis=dict(autorange="reversed"))
fig.show()

Each cell is one factor at one of its three coded levels: \(-1\) (blue), \(0\) (grey), or \(+1\) (vermillion). The blue and vermillion are the colourblind-safe Okabe-Ito pair. Reading the figure top to bottom, there is a light interpretation worth making now, leaving the mechanics to the construction above and the quantitative comparison to the next sections:

  • The runs come in mirror-image pairs. The lower block is the upper block with every sign flipped (each blue cell becomes vermillion and the reverse): this is the \(-\mathbf{C}\) half folded under the \(\mathbf{C}\) half. That mirror symmetry is the foldover, and it is what buys the clean, unaliased main effects.

  • A single centre run sits at the bottom, every factor at \(0\) (all grey). It anchors the curvature check: without a middle level there is no way to tell a bent response from a straight one.

  • Every factor is exercised at all three levels, and exactly one factor sits at \(0\) in each non-centre run (the grey staircase, which is the conference matrix’s zero diagonal). Thirteen runs cover six factors at three levels each: the economy that makes a DSD attractive as a first experiment.

dsd-run-matrix.py

The run matrix of a six-factor definitive screening design (13 runs), shown in construction order. The top block is a conference matrix \(\mathbf{C}\); the middle block is its sign-flipped mirror \(-\mathbf{C}\); the final row is the centre run. Colours are the coded levels \(-1\) (blue), \(0\) (grey), and \(+1\) (vermillion), using the colourblind-safe Okabe-Ito palette.

How to analyse the resulting data, once responses are measured, is a topic in its own right, because the structured aliasing among the second-order effects means a naive least squares fit is the wrong tool. That is taken up in Analysing data from these designs, after the OMARS family is introduced, since the same staged analysis serves both.

Readings

Contents