10.3 Exploratory vs confirmatory  10. Experimental Design  10.1 Introduction

10.2 Components in experimental design

An experiment is usually undertaken to study some general topic, such as how ice forms in clouds, how rainbands are formed, how entrainment affects droplet size distributions, how precipitation forms, or what conditions are required for contrail formation. The scientific objectives often include discriminating among a set of specific hypotheses, representing various alternative explanations. The goal in experimental design is usually to find observable consequences that distinguish among the hypotheses, and then collect measurements that can differentiate among those possibilities (or perhaps invalidate them all). This is always an interactive process: The statement of hypotheses must reflect the consequences of the physical processes in the particular application selected for observation, and the observations must be feasible. The elements in experimental design, although often presented serially, are almost always developed iteratively as compromises between what is possible and what would be decisive.

Those elements include:

• A set of hypotheses. If the hypotheses can be stated in very specific terms, the experiment often can be designed to provide critical and convincing tests that distinguish among them. For example, the hypotheses might be that various specific nucleation mechanisms will be responsible for the formation of the first ice in a particular cloud. These possibilities may have testable consequences that can be detected, and the experiment can be designed to ensure that the hypotheses can be differentiated. In another case, the hypotheses might specify the various possibilities for the dominant dynamical process leading to the formation of a rainband. It is of course never possible to prove a hypothesis, only to obtain evidence either consistent or inconsistent with the hypothesis, so the set of hypotheses for an experiment should include conventional explanations as well as new and more controversial possibilities. Often, a new hypothesis will arise during the exploratory analysis of data, but the experiment will be more convincing (and probably better designed) when the hypotheses have been stated explicitly in the experimental design.
• Experimental tests. When the hypotheses lead to different results, key features can be selected that would serve as tests. In the example of ice formation, the consequences of a particular nucleation mode might be that ice formation is governed by temperature or supersaturation or collision with aerosol particles; each of these possibilities leads to different ice formation in different parts of a cloud, and so has observable consequences. The selection of appropriate experimental tests is the key aspect of experimental design, requiring understanding of practical as well as scientific issues. One must select experimental conditions that occur with appropriate frequency, that can be recognized and probed with available instrumentation, and that provide good tests of the hypotheses.
• Measurement strategies. Many consequences that can be hypothesized are outside the capabilities of current measuring systems, so meteorological experiments must consider if the measurement strategy is practical and must identify the instruments needed to perform the experiment.
• Analysis strategies. An often-neglected component in experimental design is consideration of the analysis approach. For example, what sample size will be needed to draw conclusions of statistical significance? What tests will be applied to the data to accept or reject hypotheses? Experiments are strengthened when these can be specified in advance and considered in the experimental design.

   
 10.3 Exploratory vs confirmatory  10. Experimental Design  10.1 Introduction