RESEARCH ARTICLE

systematic process analysis: when and how to use it1

peter a. hall Harvard University, Minda de Gunzburg Center for European Studies, 27 Kirkland Street, Cambridge, MA 02138, USA E-mail: [email protected]

doi:10.1057/palgrave.eps.2210130

Abstract Challenging the contention that statistical methods applied to large numbers of cases invariably provide better grounds for causal inference, this article explores the value of a method of systematic process analysis that can be applied in a small number of cases. It distinguishes among three modes of explanation – historically specific, multivariate and theory- oriented – and argues that systematic process analysis has special value for developing theory-oriented explanations. It outlines the steps required to perform such analysis well and illustrates them with reference to Owen’s investigation of the ‘democratic peace’. Comparing the results available from this kind of method with those from statistical analysis, it examines the conditions under which each method is warranted. Against conceptions of the ‘comparative method’, which imply that small-n case-studies provide weak grounds for causal inference, it argues that the intensive examination of a small number of cases can be an appropriate research design for testing such inferences.

Keywords systematic process analysis; methods; small-n; explanation

F or securing causal explanations of social or political phenomena, how useful are research designs based on

the intensive investigation of a small number of cases? A series of works, from Lijphart’s (1971) seminal article on the comparative method to the influential text of King et al (1994), declare them inferior to designs that apply statistical methods to a large number of cases. By contrast, this article argues that ‘small-n’ research

designs can be valuable for causal infer- ence in the field of management studies and social science more generally, espe- cially if a methodological approach that I term ‘systematic process analysis’ is ap- plied to the cases. In order to make the case I outline several modes of explana- tion in social science, describe what ‘sys- tematic process analysis’ entails, show how one analysis uses it, and consider when it might most usefully be employed.

european political science: 7 2008

(304 – 317) & 2008 European Consortium for Political Research. 1680-4333/08 www.palgrave-journals.com/eps

304

MODES OF EXPLANATION IN SOCIAL SCIENCE

In what does a causal explanation for a social phenomenon consist? Perhaps be- cause this question is daunting, few empirical works consider it, but, if we are to compare methods and research designs, the issue cannot be avoided, although many aspects of it, such as the role of interpretation, cannot be covered here.2 To delimit the issues, I adopt a positivist perspective that sees the pro- blem as one of identifying a set of variables (x1y xn), understood as events or phenomena whose ‘value’ can vary across time or space, that exert a causal impact on a set of outcomes (y1y yn) the investigator is interested in explaining, as well as an appropriate theory specifying how and why these variables should affect the outcome in question. Even on this delimited terrain, it is useful to distinguish three related, but distinctive, approaches to the ques- tion: what constitutes a good causal explanation? One mode of explanation might be

described as historically specific. This is the type historians typically seek. Their objective is usually to explain the occur- rence of a specific set of events in a limited set of cases, such as the outbreak of the English Revolution in 1640 or of World War I in 1914. Events such as these are typically the product of a long chain of causal factors in which one development conditions another (x1-x2-x3y) and historically specific explanations are dis- tinguished by their ambition to identify the full set of causal factors important to an outcome, establishing not only why the outcome was likely but why it hap- pened in a particular time and place. Whether explicitly or not, many such analyses assert some order of priority among the factors cited as causes of the outcome, as when Stone (1972) classifies the causes of the English Revolution as

preconditions, precipitants or triggers. Moreover, historians are unusually attentive to the importance of context, namely to how factors interact to generate an outcome and to the spatial or temporal specificities affecting the value of each factor. Contingent events that do not themselves seem predictable often figure prominently in the causal chains cited in this mode of explanation.

However, even in narrative mode, when engaged in explanation as opposed to description, historians are doing more than listing ‘one damned thing after an- other’.3 Although they rarely use the language of ‘variables’ and often concen- trate on a single case, when making causal assertions, they refer implicitly to the operation of variables as general causes (Roberts, 1996). To say that the arbitrary efforts of King Charles to raise taxes caused discontent implies that, under a given set of conditions, arbitrary efforts to increase taxes will tend to cause discontent. Similarly, although historians wear their theories lightly, such assertions are underpinned, not only by implicit contentions about causal regularities, but by theories speci- fying why it is reasonable to see one set of factors as the cause of another. Despite its distinctive qualities, a histori- cally specific mode of explanation shares some features of all positivist explanations.

It can usefully be compared to a second modality that I will call multivariate ex- planation. Here, the objective is not to explain a historically specific event but to identify the causal factors conducive to a broad class of events. Accordingly, if historians adduce long causal chains, multivariate analysts typically attempt to identify a small set of variables that can be said to cause such outcomes in a general class of times and places, inde- pendently of the other factors that might contribute to the relevant causal chain in

peter a. hall european political science: 7 2008 305

any one case. Their objective is often also to estimate the magnitude of the effect of each variable and the confidence with which we can assert its effect. From this perspective, good explanations are parsi- monious ones that specify the precise impact of a few key variables. Needless to say, this approach to explanation tends to privilege statistical modes of inquiry cap- able of generating precise parameter estimates. However, because ‘correlation is not causation’, this mode of explanation too depends on the elaboration of a theory specifying how and why each variable should have the impact asso- ciated with it (Waltz, 1979). Finally, we can identify a third mode of

explanation that I will term theory- oriented explanation because it construes the task of explanation as one of elucidat- ing and testing a theory that identifies the main determinants of a broad class of outcomes and attaches special impor- tance to specifying the mechanisms whereby those determinants bear on the outcome. In contrast to historically spe- cific explanation, the object is not to provide a complete explanation for why one outcome occurs at a particular time and place, but to identify the most important elements in the causal chain generating this class of outcomes. In contrast to multivariate explanation, this approach attaches less value to securing precise parameter estimates for a few key variables seen as the ‘ultimate causes’ of the outcome and more value to identify- ing regularities in the causal chain through which the relevant outcome is generated. The focus is on elucidating the process whereby the relevant variables have effects. In this respect, if the multi- variate mode of explanation was usually grounded in nomological philosophies of science, theory-oriented explanation has more affinities with the critical rea- lism that succeeded nomological ap- proaches (Moon, 1975; Archer et al, 1998).

CHOOSING A RESEARCH DESIGN AND METHOD

It should be apparent that the choice of methodology and research design for any project must depend, in the first instance, on selecting the mode of explanation to be employed in it. Many factors will influence this choice, including the tastes of the investigator, the state of the exist- ing literature, and the object of inquiry. If one wants to know why an outcome occurred in a particular time and place, a historically specific mode of explanation may be most useful, as other modalities can rarely explain the exact timing or location of the relevant outcome. If one is considering a problem dominated by debate about the precise magnitude of the impact of well-known causal factors, a multivariate mode of explanation will be useful for securing the parameter esti- mates to resolve such debates. Alterna- tively, if there is contention among competing theoretical perspectives about what kinds of causal factors matter to a given outcome, a theory-oriented mode of explanation may be most appropriate.

However, the choice of a methodology must be conditioned, not only by the state of the literature, but by the state of the world, as we perceive it, and notably by the character of the causal relations in the cases to be investigated. Although the object of the inquiry is to propose and test some specific inferences about causal relations, every methodology produces valid inferences only when some assump- tions about the general structure of the causal relations to be investigated are met (see Hall, 2003). To take one exam- ple, most standard forms of regression analysis, the most popular statistical technique employed in social science, produce valid causal inferences only when several conditions are met. The method assumes unit homogeneity, namely that a given change in the value of a causal variable produces a corresponding

european political science: 7 2008 systematic process analysis306

change in the value of the outcome of the same magnitude across all cases. It assumes that the causal variables in- cluded in the analysis are uncorrelated with any causes of the outcome omitted from the analysis, that all the relevant interaction effects among the causal vari- ables have been specified by interaction terms in the regression, and that the cases are fully independent, such that the values of the causal variables or outcomes in one case are unaffected by the corresponding variables in the other cases in the analysis (Wallerstein, 2000). Although there are techniques that allow one to relax some of these assumptions in particular instances, in general, if the causal structure to be investigated does not meet these preconditions, regression analysis is unlikely to produce estimates on which valid inferences can be based. Thus, one’s presuppositions about the

type of causal factors and structure of the causal relationships likely to condition an outcome – derived from observation, existing studies and intuition – must influence the mode of inquiry to be used. Where it is thought that an outcome is determined by a small set of structural factors operating with great force and analogous effect across cases, for exam- ple, statistical methods may be effective for assessing their impact. They were usefully employed to assess the condi- tions conducive to securing stable democ- racy, when those conditions were thought to include basic socioeconomic factors such as the level of economic develop- ment, related levels of literacy, and the correlates of ‘modernisation’ (Lipset, 1959). However, when theorists began to see stable democracy as the product of an intricate strategic interaction among reformers, extremists and defenders of the old regime, statistical methods were no longer so appropriate for assessing the causal chain. Instead, analysts turned toward theory-oriented modes of expla- nation and historical methods for

assessing the adequacy of specific the- ories in individual cases (O’Donnell and Schmitter, 1986; Bates et al, 1998).

In short, despite some claims to the contrary, there is no single methodology that is invariably most powerful for asses- sing the validity of causal inferences in social science. The usefulness of any particular method and research design will depend on both the mode of explana- tion the analyst deems most appropriate and the overarching assumptions made about the structure of causal relations in the cases at hand.

THE VALUE OF SMALL-N RESEARCH DESIGNS

Some of the contexts in which small-n research designs that investigate a few cases in detail have special value can now be identified. Obviously, this approach is useful when the object is to produce a historically specific explanation for an outcome. Here, the most salient issue is whether the analyst should try to inves- tigate more than one such case, and the answer turns heavily on whether it is practicable to secure enough contextual information to establish the full causal chain in more than one case.

In political science and sociology, where multivariate and theory-oriented modes of explanation are generally preferred, the choice among research designs is more difficult to make and will be influ- enced by the considerations mentioned in the preceding section. How certain are we a priori that we can identify all the important variables with causal impact on the outcome? How numerous are they? How readily can they be measured and how consistent do we expect their impact to be across cases? Where we are reasonably confident that one existing theory specifies the relevant variables well and where they are measurable and few in number, a multivariate mode of explanation that employs statistical

peter a. hall european political science: 7 2008 307

analysis on a large number of cases makes good sense, for reasons well- presented by King et al (1994). By looking at many cases, we increase the reliability of our estimates of each variable’s impact and, by looking at a diverse set of cases, we decrease the likelihood that those estimates are being distorted by the presence of another causal variable not incorporated into the analysis. More cases increase the degrees of freedom in the analysis, allowing the analyst to incorpo- rate further variables in order to examine interaction effects, to assess competing hypotheses, or to explore variation in causal impacts among subsets of cases. However, there are many important

issue-areas in the social and political world where the conditions required for the successful use of regression-based modes of statistical analysis do not apply. In some instances, the task of producing quantifiable measures of the relevant variables requires such oversimplification that the resulting proxies distort reality beyond reasonable limits. In others, the causal structures generating the relevant developments contain so many causal variables relative to the number of rele- vant cases available that we lack the degrees of freedom necessary to employ statistical methods with validity. The pre- sence of multiple interaction effects can quickly exhaust the degrees of freedom needed to perform a valid statistical analysis. In recent years, social science has

become increasingly conscious of such interaction effects (Ragin, 1987). Several of the most prominent theoretical devel- opments in fields such as political science draw our attention to them. Rational choice models of political behaviour that view outcomes as the product of long sequences of interactions among strate- gic actors often lend themselves less readily to testing by statistical methods than did earlier causal models that attrib- uted similar outcomes to the impact of a

few key socioeconomic variables. Path- dependent models of the polity often specify an accumulation of interaction effects over time, creating so many divergent contextual effects in the cases that it becomes unreasonable to expect the same causal factor to produce similar effects in each of them (Pierson, 2000; Mahoney, 2000a). In short, despite the continuing popularity of regression ana- lysis, recent theoretical developments in social science tend to specify a world whose causal structure is too complex to be tested effectively by conventional statistical methods (Hall, 2003).

Faced with such dilemmas, analysts have turned only reluctantly to small-n research designs for solutions, because those designs have become associated with the use of the ‘comparative method’ as defined by Lijphart (1971) and a succession of other scholars (see Collier, 1991). In the terms many use to describe it, the comparative method is essentially the statistical method writ small. Their emphasis remains correlational, stressing the causal inferences that can be drawn by comparing the correspondence be- tween a small number of ultimate causal variables and a relevant outcome across the cases. Much attention has been de- voted to how the cases should be chosen, when only a few cases are available, so as to maximise the validity of this type of causal inference. Some advocate choos- ing cases that are similar on all relevant dimensions except on the values of the outcome and causal variables of interest. Others argue for choosing cases that are as different as possible in the hope that the selection will approximate a randomi- sation of other potential causal factors and that systematic correspondence can still be found between the outcomes and key causal variables (Przeworski and Teune, 1970). However, all who adhere to the conventional view assume that the basis for causal inference lies in the correlation to be found, across the cases,

european political science: 7 2008 systematic process analysis308

between a few causal variables and the relevant outcomes. From this perspective, the weakness of

small-n research designs is obvious. The conventional comparative method em- ploys the same basis for inference as the statistical method, but small-n designs lack the degrees of freedom that large-n designs provide for considering substan- tial numbers of causal variables and interaction effects among them. Seen from this perspective, it is not surprising that small-n research designs based on intensive investigation of a few cases have been considered the weak sister of statistical methods applied to a large number of cases. However, they are not necessarily so.

As George (1979), George and McKeown (1985) and Campbell (1975) have noted, when we have a small number of cases to work with, we need not approach the problem of causal inference in the corre- lational terms of the conventional com- parative method (Bennett and George, 2005). On the contrary, a small set of cases, from which many observations can be drawn, can be used as terrain for ‘process tracing’ in which many facets of the causal chain are examined. A more intensive examination of the causal chain, in turn, provides a new and different basis for causal inference, one especially well- suited to assessing the complex causal theories now prominent in many of the social sciences. In short, small-n research designs can be valuable for testing causal propositions if a method that I will term ‘systematic process analysis’ is applied in them.4

THE METHOD OF SYSTEMATIC PROCESS ANALYSIS

As a method, systematic process analysis draws heavily on longstanding conven- tional wisdom about how social science advances.5 Under one rubric or another, it

has been practiced to good effect, albeit with some variation, by many scholars, including Moore (1966), Skocpol (1979), Collier and Collier (1991), Rueschemeyer et al (1992) and Moravcsik (1998). The basic steps of the technique are as follows.

THEORY FORMATION

The investigator begins by formulating a set of theories that identify the principal causal variables said to conduce to a specific type of outcome to be explained as well an accompanying account, which may be more or less formal, about how those and other variables interact in the causal chain that leads to the outcome. In general, these theories will not only identify a few variables thought to have an especially important impact on the outcome but also outline the processes whereby those variables are thought to secure such an impact. As Cartwright (1997) notes, these theories should also specify some basic assumptions about how variables of the type on which the theory focuses operate in the world and why they have causal force, especially where alternative assumptions are plau- sible or adduced by others.

By and large, the theory should be specified as deductions from more gen- eral contentions about the world based on previous observations and axiomatic pre- mises. In this respect, any theory, how- ever novel, depends on previous investigations as well as deduction. The theory of principal interest to the investi- gator may be original or drawn from the work of others. But the crucial point is that the investigator should approach the case, not only with a principal theory, but with it and one or more other theories that could plausibly be adduced to explain the outcome. The object will be to test one theory against another.

The rationale for this injunction is the familiar point of Kuhn (1970) and others

peter a. hall european political science: 7 2008 309

that the ‘facts’ against which a theory is tested are always generated, to some extent, by the theory itself. As a result, one secures a more stringent assessment of the validity of a theory by comparing how well it explains the facts one ob- serves with how well another theory explains such facts. Moreover, as the results often reveal that a theory corre- sponds well in some respects and poorly in others with the observations made in the course of research, the analyst must then make a judgment about whether to reject the theory or to accept it and question the adequacy of the observa- tions. Such judgments are invariably better informed when the fit between the theory and the observations can be compared with the fit between the latter and the next most plausible theory. As the familiar adage has it, research in social science is most likely to advance when it focuses on a ‘three-cornered fight’ among a theory, a rival theory, and a set of empirical observations (Lakatos, 1970).

DERIVING PREDICTIONS

For each of the theories to be considered, the investigator then derives predictions about the patterns that will appear in observations of the world if the theory is valid and if it is false. Special attention should be devoted to deriving predictions that are consistent with one theory but inconsistent with its principal rivals so as to be able to discern which among a set of competing theories is more likely to be valid.6 In many instances, the most im- portant of these predictions will be speci- fied as the hypotheses to be examined in the research. The emphasis should be on deriving predictions that are as ‘brittle’ as possible, against observations and other theories. That is to say, theories should be formulated so as to yield predictions that can be shown to be false by available data and that are distinguishable from the predictions of rival theories.

MAKING OBSERVATIONS

Observations relevant to these predictions are then made of the world, drawn from the cases to be examined. Since there is often ambiguity about the point, let me note that I define a case as a unit in which the relevant outcome takes on a specific value, whether that be a region, nation, organization or other unit at a given time. Thus, comparison across cases may be across units at one point in time or within the same unit across time. An observation consists of a piece of data drawn from, or ‘observed’, in that case, using whatever technology is appropriate for securing it, whether documentary research, inter- views or computation.

It should be apparent that many ob- servations can be drawn from each case. The strength of this method rests on the multiplicity of the observations, and hence tests of the theory, that it allows. Of course, observations of the sort central to the conventional comparative method, namely ones drawn on the outcome and a small set of variables identifiable as the principal causal variables will be germane to the inquiry. But correlation between these types of variables is not the only way to assess the validity of a theory. Instead, this method assumes that ob- servations bearing on a theory’s predic- tions about the process whereby an outcome is caused provide as relevant a test of that theory as predictions about the correspondence between a few key causal variables and the outcomes they are supposed to produce. Even where the object of the analysis is to identify a few such causal variables, any theory identi- fying them as causes must also specify a process whereby they operate, and the validity of the theory can be assessed by observations designed to assess whether that process is present in the cases being investigated.

Therefore, relevant observations in- clude ones about the events that can be

european political science: 7 2008 systematic process analysis310

expected to occur if a theory is valid, the sequence of those events, the specific actions taken by various types of actors, public and private statements by those actors about why they took those actions, as well as other observations designed to establish whether the causal chain that each theory anticipates is present in the cases. This is not simply a search for ‘intervening’ variables. The point is to see if the multiple actions and statements of the actors at each stage of the causal process are consistent with the image of the world implied by the theory.7 In keeping with the advice of King et al (1994), the investigator should seek as large and diverse a set of observations as feasible from each case. Ceteris paribus, a theory that survives tests against more observations of different kinds is more likely to be valid than one tested against a smaller or more homogenous set of observations.

DRAWING CONCLUSIONS

In the penultimate stage of the investiga- tion, the observations drawn from the cases are compared with the predictions of the theories to reach a judgment about the relative merits of each theory, on the basis of congruence between the predic- tions of each and the observations. This is a matter of judgment, rather than one of tallying points of congruence, because the accuracy of some predictions will usually be more crucial to the survival of a theory than others, based on their pertinence to its core propositions and the confidence with which they can be extrapolated from the theory. As in all such enterprises, deciding

whether a theory is valid after observa- tions are made calls for a fine-grained judgment. In many instances, there will be some discrepancies between the the- ory and the observations. The analyst must then make simultaneous judgments about the plausibility of the theory and

about the validity of the observations. This is one reason why effective theory- building is as important a part of the exercise as gathering empirical data. Although the observations drawn in the study at hand must weigh most heavily, judgments about the intrinsic plausibility of a theory can also be based on the support available for its core propositions from other studies and the quality of the deductions used to generate it. Similarly, judgments about the adequacy of the observations should be based on such factors as the reliability of the methods used to secure them and the credibility of the sources. Where some observations support the theory, while others contra- dict it, judgments must be made about the trustworthiness of the data before the theory is rejected. If there are reasons to doubt the adequacy of the data or to attach high value to a theory that seems contraindicated, further observations can be made in existing cases or new cases examined to improve the judgment. As I have noted, this process of judgment can be improved by comparing the observa- tions not only against one theory but against its principal rival.

AN EXAMPLE: OWEN ON THE DEMOCRATIC PEACE

John Owen’s (1994) analysis of the ‘de- mocratic peace’ provides an illustration of how systematic process analysis can illuminate causal problems in social science. Inspired by Kant’s discussion of ‘perpetual peace’ reformulated in empiri- cal terms by Doyle (1983, 1986), the proposition that democracies are unlikely to go to war against other democracies has become one of the most prominent contentions in the study of contemporary international relations. Most efforts to assess the empirical validity of this claim use statistical methods on large numbers of cases. Although these studies show

peter a. hall european political science: 7 2008 311

that democracies rarely go to war against other democracies, they leave open the precise status of the causal claim. As the number of cases available for examina- tion is relatively small, it has been difficult for those who use statistical methods to dismiss rival explanations for their results based on contentions that the infre- quency of war between democratic states is a random outcome or an artifact of factors cited by rival realist theories of international politics. Owen suggests im- proving the basis for causal inferences about this issue by formulating a theory about the causal mechanisms that lead democracies to be reluctant to attack other democracies and examining a set of cases to see whether such a causal mechanism operates in them. This is the project on which he embarks. Here, we see one of the contexts in

which systematic process analysis is most useful, namely a setting in which the number of available cases is too small to allow a statistical analysis to control for all the potentially relevant causal factors as well as one in which many of the relevant variables do not lend themselves readily to accurate measurement. Indeed, as Owen notes, there has been contention about which states should be classified as democratic for the purposes of testing this proposition partly because there has been relatively little investigation of, and no agreement on, the causal mechanisms lying behind the proposition itself. Owen begins by developing a theory

designed to elucidate these causal me- chanisms. As is well-advised in such instances, he uses elements from existing theories to formulate his own, in this case combining structural theories that attri- bute the democratic peace to the institu- tional constraints democracy imposes on governments and normative theories that attribute it to the ideas embraced in democratic polities. He adds precision by identifying the core set of operative ideals as liberal ones and links the two pathways

into a synthetic theory based on the premise that both the institutions and ideology of democracies flow from the character of liberal ideas. He is careful to provide a rationale for why each of the operative variables in the causal chain should have the effects he posits, and he uses the historical documents of liberal- ism to show why key elements in this rationale are plausible. Here, we see the importance and value of specifying the causal process that lies behind an out- come with precision and of adducing a coherent rationale for the operation of each variable within it.

To justify the plausibility of the assump- tions made in the causal model, Owen draws on previous observations about the world and the existing literature about it. Although some have argued that that the realism of a model’s assumptions is irrelevant to its validity, on the grounds that the latter should be judged only by the accuracy of its predictions (cf. Fried- man, 1968), it strikes me as perilous for analysts of causal mechanisms to ignore the realism or plausibility of their as- sumptions. Although the accuracy of a theory’s predictions provide an important test of its validity, given how difficult it is to assess the validity of a theory, even under the best of circumstances, and the omnipresent possibility that more than one theory will generate predictions fit- ting the empirical data, we need addi- tional criteria with which to discriminate among valid and invalid theories; and, in such instances, the plausibility of a theo- ry’s assumptions, especially about how causal mechanisms operate, are useful additional grounds for making such determinations.

From the causal model he has formu- lated, Owen then develops predictions about what should be observed in the cases if his theory is correct and if it is incorrect, couched as six-key hypotheses, as well as analogous predictions for alternative theoretical approaches. The

european political science: 7 2008 systematic process analysis312

emphasis of these predictions is on the attitudes liberals will have toward foreign states of various types and the actions they will take, key components of the causal processes he posits. Owen then examines four cases in which war be- tween the US and another nation was a realistic prospect, two in which it occurred and two in which it did not, with a view to assessing whether observations drawn from these cases conform to the predic- tions of his theory.8 Broadly speaking, he finds that they do. Several features of Owen’s investiga-

tion deserve note. He increases the cred- ibility of his observations about the causal process by examining archival material about the cases and paying careful atten- tion to the perceptions and statements of the historical actors themselves. This is especially crucial here because much of his argument turns on the contention that liberals were reluctant to go to war with states they viewed as liberal and those perceptions about whether a potential antagonist was liberal did not turn en- tirely on the nature of its electoral institu- tions. In most cases, he examines developments at several stages in the causal process to see if they are congru- ent with his theory, rather than scrutinis- ing only one moment in an extended causal chain. He is especially attentive to ambiguous cases, such as that of Whilhelmine Germany, which might ap- pear exceptional from some perspectives, with a view to assessing whether such ‘hard cases’ conform to his theory. Finally, Owen is unusually balanced in

his conclusions. In contrast to many in social science who seem to believe that, in order to show that their theory is correct, they have to show that all other theories are not only incomplete but totally wrong, Owen acknowledges the insights of the realist perspective that is the principal rival to his own theory and suggests how some of its contentions can be rendered congruent with his own

theory and with the empirics of the cases, while rejecting other elements of that perspective. As a result, his work moves the relevant research program forward in a constructive way toward new types of theoretical syntheses (Lakatos, 1970).

For the ease of those who wish to consult an example, I have taken an article as an example of systematic pro- cess analysis. However, it can be difficult to report this kind of an analysis fully in an article, and some features of the method receive less emphasis here than they would deserve in a full treatment. In particular, although Owen examines the predictions of rival theories at various points, a more complete test of his theory would dictate more extensive tests of its principal rivals (see Owen, 1997). This is a facet of the method on which many who undertake it skimp. In such works, it is common to see relatively brief discus- sions of the claims of rival theories, because it can be expositionally awkward to discuss them at length, but, unless rival theories are also examined carefully (even if the results of the examination are not reported at length), it can be difficult to assess fully the causal claims of the analysis.

Needless to say, although I have taken Owen’s work as an example of fine systematic process analysis, it does not definitively settle all disputes about the democratic peace. It is in the nature of social science that there remain grounds for querying some of his contentions. However, by taking an issue often treated with correlational methods and examin- ing the causal processes behind the correlations, Owen shows how illuminat- ing this type of inquiry can be.

EMPLOYING SYSTEMATIC PROCESS ANALYSIS

To most social scientists, the account I have given of systematic process analysis

peter a. hall european political science: 7 2008 313

will be a familiar recipe, since it parallels many basic descriptions of the scientific method. The key point, however, is that the method can be used to examine the causal processes a theory invokes, allow- ing those interested in social explanation to move beyond a focus on simple corre- lations between a set of outcomes and a small set of ‘explanatory variables’ to- ward which the popularity of regression analysis and the infelicities in many con- ventional discussions of the comparative method have drawn the field. It should be apparent that systematic

process analysis is most useful when an analyst seeks a theory-oriented mode of explanation. Although it can yield assessments about the relative impact of causal factors, a key objective of multivariate modes of explanation, the estimates of this sort that it yields are usually less precise than the ones generated by a good statistical analysis. Because it depends on detailed analysis of a small number of cases, systematic process analysis cannot produce precise parameter estimates that are reliable. However, when the decisions or actions of key participants are crucial to the outcome, by comparing the statements and actions of those participants, the process analyst can often establish the relative influence various factors had over them with more precision than can be secured by statistical analysis. There are several contexts in which

systematic process analysis is especially valuable. Statistical analysis is often most useful when there is broad agreement on the basic causal processes behind an outcome and dispute about the relative impact of particular factors within it. By contrast, process analysis can be particu- larly useful when several theories alluding to rather different causal processes have been proposed to explain the same phe- nomenon, because it mobilises multiple observations to reach fine-grained

assessments about the presence of a specific causal process. In such contexts, the parameter estimates generated by statistical methods often assess the cau- sal chain in terms that are too indirect to provide reliable tests for the presence of one particular kind of chain. In instances where the causal chains are highly com- plex or do not yield specific predictions about measurable parameters, of course, systematic process analysis is indispen- sable. That is often true of processes that are path dependent or rooted in strategic interaction.

Bates et al (1998) have proposed an alternative method, based on ‘analytic narrative’ to assess the validity of the- ories in which strategic interaction figures prominently. It shares several features of systematic process analysis but differs in two key respects. On my reading, their approach does not attach as much im- portance as systematic process analysis does to testing rival theories against each other but proposes, instead, an iterative process in which one main theory is examined and actively refined when the analyst encounters data that contradicts it. As a result, although analytic narrative can be useful for refining a theory, it may not offer as stringent a test of that theory as systematic process analysis would. If the investigator’s principal theory is not tested against other theories and if it is adjusted to fit non-conforming observa- tions when they are encountered, it becomes increasingly difficult to falsify the theory using the observations, and the risk of affirming a false theory in- creases. In particular, if rational choice theories are not assessed against other theories lying outside that perspective, there is some chance that the use of analytic narratives will advance one re- search program at the cost of neglecting others that may offer more purchase over some issues (see Elster, 2000).

Even when applied to a single case, systematic process analysis offers some

european political science: 7 2008 systematic process analysis314

grounds for causal inference. Provided the principal theories being tested are formulated in terms that apply to a wide range of cases and spell out the relevant causal process in detail, much can be learned from establishing whether that process is present in a single case (Mahoney, 2000b; Becker, 1992; Eckstein, 1975). Because they are numerous and diverse, the predictions and observations made in a single case are not necessarily less informative than correlations calculated between a small number of causal variables and the outcomes in multiple cases. Where feasible, however, it is desirable to apply systematic process analysis to sev- eral cases, even if the number examined must be small to accommodate the gathering of an extensive set of observa- tions in each. Increasing the number and diversity of the cases increases the investigator’s confidence that the causal process observed is not idiosyncratic to one of them. As most of the theories generated by social science are meant to apply to specific types of cases rather than in all times and places, however, the diversity of the cases chosen should be limited to diversity within the universe of cases to which the theory is meant to apply and that universe should be clearly specified. Similarly, because the object of

the inquiry is usually to explain a parti- cular kind of outcome, there is also special value in extending the analy- sis to cases in which that outcome does not occur, as well as those in which it does, because the explanatory theory being tested implicitly contains important predictions about both types of cases. Where the values taken by a small group of causal variables is especially important to the causal process, it will be useful to examine cases displaying a range of values on those variables, because these are instances in which clear and important predictions can

be made from the theory about the correspondence between those values and the outcomes. However, one should not become a fetishist for this point (cf. Geddes, 1990). Many theories in social science do not attribute dominant causal importance to one or two variables but rather describe causal processes of a particular character, and, in such in- stances, it may be sufficient for causal inference to establish that those pro- cesses were present in cases where the outcome occurred.

In sum, systematic process analysis and small-n research designs complement each other nicely and, used together, they provide a good basis for causal inference. By using process analysis, researchers take full advantage of the wealth of detail that investigation of a small number of cases offers, and they secure more powerful grounds for causal inference than the conventional comparative method offers. This type of method and research design correspond nicely to the recent emphasis in the philosophy of science on critical realism, and they are especially well-suited to assessing the complex causal chains that many theoretical perspectives in social science, including historical institu- tionalism and rational choice analysis, have begun to posit. As such, they deserve the popularity they have long had among sophisticated social scientists.

Acknowledgements I am grateful to Hervé Dumez for com- ments on a previous version of this essay and to Sidney Verba and Robert Putnam, who may not agree with all that is written here but from whom I first learned much of what I know about social science methodology. For support while this essay was written, I acknowledge the Wissenschaftskolleg zu Berlin.

peter a. hall european political science: 7 2008 315

Notes

1 This paper was first published in European Management Review 3(1): 24–31. Reprinted with kind permission from Peter Hall and the European Academy of Management. 2 For discussion of this issue see: Roberts (1996) and Taylor (1971). 3 This phrase is usually attributed to the American author, Elbert Hubbard. 4 Although my argument is similar in key respects to the important formulations of George (1979), Campbell (1975), Bennett and George (2005), I adopt a slightly different term for it in order to associate it with the very specific conditions I consider crucial to its practice. However, I want to acknowledge here the similarity and fruitfulness of these prior formulations. 5 This section draws on Hall (2003). 6 When I use the term ‘predictions’, I refer not only (or even primarily) to future developments but to predictions about patterns observable in data gathered from past events. 7 Although not strictly entailed by the method, as Weber (1949) advises, the investigator should also ask whether each theory is consistent with the meanings the historical actors themselves attributed to their actions. 8 In a work larger than the article discussed here, Owen (1997) considers an additional eight cases, gaining further comparative leverage. Although his own theory was developed in the context of these cases, as he notes, there would be stronger grounds for causal inference if the theory had been developed in some cases and then tested in others.

References Archer, M., Bhaskar, R., Collier, A., Lawson, T. and Norrie, A. (eds.) (1998) Critical Realism: Essential

Readings, London: Routledge. Bates, R., Griet, A., Levi, M., Rosenthal, J.-L. and Weingast, B. (1998) Analytical Narratives, Princeton:

Princeton University Press. Becker, H. (1992) ‘Cases, Causes, Conjunctures, Stories and Imagery’, in C.C. Ragin and H.S. Becker

(eds.) What is a Case?, New York: Cambridge University Press, pp. 205–216. Bennett, A. and George, A. (2005) Case Studies and Theory Development in the Social Sciences,

Cambridge, MA: MIT Press. Campbell, D.T. (1975) ‘Degrees of freedom and the case study’, Comparative Political Studies 8:

178–193. Cartwright, N. (1997) ‘What is a Causal Structure?’, in V.R. McKim and S.P. Turner (eds.) Causality in

Crisis? Statistical Methods and the Search for Causal Knowledge in the Social Sciences, Notre Dame, IN: Notre Dame University Press, pp. 342–358.

Collier, D. (1991) ‘The Comparative Method: Two Decades of Change’, in D. Rustow and K. Erickson (eds.) Comparative Political Dynamics, New York: Harper Collins, pp. 7–31.

Collier, R.B. and Collier, D. (1991) Shaping the Political Arena, Princeton: Princeton University Press. Doyle, M. (1983) ‘Kant, liberal legacies and foreign affairs Part I’, Philosophy and Public Affairs 12:

205–235. Doyle, M. (1986) ‘Liberalism and world politics’, American Political Science Review 80: 1151–1169. Eckstein, H. (1975) ‘Case Study and Theory in Macro-politics’, in F. Greenstein and N. Polsby (eds.)

Handbook of Political Science, Vol. 1, Reading, MA: Addison Wesley, pp. 79–138. Elster, J. (2000) ‘Review essay: ‘analytical narratives’’, American Political Science Review 94: 685–695. Friedman, M. (1968) ‘The Methodology of Positive Economics’, in M. Brodbeck (ed.) Readings in the

Philosophy of Social Science, New York: Macmillan, pp. 508–529. Geddes, B. (1990) ‘How the cases you choose affect the answers you get’, Political Analysis 2: 131–149. George, A. (1979) ‘Case Studies and Theory: the Method of Structured, Focused Comparison’, in

P. Larson (ed.) Diplomacy: New Approaches to History, Theory and Policy, New York: Free Press, pp. 43–68.

George, A. and McKeown, T.J. (1985) ‘Case studies and theories of organizational decision-making’, Advances in Information Processing in Organizations 2: 21–58.

Hall, P.A. (2003) ‘Aligning Ontology and Methodology in Comparative Research’, in J. Mahoney and D. Rueschemeyer (eds.) Comparative Historical Analysis in the Social Sciences, New York: Cambridge University Press, pp. 373–406.

King, G., Keohane, R. and Verba, S. (1994) Designing Social Inquiry, Princeton: Princeton University Press.

Kuhn, T. (1970) The Structure of Scientific Revolutions, Chicago: University of Chicago Press.

european political science: 7 2008 systematic process analysis316

Lakatos, I. (1970) ‘Falsification and the Methodology of Scientific Research Programs’, in I. Lakatos and A. Musgrave (eds.) Criticism and the Growth of Knowledge, Cambridge: Cambridge University Press.

Lijphart, A. (1971) ‘Comparative politics and the comparative method’, American Political Science Review 64: 682–693.

Lipset, S.M. (1959) ‘Some social requisites of democracy: economic development and political legitimacy’, American Political Science Review 53: 69–105.

Mahoney, J. (2000a) ‘Path dependence in historical sociology’, Theory and Society 29: 507–548. Mahoney, J. (2000b) ‘Strategies of causal inference in small-Nanalysis’, Sociological Methods and

Research 28: 387–424. Moon, D. (1975) ‘The Logic of Political Inquiry’, in F. Greenstein and N Polsby (eds.) Handbook of Political

Science I, pp. 131–195. Moore Jr, B. (1966) Social Origins of Dictatorship and Democracy, Boston: Beacon Press. Moravcsik, A. (1998) The Choice for Europe, Ithaca: Cornell University Press. O’Donnell, G. and Schmitter, P. (eds.) (1986) Transitions from Authoritarian Rule, Baltimore: Johns

Hopkins University Press. Owen IV, J.M. (1994) ‘How liberalism produces democratic peace’, International Security 19: 87–125. Owen IV, J.M. (1997) Liberal Peace, Liberal War: American Politics and International Security., Ithaca,

NY: Cornell University Press. Pierson, P. (2000) ‘Increasing returns, path dependence and the study of politics’, American Political

Science Review 94: 251–267. Przeworski, A. and Teune, H. (1970) The Logic of Comparative Social Inquiry, New York: Wiley. Ragin, C.C. (1987) The Comparative Method, Berkeley: University of California Press. Roberts, C. (1996) The Logic of Historical Explanation, University Park, PA: Pennsylvania State University

Press. Rueschemeyer, D., Stephens, E.H. and Stephens, J.D. (1992) Capitalist Development and Democracy,

Chicago: University of Chicago Press. Skocpol, T. (1979) States and Social Revolutions, New York: Cambridge University Press. Stone, L. (1972) The Causes of the English Revolution, New York: Harper and Row. Taylor, C. (1971) ‘Interpretation and the sciences of man’, Review of Metaphysics 25: 3–51. Wallerstein, M. (2000) ‘Trying to navigate between Scylla and Charybdis: misspecified and unidentified

models in comparative politics’, APSA-CP: Newsletter for the Organized Section in Comparative Politics of the American Political Science Association 11: 1–21.

Waltz, K. (1979) A Theory of International Relations, Reading, MA: Addison-Wesley. Weber, M. (1949) The Methodology of the Social Sciences, New York: Free Press.

About the Author

Peter A. Hall is Krupp Foundation Professor of European Studies at the Minda de Gunzburg Center for European Studies of Harvard University. His publications include Changing France (with P. Culpepper and B. Palier), Varieties of Capitalism (with D. Soskice), The Political Power of Economic Ideas and Governing the Economy as well as many articles on European politics.

peter a. hall european political science: 7 2008 317

Career Research Paper

Career Research Paper

Designed to help you research more about your potential career

Give you practice in the process of writing a paper

Help you learn how to research

Encourage the use of the Writing Lab

We will go over resources for potential career shortly

Career Research Paper (cont.)

Several steps, with due dates throughout semester

80 points total

You will be required to cite your sources

Career Research Paper Details

Choose a career

Discuss:

Education needed

Future career growth

Necessary job skills

Other important information (such as hours, location, why you want this career,how it fits in with your goals, etc.)

Career Research Paper Steps & Deadlines

Outline – March 8

Rough Draft – March 29

Writing Lab Signature – April 12

Final Draft – April 26

Career Research Paper: Outline

Introduction

Write Draft Thesis Statement

I. Education

List the educational steps needed

B.

II. Future Career Growth

Potential career growth according to http://www.bls.gov/ooh/

B. List future promotions you hope to attain

III. Necessary Job Skills

List Skills

IV. Personal

A. What personal aspects will you talk about?

Conclusion

Include a works cited page of at least two sources.

Due March 8!

INCLUDE LIST OF AT LEAST TWO SOURCES

Example: Career Research Paper: Outline

I. Introduction

After researching the field of higher education, I understand I will eventually need a PhD, the potential growth for my career, and what skills I will need to develop. I also have a deeper understanding of why I am passionate about this career.

II. Education

PhD: Education or Higher Education or Education Administration

On the job experience

III. Future Career Growth

A. Projected career growth in the field

B. Director of Student Development/Student Leadership

C. Dean of Student Affairs

D. Executive Director of Nonprofit

Career Research Paper: Outline (cont.)

IV. Necessary Job Skills

A. Program development

B. Student development

C. Diverse population

D. Supervision

E. Communication

V. Personal

A. Passionate about educational access for all people

B. Enjoy working with at-risk populations

C. Will have to decide eventually which direction I want to go

VI. Conclusion

Career Research Paper: Rough Draft

Using outline, complete a typed draft

Times New Roman, Double-spaced, Font 12

Grammar/Spelling

Must include at least two sources

Works Cited Page

Parenthetical Citations

Due March 29!

Career Research Paper: Writing Lab Visit

Incorporate the suggestions I gave you into your paper

Visit the Writing Lab in TC or use Online Writing Lab (OWL)

Must show me the signature or

printed out OWL returned copy

Must show me by

April 12!

Career Research Paper: Final Draft

Must incorporate feedback given from Writing Lab or OWL

Turn in final copy in class or on Blackboard

Must include:

Outline

Rough Draft signed by me

Draft signed by Writing Lab

Works Cited Page

Due April 26!

Resources for Research

CHOICES Planner (MyJeffco)

Career Development page on Jeffco website

O*NET Online (www.online.onetcenter.org)

Occupational Outlook Handbook (www.bls.gov/ooh/)

Career One Stop (www.careeronestop.com)

Jobs.mo.gov (Job Seeker Resources)

Library search engine

Resources for Citing Sources

Purdue OWL

https://owl.english.purdue.edu/owl/

Jefferson College Writing Lab

Jefferson College Library – LibGuides

http://jeffco.libguides.com/citingsources

Career Research Paper Checklist

____ Outline

____ Rough Draft, Signed by Instructor

____ Draft with Writing Lab Signature

____ Final Draft

____ Works Cited Page with at least two

sources (MLA Format)

Documentation

Capital Plan

Loan Scenarios

Savings Scenarios