Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

The Future of Strategic Planning in the Public Sector: Linking Strategic Management and Performance Poister, Theodore H Public Administration Review; Dec 2010; 70, S1; ProQuest Central pg. S246

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Technological Forecasting & Social Change 85 (2014) 134–152

Contents lists available at ScienceDirect

Technological Forecasting & Social Change

Collaborative foresight: Complementing long-horizon strategic planning

Kirk Weigand a, Thomas Flanagan b, Kevin Dye c, Peter Jones d,⁎ a Air Force Research Laboratory, Dayton, OH, USA b Institute for 21st Century Agoras, 117 Highland Avenue, Barrington, RI, USA c Institute for 21st Century Agoras, 132 Rhoda St., Quincy, MA, USA d OCAD University, 7 Fraser Ave., Toronto, ON, Canada

a r t i c l e i n f o

⁎ Corresponding author. Tel.: +1 416 799 8799. E-mail addresses: [email protected] (K. W

[email protected] (T. Flanagan), kevin@globalago [email protected] (P. Jones).

0040-1625/$ – see front matter © 2013 Elsevier Inc. A http://dx.doi.org/10.1016/j.techfore.2013.08.016

a b s t r a c t

Article history: Received 12 August 2012 Received in revised form 10 June 2013 Accepted 16 August 2013 Available online 13 September 2013

An action case study demonstrates an effective integration of collaborative planning using long-range foresight in a hierarchical government research organization. The purpose of the study was to evaluate the effectiveness of collaborative, bottom-up strategic planning as a complement to top-down strategizing. Large research institutions plan investment over long time horizons and must cope with significant uncertainty, complexity, and mandate changes. Collaborative foresight enhances organizational resilience by improving ideation, problem definition, and consensus in long-horizon strategies. It increases the variety of perspectives in scenario creation, resulting in improved strategic options. Structured Dialogic Design (SDD) was employed as a complementary strategic planning method to the mandated Capabilities-Based Planning (CBP) process. The two methods were conducted in parallel sessions with different organizational participants, strictly limiting information sharing between teams. Participants using SDD to plan efficiently produced a detailed structure representing long-horizon strategic challenges and solution ideas. This collaborative foresight approach demonstrated strong consensus for organizational priorities defined in scenarios and investment pathways. The SDD method demonstrated that transactive and generative planning integrated with traditional rational planning and surpassed it by incorporating deep tacit knowledge from diverse participants. It also fostered organizational cohesion through facilitated collaboration in the planning sessions.

© 2013 Elsevier Inc. All rights reserved.

Keywords: Strategy Strategic foresight Collaborative planning Structured dialogic design Influence mapping

1. Introduction

This study evaluated the effectiveness of facilitated collab- orative foresight within a hierarchical organizational culture defined by a strong preference for rational, top-down strategic planning. The study was conducted in concert with a management proposal to merge six branches within a division of a large government research and development (R&D) organization. The study used collaborative foresight approach to strategic planning. Its purpose was to elicit a useful portfolio of future technology proposals for current investment

eigand), ras.org (K. Dye),

ll rights reserved.

decisions applicable to a 20-year R&D strategic horizon, within the context of a newly reorganized R&D division.

The Sensors Directorate of the Air Force Research Laboratory (AFRL) employs traditional Capability-Based Planning (CBP), a rational planning process well understood by management. CBP is valued for the planning function of strategy-to-task alignment, which allocates work packages to strategic commitments.

Managers and senior technical advisors typically lead the planning for multi-year research investment in any large R&D organization, with only indirect inclusion of line personnel or junior staff. However, several risks are acknowledged when engaging more senior personnel. For one, primarily due to management time commitments, more senior members tend to expedite decisions under time pressure. An expectation for

135K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

rapid closure encourages efficiency, and a group may therefore avoid challenging the dominant paradigms of programs and forecasts. In large institutions, the inclusion of only managers and technical advisors may lead to group- think and other group bias pathologies as they tend to rely on well-understood planning assumptions and share similar worldviews. Rational top-down planning may not be condu- cive to anticipating unforeseen shocks and critical uncertainties within long planning horizons. There are few accepted methods in the rational planning approach that enable managers in planning coordinator roles to pierce the embedded practices in a large hierarchical organization.

The traditional capabilities-based planning team consisted of managers and technical advisors, all experienced in long- range strategic planning. The CBP method was employed and aligned with the Strategy-to-Task framework [1] as shown in Fig. 1.

The Strategy-to-Task framework functions as a strong set of constraints for aligning program-level objectives with national level strategy from executive planning levels. Capabilities-based planning is a “top-down” approach to strategy, a hierarchical model intended to maintain strategic intent from the top-level objectives to the lowest “task” level in the execution. CBP and the strategy-to-task approach are powerful tools for strategic alignment and evolutionary improvement. They may have limited ability when applied to early lifecycle planning of emergent innovations, especial- ly for long-term R&D programs in fields that are rapidly evolving.

The prevailing military culture, even in its research organization, reflects an organizational hierarchy and bu- reaucracy that may impede collaboration in strategic plan- ning [2]. Yet in the present case, the planners were concerned with the risk of CBP producing a program investment plan insufficient to the complexity and uncertainty of long-term strategy. To evaluate a dual-track planning process as a process improvement, AFRL management agreed to a sepa- rate “bottom-up” planning session to complement the ac- cepted top-down CBP method. The planners recognized that top-down planning for a 20-year R&D investment horizon could risk overlooking critical emerging trends in technology and research that a more diverse group might better inform. Therefore, AFRL managers selected Structured Dialogic Design1 (SDD) as a bottom-up planning method to increase the variety of perspectives and inspire collaboration across competing organizational groups to improve overall invest- ment planning quality. SDD was applied as a collaborative foresight methodology.

Both long-term strategic foresight and near-term invest- ment scenarios were needed to complement the CBP ses- sions. Strategic foresight methods can be employed for identifying strategic options in highly uncertain future contexts. According to the well-known “diamond” of Popper [3] foresight methods often employ a mix of both evidence (e.g., horizon scanning) and creative methods (e.g., scenario fiction). Strategic foresight also blends both expertise (e.g.,

1 Structured Dialogic Design (SDD) is a term of art referring to the contemporary form of the methodology practiced as Interactive Manage- ment (Warfield and Cardenas, 1994). It is a registered service mark of the non-profit Institute for 21st Century Agoras.

Delphi panels) and interaction (e.g. workshops). However, participatory collaboration among mixed participants is rarely indicated as a methodology for strategic foresight, even in the more creative techniques.

Ringland [4] suggests that senior management might adopt strategic foresight for surfacing assumptions and mental models, encouraging reflection, understanding com- plexity, and extending collective vision beyond the bound- aries of organizational knowledge. Miles [5] and others have developed and advocated foresight methods for anticipating impacts of technology on markets, organizations and gov- ernment policy. A systematic review of scenario methods [6] analyzed 101 source articles to map applications of scenarios in foresight and decision making across all reported sectors. No mentions of collaboration among diverse participants are found among the taxonomies and reviews. Foresight and scenario development are predominantly led and formulated by management. A key exception is the TIPS (transdisciplin- ary integrated planning and synthesis) process designated for multi-sectoral high-complexity strategic planning and decision making [7]. In complex cross-sector engagements where participants may have conflicting viewpoints and interests, a collaborative planning and consensus approach has been found helpful [8,9]. Yet, collaborative planning relies on bottom-up stakeholder collaboration as a way to understand current stakeholder values and to reach consen- sus in near-term action planning, not necessarily for im- proving the quality of plans and outcomes.

Institutional biases persist in privileging top-down deci- sion making in these large organizations. Few normalized methods for collaborative, “bottom-up” approaches to fore- sight are recommended in current strategic planning prac- tice. For the purposes of this research, organizational collaboration can be defined as a communicative practice engaging multiple participants working together to realize shared outcomes. Collaboration can be viewed as a spectrum of engagement, from the most elementary forms of “working together” to a deep involvement of participants over an extended period of time, with anticipation and mutual understanding of objectives and values. The purposes of “bottom-up” collaboration are to increase the diversity of perspectives, the novelty of ideation and productive creativ- ity in work practices. The bottom-up style of collaboration is inspired by a democratic notion of engagement where power and status differences are minimized for the sake of productive ideation and effective outcomes for complex or uncertain problem areas.

In the case organization's typical strategic planning meetings, managers engaged the more senior staff and technical advisors. Sessions were facilitated by managers and technical advisors in a process often referred to knowingly, and not pejoratively, as a BOGSATT (“Bunch of guys/gals sitting around the table talking”). Several facilitated approaches to collaborative foresight, including structured brainstorming workshops, Future Search [10], and SDD [11] had been evaluated. SDD is a highly-structured facilitated method that evolved from Interactive Management, based on Warfield's social systems theory and methods [12]. SDD was proposed as a bottom-up, collaborative foresight process to complement the mandated “top-down,” rational planning process, drawing from bench-level staff instead of managers

Fig. 1. Strategy-to-Task framework. From Lewis and Roll [1].

136 K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

and technical advisors. The complementarity was based on the understanding of the expected efficiency and acceptabil- ity of both process and products to the organizational culture.

SDD engaged primarily bench-level staff to explicitly contribute to strategic planning, who, by traditional practice, were never involved in planning. This approach had the potential of revealing and questioning status quo assump- tions and surfacing novel challenge and solution ideas. To ensure some alignment between the two planning processes, the planners established a baseline set of categories in the CBP sessions and employed them as content guidelines to ensure the collaborative foresight process was accounting for several core functions necessary in the resulting R&D investment plan.

Convening a complementary planning series was a significant organizational exercise, requiring full manage- ment support and communication of the expected value. The SDD facilitators informed senior staff to aid their under- standing of the new process, of the time levy required for assigned participants, and to coordinate planning deliver- ables. Management did not participate in the collaborative foresight sessions, except as auditors. The planning results and content were based solely on the statements of AFRL participants recorded in SDD dialogue sessions. Management therefore faced the risk that so-called “bottom-up” planning products could either disagree with top-down planning, reflecting organizational misalignment, or merely echo the content of the CBP process, leading to an untested perception of validated superiority of the mandated process.

Upon completion, planners auditing the sessions unan- imously agreed that SDD generated critical proposals that were not addressed by the CBP process. Additionally, they applauded the rapid completion and high performance

output of the SDD collaborative foresight approach and recognized the value of available process performance metrics.

Findings of the audit revealed three unique contributions of SDD compared to the traditional CBP process:

1) The immediate generation of process artifacts, including the challenge map, solution map, and cross-impact map based on strong agreement of the team;

2) The small group generation of 3 scenarios during the workshop with high esprit de corps, originality and tied to identical input criteria; and

3) The post-workshop construction of strategic pathways based on cross-impact mapping of workshop deliverables.

2. Research approach

This action case study evaluated the comparative effec- tiveness of a collaborative approach to long-range, strategic research planning with a diverse group of early-to-mid career technical participants at a “bench-level” or working level in the organization.

The collaborative foresight planning process was deployed during a year-long engagement of the authors with the AFRL organization. AFRL managers decided to proceed with both traditional CBP planning and the SDD collaborative foresight planning in parallel. The authors recognized the opportunity to conduct a comparative case study. There was a common interest in comparing the organization's response as well as comparing the outcomes. Data collection built into the SDD process and software enables researchers to analyze planning content as an action case study.

137K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

2.1. Organizational action case study

The study methodology employed an action research method known as the action case method. This method is consistent with the research intent to both understand behavior (an interpretivist orientation) and to enact and evaluate change in participation with the organization leadership (an action orientation).

A participatory action research (PAR) approach involves interventions on organizational problems within a practical context [13]. Participatory action research is premised on Lewin's proposition that “causal inferences about the behav- ior of human beings are more likely to be valid and enactable when the human beings in question participate in building and testing them” [14].

As a complementary planning process, SDD sought greater variety of participants to enable a wider range of perspectives. “Bench-level” technical staff were engaged as participants to develop alternative strategic inputs to their management. SDD also fostered intra-division participation. The participation of bench-level staff en- abled young practice leaders and future organizational leaders to represent emerging views and expertise in a relevant management context. These two organizational development and human resource management objec- tives constituted significant differences from traditional CBP.

Adopting SDD as a complementary planning approach was conceived as an experiment in organizational practice, following the action case method [15]. The action case provides a basis for both organizational understanding and intervention. The action case method fulfills three research interests — organizational change, managerial learning, and shared understanding of the problem domain. As with other action research methods, it combines techniques to facilitate both organizational intervention and the evaluation of the case study. Using a well-structured process reduces risk and increases predictability for management. Finally, a “soft” or interpretive case study approach enables critical interpretation to evaluate systemic problems and behav- ioral interactions.

As part of an action research process, SDD engages participants as co-researchers and co-interpreters of the organizational strategy. It does this by explicitly co-creating a structured vision of the organization's future [16] as articu- lated in scenarios, new requirements and categories, and influence maps. SDD is facilitated using group management software, well-defined scripting, and protocols that collect participant responses to prompted questions. This data collection and process management provide a higher degree of micro-behavioral observation and data reduction than typically found in published action research. While the authors, as SDD facilitators, did not advise on any of the planning content, they engaged closely with AFRL man- agers and their planners to design the workshops to achieve time, outcome, and process effectiveness objectives. The authors in effect served as organizational change consul- tants or agents [17], a role acknowledged in the planning and research reporting. As action research, SDD is compat- ible with the boundary-spanning, trans-disciplinary knowl- edge co-production objectives of the current case [18].

2.2. Parallel planning methodology

Two SDD workshops were conducted in parallel with CBP BOGSATT sessions as shown in Fig. 2.

Over a 40-day period the CBP and SDD processes identified ideas as goals, scenarios, and priorities for 1) defining chal- lenges and 2) formulating solutions. SDD started immediately after the completion of the second CBP strategy session. The lead planner organized and facilitated the CBP workshops. The two planners also participated as auditors (providing limited, on-call planning scope and framing) in the SDD workshops. The outputs from facilitated CBP sessions were restricted and not provided to the SDD participants. The only exchanges prior to the SDD workshop were an overview presentation from the lead planner and a briefing of the future operations scenario from management's chief technical advisor.

The SDD collaborative foresight process required contin- uous preparation and development over the 40 day period and the engagement of two 2-day workshops for selected organizational participants. A variety of participants were sought across engineering, science, and planning functions from non-management, staff roles in six branches of the organization. Twelve candidates agreed to participate in each of the two workshops. The two SDD workshops were similar in intent to the CBP sessions, focused in two stages on the problematique (definition SDD1) and solution space (design SDD2). SDD1 was held to identify significant findings and construct a map of science and technology challenges over a 20-year horizon. SDD2 generated salient solutions proposals that would best address the network of challenges. These science and technology challenges and solutions were identified and collaboratively mapped into strategic path- ways focused on the most promising programs for invest- ment decisions.

The planners audited and assessed both the CBP and SDD processes, providing external data collection and providing continuity between the planning processes.

2.3. Capabilities-based planning

For both military and research planning purposes, the US Air Force mandates the Capabilities-Based Planning (CBP) process [18]. CBP is a general framework that provides flexibility for its particular use in local organizations. CBP is widely employed in part due to its explicit support for the strategy-to-task doctrine, a management method employed to unify planning, budgeting, and resource al- location [1,19].

This integrated planning method was developed in part to resolve cost management issues in R&D investment planning. The CBP method addressed the need for cost management of high priority science and technology projects. With a wide range of technology projects competing for increasingly- limited present and future budgets, SDD was accepted as a complementary planning process, as a collaborative fore- sight approach to identify best candidate investments for long-horizon planning.

CBP establishes principles that can be implemented through iterative dialogue and local variations in practice. The approach requires that participants meet frequently (if briefly) and reach decisions through deliberations among

Fig. 2. Parallel planning and management processes.

138 K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

subject matter experts and management-level planners. Usually a facilitator (often a manager in the organization as in this case) tracks points as they occur during the dialogue and guides the group to a summary. Consensus may not always occur on specific points, but the overall process is intended to reflect agreement on the interpretation of strategy.

An iterative planning process like CBP is challenging for participants, and to facilitate. As new information emerges and some understandings achieved in earlier dialogue change, the record of that basis for understanding must be revised. In planning sessions, evolving proposals and chang- ing meanings are known to fatigue and confuse participants, exacerbating the latent group tendency toward groupthink. Records of deliberations become expansive and onerous to track. Finally, truly innovative ideas held by individuals can become marginalized or ignored in the quest for consensus.

In this case, the planners used facilitated CBP workshops to address the planning challenge with managers and technical advisors from six branches of their division. The CBP product of the participants ultimately requires a skilled facilitator to consolidate the results as deliverables on behalf of the group. In the typical approach, the facilitator may unknowingly act as a gatekeeper on the expression of ideas produced by the group. The findings of the CBP workshops constituted a baseline (not presented in this article for proprietary reasons) made available upon completion for comparing outcomes of the two strategic planning processes.

2.4. Structured dialogic design

SDD is based on systems science and group behavioral research, and is a canonical method among a class of dialogic

design methods. The methodology is process oriented, focused on managing the dialogue process to ensure the quality of performance. The SDD method embodies principles of dialogic design science developed over several decades of research and practice [11]. These draw from Warfield's science of generic design [12], a Peircean domain of science model [20] and empirical study of applications of Interactive Management. SDD adopts deliberative techniques that obvi- ate groupthink, and collects data that measures collective learning. The method consists of scientifically-validated techniques that enable a diverse group of participants to generate the variety of perspectives necessary to describe and understand complex situations. These techniques have been refined by years of evaluation to minimize the time needed for effective and predictable deliberation.

The SDD facilitators conducted a multi-stage engagement as both strategic planning and action case study. SDD employs a version of Interpretive Structural Modeling (ISM) method [21], which constructs and presents an influence map from salient (voted) participant inputs. SDD applications have been employed in a wide range of collaborative planning problems and domains. A few examples are urban planning [22], indigenous governance [23], large-scale defense policy plan- ning [24], resource management planning [25], technology roadmapping and R&D strategy. There are several good precedents for the adaptation of SDD in defense planning, including five years of applications with 300 program man- agers, culminating with the passage of U.S. Public Law 103-355, the Federal Acquisition Streamlining Act of 1994. These applications were conducted from 1988 to 1993, at the Defense Systems Management College (DSMC) of Fort Belvoir, Virginia in conjunction with the Institute for Advanced Study in the Integrative Sciences at George Mason University (GMU) [24].

139K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

In the current case, SDD was employed as a foresight application to find a problematique [26,27] or future diagnosis. SDD was selected to collaboratively produce a foresight model of significant anticipated problems and solutions for a loosely-defined future security scenario. A distinguishing feature of using SDD for collaborative foresight is that the ideation of problems, or barriers, and solutions to a desired future state, is elicited from bench-level personnel, rather than the managers and technical advisors responsible for the plan. This bottom-up planning process explicitly drew participants from front line personnel rather than more senior staff employed in CBP planning. This provided a higher variety of inputs into problem definition and anticipatory solutions.

SDD workshops were facilitated and recorded in part by the Cognisystem software developed by Christakis [11]. The Cognisystem supports a set of group processes used in most applications including Nominal Group Technique [28], Op- tions Fields and Options Profiles, and Interpretive Structural Modeling (ISM). The ISM method models the contributions to strategic dialogue by representation of a statement's leverage or influence in the network of all propositions. The ISM algorithm presents the participants' dialogue results in a directed graph, as a systemic influence map. While similar products may be produced via rational planning, SDD facilitates equitable input into the planning process, and leads to more coherent outputs based on consensus.

2.5. Dialogic design process participants

Three AFRL planners sponsored the SDD workshops and served as a steering committee to guide the selection of participants for both planning activities. Two of these planners audited the SDD workshops and evaluated the SDD deliverables.

Workshops were planned and convened by a practiced SDD facilitation team that included the authors (primary advisors) in coordination with the AFRL managers and bench-level staff. This team included the SDD process designer, two associate process facilitators, a recorder and an AFRL senior engineer who served as a liaison (a broker role) with AFRL managers. Participants consisted of two separate groups of AFRL division personnel chosen by the planning manager to participate in either the SDD process or the CBP process.

SDD participants were selected to satisfy the requisite variety of knowledge sufficient to collectively understand the complexity of the emerging problem area. Requisite variety [29] is a functional relationship from control systems theory. One definition of requisite variety is “the minimum number of states necessary for a controller to control a system of a given number of states.” The ability to “control” or anticipate the states in a future system is governed by the diversity in the control system. In our case requisite variety is pursued by selecting a set of SDD participants with the greatest variety of perspectives and knowledge of the technology and domain problems. As a social process it may be imperfectly achieved, however the diligent attempt to establish requisite variety results in deeper consideration of the role and representation of workshop participants. Achieving requisite variety in the planning process demands the participation of multiple levels of hierarchy and

multiple disciplinary perspectives. The bottom-up approach was achieved through inclusion of bench-level staff from lower levels in the organizational hierarchy.

Requisite variety requires the selection of participants that have both experience and knowledge in the problem domain. This ensures that organizational values, ethical principles, and disciplinary views are integrated and valued in a structured dialogue. Managers aimed to recruit 15 participants for two 2-day SDD workshops held about three weeks apart. Twelve bench-level participants were identified from across the organization for each of the SDD workshops. This included senior and journeyman engineers and scientists from core disciplines (electrical and mechanical engineers, software designers, and computer scientists). Participant variety was enhanced by drawing from multiple branches or project teams within the division and from across various stages of career (and age). Few of the participants had worked together or knew each other prior to the workshops.

2.6. Process application design

Dialogic design workshops are configured for each type of application through the modular combination of collabora- tive methods. The methods are selected for their fit to the desired organizational outcomes. Three alternative applica- tion configurations were developed and evaluated for use in this case. They were named: a) Extending Top Down Planning, b) Strategic Roadmaps, and c) Multiple Bottom-Up Roadmaps.

The Bottom-Up Roadmaps application was selected. The final selection was based primarily on the intent to maximize the variety of contributions to the parallel planning process. There was not a desire to extend the CBP process nor to develop a single roadmap view. This application intended to discover detailed foresight insights from technical staff based on deep knowledge of interdependencies. A desired outcome was to discover latent challenges that might offer significant leverage in the final investment proposal. (Leverage is defined as the ability of one action to significantly influence multiple options.) This foresight application was intended to create a richer understanding of technical challenges than available with the CBP group alone.

The process application was designed to generate four types of content as SDD planning:

• Near/Mid/Far Goals, • Technology Challenges, • Solutions and Inventions and • Technical Areas of Investment.

This design was driven by the need to enable a diverse group to collaboratively cultivate a sense of cross-functional technical challenges in achieving long-term goals. The pro- spective interdependencies of these challenges needed to be assessed in order to determine the most highly leveraging investments.

2.7. Structured dialogic design method

SDD employed in any of its archetypal forms (diagnosis, system design, complex problem definition, alternative future co-creation, resolution) reuses a common process framework.

140 K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

A series of corresponding language patterns (Fig. 3) are produced that are tightly coordinated to facilitate interactive learning and produce a series of representations of individual and collective contributions.

The workshop process as conducted was defined in deliberations with the planners and SDD facilitators. Patterns 1–4 were selected for both SDD workshops (Definition and Design). Patterns 5 and 6 were employed only in the second (Design) workshop for scenario generation and cross-impact linking ideas about solutions to ideas describing challenges. These patterns are described as follows:

1. Generation of responses: nominal group technique Participants wrote up to 5 individual ideas in response to the triggering question, following the steps of an en- hanced nominal group technique. Through several rounds, ideas were clarified in dialogue with the original authors to achieve whole group understanding. Ideas were collected via the Cognisystem software and were printed to display them in order on one wall. A summary document of all ideas and their clarifications was printed for all participants. Clarifications were recorded by the facilitators, added to the challenge ideas, and distributed as a handout for reference in dialogue.

2. Clustering: options field and options profile Participants collaborated to construct groups of state- ments with features in common as clustered sets. Clustered groups were labeled by consensus. Participants assigned one voting “dot” sticker to each of their 5 most preferred ideas for group consideration in the context of the triggering question. Votes were tallied, and a rank of preference votes was reported to the group. All statements with at least two dots were selected for initial inclusion in the structuring.

3. Structuring: interpretive structural modeling (ISM) The Cognisystem software was to structure influence decisions made by the group using ISM. Statements (challenges or solutions) were selected for ISM structuring in order of participant preferences. The software prompted comparisons based on the ISM matrix algebra algorithm to form the influence structure of all statements entered. Verbal protocols were elicited from participants in the form of warrants (arguments) for their choices in pairwise

Fig. 3. Series of methods in Struc

comparison of statements to establish influence relation- ships. The Cognisystem then converted the matrix of assessments into an influence map (a directed acyclic graph). The map was printed out for distribution to all participants and was displayed on a screen for group interpretation and evaluation.

4. Interpret group learning Root cause statements or deep drivers (at the base of the structure) were specifically compared with ideas previ- ously rated as highly important. In complex situations, groups invariably discover that highly preferred ideas are not necessarily deep drivers. As a result, preferences adopted on the basis of popular preference alone, (as is often the case in other planning methods), may be “erroneous priorities.”

5. Scenario narratives Participants assembled into 3 cross-functional working groups to develop brief scenarios. This provided a narrative account of the strategic pathway each working group chose to present as the best strategy available based on all items in the structure. The working groups presented their narrative presentations in plenary to internalize their understandings of the influence map. Each working group was in that way exposed to two alternative investment pathways beside their own.

6. Cross-impact analysis At the conclusion of the second workshop, the relationship of ideas – from solutions to challenges – was an analyzed and mapped. This composite or superposition structure of ideas provided the basis for tracking the reachability of challenges from prospective solutions. The superposition structure was compared to the strategic pathways to discover common themes. These formed the basis of R&D program and investment areas.

2.7.1. Two-phase planning process Two workshop cycles, SDD1 and SDD2, were convened as

the definition and design phases as shown in Fig. 2. Each workshop required two days of continuous meeting time, with a total of 28 h.

As shown in Fig. 2, the two-phase collaborative foresight process was designed to address two outcomes: 1) Technol- ogy Challenges faced in the future by the R&D customers and

tured Dialogic Design [11].

141K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

2) Technology Enablers defined as solutions to these chal- lenges. In pre-workshop preparation, the SDD facilitators and planning manager iterated and tested versions of the triggering question (the initiating prompt to generate responses in the SDD workshops).

A consistent presentation of an envisioned 20-year mission scenario was prepared for both parallel planning processes. The initial parallel planning approach required the CBP and SDD efforts to exchange interim work products with each other following each workshop, similar to a Delphi consultation. While the initial intention was to maximize total available knowledge across planning modalities, the planners later decided to isolate the parallel planning teams and to not explicitly share any planning content.

Both planning modalities were guided by simple initial directions. A recently revised division mission statement was presented to all groups: “Research, develop, and transition adaptive spectrum warfare and open, composable architecture technologies to enable resilient mission assured warfighting capabilities”.

Three R&D program capabilities further defined the research mission as:

• Fractionated Cyber Effects, with • Cognitive, Autonomic Response, through • Trusted Architectures.

An identical Air Force operational scenario for a 20-year horizon was presented to both CBP and SDD planning teams. The generic scenario did not elicit any specific requirements uniquely associated with either team's expertise.

2.7.2. Challenge definition phase The vision and 20-year scenario were presented to SDD

Definition workshop participants in a pre-session briefing. For each of the three capabilities, attributes were generated for near, mid, and far term planning.

Consistent with the SDD application design, for each workshop a specific triggering question was composed in close consultation with the planning program managers. A triggering question is characterized by careful framing and worded with strategic ambiguity to elicit well-considered responses from participants. A well-defined triggering ques- tion sets clear boundary conditions but also provides latitude to encourage contribution of far-reaching, novel ideas consistent with the problem. The triggering question for the challenge definition phase workshop was:

“What are the Air Force science and technology (S&T) challenges we must face in addressing anticipated capabil- ities over the next two decades?”

The planning horizon of “two decades” was formulated to require participants to adopt a long horizon view of technology challenges that would emerge over the period. Participants formulated and clarified 66 challenges in a 5 hour period.

2.7.3. Design phase with scenario generation The triggering question for the Design workshop (SDD2)

was solution-oriented: “What are the technology options required to address the system of S&T challenges facing the organization?”

This triggering question specifically invoked the “system of challenges,” anticipating a cross-impact analysis of solu- tion proposals mapped to challenges in the system network. This session was concluded by small team selection of solution sets which they presented in the form of scenarios.

2.7.4. Cross-impact analysis Following the solution design workshop, a cross-impact

analysis mapped the relationships between the definition phase “problematique” and the technology enablers identi- fied as solutions in the design phase. This superposition of design solutions to challenges was conducted independently by the facilitators, without stakeholders. The purpose was to analyze and map the relationships between challenges and solutions. The superposition is formulated by associating the solutions with the most leverage (deeper in the network hierarchy) with their corresponding deep driver statements in the problematique. Influence pathways are connected to form a single integrated influence map as shown in Fig. 4.

Each challenge idea (pink) or solution idea (blue) selected in the final mapping was printed, and fastened to the relative location indicated by the ISM map generated by the Cognisystem software. The two influence maps are joined together by analyzing the solution-to-challenge relationships between the two maps. The cross-impact map was prepared as a full-size wall display visible to all participants in the room. The map exhibits too many nodes, lines and relation- ships to be displayed in electronic form to participants for their collective sensemaking. The entire map must be easily read and modified by participants to enable collective assent to the mapping. The constituent maps are located in the Findings section.

2.7.5. Resources The SDD facilitators coordinated the workshop roles of

workshop convener, small group convener, session recorder, wall display manager, software technician, voting manage- ment, and process manager. The Cognisystem software (Leading Design International) was installed on a single PC computer, a second PC was dedicated to displaying group vote results through a projector, and a third supported data entry of verbatim session dialogue and warrants (rationale for votes). Two projectors were used to display the Cognisystem results and voting options and results (in the structuring process). A dedicated laser printer was attached to the main PC, and two colors of paper were used to distinguish challenges (pink) from solutions (blue). Three walls with whiteboards of 25–30 ft in length were available in the workshop room, as well as sticky notes and green masking tape.

3. Theory

This case presents a theory of strategic planning as a process of organizational sensemaking in response to shared beliefs about the future environment by allocating commit- ments and resources that respond most effectively to high-concern future situations. Organizations are compelled by external forces (perceived as change) and respond to increasing complexity and uncertainty by positioning the organization to select from strategic options for possible outcomes. The preferred mode of sensemaking supported by

Fig. 4. Cross-impact mapping of technology enablers to S&T challenges.

142 K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

the organizational culture was that of facilitated “BOGSATT” meetings associated with Capabilities-Based Planning, which had been considered successful in many past situations.

The rigorous facilitated approach of Structured Dialogic Design was considered compatible with the R&D organizational culture. The scientifically-grounded foresight methodology of SDD supported management's desire to make strategic investment decisions in response to the best organizational knowledge informing these complex situations. The planning uncertainty and technological complexity of large research programs require managers to adopt strategic planning ap- proaches sufficient to address anticipated complexity.

Three positions of management in the given case sup- ported the parallel use of collaborative foresight planning within CBP:

1. The necessity for a wider variety of disciplinary perspec- tives given the inability of individual experts to adequately specify technology solutions in a 20-year horizon.

2. The necessity of incorporating a bottom-up planning process into strategic management to ensure a sufficient granularity of understanding in the implications of technology choices.

3. The requirement for sufficient process structure to ensure that maximum productivity and attention is maintained in working through complex planning issues.

According to management and systems theory, these requirements are well supported by collaboration, with one exception. Research supporting claims for collaborative planning remains thin and inconclusive. Management theory has overlooked collaborative (“bottom up”) advising and planning as a strategic lever. Management research promot- ing co-creative practices [30–32] endorses co-production as a customer engagement process but not as a management practice within organizational decision making or strategy. Given the theoretical position that collaborative knowledge production with customers co-produces higher-value offer- ings in a market, why would both management practice and the literature not develop an analogous co-production practice in strategic management?

3.1. Management's operational theory

Planners in the case organization essentially accepted a theory that collaborative foresight would reduce risk in R&D investment by convening a separate, complementary planning process as a contingency for known limitations of rational

planning. Additionally, management also indicated that inclusion of junior and mixed levels of bench-level staff could improve the novelty and robustness of science and technology foresight. A secondary aim was to develop the management skills of “junior force” and journeymen scientists and engineers by affording full participation in a collaborative planning exercise.

Planners indicated their openness to organizational change by adopting novel SDD foresight methods in a complementary parallel planning process. Yet this was constrained by a strict inward focus on generating planning outputs in the SDD process and a limited interest in prompting organizational change through the learning gained through the process of bottom-up planning. These and other limitations were in- formed by retrospective sensemaking [33], wherein organiza- tional behaviors were selectively associated to explain research observations. The lack of interview data renders much of this retrospection speculative. Theoretical claims are therefore limited to those that may be demonstrated from observations in the action case study.

3.2. Organizational learning theory

The core function of the R&D enterprise is organizing to research and explore new technology concepts. Organiza- tional learning that facilitates a shared understanding of the possibilities and consequences of technology requires social- izing knowledge within core disciplines and between levels of management.

The intervention approach of action case study research is consistent with organizational learning theory [34], as action research requires stakeholders to reflect and evaluate their participation and development. Reflective inquiry, as required in SDD, represents an organizational learning intervention:

“Organizational learning must concern itself not with static entities called organizations, but with an active process of organizing which is, at root, a cognitive enterprise. Individual members are continually engaged in attempting to know the organization, and to know themselves in the context of the organization. At the same time, their continuing efforts to know and to test their knowledge represent the object of their inquiry. Organizing is reflexive inquiry.” (34: pp. 16–17)

A guiding assumption in organizational theory is the belief that research management practice may be systematically

143K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

improved to significantly advance organizational R&D perfor- mance with respect to accepted (and often imposed) policy measures. The major guiding assumptions are that research innovation productivity (return on research investment) can be achieved by improved management practice [35–40] and strategic management and planning [41]. Improving the management of management is a front-end activity with very high potential return on organizational investment [40,42–45]. However, the thrust of this stream of management research suggests that these improvements in practice are related to advancements in individual management practice or styles of management.

3.3. Multi-modal strategy in research management

Any strategic organizing method embodies an inherent theory of practice and a rationale for its expected effective- ness. Methods and practices aimed at improving research and innovation management are often revisited and refined (e.g. [46–49]) with the aim of improving the method for greater applicability. Practical methods are evaluated with respect to their effectiveness for a given application.

Structured planning methods that organize strategies, programs, and work packages [49–51] are primarily methods for implementation, not innovation. However, planning for innovation is more desirable in an R&D organization that, by definition, is not responsible for implementation. SDD introduces collaborative practices that enhance R&D innovation management by:

• Improving collaboration by engaging diverse perspectives in R&D program planning

• Improving the capacity to address increase in program complexity over time

• Coordinating interconnections with other projects in an investment portfolio

• Planning more frequently with multi-disciplinary teams for addressing uncertain environments

• Planning for management of programs facing rapid tech- nological change.

In conventional large organizations, strategic planning produces better outcomes when using both top-down and bottom-up planning modalities. The deliberate employment of complementary planning modalities, both top-down and bottom-up, is correlated with better strategic outcomes [41]. These modes of planning are not treated as independent methods, but are selectable as management “levers” for specific organizational purposes and can be integrated. Dialogic design applications are compatible with both top-down and bottom-up planning modalities and can complement either. SDD in this case was found to complement “traditional” command, symbolic and rational planning modes, the modes represented by AFRL's expert-led capabilities-based planning (and strategy-to-task doctrine). The command mode represents a top-down vision, leading to planning as translation and implementation of an overarch- ing strategic framework. A symbolic strategy aligns to normative organizational values and guiding principles established by leadership. Rational planning is based on the assumption that strategic options are best identified by expert definition and careful analysis, as supported by

environmental scanning, goal setting, mapping and evaluat- ing alternative actions, and the strong narrative of compre- hensive reports.

The generative and transactive planning modes are recommended for organizations where long-horizon strate- gies (10–20 years) require management to stake positions in the present to prepare for uncertain futures for both expected technological evolution futures and unexpected contingen- cies. Generative planning is a mode characterized by more creative and grassroots innovation within an organization. Generative strategies deal best with turbulent environments and rapid technological change. Transactive planning is an adaptive and participatory approach to strategy formulation as a process. It is most suitable for organizations where domain knowledge makes a significant difference, and multiple stakeholders must be consulted to understand complex and dynamic scenarios.

Hart [41] suggests rational planning will be more effective when combined with (complemented by) “distal” or dissim- ilar strategy modes. Any strategic method employed as the sole approach may lead to erroneous decisions due to cognitive and organizational biases of that mode and the gaps and blind spots of the approach. SDD was employed as a transactive mode in the current case, with its collaborative formation of scenarios and strong process model of engage- ment with stakeholders selected for their domain knowledge. In the AFRL case, SDD was designed for a collaborative foresight application, and most of the planning modes to some extent. It combined generative methods for eliciting unique personal knowledge, and sessions were organized to honor the cultural symbolism of the organization. However, it was primarily a complementary method to the traditional rational planning mode.

3.4. Complexity drives boundary spanning

Organizations are coming to recognize the problems they now face are increasingly complex. For example, U.S. cyberse- curity R&D priorities include the major research thrust of “Catalyzing integration across the game-changing R&D themes, cooperation between governmental and private-sector com- munities, collaboration across international borders, and strengthened linkages …” [52]. The interdependencies inherent in complex systems drive a need for boundary-spanning collaboration. Boundary-spanning research activities result in the social production of knowledge that can improve organiza- tional response to complexity [14].

More fundamentally, the production of knowledge is changing as the problems being addressed are now seen as more complex and therefore require more sophisticated means of interface across disciplines and functional areas in research. A key determinant of organizational innovation is the ability to maintain and properly manage a complex division of labor represented by requisite variety [53] across disciplines and perspectives.

3.5. Dialogic design science

The goals of SDD applied to collaborative foresight are oriented toward envisioning shared future outcomes and planning based on highly leveraged options. The process of

144 K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

developing a shared, consensus model of the foresight map and options is one of iterative dialogue and the exchange of individual representations (rationale or warrants [54]) that result in shared mental models. According to some organi- zational theorists [55–57] dialogue (reflexive conversation) is necessary for organizational learning and effective process management. In this context, dialogue is defined as “the participation of observers engaged in creating meaning, wisdom, and action through communication and collabora- tive interaction” [11].

SDD is premised upon dialogic design science, which defines a theoretical basis for a class of foresight, planning and systemic design methods employed in participatory decision making. This science is based on a taxonomy of distinctions derived from seven axioms and seven “laws of dialogue.” The axioms distinguish elemental functions that must be observed in any productive stakeholder dialogue, such as authentic engagement, respect of cognitive limita- tions and ensuring stakeholders are invested in the out- comes. The laws are drawn from seven bodies of published science and refer to related theories of effectiveness observed in SDD practice. Background in the theoretical basis of the science of dialogic design is referenced earlier [20–27].

3.6. Planning process effectiveness

A planning process can be evaluated according to its ability to express appropriate organizational actions that effectively cope with the complexity of expected future situations. Complexity is defined theoretically as a combina- tion of the diversity of perspectives, the number of distinctive ideas, and degree of interrelatedness [11]. Performance effectiveness is measured by operating according to predict- ed cost, ability to reach closure in a predicted timeframe, and rate of convergence toward strong agreement.

A proxy measure of coherence is the topological connect- edness of the observations through dialogue about their relationships. Strong agreement is defined as greater than two-thirds agreement without any strongly voiced disagree- ment and is measured by the degree of agreement in assessing the importance and influence of observations with respect to other observations. Comprehensiveness is judged by the degree of entailment of the baseline planning categories. Divergence from the baseline is taken to confirm a possible new insight. Group learning is measured by the shifting of individual preferences throughout the planning process as well as when individuals propose significantly different challenges or solutions from the baseline [58]. The degree of organizational learning can be assessed over time as the strength of memory of the narratives and scenarios developed in joint workshop exercises.

Collective decision making based on attempted consensus or explicit voting is fraught with subjective representation and cognitive bias. The well-known problems of groupthink [59,60] and lesser-known group cognitive biases of spread- think and clanthink [61] are pervasive in group decision making and yet unrecognized by participants. Groupthink is a phenomenon that significantly affects the individual and group self-perception that consensus has occurred. Spread- think refers to the measure of individual variability or differentiation across all members in a group session. Clanthink

refers to individuals aligning and voting with members of their immediate work group or discipline. In unstructured dialogue, high status individuals, managers and small voting blocs within large group sessions can influence group decisions and create the appearance of consensus.

4. Findings

This case study supports three claims: a) construction of robust stakeholder consensus using a collaborative foresight model, b) resolution of high leverage planning priorities, and c) efficient and effective collaborative design.

4.1. Robust consensus with collaborative foresight

In this case study, consensus agreement among the participants was produced over time and indicated in several artifacts: Prioritized maps of key challenge ideas, detailed group scenarios, and investment pathways. Collaboration effectiveness was assessed by demonstrable coherent agree- ment and the formation of group identity in a new or- ganization composed of formerly competing teams.

Consensus emerges as a result of shifts in preferences that are expressed throughout deliberation. Preference shifting was observed as participants selected ideas offered by others in preference to some of their own ideas. From an initial idea pool of 4.8 distinct challenges per person, the group converged over 120 cycles of pairwise (influence) voting to agree upon 5 challenges that were agreed to represent “deep drivers” of the foresight model [see Fig. 5]. This approach is in contrast with a single winner-take-all, up/down vote for a view constructed through unstructured deliberation.

In contrast, no such evidence is available for the CBP process. The SDD findings encompassed many of the same concerns as the CBP outcomes, but the following example reveals the significance of identifying deep drivers (leverag- ing ideas) in an influence map. SDD discovered several critical emerging ideas that were not identified within the CBP process or the planning boundaries articulated by program managers. For example, one high-leverage solution idea proposed was R&D into commercial or “COTS” technol- ogies. Team learning based on the convergence of preference and generation of new insights proceeded at a much faster pace than reported with the CBP process.

4.2. High-leverage strategic priorities

High leverage priorities are the challenges (problems) and solutions (options) agreed to as having the most influence within the network defined by the triggering question, mathematically determined in a reachability matrix [62]. Reachability is defined as the count of the number of arcs between two nodes in an acyclic graph structure. A structured pairwise comparison is conducted for a plurality of comparisons, as constrained by the transitive logic of the ISM algorithm. The Cognisystem software presents a series of pairwise comparisons of group-selected challenge or solution items. Participants commit agreement on the strength of an influence relationship between each item in a pair with a supermajority vote on the presence or absence of significant influence.

Fig. 5. Influence map of 20-year strategic challenges.

145K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

A directed graph is formed, displaying the highest- leverage priorities at the base of the map. This method resulted in the map shown in Fig. 5 (coded with numbers only to hide the statements, some of which are sensitive to public release). This map was reviewed by all participants as consistent with their voting patterns. The selected challenges were drawn through aggregated individual voting from a pool of 67 unique, clarified challenge ideas considered by the participants.

Five “deep driver” challenges (at levels VII and VIII in the map) are at the root of influences propagated throughout the system. Here the problem idea with the most leverage (at Level VIII) was “validation and verification of complex software systems”. Yet this problem idea entails core compe- tencies largely beyond the purview of the research organiza- tion defining the strategy. It was revealed as a strategic vulnerability of significant leverage, with significant agree- ment. Also, three challenges at Level VII are root level ideas

and exert similar influence in the network, as indicated by the connecting arcs.

Fig. 5 reveals strong agreement on influence propagated through a structure of 20 challenge ideas, selected from a field of 67 challenges. At Levels VII and VIII, we see five “deep driver” challenges that have significant leverage upon challenges at the upper levels of the map (i.e. lower Level numbers have less leverage and are higher in the structure).

Recommended strategic investments included the inte- gration of cyberspace and avionics spectrum electronic research to enable better sensing capability, improving formal methods, verification, validation, and cognitive sys- tems research to enable more trusted systems.

4.3. Cross-impact of challenges to solutions

An analysis of cross-impact [63] was performed by mapping an overlay of the connection of the solutions

146 K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

(in the Design influence map) to their corresponding challenges relationships in the Definition map. Four types of ideas were identified across strategic options: Solutions (blue), Challenges (pink), Goals (yellow tags) and Touchpoints (blue tags).

Five Goal statements (34, 55, 56, 61, 63) were identified as distinct from challenges, as these were judged to have restated mission or desired outcomes without identifying a specific challenge or problem. This situation was an under- standable consequence of the ambiguity designed into the triggering question and the decision to isolate the SDD planning team from ideas generated in the parallel CBP team. As goals are “highly influenced” by challenges and solutions, they typically become “driven” by the more influential statements toward the top layers of the influence map.

Four Touchpoint statements (1, 22, 54, 56) were initially proposed among the challenges and later were judged to represent another organization's remit. Two touchpoints required explicit interface with another organization for their accomplishment. Touchpoints indicated crucial insights, as they define problems requiring shared action or future collaboration between divisions or units. However, in the well-demarcated boundaries of the current case, touchpoints represented outliers that were in effect offloaded to another organization.

There were 2 Cycles (11, 4, 28 and 6, 1), defined as mutually-dependent ideas represented together as two or more statements within a single box in the graph. A cycle represents a reciprocal influence of the challenge ideas upon each other and a possibly combined influence on upstream (influenced) challenges. A cycle appearing in an influence map suggests the ideas are very closely related, or might best be managed under a single function.

4.4. Strategic pathways

Three strategic investment pathways were elicited from the cross-impact analysis, to illustrate possible strategy scenarios based on leverage among solutions to challenges. The members of each pathway were selected from the mapping of relation- ships of solutions (blue) to challenges (pink) as shown in Fig. 6. Strategic pathways were constructed as scenarios by partici- pant working in small groups to represent what they believed were ideal configurations.

Based upon reachability analysis (leverage of solutions to influence the network), the strategic pathway shown in Fig. 6 was considered a leading candidate (Cognitive Systems Research to Enable Trusted Systems).

The strategic pathway is based on the solution “Establish a deep root of trust at physical layer” — a solution statement that effectively enhances the two “cycles” it influences at the next stage. The first cycle included the ideas “Develop cognitive systems” and “Instrumentation to measure and control the spectrum”. This cycle addressed three challenges:

• Validation and verification of complex software systems, • Improve ability to handle exponentially increasing data streams and

• Develop more accurate sensors.

This sub-path was also represented in the second of three strategic pathways defined in the cross-impact analysis. The

solution cycle of “Advanced authentication and encryption techniques” and “Novel/strong authentication techniques” was judged to significantly advance the challenge related to “Trust of elements information is sent or received from”.

Pathway 3 had the highest reachability score (8) of the three pathways derived from the analysis. Reachability is a measure of the overall impact on the influence that extends through the system, and thus the pathway with the greatest reachability is a candidate for central consideration within research program design. The composition was evenly divided across solutions and challenges, and was judged as effectively addressing more challenges than the other pathways.

These analyses were made based on the scoring of reachability drawn directly from the matrices mapping the Solutions to Challenges in the cross-impact map. While these analyses were not based upon expert assessment of the underlying investment targets, they were considered credi- ble scenario alternatives based upon the participants' voting patterns in the workshops.

4.5. Solution scenarios and narratives

Summaries of findings were independently constructed in the form of scenarios by three diverse teams of participants. Two solutions were selected by all 3 teams, and five solutions were selected in common by 2 teams. Scenario teams chose only 11 solutions from the total 40. These scenario narratives might be considered the “people's choice” solution sets, as they indicated significant common adoption of 11 of the solutions for 3 different scenarios.

All three of the scenario teams selected the following solutions:

• Developing cognitive systems — filtering & prioritizing data before human eyes (a deep driver, highly influencing and highly leveraging) and

• Dynamic spectrum situational awareness (not a deep driver, highly influenced).

Two of the three scenario teams selected:

• Instrumentation to measure & control the spectrum/cyber requirements,

• Develop advanced authentication & encryption techniques for airborne systems,

• Novel/strong authentication techniques, • Improving & adapting standard hacker techniques and • Combine or integrate cyber/EW effects testing.

Comparison of the freely-selected scenario statements with the cross-impact mapping shows that 4 of the 7 were also selected in cross-impact mapping, further validating consensus in the application of these solutions in their scenarios. Reachability analysis shows the item Dynamic spectrum situational awareness was highly influenced (as decided by the participants in a series of supermajority votes on pairwise comparisons) and was discovered to be an outcome of the solution-challenge network (a goal) and not a deep driver.

Establish a deep root of trust at physical

layer

Advanced authentication techniques for airborne systems

IN CYCLE WITH Novel / strong authentication techniques

Develop cognitive systems

IN CYCLE WITH Instrumentation to measure & control the spectrum/cyber requirements

Develop more accurate sensors

Improve ability to handle exponentially

increasing data streams

V & V of complex software systems

Trust of elements information is sent or received from

Fig. 6. Strategic pathway 3: Cognitive systems research to enable trusted systems.

147K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

4.6. Analysis of influence vs. importance in collective judgment

Consensus can be achieved only through the evolution of a group's understanding through learning processes. In SDD, learning is indicated by preference shifting (and vote changing) in the dialogue. Empirical studies have shown that the typical group aggregated voting on ideas based on “importance” (e.g. dot voting) in stakeholder planning sessions results in “erroneous priorities.” The histogram (Fig. 7) illustrates the paradox known as the Erroneous Priorities Effect [63] and is invariably revealed when analyzing stakeholder behavior in SDD. Two types of error are understood: 1) identifying a statement as significant when it results in having no or only marginal influence on other challenges in a problem system, and 2) identifying a

Fig. 7. Change of initial votes from important to influential priorities.

statement as insignificant when it has deep (but often unintuitive) influence in the system.

Solutions included in the influence map by preference (dot) voting were plotted against the influence or “reachability” of each solution within the map for the Design workshop (SDD2). The initial “importance” votes are plotted by the line, showing here that 3 ideas with ultimately very little influence on the map were collectively voted as having the highest importance. Measures collected from the Cognisystem reveal that these solution statements (9, 10, and 22) had little or no reachability to other solutions. The plot also shows that three of the highest voted statements (3, 13, 14) were initially voted as reasonably important (4 dot votes), indicating congruence of the group in recognizing the systemic significance of these solutions.

4.7. Organizational learning in workshop process

Most of the participants (53%) in the collaborative foresight sessions had never interacted with each other previously in the course of their professional work. Through interactions within the workshops, new collaborative bridges were established between groups that formerly were in open competition for organizational resources. Collaboration was established in the process through in-depth dialogue engag- ing their authentic perspectives on shared challenges and solutions.

The workshops facilitated the creation of an autonomous environment away from the culture and context of the everyday work setting. This environment was explicitly created by the SDD facilitators to be conducive to the structured exploration of individual expert knowledge and to afford expression of deeply-held values and concerns. Facilitators carefully attended to room selection, lighting,

148 K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

seating, daily and hourly schedules, meals and breaks, and other physical and environmental requirements. SDD facili- tators also prepared and guided the social and relational aspects of collaborative engagement, from the seating order of individuals seated in the (U-shaped) table arrangements, to enhancing variety in breakout sessions by selecting new participation groups, to facilitating a balance of responses from all participants during structured dialogue.

Furthermore, the SDD process offered opportunities to promote decision making leadership by selecting participants with less seniority to lead scenario sessions. These individ- uals presented summary narratives of their scenario models, and were formally recognized in the workshop. These were the types of opportunities that would rarely be offered within traditional top-down CBP.

Post hoc evaluation of the strategic planning indicated that SDD significantly improved the definition and structur- ing of strategic challenges. The process enhanced organiza- tional understanding of potential strategic options and their interdependencies, enabled strong agreement for collabora- tive action, and facilitated coherent agreement and group identity in a newly designated organization composed of formerly competing branches in the case organization. These findings suggest that collaboration methods such as SDD significantly contribute to overcoming the management challenges of long-term strategic planning for complex and uncertain high-risk future requirements.

5. Discussion

Strategic foresight [64,65] and scenario planning [66] construct narratives that enable decision makers to evaluate potential strategic options in the face of probabilistic events bundled into meaningful scenario forms. Typical foresight scenarios are constructed to communicate possible future situations and outcomes for long-horizon planning. A collaborative foresight approach to planning enables man- agers to make decisions based on a collective formulation of the most relevant knowledge to inform the complex, emerging, and ambiguous situations characteristic of R&D strategy and long horizon planning.

While valuable as long-range thinking and decision pro- cesses, most foresight methods and futures-oriented planning introduce unrecognized cognitive and group biases, often deeply embedded in collective assumptions. A primary source of bias is the reliance on organizational leaders, recognized technical advisors, and professional planning advisors to produce future narratives. Group pathologies such as group- think and “clanthink,” the “garden path” confirmation bias, and professional disciplinary biases are rarely indicated as process quality concerns. Managers and technology advisors are unlikely to identify their own involvement as biasing, and planners may not understand how to select participants to achieve requisite variety in a strategic or planning context. Several qualities of SDD used for collaborative foresight directly address these group process concerns.

5.1. Complementarity of planning modalities

Planning current strategies to meet the highly uncertain requirements for a 20-year R&D management timeline

presents a wicked problem involving changing societal trends, technologies, defense capabilities, and political de- mands. The demands of high uncertainty and a complex operational environment require sufficient cognitive variety to address at least the identified components of systemic complexity. Planning teams should avoid reductionism when facing critical uncertainties, as the case when a single, dominant narrative becomes extrapolated into the future or built into planning doctrine.

R&D strategy encounters cognitive process and knowl- edge limits when reliance is solely on management and their advisors to inform planning. Rational and command strategy modes in particular explicitly draw from top-down opinion, such as the more senior technology advisors guiding the CBP planning in the present case. These modes avoid inclusion of bench-level and outside stakeholders representing different “future users” invested in the problems identified as chal- lenges or solutions to a strategy.

As understood from tacit knowledge theory, individuals have a limited capacity to express intuitive knowledge or deep expertise in explicit terms. Dialogic approaches such as SDD tap into both individual expertise and deeply held values of tacit knowledge, which are not typically surfaced in rational planning. The action case study suggests that collaborative and structured dialogic methods capture di- verse perspectives and promote coherent group reasoning. The structured collaborative method of SDD delivers an integrated strategic map of influence relationships. The influence map not only preserves the collectively highest- valued participant knowledge, but associates strategic knowledge within an influence map of relative leverage. The collaborative foresight application of SDD generates multiple strategic roadmaps from different perspectives, contributing several plausible and integrated strategies to aid organizational decision making.

5.2. Planning process impacts

An organization's strategic response to future require- ments is essentially a social project based on individual judgment, group sensemaking, and individual persuasion. These social factors are well controlled within rational and command planning. Managers may consider collaborative planning approaches risky. Planners relinquish some control by sharing strategy-making responsibility with bench-level staff who may not hold corresponding institutional authority. Bottom-up, collaborative processes typically do not employ the sophisticated planning models available to strategy advisors, and this lack of professional appearance also tends to cast bottom-up planning as an inferior approach.

As a systemic methodology, SDD contributes formal process and theory to bottom-up collaborative planning. A thoroughly evaluated set of methods are facilitated within participatory workshops. Even if the perspectives of bench-level planning participants appear incommensurable, the team's transactions are communicated via multiple artifacts that effectively transfer a coherent representation of the bottom-up team's idea map to top-down strategic planners [11,63].

Combining collaborative, bottom-up and rational, top-down modes can be unsettling for precisely the reasons that call for their integration. Problem definition is critical, as any planning

149K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

(model building) is both constructive and destructive. When- ever a model of the world is constructed, that model adoption precludes and thereby destroys the adoption of other potential alternatives [67]. A highly complex problem viewed from a single reduced perspective may appear, not complex, but merely manageably complicated, and thereby addressable by traditional modes of rational planning.

Rather than replacing traditional rational modes with collaborative or dialogic modes entirely, we advocate com- plementarity and integration between bottom-up collabora- tive planning (and foresight) with hierarchically-driven, top-down rational planning.

5.3. Effective collaboration and consensus

This case illustrates collaboration in a complex foresight activity across multiple groups in a government R&D organization. The comparison of complexity addressed between the collaborative (SDD) and rational (CBP) modes falls then to the number of distinct ideas addressed and the explicit degree of interrelatedness considered. The rational mode produced a deeper, more nuanced strategic view among a smaller set of key ideas, with less emphasis on the interrelatedness of challenges and solutions. In a roughly equivalent time, but with twice the participants, the collaborative foresight process addressed a much greater structural and temporal complexity of ideas in a long-horizon planning scenario.

SDD's unique contribution is achieving agreement on collective decisions by integrating all individual contribu- tions. This enables an increase in requisite variety of diverse participants that improves the group's synergy while reduc- ing the need for compromise. SDD facilitates the co-creation of consensus by associating individual ideas within an influence map of their relationships, allowing decisions to be made based on their leverage toward desired effects. Quantitative measures are continually collected to reveal when participants strongly agree. The structured facilitation of SDD also diminishes group herding effects, avoided by the large number of incremental supermajority votes, each of which are focused only on a localized comparison to determine their influence.

6. Conclusions

6.1. Collaborative reasoning for critical uncertainty

We have reported on the application of SDD for collabo- rative foresight to enhance the quality of long-horizon strategy to facilitate management decision making for R&D investment planning. The methodology underpinning this approach has a 35-year history of applications in strategic management and large organizational decision making [11,58]. It also has an equally long history of civil and emancipatory applications employing the same process technology and facilitation model. SDD has been adopted for indigenous civil society development and international peacemaking negotiations, for applications ranging from disease elimination (World Health Organization) to demo- cratic development of educational systems. The large orga- nizational context is perhaps less commonly reported in the

literature, but provides an important domain of application for this evolving methodology.

The applicability of SDD to AFRL's interest in long-horizon foresight draws on its power to surface and test overlooked assumptions and constraints, as well as its primary purpose of identifying effective and novel solution ideas. It facilitates organizational assessment of critical uncertainties in the long-term that may be surfaced by bench-level organization- al actors with special understanding.

However, this collaborative foresight application of SDD (as any planning process) is limited by the framing of its purpose and the boundaries of the reference system by which participants are selected. The triggering question itself is framed and determined by planners a priori and not by participants. The ability of a bench-level team to expand the dialogue beyond this frame is limited by the planner's scope. Therefore a single series of SDD engagements will not always suffice to emancipate the reasoning and normative concerns of organizational participants if they breach the boundary conditions of the frame. The game-changing nature of such a violation of assumptions may be of great value in strategic foresight and can be disruptive to the organization [70].

The larger issue of the applicability of complementary planning modes in large R&D organizations is beyond the scope of this case study. The determination of the research framing for long-horizon planning remains in the scope of management and its responsibility to propose appropriate allocations for national-scale research budgets.

SDD as collaborative foresight enables a larger proportion and variety of organizational actors to participate in ideation, reasoning and structured planning. It visualizes planning outputs in a structural influence map revealing systemic relationships, not merely a list of planning priorities. But SDD has real organizational limits: as a planning modality, it is not sufficient to cause changes in organizational culture nor will mere use of this planning method redefine management's stance with respect to inter-organizational power and authority structures.

On the other hand, management-led organizational change will be well-supported by complementary planning using collaborative foresight application of SDD, especially in facing the complexity of large hierarchical R&D organizations. In large hierarchies, a top-down change initiative generally leads restructuring the organization to better fit foreseeable political and economic shifts that portend a larger change of fortunes on which the very future of the planning process hinges. The possibility of significant external change – sociopolitical, geopolitical, and global economic changes – may be introduced into top-down planning scenarios. However, in our observations the capacity of large organiza- tions to respond to these external trends is limited by the prevailing management perspective. Bottom-up planning if given mandate and scope may improve management's ability to respond to societal change.

6.2. Modalities of planning and foresight

Building on Hart's typology of strategy-making [41], Brews and Purohit's [68] four modalities of planning – rational, transactive, generative and symbolic – indicate that higher performance is correlated with increased use of

150 K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

generative and transactive modes. SDD was classified in this organizational context as transactive and generative planning, based on the affirmative approach to inclusive recruiting and the participatory planning process, and the intentional selection of participants to reflect diversity of expertise and roles. Generative planning was evidenced by the formulation of novel concepts not described in the traditional planning attributes and themes [41].

Capabilities-based, top-down, rational planning appears complementary with generative/transactive, bottom-up plan- ning. Since the “bottom” of a hierarchy is necessarily wider than the top as noted by Warfield and Espejo, any approach that seeks wider interaction with more diverse stakeholders necessarily results in pulling from lower in the organiza- tional hierarchy [69,70]. Given Brews and Purohit's conclu- sion that productivity may increase more for firms that use transactive and generative planning and the inability of CBP to carry out these modes of planning, some need in this case appears to exist for a complementary, non-top-down planning approach that can carry out the generative and transactive functions. SDD is viable as one such approach to improve organizational productivity, assuming that this aspect of firm theory can be generally applied to government organizations.

Symbolic planning for large R&D organizations should be recognized as a planning mode for management direction that ensures alignment to strategy, following Strategy-to- Task doctrine. Rational planning practices are employed to negotiate the structural and budget requirements for short and long-term investment scenarios. However, rational planning employed without complementary bottom-up practices yields brittle, “top heavy” plans. Rational plans provided as sole guidance are insufficient for the higher technical and organizational performance (associated, not causal) necessary in unstable and complex environments.

Conducting collaborative, transactive and generative plan- ning activities in a rational and symbolic planning organization will fail to change organizational practices if there is no expectation of implementation from the bottom-up planning effort. Our case shows the breakdown where meaning creation and production in the collaborative planning team could only create meaning at the micro-level but it could not produce that new meaning socially in the wider organizational culture [70].

6.3. Theoretical and practical implications for managers of R&D organizations

Improved R&D planning benefits from planning methods that are transparent and actionable. While collaborative visioning and dialogic practices are effective for raising and validating important ideas, many such approaches reportedly fail to converge on a central concern for actionable decision making by those who seek to improve planning theory and practice [71,72]. In this case the collaborative planning resulted in empirical evidence for consensus on a strong set of inter-related strategic options that both included and surpassed the span of recommendations generated by the rational CBP team.

The significant improvement of planning represents a highly visible management achievement (e.g. [73–77,37]). By mandate and by nature of discipline, managers respond to

uncertainty by restoring stability and integrity to organiza- tional systems of which they have control. By raising an entire, highly visible, planning track, not anticipated by the rational planning team, the SDD planning process may have created more uncertainty from management's perspective and thus pressure to either bring it into alignment or suppress it. By fostering independence of the collaborative and rational planning teams, in order to avoid groupthink, their work products were positioned organizationally for a potential conflict between their results. Since the CBP and SDD teams were not to be combined into a common dialogue, the results of the ‘experimental’ collaborative team did not confront the dominant paradigm. The traditional CBP process expended a great deal of time on the overall form and categories of their recommendations which were dissemi- nated as part of the existing process. In comparison the results of collaborative foresight tersely expressed the complexity of the SDD team's perspective raised distinct ideas resulting from their independent dialogue.

Comparison with a parallel strategic planning and fore- sight initiative validated that the SDD methodology achieved a superior planning product with wider organizational consensus. The novel outputs of this collaborative foresight planning process established transactive and generative planning rationale for the strategic management of invest- ments and activities in a critical new organization in a larger R&D enterprise.

Acknowledgments

This work is approved for public release by the Air Force, Approved Date: 2012-08-03, PA Approval Number: 88ABW-2012-4261. The authors would like to express their sincere gratitude to the Air Force Research Laboratory for sponsorship of this work. The Air Force Research Laboratory makes no claims as to the quality of this work and did not influence the interpretation of the findings of this study. The authors assume full responsibility for the content of this article.

The authors would like to acknowledge Dr. Alexander Christakis, who provided invaluable support in the design and formulation of the originating project for this research. We also thank Jeff Diedrich, member of the dialogue management team, who was instrumental in the facilitation and analysis.

References

[1] L. Lewis, C.R. Roll, Strategy-to-Tasks: A Methodology for Resource Allocation and Management, RAND Technical Report P-7839, 1993.

[2] A. Bahrani, J. Yuan, C.D. Emele, D. Masato, T.J. Norman, D. Mott, Collaborative and Context-aware Planning, Proc. of the Military Communications Conference, 2008. 1–7.

[3] R. Popper, How are foresight methods selected? Foresight 10 (6) (2008) 62–89.

[4] G. Ringland, The role of scenarios in strategic foresight, Technol. Forecast. Soc. Chang. 77 (2010) 1493–1498.

[5] I. Miles, The development of technology foresight: a review, Technol. Forecast. Soc. Chang. 77 (2010) 1448–1456.

[6] C.A. Varum, C. Melo, Directions in scenario planning literature — a review of the past decade, Futures 42 (2010) 355–369.

[7] A. Wiek, A.I. Walter, A transdisciplinary approach for formalized integrated planning and decision-making in complex systems, Eur. J. Oper. Res. 197 (2009) 360–370.

151K. Weigand et al. / Technological Forecasting & Social Change 85 (2014) 134–152

[8] P. Healey, Consensus-building across difficult divides: new approaches to collaborative strategy making, Plan. Pract. Res. 11 (1996) 207–216.

[9] J. Friedmann, Toward a non-Euclidian mode of planning, J. Am. Plan. Assoc. 59 (1993) 482–485.

[10] S. Janoff, M. Weisbord, Three perspectives on future search: meeting design, theory of facilitating, global change strategy, Scand. J. Organ. Psychol. (2003) 13.

[11] A.N. Christakis, K.C. Bausch, How People Harness Their Collective Wisdom and Power: To Construct the Future in Co-Laboratories of Democracy, Information Age Publishing, Greenwich, CT, 2006.

[12] J.N. Warfield, A Science of Generic Design, Iowa State University Press, Ames, IA, 1994.

[13] R.L. Baskerville, Investigating information systems with action re- search, Commun. AIS 2 (3) (1999)(Article 4).

[14] C. Argyris, D. Schön, Participatory action research and action science compared, in: W.F. Whyte (Ed.), Participatory Action Research, Sage, Newbury Park, NJ, 1991.

[15] K. Braa, R. Vidgen, Interpretation, intervention, and reduction in the organizational laboratory: a framework for in-context information system research, Accting. Mgmt. Info. Tech. 9 (1999) 25–47.

[16] A. Gregory, Systems thinking for strategic development, Proceedings of the International Society of Systems Sciences 2010, Waterloo, ON Canada, July 18–23, 2010.

[17] Action and Interactive Research — Beyond Practice and Theory, in: K.A. Nielsen, L. Svensson (Eds.), Shaker Publishing, 2006.

[18] N. Adler, M. Emquist, F. Norrgren, The challenge of managing boundary-spanning research activities: experiences from the Swedish context, Res. Policy 38 (2009) 1136–1149.

[19] T. Tagarev, Capabilities-based planning for security sector transforma- tion, Inf. Secur. 24 (2009) 27–35.

[20] J.N. Warfield, The domain of science model: evolution and design, Proc. 30th Annual Meeting, Society for General Systems Research, Intersys- tems, Salinas, CA, 1986, pp. H46–H59.

[21] G.G. Lendaris, On the human aspects in structural modeling, Technol. Forecast. Soc. Chang. 14 (1979) 329–351.

[22] R. Fitz, J. Troha, Interpretive structural modeling and urban planning, Proceedings, 1977 international Conference on Cybernetics and Society, IEEE-SMCS, October 1977.

[23] A.N. Christakis, L. Harris, Designing a transnational indigenous leaders interaction in the context of globalization: a wisdom of the people forum, Syst. Res. Behav. Sci. 21 (2004) 251–261.

[24] H.C. Alberts, Redesigning the United States Defense Acquisition System, Ph.D. Dissertation Department of Systems Science, The City University, London, UK, 1995.

[25] A.N. Christakis, The national forum on non-industrial private forest lands, Syst. Res. 2 (1985) 189–199.

[26] J.N. Warfield, A.R. Cardenas, A Handbook of Interactive Management, Iowa State University Press, Ames, IA, 1994.

[27] H. Özbekhan, On some of the fundamental problems in planning, Technol. Forecast. 1 (1970) 235–240.

[28] A.L. Delbecq, A.H. Van de Ven, Nominal group techniques for involving clients and resource experts in program planning, Acad. Manag. Proc. (1970) 208–227.

[29] W.R. Ashby, Requisite variety and its implications for the control of complex systems, Cybernetica 1 (2) (1958) 83–99.

[30] J. Fuller, Refining virtual co-creation from a consumer perspective, Calif. Manag. Rev. 52 (2) (2010) 98–122.

[31] C.K. Prahalad, The blinders of dominant logic, Long Range Plan. 37 (2) (2004) 171–179.

[32] S.L. Vargo, P.P. Maglio, M.A. Akaka, On value and value co-creation: a service systems and service logic perspective, Eur. Manag. J. 26 (2008) 145–152.

[33] K.E. Weick, K.M. Sutcliffe, D. Obstfeld, Organizing and the process of sensemaking, Organ. Sci. 16 (4) (2005) 409–421.

[34] C. Argyris, D. Schön, Organizational Learning: A Theory of Action Perspective, McGraw-Hill, New York, 1978.

[35] D.G. Reinertsen, Taking the fuzziness out of the fuzzy front end research, Technol. Manag. 42 (1999) 25 31–25.

[36] G.A. Stevens, J. Burly, Piloting the rocket of radical innovation, Res. Technol. Manag. 46 (2) (2003) 16–25.

[37] B. Verworn, C. Herstatt, A. Nagahira, The fuzzy front end of Japanese new product development projects: impact on success and differences between incremental and radical projects, R&D Management 38 (1) (2008) 1–19.

[38] K. Clark, C.S. Wheelwright, Leading Product Development, Free Press, 1995.

[39] A. Cottam, J. Ensor, C. Band, A benchmark study of strategic commitment to innovation, Eur. J. Innov. Manag. 4 (2001) 88–94.

[40] R.E. Morgan, P. Berthon, Market orientation, generative learning, innovation strategy and business performance inter-relationships in bioscience firms, J. Manag. Stud. 45 (8) (2008) 1329–1353.

[41] S.L. Hart, An integrative framework for strategy-making processes, Acad. Manag. Rev. 17 (2) (1992) 327–351.

[42] D. Nobelius, L. Trygg, Stop chasing the front end process — management of the early phases in product development projects, Int. J. Proj. Manag. 20 (2002) 331–340.

[43] M.K. Perttula, Implications on cultural and formal processes of the front-end of new product development, Proceedings of 2nd World Conference on Production and Operations Management, Cancun, Mexico, April 30th–May 3rd, 2004.

[44] M.A. Williams, A.K. Kochbar, C. Tennant, An object-oriented reference model of the fuzzy front end of the new product introduction process, Int. J. Adv. Manuf. Technol. 34 (7–8) (2007).

[45] M. Backman, S. Börjesson, S. Setterberg, Working with concepts in the fuzzy front end: exploring the context for innovation for different types of concepts at Volvo Cars, R&D Management 37 (1) (2007) 17–28.

[46] A.L. Page, Assessing new product development practices and perfor- mance: establishing crucial norms, J. Prod. Innov. Manag. 10 (1993) 273–287.

[47] A. Oke, Making it happen: how to improve innovative capability in a service company, J. Chang. Manag. 2 (3) (2002) 272–281.

[48] R. Adams, J. Bessant, R. Phelps, Innovation management measurement: a review, Int. J. Manag. Rev. 8 (1) (2006) 21–47.

[49] J.I. Igartua, J.A. Garrigós, J.L. Hervas-Oliver, How innovation manage- ment techniques support an open innovation strategy, Res. Technol. Manag. (May-June 2010) 41–52.

[50] R.G. Cooper, E.L. Kleinschmidt, Winning businesses in product develop- ment: the critical success factors, Res. Technol. Manag. (July-August 1996) 18–29.

[51] R.G. Cooper, From experience: the invisible success factors in product innovation, J. Prod. Innov. Manag. 16 (1999) 115–133.

[52] Executive Office of the President, National Science and Technology Council, Trustworthy Cyberspace: Strategic Plan for the Federal Cybersecurity Research and Development Program, , December 2011.

[53] J. Hage, G. Jordan, J. Mote, Y. Whitestone, Designing and facilitating collaboration in R&D: a case study, J. Eng. Technol. Manage. 25 (2008) 256–268.

[54] S.E. Toulmin, The Uses of Argument, 2nd ed. Cambridge University Press, Cambridge UK, 2003.

[55] E. Schein, On dialogue, culture, and organizational learning, Organ. Dyn. 22 (2) (1993) 40–51.

[56] M.R. Weisbord, Productive Workplaces Revisited: Dignity, Meaning, and Community in the 21st Century, Jossey-Bass/Wiley, New York, 2004.

[57] D.A. Schön, The Reflective Practitioner: How Professionals Think in Action, Basic Books, New York, 1983.

[58] A.N. Christakis, K.M.C. Dye, Collaboration through communicative action: resolving the systems dilemma through the CogniScope™, Systems, J. Transdiscipl. Syst. Sci. 4 (1) (1999) 9–32.

[59] I. Janis, L. Mann, Decision Making: A Psychological Analysis of Conflict, Choice and Commitment, The Free Press, New York, 1977.

[60] J.N. Warfield, Spreadthink: explaining ineffective groups, Syst. Res. Behav. Sci. 12 (1) (1995) 5–14.

[61] J.N. Warfield, C. Teigen, Groupthink, clanthink, spreadthink, and linkthink: decision-making on complex issues in organizations, Technical Report, 31, Institute for Advanced Study of the Integrative Sciences, George Mason University, 1993, pp. 4–5.

[62] J.N. Warfield, Societal Systems: Planning, Policy, and Complexity, Wiley Interscience, New York, 1976.

[63] A.N. Christakis, K. Dye, The Cogniscope™, Lessons learned in the arena, in: M. Jenlink, B.H. Banathy (Eds.), Dialogue as a Collective Means of Design Conversation, Springer, 2008, pp. 187–204.

[64] R. Nelson, Extending foresight: the case for and nature of Foresight 2.0, Futures 42 (2010) 282–294.

[65] T. Heger, R. Rohrbeck, Strategic foresight for collaborative exploration of new business fields, Technol. Forecast. Soc. Chang. 79 (5) (2012) 819–831.

[66] J. Bootz, Strategic foresight and organizational learning: a survey and critical analysis, Technol. Forecast. Soc. Chang. 77 (2010) 1588–1594.

[67] A.H. Whitehead, Process and Reality, The Free Press, New York, 1978. [68] P. Brews, D. Purohit, Strategic planning in unstable environments, Long

Range Plan. 40 (2007) 64–83. [69] J.N. Warfield, Measuring complexity, Technical Report, Integrative

Sciences, Inc., 2001 [70] R. Espejo, The footprint of complexity: the embodiment of social

systems, Kybernetes 33 (3/4) (2004) 671–700. [71] T.M. Amabile, R. Conti, H. Coon, J. Lazenby, M. Herron, Assessing the

work environment for creativity, Acad. Manag. J. 39 (5) (1996) 1154–1185.

[72] W.V. Dyck, P.M. Allen, Pharmaceutical discovery as a complex system of decisions: the case of frontloaded experimentation, E:CO 8 (3) (2006) 40–56.

International of Dayton, Ohio.

152 K. Weigand et al. / Technological Forecasting

[73] D. Dougherty, L. Borrelli, K. Munir, A. O'Sullivan, Systems of organiza- tional sensemaking for sustained product innovation, J. Eng. Technol. Manage. 17 (2000) 321–335.

[74] L. Dwyer, R. Mellor, Organisational environment, new product process activities, and project outcomes, J. Prod. Innov. Manag. 8 (1) (1991) 39–48.

[75] K. Atuahene-Gima, An exploratory analysis of the impact of market orientation on new product performance: a contingency approach, J. Prod. Innov. Manag. 12 (4) (1995) 275–293.

[76] A.J. Shenhar, A. Tishler, D. Dvir, et al., Refining the search for project success factors: a multivariate, typological approach, R&D Manage- ment 32 (2) (2002) 111–126.

[77] S.E. Reid, U. Brentani, The fuzzy front end of new product development for discontinuous innovations: a theoretical model, J. Prod. Innov. Manag. 21 (2004) 170–184.

Kirk Weigand, Ph.D. is a collaborative systems engineer for the Air Force Research Laboratory. His main interest is in collaborative, human-machine, mixed-initiative decision support as applied to net-centric sensor systems. Dr. Weigand led research of interdisciplinary dialogue workshops using organization development and design methodologies. He is also seeking to advance sensemaking research through application of process philosophy and artificial intelligence to improve mission assurance and trust of complex defense systems.

Thomas Flanagan has a Ph.D. from Wesleyan University and an MBA in the Management of Technology from the Sloan School of Management at MIT. He is President of The Institute for 21st Century Agoras, the international nonprofit organization for the application of Structured Dialogic Design, and he runs a regional SDD practice center based in Rhode Island and Massachusetts.

Kevin M.C. Dye co-founded two companies in Decision Support Systems and relaunched a consultancy focusing on inter-organizational collaborative planning. At United Technologies Research Center he led a process redesign team for the Advanced Technology Program's Rapid Response Manufactur- ing Consortium in which he was introduced to the practice of Interactive Management (a predecessor to Structured Dialogic Design). He graduated as the Ingersoll Rand Award's Senior in Mechanical Engineering at Northeast- ern University and was a Sloan Visiting Fellow at MIT.

Peter Jones, Ph.D. is an associate professor at OCAD University, Toronto, and is a senior fellow of the Strategic Innovation Lab. He teaches in the Strategic Foresight and Innovation MDes program, and leads research and system design in knowledge-based organizational strategy, knowledge practices in professional work, and information significance in collaborative sense- making. He conducted this study as a principal consultant of Dialogic Design

& Social Change 85 (2014) 134–152