Design and application for a replicable foresight methodology bridging quantitative and qualitative expert data

Abstract

Producing foresight knowledge frequently requires both evidence and creativity based expertise. This paper offers assessment of an electronic expert Group Support System (named «E-lab») facilitating efficient foresight knowledge production by using and coalescing a series of quantitative and qualitative methods and techniques, challenging the need for better understanding of foresight methodology assumptions and options. Observations and assessments of the E-lab methodology have been made in several expert workshops conducted both at the regional, national and European level over a ten years’ period (2003–2013) and across a series of foresight themes; e.g. European hydrogen society, Nordic addiction policy, National knowledge policy, Images of addiction and lifestyles in Europe, and University future governance. This methodology research adds to the body of knowledge on the effects of virtual group meetings encompassing a mixed method information platform often generated and shared in expert foresight exercises.

Introduction

Foresights - rigorous imagining is the driver

Participative expert foresights are intriguing and captivating, since they capitalise on intellectually virtuous thought processes about the future [1]. The intention is neither prediction nor falsification; rather, it is about crafting robust images of the future built on the perceptual and creative capacity of the experts. Nevertheless, every imagining starts in the present and most often includes considerations of the past. Thus, the baseline is grounded on factual observations of what we habitually call evidence-based ‘conditional facts’. Furthermore, signs of forerunners of the future may be spotted. Accordingly, both qualitative and quantitative data help experts relate to the imagining of domain futures.

However, the need is prominent to optimise the time budget and the number of experts in such foresights. Presumably, we involve domain experts, since they hold explicit and implicit knowledge conducive both to assess the evidence of a present baseline and to elaborate on future images and their trajectories. In doing so, experts attempt to create, analyse, combine and apply both numbers and narratives.

Arguably, the integration of qualitative and quantitative information is a major endeavour in foresight analysis. Both approaches have their assets. Qualitative approaches can have a richness that is not assured by quantitative methods. They can explore relationships and trends for which little or no numerical data is available, including shocks and discontinuities; they can more easily incorporate motivations, values, and behaviour; they can create similes that capture the imagination of those for whom they are intended. Quantitative foresights, when done properly, provide a rigour, precision, and consistency that comes from their numerical and mathematical underpinnings. Their ontological and epistemological suppositions are made explicit and are, thus, open for critical analysis [1].

«Futures literacy», a concept introduced by Miller [2], calls for a powerful blend of methods and ways of inventing imaginary futures, including the development of hybrid approaches. Miller claims that the foresight process must be designed using a threefold framework that pays equal attention to a narrative capacity, a collective intelligence and a capacity to reframe. It is urgent to take full account of the insights that may be gained through the inclusion of interpretive qualitative approaches when doing quantitative foresight exercises, because the latter may tend to underestimate the contribution and need for triangulated approaches in practice and vice versa. Also, rigorous imagining about the future may hint at the quantising of qualitative data to benefit from the quantitative rigour when building scenarios, roadmaps or other foresight deliverables.

From such a hinterland, this article first discusses the scientific justification for a hybrid foresight methodology applying both qualitative and quantitative expert data, how to organise an electronic support platform for such an endeavour and its practical benefits. Then the philosophical frameworks that delineate assumptions about the paradigmatic approach related to the customary foresight toolbox are briefly discussed. Subsequently, two pillars of an expert foresight methodology – the Nominal Group Technique (NGT) and the Group Support Systems (GSS) - are discussed, linking it to the conversion of expertise into novel knowledge and data about the domain futures. Finally, lessons learned for design and application of an expert-based foresight methodology are discussed.

Modus operandi of foresights

In ‘Foresight for Europe’, a high level expert group (HLEG) defined foresights as a [3]:

… systematic, participatory, future intelligence gathering and medium-to-long-term vision-building process aimed at present-day decisions and mobilising joint actions.

Anchored in this definition, the HLEG launched a conceptual triad of foresight activities, called ‘thinking, debating and shaping the future’. Expanding on these core concepts, we may assume that thinking the future is the cognitive dimension of foresight. It is about envisioning possible futures and creating general awareness of trends and future issues by building scenarios, forecasting, technology assessment and different types of future studies. Debating the future is the normative or value-based dimension of foresight. It is about open discussion between the different stakeholders in the form of panels, for example, in order to create a shared understanding. Shaping the future is the action-oriented or pragmatic dimension of foresight. This dimension is about defining strategies and making real joint actions happen by feeding foresight results into decision making, but it may also help participants themselves to develop or adjust their own strategies.

Godet [4] reasons along the same lines when justifying his ‘Prospective’ configuration, referring to the ‘Greek triangle’ as the pathway from ‘anticipation to action through appropriation’. Anticipation encompasses a global vision of the domain environment: past, present, and future. Key figures and information, calibrated against uncertainty, step changes and possible chocks must be sufficiently synthesised at this stage. Next, the appropriation, i.e., the collective mobilisation and buy-in of core players in an open dialogue, intends to result in unequivocal images of domain futures. Finally, action must follow, at first in the form of a plan having milestones and measures, later as implemented actions to socially construct the future. The triangle must be balanced to function properly. Anticipation may imply the use of experts; appropriation relies on vested stakeholders and action on the will and power to act purposely to shape the future.

In the context of expert group foresights, all three aspects mentioned by Godet and the HLEG are prominent: thinking creatively, debating constructively and shaping plausible futures.

A support platform for expert foresights

What I talk about when I talk about foresight methodology

It has become quite conventional to use the terms «method» and «methodology» as synonyms. However, there are convincing philosophical reasons for distinguishing the two. Method connotes a systematic and repeatable way of doing something - a procedure. Methodology connotes a discourse about methods, i.e., an argumentation about the adequacy and appropriateness of particular combinations of research principles and procedures. As such, a methodology is methods of knowing, it reveals both ontological and epistemological implications, and it does not come without its own kind of philosophical baggage.

Foresights have derived their current, often implicit, methodologies from a range of disciplines and practices, embracing a variety of episteme, and as they increase world-wide, they produce a massive volume of information and data on the future. Therefore, it is a need to develop ways to integrate qualitative and quantitative approaches that may decrease the gap between the complexity, ambiguity and uncertainty of the foresight data presented and the analytical tools used to address them.

Ideally, we look for the in-depth, contextualised, and natural but more time-consuming insights of qualitative foresight research, coupled with the more time-efficient but less rich or compelling predictive power of quantitative foresights. Such a hybrid approach is far more comprehensive than attacking the challenge of future imaging from only one point of view. Emergence of strategies and tools for blending quantitative and qualitative methods and types of data allow for the crossing of disciplinary boundaries like never before. This may favourably support the strengthening of futures literacy.

The Nominal group technique

Expert group work that involves consensus from a range of alternatives is frequently laborious, inefficient, and frustrating. Consensus methods are a means of dealing with conflicting scientific evidence. They allow not only a wider range of study types to be considered than is usual in statistical reviews but also a greater role for the qualitative assessment of evidence. The three best-known consensus methods are the Delphi process, the Consensus Development Conference and the Nominal Group Technique (NGT), the latter also known as the «Expert panel». The consensus method is a structured method of group decision-making that allows a rich generation of original ideas, balanced participation of all members of the small group, and a rank-ordered set of decisions based on a mathematical voting method. Potter et al. [5] hold NGT to a have a solid advantage over group decision- making methods like Delphi, Focus groups and Brainstorming on a series of attributes, inter alia, generating an abundance of ideas in a short time, encouraging equal contributions from all participants and measuring the relative importance of ideas generated.

The NGT expert panel uses a highly structured meeting to gather information from relevant experts (usually 9–12 in number) about a given issue. A nominal group meeting is facilitated either by an expert on the topic or a credible non-expert and is structured to elicit the expertise in the most productive way and in its most appropriate form. The technique is similar in some ways to «brainstorming», but its structure calls for all participants to take an active part in the process. It also requires participants to use their brains and mental capability in a series of different ways, i.e., to individually generate new ideas, to silently assess the ideas of others, to jointly examine the implications of new ideas with others, and to formally evaluate a series of options. The results of using this technique are, typically, semi-quantitative or a hybrid. The NGT is often used to uncover new business opportunities, to identify new solutions to old problems, or to project future developments^1,2.

Group support systems

Group support systems (GSS) may be a way to help us out. These are collaborative software tools that can be used to focus and structure a team’s deliberations, while minimizing costs and distractions among teams working collaboratively. Several advantages are present, such as anonymity, parallel communication, automated record keeping, more structure, and increased productivity [8]. In expert group foresights, the exercises most often take place at the same time and place, not necessarily excluding face-to-face interaction, but mostly being virtual by means of computer-mediated communications to support the group’s work. To be useful to expert foresights, GSS software must offer the experts a range of applications, from creative techniques to categorising to ranking and scoring alternatives. A GSS must, de facto, have an architecture that could include all methods and techniques usually mentioned as foresight tools [9–11].

Generally, GSS consists of a set of hardware, software, language components, and procedures that support the participants’ knowledge creation processes [12], and they are categorised within a time-place paradigm. In most expert foresights, time and number of experts are urgent factors. The participative process is under time constraints to be completed, thus pressing to minimize cost and obtaining the data as quickly as possible. Small sized, short and intensive workshops assisted by electronic group support systems are often deemed appropriate. The challenge is to compose the GSS in such a way that it supports the knowledge creation and the consensus building of the foresight expert group, while optimising the input of time and expertise resources.

A GSS named «E-lab»

The «E-lab»³ electronic laboratory is a group support system for efficient expert meetings and workshops, initially developed and tested during the early years of the 2000s. Arguably well suited for foresight expert exercises, it is based on a standard GSS platform combined with the design of the Nominal Group Technique (NGT). It consists of a methodology database and a set of laptops in a local area network, most often arranged at a horseshoe-shaped table in a meeting room.

An experienced facilitator and a moderator guide the introduction to and the use of the technical platform, the domain objectives, the agenda topics and the time schedule. E-lab is portable and can be set up anywhere. It allows for parallel input of data from all participants, anonymity, instant availability of input data, and ideas structured in a stepwise manner. Participants can simultaneously and concurrently generate and communicate ideas, comments, oppositions, etc. This eradicates waiting to take turns to ‘speak’, and it facilitates electronically storage of all input data. The E-lab methodology is nominal in the sense that it presupposes the composition of a small expert group that is summoned to solve a specific task and in which there is limited room for interpersonal or informal group interaction outside the framework and the agenda of the electronic expert meeting itself.

In a nutshell, as Fig. 1 depicts, E-lab is modelled according to the basics of standard consensus methodologies with laptops in a network, operated by a facilitator and a moderator, careful peer selection of experts, with tools tailored to the issue and often implying a jury for assessing the primary output from the expert group assignments.

Fig. 1

The E-lab technical platform

In a face-to-face meeting, only one person speaks at a time while other participants are inactive and listen. In some instances this can be very time-ineffective, especially in brainstorming-type meetings. With E-lab, the participants contribute concurrently and are able to read and comment on other participants’ input instantly and continuously. Since communication is done through the laptops, all input is anonymous for everyone except for the facilitator who overviews and coordinate all steps via the server. This anonymity secures more ‘freedom of speech’ and, in some cases, less inhibition. Often in face-to-face meetings, someone might presume a dominant role and behaviour that might affect the rigorous imagining of ideas that usually is the essential starting point for a foresight exercise. This distracting situation is avoided in E-lab exercises. Evaluation and voting are done in a democratic way so that the opinion and assessment of the group as a whole is produced. Finally, a report is generated at the end of the meeting containing everything that has been written. In this way the group is not dependent upon a secretary who has tried to pick out what s/he thinks are essential elements of the expert meeting.

As a methodology, E-lab encompasses methods, tools and procedures that facilitate consensus building and informed decision making among experts in a field. By itself, it is one of several group techniques developed for situations where individual judgments must be combined to arrive at informed decisions that cannot be made by one person alone and for which there is insufficient scientific information or an overload of often contradictory information.

Supplementing the longer tradition of quantitative approaches qualitative methods are increasingly recognised as valuable in foresights. Nevertheless, practitioners face difficult decisions in their choice of method and in the process of analysis. The Nominal group technique [6] combines quantitative and qualitative data collection in a group setting and avoids the problems of group dynamics associated with other group methods. Idea generation and problem solving are combined in a structured group process, which encourages and enhances the participation of group members.

The expert group itself must be carefully composed for the purpose of the problem solving and to allow experts of different shades and levels of expertise to participate. Likewise, the agenda must state clear definitions of objectives, scope and tasks of which the group must be informed. Table 1 below shows five features that characterise the E-lab methodology as a group knowledge-producing foresight process.

Table 1 The E-lab Methodology Framework

As will become apparent, a methodology framework like the E-lab is confronted with a multiplicity of viewpoints on the role of paradigmatic approaches connected to the creation and application of qualitative and quantitative data, the philosophical assumptions related to the concept of and knowledge of the future, the iterative conversion of expert knowledge applied, and on the choice of effective, productive and reliable workshop procedures.

Of its own accord, the E-lab twinsets the endeavour of the foresight triad, i.e., thinking, debating and shaping the future [3, 4].

Experto credite?

Knowledge elicitation

Experts play an important role in foresights. However, the nature of expertise varies. Usually we regard expertise as knowledge possessed by the individual about a particular area that s/he holds greater knowledge about than lay people. Such expertise, although it is individually based, is often a result of cultural and social conventions, such as education and mastery. Yet, this knowledge is relevant in terms of sharing only if the individual is able to convey what s/he knows to others. When this happens, knowledge is being translated into information, which is easily transferable to other experts in the group.

On a collective or group level, the knowledge creation process seems to be less certain. However, some groups or communities engaging in foresights are able to create a common platform or a mental and cognitive scheme sufficient to produce a scenario or a similar foresight outcome. Such collective consciousness may take on a form of cultural or world view, i.e., some taken-for-granted, implicit and consensus-based understanding of how one should behave in the domain group foresight context. The notion of expertise can refer both to the domain capability of a person, a group or an expert system. In foresight exercises, the experts may be supported by a tailor-made GSS (such as the E-lab) to generate and deliver appropriate and large amounts of data, both qualitative and quantitative.

The E-lab foresight sessions are nurtured on a balanced blend of expertise and imbed a series of knowledge conversion and elicitation techniques [1]. All techniques were converted into electronic templates concentrating on domain elements. This approach also supported brainstorming, developing univocal terminology and categorising and evaluation of ideas, using multiple criteria and techniques, depicted in Table 2.⁴

Table 2 The E-lab Expertise Conversion Process

Often experts are involved in a semi-structured, technical creativity group activity, most often used to produce new theories, models or ideas for innovations and improvements. Crafting foresights – as opposed to relying purely on method and technique – means being careful about how we notice, bring to attention and shape knowledge about domain futures. In E-lab exercises, group members are encouraged to put forward ideas about a problem and how it may be solved, to generate as many (including unusable) ideas as possible. However, the problem solving may crash due to both individual and group-based phenomena. As depicted in Table 2 above, the knowledge conversion goes through stages from being implicit to conceptual: the individual expert is making explicit what previously has not been put forward in words or numbers and communicating these to the group. Next, the explicated ideas are combined and ordered, bringing them up to a systemic level, preferably conducive to building consensus for further steps in the foresight exercise [1].

Being an expert means that the person holds some unique core knowledge. Meeting other experts with confronting views and expertise could freeze the frame of reference used by the experts. Since E-lab often employs expert groups where the number of participant is small and comes from various turfs of life, there could be lack of trust between participants who have never met before or who see themselves as competitors. For instance, Kuusi [13] refers to the reluctance of knowledge sharing because of business secrets, competing interests, etc., in foresight exercises.

The experts could also stick so hard to their particular mind sets that they do not see or recognise other perspectives, even though they might hear them in the group and think they have considered them. Such a sticky and tamper-proof mindset will prevent considering novel options and solutions with an open mind, thus undermining the productivity and the essence of the expert group interaction output. In such circumstances, we sense more deeply rooted mindsets, not only opinions sticking to fads or to riding ad hoc hobby horses.

Other obstacles to the knowledge elicitation that may arise in the E-lab and similar methodologies are connected to the «groupthink» phenomenon, i.e., a situation where ‘loyalty requires each member to avoid raising controversial issues’ [14, p. 12]. Some deem groupthink no better than a myth, others deem it a brilliant construct [15]. Thus, the composition of the expert group and its ways of communication are extremely important, since the output depends on the range of perspectives, knowledge and experiences and is vulnerable to one-eyed views. When an expert group work together and is too homogeneous, either by which stakeholder segment the participants are recruited from or by professional education and training, they may end up as «thinking as one». This may result in constricted ideas and perspectives, not in an open debate on key assumptions, trends or possibilities (e.g., wild cards), etc. Surely, such a perverted atmosphere detracts from what can be deemed a novel output from an expert group exercise. Arguably, the E-lab helps avoid such fallacies.

Converting expertise to narratives and numbers

Weinstein [16] argues that there are two kinds of expertise: epistemic and performative. The former’ expertise is a function of what they know, the latter’s of what they do. Epistemic experts have the capacity to provide strong justification for a range of propositions of the future, while performative experts can execute their skill well according to rules and the virtues of a practice. Presumably, applied foresights epistemic expertise is required when developing rigorous qualitative and creative imagining (e.g., brainstorming, scenarios, futures workshops), while performative expertise is required when doing quantitative ranking and assessments (Delphi, technology roadmaps, forecasting, etc.). Expert-based foresights will undoubtedly need support from both strands of expertise. However, when both are present in foresight workshops, epistemic and performative experts may legitimately disagree with one another, since the two senses are conceptually and logically disparate. This confrontation of views may be a virtue of an expert foresight group in the rigorous imagining process.

The E-lab methodology assumes the presence of both senses in its five-step procedure, encompassing both qualitative and quantitative knowledge creation, depicted in Fig. 2 below.

Fig. 2

Building the E-lab Groundwork for Hybrid Foresights

Most E-lab exercises comprise all five stages and processes: moving from a looser to a firmer conceptualisation (converging and elaborating) to more faceted viewpoints and aspects of the core topic (diverging), then further on to the broadened understanding of relationships among concepts (re-organising), then to an ordered insight into possible consequences for the domain future (assessing), and, lastly, to reach an acceptable level of agreement among the experts (consensus building). Briefly, steps 1–3 involve qualitative methods and data, step 4 transforms these data into figures and quantities, while step 5 represents a more hybrid knowledge data formation stage, usually creating both narratives and numbers, though with a prominence to the quantified output. The procedure runs like this:

• Step 1: In practice, the first step comprises getting to know the E-lab platform, discussing the workshop’s key objectives and agenda, and being introduced to each other on the laptop screens. If needed, core theme, objectives and deliverables are adjusted before moving on to the next step. The experts may be confronted during this session with a statement expressing the core challenge of the workshop that they have to debate. Pros and cons are usually generated in large numbers.

• Step 2: Brainstorming on factors and actors that impact on the domain challenge is most often the next step, although occasionally a short oral introduction to the baseline domain is presented prior to the creativity session. This idea-generation activity may rely upon various creativity techniques to stimulate simultaneous generation, elaboration and expansion, and communication of ideas. These results are vital to the foresight process, since they constitute the basis for generating driving factors and trends to be applied in the subsequent stages of rigorous imagining and scenario-building. All kind of factors and actors and their relationships are listed. Most often this session runs through stages of ideas generation and elaboration and often produces a list of 100–150 domain statements.

• Step 3: The results from the previous step are used to converge and organise the tentative list of drivers and trends. In this step a joint list of ideas are organised by applying the NGT procedure. Ideas are structured and grouped, some ideas are subsumed, redundant ideas are deleted, overlapping and interrelated ideas are merged, and reorganised ideas are levelled into a hierarchy. A common and shorter list is emerging.

• Step 4: Evaluation and analysis of the reorganised ideas occur in the fourth step by applying quantitative assessment and ranking techniques focusing on probability and impact of the core issues. Most often, the pool of ideas on factors and actors, etc., that were developed through steps 1–3, is scored on a 1–10 scale, then ranked by use of simple statistics (mean/standard deviation) and subsequently entered into a hierarchy.

• Step 5: Finally, the voting for ‘best practice’ and consensus building take place in step 5, including the sketching of mini-scenario narratives⁵. The mini-scenarios are usually developed based on items from the ranking session deemed to be significant in terms of impact and plausibility. Average and standard deviation are used as simple statistics to check on the most noteworthy elements as assessed by the whole group of experts.

Besides this five-step procedure, a baseline section (Step 0)may be added that presents the relevant status quo kick-off information and also more steps to succeed the final step 5, e.g., building mini-scenarios or proceeding to build main scenarios and extended story lines and other narratives. As Fig. 1 shows, step 5 may give output to a secondary use of the raw text and data file to produce a more comprehensive and publishable output.

Observations from the field

Observations and assessments of design, input, processes and output have been made in a series of E-lab foresight workshops undertaken by domain experts⁶ over a ten year period (2003–2013) in such diverse fields as hydrogen technology, national research policy recommendations, drugs policy, lifestyles and addiction, and university governance. Accordingly, the anatomy and the applications of the E-lab system have so far been reported and presented as conference papers or articles on seven different occasions [1, 17–22]. Every workshop was designed on the step-wise platform outlined in Fig. 2, and most of them involved 12 experts and lasted 1–2 working days.

Depending on the issue addressed, a series of foresight methods and techniques were applied in various E-lab sessions. Common for the workshops is the high concentration of domain experts assembled in one room to communicate via laptops and the extensive use of thematic brainstorming in which ideas were organised and ranked according to consensus-building procedures. The workshops were all designed for collective handling of rather complex and often unstructured issues and problems, and the participants were given a detailed agenda specifying the objectives, tools and time constraints. Orderly attempts were made to envisage the longer term future of the system under study and to identify the trends, issues and events critical to the domain addressed. Several of the exercises applied the use of backcasting and scenario building, providing visions about desired futures to which significant actors may adapt and for which they may eventually develop measures and sketch roadmaps and trajectories. Because all exercises produced a combination of qualitative and quantitative data, they functioned, as such, as an observational arena for understanding the practicing of the ‘Greek triangle’ [4] when creating and sharing novel knowledge.

Group creativity tools were used that focus on a variety of aspects of creativity, including techniques for idea generation and divergent thinking, methods of reframing problems, changes in the affective environment, and so on. ‘Brainstorming’, ‘Think outside the box’ (analogy/metaphor), ‘Wild cards’, ‘SWOT analysis’ and ‘Backcasting’ have most frequently been applied. Next, idea-organising techniques were applied, such as ‘Mind Mapping’ and various checklists to relate and combine ideas.

Then, assessment of ideas occurred in a subsequent session during which the same group of participants was invited to rank and evaluate the generated ideas. Because the objective of the brain storming session is to gather as many ideas as possible, trimming down of ideas was done at the end of the session. Commonplace procedures of voting and ranking the reorganised list of ideas have been followed. An electronic voting system specified the form of the ballot (the set of allowable votes) and the algorithm for determining the outcome (tallying method). Usually «range voting»⁷ was applied, in which the experts score or rate each option on a range, and the option with the highest total or average score wins. Such range voting allows the experts to express preferences of varying strengths. Tabulations of the voting data were stored in a memory component of the server and as printed copy. Based on the outcome, consensus was usually built around the top ranked alternatives.

Our observational data indicates that the E-lab avoids the usual individual and group-based obstacles connected to the experts’ mindsets and the groupthink trap. If the individual experts have rigid mindsets, this has not been evident in the problem-solving exercises. Rather, the voting on the final issues of the group actions indicates that the experts think and argue on the basis of their individual expertise, shown by a normal distribution of mean and standard deviation. However, they have recorded their own stories and their common discussions, signifying a «group-intelligence» in which each member of the expert group has contributed to the final output. The result points to a synergistic effect in which the group’s contribution is more than the individual parts. Such a group synergy effect is vital to the nominal method; in fact, it is the method’s very essence. The E-lab concept is apparently conducive to such a process.

While the idea generation and organising stages mostly deal with qualitative data, the assessment, ranking and voting transform qualitative statements and ideas into numbers. What follows is the more substantial expert work, applying methods as diverse as focus groups, relevance trees, value chain analysis, technology or policy assessment, trend analysis, roadmapping, futures workshops, Delphi, gap analysis and, occasionally, web-based scanning, literature review and crowd sourcing (e.g., Wikipedia). Overall, the specific foresight toolbox composition in the various E-lab expert workshops was selected and tailored according to the theme, combining techniques from both the expertise-interaction and the creativity-evidence dimensions [10, 23].

Discussion

A «qual-quan» schism of foresight data?

Foresights comprise projections⁸ and explorations of different futures and alternatives based on insights from today’s knowledge and expectations of the future. In many cases foresight exercises use expert-qualified estimates of, for instance, technological developments. The foresight process also includes the understanding of the opportunities and possibilities of a technology, a system or an organisational device, however uncertain, as well as the pitfalls.

Therefore, the foresight process is a way to define the system and a constituting of the field with regards to stakeholders, system boundaries, possible consequences and impacts of the field and its development. It is also – if the primary goals and objectives are met – a roadmap for choosing the desired way and balance to attain the potential benefits and to minimize or exclude possible negative outcomes for the stakeholders. The ultimate success of a foresight project lies, as in most projects, in the extended outcome; that is, the implementation of foresight knowledge in policies, strategies and actions.

Some researchers think that a true and fruitful amalgamation of quantitative and qualitative approaches is not feasible; that is, the quest for meaning and the quest for measurement are incommensurable [24]. Karlsen and Karlsen [3] investigated the inherent ontological and epistemic premises embedded in the application of quantitative and qualitative foresight methods and tools, offering taxonomies for the classification of the most commonly-used approaches according to criteria such as mobilisation, scope and complexity. They claim that an optimal combination of qualitative and quantitative foresight methods’ approaches is only limited by the ingenuity of the researchers themselves, not by the intrinsic characteristics of the approaches. In the world of foresight methods, they imagine the future as uncertain regularities of which there is no objective knowledge and where (qualitative) narratives and (quantitative) numbers supplement each other. Gorard [25, p.249] finds it paradoxical that many researchers think it is reasonable to claim that other ‘researchers should ignore either evidence in the form of text or evidence in the form of numbers’. In the context of mixed methods, framing the qualitative-quantitative distinction, he claims that:

The schismic classifications of qualitative and quantitative work are unjustifiable as paradigms. They are not based on the scale of the work or on different underlying logic of analysis. They are pointless.

The «q-word» dichotomy does not represent two different worlds in the ontological and epistemological sense or opposing paradigms in the Kuhnian sense as socially accepted assumptions tending to appear in ‘normal science’[26]. However, they signify options for robust research designs as independent of methods. No research design, including foresights, implies either qualitative or quantitative data. Observations from the many E-lab exercises applying both kinds of data underpin the assertion that, ‘in practice, qualitative and quantitative are not conducted in differing research paradigms’ [25, p.244].

Multi-methodology (mixed methods) is an approach that combines the collection and analysis of quantitative and qualitative data within the framework of a single and unified study. In the world of mixed methods, three philosophical stances have emerged combining quantitative and qualitative data in one study’s evidence. A dialectical pluralism stands at the nexus of the constructivist and post-positivist paradigmatic school [27], enabling the quantitative and qualitative data to be in conversation with each other. A pragmatic school is directed by the appropriateness of methods related to the research question or problem to be addressed [28]. A transformative school is focused at methods conducive to the strengthening of human rights and social justice [29]. The combination of quantitative and qualitative methods and data in expert foresights supports the assumption that the nature of imagining the future also needs evidence gathered about the past and present. Thus, all three philosophical stances may be found in expert foresights.

Besides, even elements of the images of future may be represented by both narratives and numbers. Such integration of multiple forms of data can be done in three principal ways: merging, connecting and embedding data [30]. Merging data implies that texts and images are combined with numeric information, e.g., by counting the occurrence of themes in a text or narrative, thus transforming aspects of the written statements into numbers. Connecting data involves analysing a dataset in an initial phase and using the analyses as input to the subsequent phase of the project. Embedding data denotes that a dataset of secondary priority is amalgamated within a more comprehensive primary scheme.

«Foresight knowledge», as a form of strategic expertise or insight deemed ‘necessary for agenda setting, opinion formation and vision development and problem-solving’ [31], is, per se, non-verifiable in nature. Therefore, knowledge about the future cannot deliver ‘hard evidence’. It does not give a representation of a physical or empirical reality, it cannot be predicted, just imagined and perceptually explored [1, 9]. Rather, the ambition of an expert foresight is to make this imagination robust, plausible and implementable; it involves creativity; and, at its best, it spans the boundary of perspectives and expertise. Futures literacy [2] is targeted toward various sources of information, opinions and assessments and many levels of analysis that may impact on the future image-building process. The quantitative methods open the measuring of known phenomena and central patterns of association. The qualitative methods allow for and tolerate the naming and description of unknown processes, explanations of why and how an imagined event or state will occur, and the range of the potential impacts.

Arguably, a mixed methods approach combines the best of the two methods’ worlds [32]; it represents the intentional collection of both quantitative and qualitative data and the combination of the strengths of each to address the theme of the domain future. In short, mixing or integrating research strategies (qualitative and/or quantitative) in research undertakings is considered a common feature of all good research, including foresights and futures studies. Such an approach underpins the E-lab methodology.

Lessons learned for expert foresights

The Achilles’ heel of expert group foresights is to be found in the initial creation of fruitful ideas. In fact, the initial thinking and instantaneous idea generation becomes a ‘conditio sine qua non’ to ensure a basis for the succeeding stages of debating and shaping the future. The knowledge must be produced here and now in the expert workshop, not prepossessed. Besides, it must be acknowledged by the experts themselves as a result of the dialogue and interaction during the nominal group meeting and deemed a mix of well-known and novel knowledge. An empirical observation during the series of E-lab sessions was that teams of experts working in such settings appeared to establish a collective cognitive basis wherein common frames or mindsets emerge to describe and analyse uncertain issues. This process is not necessarily a consensus activity in itself, ending an initial group commotion. Rather, it is a form of contesting of information and individual expertise, which is necessary for further knowledge sharing and knowledge creation processes [1]. One of our key observations is that the experts during the sessions, both on and off screens, engage in a continuous debate regarding the identification at hand, wherein they oppose each other, discuss, and confront. It seems that this debating activity is a prerequisite for the expert group to come up with something new. In other words, these social activities (debating, confronting, and contesting each other) seem to reveal and embed implicit knowledge held by the individual expert but which facilitate context-specific knowledge sharing and the creation of something original, unique and collective to add to the state-of-the-art.

Computers contribute to creativity, and electronic brainstorming appears to be a useful device for generating a surplus of ideas in groups. Paying attention to others’ ideas on the screen leads to production of more ideas, to the eluding of similar ideas from various participants, and helps individuals focus their attention on the ideas. Being exposed to a flow of ideas generated by other group members apparently stimulates the creation of one’s own ideas. Moreover, it is productive, since it also reduces the detrimental blocking effects of verbal brainstorming and allows synchronous input from the group members. It outperforms similar face-to-face techniques, its positive effects increases with group size, and the group members were more satisfied when they used electronic brainstorming than when they used verbal brainstorming [33–38].

Accordingly, the E-lab electronic workshop system has been designed to enhance group effectiveness, efficiency and satisfaction⁹. Its software incorporates electronic brainstorming functionality and is valuable in foresight exercises where anonymity, ranking and voting are needed and time budgets are constrained. It delivers increased interactivity and participation by parallelisation, increased openness and less personal prejudice through anonymity, more sophisticated analysis by voting and analysis in real time, and, by the end of the day, automatic, comprehensive, neutral documentation retrievable from the computer [1]. Additionally, participants have control over their activity during an E-lab session and can attend to others’ ideas while also creating their own, continually exposing participants to a flow of ideas. Also, nominal groups, as used in the E-lab expert workshops, are found to be more productive in quality and quantity than interactive groups, and they present advantages over face-to-face groups in terms of the number, quality and originality of the generated ideas [39].

As a GSS-supported approach, E-lab offers the opportunity to strengthen relations and interactions across individuals and institutional borders, so that knowledge and ideas may be freely shared among the participants. In this manner the knowledge system as a whole may become quite efficient. In such expert exercises, the combination of analyses and communication processes may fulfil several functions. First, a collective and consultative process is established and the process itself is prone to become equally important to the written output. Second, it helps to identify and emphasise (potential) emerging futures that are deemed necessary to launch novel solutions. The methodology may be further useful in providing decision support and setting priorities for measures to meet future needs.

Arguably, what we observe in a typical E-lab expert workshop resembles elements of the SECI-model¹⁰ of the shared knowledge-creation process presented by Nonaka and Takeuchi [40]; that is, the initial sharing of knowledge is succeeded by efforts to create and justify new concepts, building proto- and archetypes that finally result in some reinforced and collectively accepted new knowledge and understanding. The SECI-model has also been applied in foresight exercises outside the E-lab context [41].

Conversely, there are critics of the SECI knowledge creation model. Bratianu [42] argues that only externalisation (from tacit/implicit to explicit knowledge) and internalisation (from the novel explicit to a higher level of tacit, operational knowledge) are true conversion processes. Socialisation and combination are knowledge transfer processes, though vital in the E-lab expert group interaction. Gourlay [43] has written one of the most extensive critiques of Nonaka’s knowledge creation theory. Amongst other arguments, he claims that Nonaka omits the inherently tacit knowledge and points out that different kinds of knowledge may result in opposite kinds of behaviour, i.e., non-reflective versus reflective knowledge. From the perspective of the expert meeting, reflective behaviour (of which we appear to know the most) is paramount to the foresight knowledge creation.

Van der Steen and van Twist [44, p. 35] brand ‘anticipatory knowledge’ as knowledge that encompasses long-term trends, developments and dynamics about the future. Anticipatory knowledge is performative, produced in a mix between traditional and creative techniques, lacking an evidence base and speculative in nature. When successful, they must position their anticipatory knowledge production close to the sites of power, adopt a fine-tuned timing, rely on a profiled professional composition and balance of the expert group, deliver rich narratives and plausible and implementable trajectories, and obtain a wide dissemination of the expert study [44, p. 40–41]. As such, this resembles what is required from Godet [4] in terms of anticipation, appropriation and action.

Conclusion

This article has described the practices that occur in the context of amalgamating qualitative and quantitative data created and applied in nominal expert group foresight exercises within a dedicated GSS design. Besides a methodology conducive to transforming qualitative statements and topics to both numbers and narratives, these practices also included a series of methods and techniques out of the foresight toolbox. If planned well, an expert group may profit from the creative production of ideas to the organised and prioritised weighting of items needed to build appropriate domain future images (e.g., scenarios).

The E-lab demonstrates cost, time and manpower efficiency and encourages the elicitation of individual expertise into combinations of collective and consensus based and novel narratives on the future. It is flexible as to what foresight tools may be applied, its 12 + 1 laptops are portable, easy to move around and to install locally. Arguably, the E-lab also helps develop sense-making frameworks and meaningful narratives, stimulates the production of a collective understanding of evidence and ideas and of the critical assessing of the present as a baseline for the future trajectories. In essence, this is what futures literacy is about - a learning process that embraces the complexity and uncertainty of the future in new ways [45].

Expert foresight is about collaboration and participation, synergies resting on the collective sharing of insights, and mutual stimulation of ideas. When thinking, debating and shaping the future, this may be an appropriate way to apply domain expertise. The E-lab methodology enables the experts to be both concurrent and connected, thus effectively and extensively sharing the relevant knowledge and experience conducive to the domain challenge [46].

However, the E-lab is not unique, it is replicable. Such foresight platforms may be constructed by many different GSS technologies and many variations of the Nominal group technique [8, pp. 123–124]. Nevertheless, similar set-ups will need a robust design that takes into account the q-word schism, the proper composition of the group of experts, the communication platform and the knowledge elicitation techniques and processes. As a robust foresight methodology, i.e., as ‘methods of knowing’ about the future, it may be worth replicating. The E-lab seemingly offers variations on a theme, played on the score of the expert foresights.