The futures Map and its quality criteria

We consider that Bell’s definition characterizes the identity of futures studies though we would replace the word “discover” with the concept “construct” The purposes mentioned are well in line with the purposes of two main approaches of futures research: the scenario approach and Delphi surveys. The scenario approach and the related methods, first of all the morphological matrix- method (or the futures table-method), are used to construct or to propose possible futures. Scenario methods also provide systematic frames for the examination and evaluation of futures. Some versions of Delphi surveys can also be used to invent futures but most Delphi versions are mainly used for other purposes (e.g., sense-making, assessments, see [11] or [12]). Experts use Delphi studies to evaluate how probable and preferable or how important some specified futures are.

The Futures Map includes all possible futures (‘futuribles’ ⁶) as identified during the research process. Any picture of the future, when assessed according to time frame and desirability, has a definite place in the Futures Map. It may even occur that two pictures of the future occupy the same point in the two dimensions of time and desirability since they have identical assessments. Nevertheless, they can be completely different according to depth and content. We can place them differently in the third dimension. Pictures of the future are descriptions of a future state of affairs, sometimes rather lengthy narrative ones, sometimes rather short ones with only some key figures about society, economy etc. When placing the pictures of the future into the map, many dots and spaces in the Futures Map may be left empty since there is no corresponding picture of the future.

In our basic illustration of the Futures Map, any future or picture of the future is characterized by two parameters or dimensions: when it is assumed to realize (time of realisation, x-axis) and how the users of the map will appreciate its realization (desirability, preferred futures), see Fig. 1. The higher a point on the y-axis the more desirable the picture of the future is. That means normative as well as non-normative questions are considered. Another parameter (not shown in the illustration below) is possibility: Whether a picture of the future may – according to our best knowledge – realize or whether it is only an abstract option that does not fit reality (utopia, pipe dream…). We reserve the third dimension for the free arrangement of pictures of the future and pathways connecting them. This third dimension spans the ‘depth’ of the terrain.

In this article, we will not describe in detail how we have integrated the basic concepts of futures research into our Futures Map frame. The detailed discussion is presented in an article in Futura [31]. Instead, we describe and motivate the Futures Map frame comparing the Map approach and the various versions of the scenario approach.

The descriptions of the scenario approaches are typically descriptions of scenario processes. For example Steinmüller [37], has described the basic steps of the scenario approach with eight basic steps. The first step is the structuring of the research field and the final stage is to look for suitable solutions to the issues or problems identified based on the scenarios and disturbing wild cards. From the point of view of the Futures Map frame, the scenario process of [37] can be seen as the integrated construction process of the Futures Map and the decision making based on it.

Besides Kahn, also many other prominent futures researchers e.g., Michel Godet [20] have defined the scenario similarly. Like most scenario approaches, we also suggest that any possible future requires a kind of looking backwards from the picture of the future, a kind of backcasting from the (picture of) the future to the present evidence. John Robinson [36] suggested backcasting as a normative and design oriented scenario approach that unlike the French prospective approach does not evaluate probability of scenarios. Recently, the backcasting approach implies a broader approach where all kinds of scenario paths starting from some picture (s) of the future are back to the present discussed.

Many scenario approaches use the concept of the scenario funnel. For example, this concept is especially important in the scenario approach of [39]. The basic idea of the scenario funnel is that the farther we gaze from today’s standpoint towards the future the more possibilities are open [29]. In Fig. 1, this idea of the funnel is taken into account in the scenario paths of the illustration. They all start from a “now” point and the range of the scenario paths increases with time. Scenario funnels are often used in a way that the most probable scenario is in the middle of the funnel. This idea does not work in Fig. 1 where the possible scenarios are presented in the order in which the most desirable scenarios are at the top.

The Futures Map is first of all the tool for the decision making of its customers. This feature is stressed in Peter Schwartz [38]’ scenario approach. According to him “Scenarios are tools for ordering one’s perceptions about alternative future environments in which today’s decisions might be played out.” You can see the similar point of view in the much-used statement: “There is no need to know the future, but to be prepared for the future” [5, 23]. The final position, the picture of the future where one arrives, depends on the choices the actors make on the way, on their purposes and capacities. Theodore Gordon [22] expressed this important starting point of futures research in the following way: ‘There is a future without action, and a different one with it. Thus futures research and predestination are, at least on the surface, antithetical’. In other words: people decide on the future path by action or inactivity – there is always an alternative. Of course, the futures map is often based on causal processes that do not function as the map assumes. Concerning any action based on the map, the actor has to be prepared that her or his expectations concerning the impacts of his or her action will not realize.

Other key concepts of the Futures Map that are also relevant for many scenario approaches are the vision, acceptable futures, the aspiration level of the actor (s), the accessibility and the trend or the business-as-usual scenario. Concerning their definitions in the Futures Map frame we refer to [31].

How can the relationship between the Futures Map frame and various scenario approaches be summarized? We consider that the Futures Map frame provides the well-working frame that defines in a consistent way key concepts used in various scenario approaches and in other kinds of futures research efforts.

In particular, the frame is useful for the classification of the time horizons used in futures research.

In futures research, the time horizon is one of the key issues. A rather common opinion among futurists seems to be that the time horizon of scenarios is longer than the time horizon of roadmaps or plans. Scenarios might have the time horizon of 100 years. The time horizon of the planning horizon/ the road map is nearly never longer than 20 years. From the point of view of futures research, the following statement of the English Wikipedia (“Time Horizon” 1.10.2015) is very questionable: A time horizon, also known as a planning horizon…

In the Futures Map frame it is possible to solve the conceptual problem concerning scenarios, roadmaps and plans using the distinction between the planning and the mapping horizons. The mapping horizon is the anticipation horizon of the Futures Map (of the possible futures). As in picture 1, a scenario path might de facto end already before the mapping horizon. For example in the Futures Map of a company, the bankruptcy of the company might be an end point for the individual company, but in fact, the scenario without this company might be going on, unconsidered by the managers of the company. However, the time horizon of most scenarios is defined by the mapping horizon. As in the picture, there might be many scenario paths to the same end point of the mapping horizon.

We connect the planning horizon of the Futures Map to the concept and method of the roadmap. In the picture, the dotted line that starts from the ‘now’ box (the present state of affairs) illustrates a roadmap. During the time frame of the planning horizon, the involved actors are committed to follow the specified road of the map – the roadmap. For example, a company decides that during the next 2 years all mobile phones they produce follow the same specified standards. A roadmap requires a vision or some other orienting picture of the future in the mapping horizon, planning horizon (time frame) and a planning horizon goal.

Typically, it is reasonable to create and design the road map for a shorter period than the mapping horizon of the Futures Map. In the illustration, when one has followed the road of the map to the end of the dotted line, the world is not the same as in the ‘now’ situation any more – it has changed and every new action starts under different conditions. At the end of the planning horizon, new evaluations of the vision and the acceptable futures are needed. Earlier adaptations are possible and recommended (e.g., in yearly strategic meetings adaptations in the roadmap can be made).

Basic dictionary meanings of ‘valid’ are sound, just or well-founded [40]. In logic it means that conclusions are based on established / agreed-on premises. In empirical sciences, the validity of generalizing theories or statements is, however, a matter of degree because you can never be sure that a general statement is true. Testing the theory, you can possibly falsify some of its conclusions but if no better theory is available you are justified to conclude that the theory is more valid than that of your competitors. The suitable definition of pragmatic validity in empirical sciences is the following: valid proposition, inference or conclusion is the best available approximation of the truth.⁸

Using the above definition of validity, we conclude that one of the key tasks of scientific futures research is to improve the pragmatic validity of Futures Maps. This focus is especially useful for the evaluation of the quality of the scenario process. The Futures Map approach looks at the quality of the scenario process from three points of view: the (internal) validity of the construction process of the Futures Map; the (external) validity of the constructed Futures Map; and the quality of the decision making process based on the Map. As we discussed above, the validation concerns the whole Futures Map. The validation increases or decreases the plausibility or probability of its single elements e.g., the scenarios.

Any empirical science defines its own criteria of pragmatic validity.⁹ A common distinction in empirical sciences is the distinction between internal and external validity. Internal validity means that the research results are obtained using sound research methods. Besides the sound use of its various methods, we assume that the internal validity in futures research requires well-organized processes. Like many scenario process versions, they should be activity-oriented and contribute to the ‘shaping of the future‘as pragmatic and organizational approaches. The EFFLA Group [14] has suggested that foresight (in the sense of ‘applied futures research in the policy context’ but including open debates, see the definition above) should always be a process that integrates strategic intelligence, sense-making activities and the link to the policy cycle (see Fig. 2).

In the case of futures research, the external validity means that there are sound reasons to generalize – or to make abduction¹⁰ - from past and present facts to futures relevant conclusions. The Futures Map summarizes these generalizations or abductions. External validity therefore means that the research results, the Futures Map, is supported by facts and observations like existing trends, weak signals etc. and well-established theories concerning the relevant causal processes. The pragmatic validity of the Futures Map increases if relevant actors are able to use it. If most relevant actors do not get the map or do not understand its messages some opportunities, which are otherwise accessible and are identified in the map, might not be part of accessible possibilities. This concerns, of course, also the avoidance of bad possibilities.

An important idea of the suggested criteria is that no Futures Map can be a high quality map in all criteria. The researcher has to select those criteria of external validity in which his or her Futures Map tries to be especially good. From this follows that there is no absolute ranking order of the Futures Maps based on the criteria. You are just able to say that the Futures Map 1 (FM1) is better than FM2 in the external validity if it is better in at least one criterion and as good as FM2 in other criteria.

One key concept related to the criteria of the pragmatic external validity is the concept “relevancy”. In the Futures Map frame, the relevancy is the customer related concept. Some feature of the Futures Map is relevant if it includes information that is useful for the customer’s choices. Though the concept of “relevancy” is mentioned just in criteria 1–4, the Futures Map cannot be relevant if it is not understood by customers of the Map (the criteria 5 and 6).

What is the connection between the relevancy and the pragmatic external validity? A way to define the pragmatic validity is the criterion that the customer’s expectations based on the Map will realize. There is, however, a way to express the quality requirement of the Map so that both the validity and the relevancy aspects are present: If the content of the Map is both valid and relevant the customer is able to make choices that she or he will not regret. Because the customer might be happy also in the case of the unexpected development this criterion is more general and suitable than the realized expectations. For example, the result might be better than anticipated. Actually, [30] has suggested that the regretted or not-regretted choices can be seen as the general epistemological basis of the futures related learning and knowledge.

We can improve the external validity of the Futures Map in a way that does not depend on specified customers. Not depending on the specified customers, we can in principle test the possibility of any assumed hypothetical causal process of the Futures Map. Of course, in the case of complex processes with many kinds of interacting causal processes, this is not often possible in practice. On other hand, there is the customer specific aspect that combines the validity and the relevancy. If the Map includes irrelevant causal processes or futures it is less suitable for the not-regretted choices of the customers because of the information overflow. The difficulties in understanding and the information overflow are taken into account in criteria 5 and 6.

The simple way to classify causal processes / futures into relevant and not relevant is by asking the customers. However, this does not guarantee the fulfillment of the not-regretting requirement. The customer might have an opinion on the relevance that is based on lacking knowledge. For example, we are justified to assume that many of the people who were killed by lung cancer in the 1950s would have stopped smoking based on our recent knowledge concerning the dangers of smoking. However, as the case of smoking nicely illustrates, even very well-justified evidence does not necessarily change the perceived relevance evaluations and related choices. Even if a person is not ready to accept the valid evidence, the Futures Map that takes the evidence into account improves the Futures Map from his or her point of view ¹² We can conclude that focusing on the not-regretting criterion the objective or scientific validation of the futures map is possible both from the point of view of the not customer specific and the customer specific relevancy aspect. ¹³

For the identity of futures studies the first criterion is especially important. According to the Futures Research Methodology 3.0 of the Millennium Project, ‘perhaps the most commonly understood reason for the use of futures methods is to help identify what you do not know, but need to know, to make more intelligent decisions.’¹⁴ . The identification of new possible scenario paths shows relevant gaps in our knowledge and in that way the validity of the Futures Map increases whenever it is able to evoke more (relevant) possible futures. ¹⁵ The validity improves even without critical fact-based examination of the suggested new possible paths (criterion 1). However, if the map includes, ceteris paribus, the most relevant path it improves the validity of the map (criterion 2).

As Malaska and Virtanen [32] remarked, futures are just partly determined by the known facts. It is possible that just some weak signals give indications of the most relevant futures. So it is important to take into account aall causally relevant past or recent facts (criterion 3). But if you are able to construct a map of a few scenario paths that takes account of a given set of facts and observations and gives an interpretation of their effects, it is ceteris paribus better than a map that needs more scenarios for its interpretation (criterion 4).¹⁶ A nice example of the use of criterion 4 (from another field) is linear regression analysis that interprets the variance of past evidence with a trend assuming that the trend will continue also in future.

Even if a futures researcher has identified causally relevant possible futures and has taken into account even weak future signals something more is needed for a valid Futures Map. The validity of the Futures Map requires more than the correspondence of the Futures Map and past or present facts. Assuming that users or customers of the Futures Map themselves are the best experts of choices that they will not regret (compare [30]), they have to understand its relevant messages (the criterion 5). Criterion 5 is especially relevant when the common understanding of possibilities and relevant past facts are important for common choices of actors. However, in some situations, it might be important that just key customers understand the map and those who have an interest in hindering most favorable futures of the map’s customers do not understand it (criterion 6).

There are connections between the suggested criteria of external validity and the internal validity of futures research processes. Typically, the identification of potentially important paths (criteria 1 and 2) belongs to the first phase of the futures mapping process. In Fig. 2, it is Strategic Intelligence. The collection of potentially futures-relevant facts belongs also to the strategic intelligence phase but the interpretation of their relevance in various scenarios belongs to Sense Making (criteria 3 and 4).

Criteria 5 and 6 are related to all stages but they are especially important in the phases Sense Making and Selecting Priorities. A good scenario is not just possible and consistent but it should also be believable, trustworthy and interesting. This is an especially difficult challenge concerning those possibilities that challenge recent values and beliefs of the Futures Map’s customers. In order to manage this challenge, the futures researcher has to make the customers understand their prejudices. In practice, this challenge is highly related to the use of interesting and convincing metaphors. To this end, Causal Layered Analysis is a suggested approach [25].

The validity criteria function pairwise so that criteria 1 and 2; 3 and 4; and 5 and 6 define basic dimensions in validity evaluations. These basic dimensions correspond to the “leading questions” of research as identified for typical research styles suggested by Jerome Ravetz [35]. According to Ravetz the leading question is ‘what/how’ for research, the outcome of which is a statement intended to be factual. What/how research combines substance and agency: ‘what is this made of’ or ‘how does this cause that’. These questions are relevant especially concerning the validity of criteria 3 and 4. Statistical forecasting methods - e.g., regression analysis - are useful in predictions based on causal connections between specified variables. In these kinds of anticipations, criterion 3 means that scenario paths of the Futures Map FM1 take into account more past trends of possibly relevant variables than the scenario paths of FM2.

The validity of criterion 4 means in these kinds of anticipations that FM1 causally interprets the past variance of facts with fewer trends about ‘how the future might develop’ than FM2. For example, think about weather forecasts. In the 1950s, the forecasts provided many possible scenarios for the next few days. Now we have a much more narrow scope of possible weather scenarios for the next 3 days, based on much richer past evidence and rather better models than in the 1950s. So recent weather Futures Maps are more valid based on both criterion 3 and criterion 4.

Ravetz’ second style of research is focused on ‘how/why’ questions. This style accepts concepts like ‘final course’, ‘function’ or ‘purpose’. According to Ravetz, here the point is the design that is able to perform a given function, to do its job. ‘How/why’ questions are reasonable only in the framework of actors with their interests (compare [30]). The questions of type “why” are especially relevant for the validity criteria 5 and 6. If an actor is not able to understand how his or her interests are connected to a possible future he or she is not interested in promoting that future.

Concerning the validity criteria 1 and 2, Ravetz’ third research style is the most relevant. According to Ravetz, ‘what if?’ expresses the spirit of creativity, of inventiveness, forays into an unknown that is passive and expectant. For Ravetz the role of ‘what if’ questions was first of all the management of treating ignorance related to the impacts of new technologies and environmental hazards e.g., possible hazards of new chemical plants. We can, however, generalize ‘what if’ questions to concern also positive futures that are possibly just based on weak signals. ‘What if?’ questions challenge prevailing anticipations and action routines and, actors realize new options for action based on them. The focus is actually at least as much on new questions as on the suggested answers. This idea is stressed by Michel Godet [21] who cited Woody Allen: ‘The answer is yes, but what was the question?’

Next we will compare the validity criteria of the Futures Map with other suggested quality criteria of futures research.The quality problem on the general level is discussed in many books that are used in academic teaching or as guides to practical work of futurists. A main theme of Wendell Bell’s Foundation of Futures Studies [2] is the nature of futures studies as a scientific activity.¹⁷ For technology foresight directly linked to policy-making, the major reference for ‘practitioners’ internationally is “The Handbook of Technology Foresight’ [16].

The journals of the field, especially Technological Forecasting and Social Change, Futures, Foresight and The Journal of Futures Studies and the new European Journal of Futures Research or the Zeitschrift für Zukunftsforschung (published in German) have served as platforms for discussing good practice in futures research. The international Millennium Project has an acknowledged role in the development of shared understanding of the field. Though the Millennium Project is open for both scientific futures research and artistic futures studies, the quality issues are met especially in its Delphi related activities.

On these three levels, the importance of the internal, external and customer specific quality criteria differ.

Up to now, futures researchers have found the philosophical basis for their research efforts mainly from two sources: critical realism and constructivism. Wendell Bell [3] has been the most prominent promoter of critical realism in futures studies. Some radical constructivists have connected critical realism to ‘positivist’,‘behaviorist’, ‘empiricist’, or ‘technocratic’ approaches. According to Jakil [26], many constructivists share their philosophical basis with ‘post-modernistic’, ‘post-positivistic’, ‘post-empiric’, ‘interpretative’, ‘discursive’ or ‘deliberative’ approaches. Less radical constructivists used to think that both critical realism and constructivism are feasible philosophical starting points for futures research. The common feature of constructivist epistemology and critical realism is the idea that human knowledge is based on continuously developing conceptual systems. Theories or constructions are “true” or “working” as long as they are not shown to be untrue or not-working. A philosophical approach that shares this common starting point of constructivism and critical realism is the learning based paradigm of the General Theory of Consistence (GTC) [30]. When the six criteria of the pragmatic external validity of the Futures Map were first suggested in 2011, they were presented in the frame of that philosophical approach.

The criteria of the internal and external validity of the Futures Map belong to the level of pragmatic general quality criteria. In the present state of the art, they just provide a simple, theoretically sound and systematic heuristics that is suitable e.g., for the evaluation of student works in futures research. Another practical and more detailed portfolio of general quality criteria has been suggested by the Task Force Standards (TSF) of the German Netzwerk Zukunftsforschung.¹⁸

According to ([18] p.12), the standards of the TSF should inspire quality focused discussions in professional futures research. They aim to support foresight professionals and other practitioners in planning and realizing scientific futures studies in assessing their quality, they are also thought to support training and education in the field.

As we have already discussed previously, there is a close connection between procedure and product and therefore the two sets of criteria/ standards are interrelated. They are not identical but support each other – as will be discussed in the following.

How are these standards linked to the quality criteria of the construction process of the Futures Map, the six external validity criteria of the Futures Maps and the quality criteria related to the use of the Map in decision making?

The Futures Map frame shares the constructivist epistemology of the TFS standard 1.1 Also from the point of the validation of the Futures Map, the reasonable requirement is that futures researchers should always make clear that they construct their statements based solely on present knowledge, not on any direct ‘fore-knowledge’ of future objects and that their research results do not represent the future as it will be not true for some time, but present assumptions about the future (images, constructs).

The distinction between possible, probable and preferable futures (the standard 1.2.) is in many ways taken into account in the Futures Map frame and in its validity standards. In crude terms, the external validity criteria 1 and 2 are related to the possible; 3 and 4 to the probable or plausible; and 4 and 5 to the preferable futures from the point of view of the Map’s customers.¨

For scientific futures research the crucial question is under which conditions a statement concerning the future represents knowledge. As was remarked at the beginning of this article, futures researchers can just evaluate if the actor (e.g., an expert) has good justifications for the statement.¹⁹ This is subject to the standard 1.3 ‘Argumentative verification’: The assumptions underlying the statement about the future and the methodological integration of these assumptions should, in principle, be open to discourse, to examination, to approval or refusal by others.

Futures Maps are generated for actors and their decision making. The validation procedure with Futures Maps serves in this way decision making and action focus of the standard 1.4.

An important choice of scientific Futures Research is the choice between interdisciplinary and transdisciplinary research approaches (standards 1.5 and 1.6). Both aspects are important from the point of view of scientific futures research. Transdisciplinarity suggests a research strategy that crosses many disciplinary boundaries to create a holistic approach and even goes beyond that. Transdisciplinary research, as a rule, includes non-scientific experts from relevant domains of practice, who bring in their perspectives and knowledge. Interdisciplinarity accepts different specific scientific perspectives in the pursuit of a common task.

More valid Futures Maps define a distinct transdisciplinary field of futures research. In this way, futures research is proceeding towards the target suggested by Bell. According to Bell [3], futures studies should not only ‘futurise’ existing fields of research but should incorporate more principles of scientific futures research into the perspectives, theories, teaching, and research of ‘old sciences’. Bell ([3] p.59) considered that without this type of distinct transdisciplinary field, the ‘futurizing’ of other fields of science will be difficult or even impossible.

Any kind of scientific work requires explicit aims and framework conditions as well as transparency and comprehensibility. The standards 2.1 and 2.2 are, however, difficult to achieve without a sound theoretical basis (standard 2.3). This standard is related to the philosophical basis that we discussed above.

We consider that the prospects of scientific futures research will depend first of all on scientific methods that organize research processes well (the internal validity) and result in more valid Futures Maps (the external validity) (the standard 2.4). In the next chapter, we briefly evaluate how different methods are suitable for different purposes and what their strengths and weaknesses in the validation of the Futures Maps are. The standards 2.5, 2.6 and 2.7 - i.e., conceptual quality, including procedure according to the state of the art; scientific relevance and code of conduct - are closely related to the methodological choices.

Futures Maps are customer specific. If the Futures Map takes into account customers’ interests and key customers understand it, the validity of the Futures Map increases. In this sense, futures research is first of all an applied science serving actors in their decision making. Relevant actors have to understand the validated Futures Maps (validation criterion 5). In some cases, it is required that just key customers of the study understand the map (validation criterion 6). Sometimes these key customers might even require that e.g., their competitors are not able to use the map. This does not imply incomprehensibility of results for third parties, but usually confidentiality: the resulting intellectual property belongs to the organization that commissioned the study. Confidentiality makes applied research different to academic research. As a rule it excludes validation procedures like peer reviews and the integration of the results into the knowledge corpus of the field.

The standards 3.1–3.5 are especially important for those who are practitioners in applied futures research. However, besides good project management, any futures study has to take very seriously the standards 3.2 and 3.3 into account. If panellists of a Delphi study notice that the study has practical relevance, usability and impact they are typically more ready to provide high quality arguments and judgments.