- Open Access
Integrating Delphi methodology to some classical concepts of the Boston consulting group framework: arctic maritime technology BCG Delphi foresight—a pilot study from Finland
© The Author(s) 2015
- Received: 21 October 2014
- Accepted: 9 December 2014
- Published: 29 January 2015
This article presents a way how the Boston Consulting Group (BCG) analysis tools can be combined with the Delphi methodology, and especially with the BCG portfolio matrix analysis. The Delphi Method is one of the most well-known futures research methods. It is an expert group interview process, which is based on two or more iterative rounds of expert interviews. Typical aspects of the Delphi methodology are anonymity and the conventional emphasis of consensus of experts with associated statistical evaluations. A conventional definition of the business portfolio is that it is the collection of businesses and products that make up the company. This article includes an experimental pilot study of the Finnish maritime cluster and its technological future options where integrative BCG Delphi methodology was used. Detailed pilot study results and learning are reported in this article. The study reveals a high potential of the BCG Delphi studies in the industrial and corporate foresight fields. Especially, futures of industrial product and service portfolios can be diagnosed and forecast by this novel methodological approach. The Delphi methodology is according to a metaphor “one of the best and constantly evolving computer’s operating system”, which can be linked to many other “programs”, among others to the conventional BCG tool package. To be strategically oriented from the BCG business perspective, the key issue strategically for European cities and regions is the strong local existence of Cash Cows (good and successful companies). Decision-makers in cities and regions should be interested in the strategy of optimum Cash Cow flows if they want to develop their cities and regions in the long-run. Thus, products with high market share and slow, but stable growth trajectory are Cash Cows.
- Boston consulting group matrix
- Delphi methodology
- Product portfolio matrix
- Industrial foresight
- Corporate foresight
- Arctic marine cluste
- Regional cluster analysis
This article aims to present how the Boston Consulting Group (BCG) analysis tools can be combined with the Delphi methodology, especially with the BCG portfolio and matrix analysis. The idea of this article is to discuss how the BCG’s product portfolio analysis and the Delphi methodology can be combined in a concrete spatial planning process. In this article, our case is the Finnish Arctic maritime cluster. This paper can provide useful insights for regional planning authorities, management consulting companies, corporate foresight experts and industrial foresight experts in Europe and elsewhere.
This paper outlines the idea to combine the BCG Matrix analysis with the Delphi methodology. The scientific motivation of this approach is that in many cases the positioning of products and services of the BCG matrix is based on very limited strategic conversations of experts. Using the Delphi approach will probably increase the validity of product positioning in the BCG matrix. Another methodological benefit of our approach is that it allows a broader base of expertise to be used in the construction of the BCG matrix. Methodologically, it is possible to present experts’ judgments about current and future aspects of products, although the Delphi methodology has been conventionally used in futures-oriented foresight analyses. The advantage of the Delphi methodology is based on the possibility to evaluate complex issues during various evaluation rounds. This methodological aspect is highly relevant for all evaluations, whether they focus on current issues or future issues. From this methodological verification perspective, a novel combination of these analysis tools makes a lot of sense from a scientific perspectives of accuracy and objectivity.
Various industrial stakeholders of pilot R&D projects have kept new analyses and results insightful and strategically relevant for them. This kind of methodological approach may have been used in SWOT and BCG analyses but as far as we know these pilot cases have not been reported in an explicit way in scientific foresight journals. From this novelty perspective, our contribution provides scientific value added and new insights to the international foresight research community. The obvious scientific benefit of our approach of the Delphi evaluation of the BCG matrix analysis is that the accuracy level and objectivity level of evaluations increase.
The Boston Consulting Group (BCG) has developed several methods for companies operating in the field of strategic management. Consulting companies (like McKinsey & Co, Arthur D. Little etc.) have always provided management consulting services, techniques and tools for companies and corporations. Each company provides its package of management services. Many strategic decisions are concerned with the linkage between organizations and their environment. The literature of strategic management emphasizes the need to understand how organizations relate to, and interact with their environments. The literature stresses also the necessity of maintaining a match between the products offered by companies and the opportunities created by changing environmental conditions. Both serendipity and strategic approaches are used in business life [1–3].
The BCG approach is based on the following assumptions: (1) various strategic business units can be identified, (2) someone can manage these business units, (3) there exists an experience curve for each market so the business unit with the most cumulative experience has the lowest costs, (4) the experience curve can be used to generate a positive relationship between relative market share and profitability (or cash flow) because there is a high correlation between cumulative output and relative market share, (5) the relative market share of a unit is the key to profitability and (6) high market share units generate the cash flow that can be invested in the unit, used for corporate expenses or transferred to other businesses. The BCG assumes that the total resources available to a company are limited. That is why there must be a balance between cash used and generated in a company [4, p. 48]. This means that a company cannot undertake all profitable investment strategies and plans. These assumptions are very realistic from the perspective of regional planning of businesses.
Many BCG analyses have remained in use and are conventionally utilized in many companies and corporations [4–6] and have also been applied in the areas of spatial planning and strategy formulation [6–9]. Many of the BCG analyses make use of expert information. Typical methods are the Product Life Cycle and Portfolio Analysis . Thus, the Delphi methodology is a natural choice to combine expert information with the Boston Consulting Group framework. In this article, we elaborate the use of these two effective methodologies.
In the field of futures research, several methods have been developed for anticipating the future and for active decision making and leadership. Futures research methods are used in most cases to contextualize expert information and knowledge. Futures research methods have generally formed as a separate research field since World War II [10–12].
utilization of context dependent expert knowledge,
evaluation of different kinds of potential changes in the operating phase,
evaluation of future opportunities and strategic options,
the selection of strategic alternatives, and
delivering a decision support for decision makers.
Thus, the above-mentioned characteristics of the inquiry are common for both the BCG approach and the Delphi methodology. Of course, the Delphi methodology is not always used for strategic purposes, but in this article we shall focus on this specific aspect of the Delphi methodology. In this article, a closer inspection of the BCG portfolio analysis and the Delphi methodology will be performed. We believe that this kind of methodological synthesis is very beneficial for European planning and industrial studies everywhere in Europe and also in other spatial contexts. In general, we believe that European clusters and regions need a sharper strategic focus. For example, European regional smart specialization strategies could get benefits from our BCG Delphi approach, which combines the Delphi methodology and BCG analysis. Thus we do not categorize BCG analysis to be methodology as such. This novel methodological combination means combining two very strong methodological traditions. From this methodological perspective, this article provides a new approach to spatial planning and management. This novel approach allows many generalizations, and it is also a flexible approach as long histories of both methodologies show to us. Both tools work quite well, despite all criticism.
As we know, expert information can be utilized, for example, in policy-making seminars and workshops. These forms of strategic collaboration, however, have their own problems which prevent the expert group from operating democratically. For example, as a principle, all the opinions of the group should be looked after and taken seriously. However, a strong impact of some personalities can dominate the group discussion. Or some experts do not dare to bring all the honest opinions out of fear, or losing face, or fear of losing their jobs. The Delphi method was originally developed to address these kinds of group-thinking problems. This has made the Delphi teamwork a traditional and widely used method because this methodology allows anonymity of the experts [15, 16].
When we are utilizing the Delphi methodology, expert knowledge can be used more efficiently due to the anonymity of experts. All participants can have their views processed in the Delphi process. The Delphi methodology has also a feature of iteration rounds. There are typically several rounds of expert interviews, and between them the Delphi manager provides summaries of expert communication and feedback. These emerging new expert ideas offer the critical possibility of processing results and making corrections when they are needed. As we know, this is not possible in traditional statistical surveys. The feedback mechanism is an advantage of the Delphi methodology [17, 15, 16, 18–20, 12].
Interview rounds, stretching between the previous rounds of feedback, support the learning process and offer anonymity, combined with the possibility of opinion change, if necessary. It is obvious that the Delphi methodology and the integration of the Boston Consulting Group framework will bring many benefits for spatial planning and management processes.
This article examines, in particular, the BCG’s product portfolio analysis and an integration of the Delphi methodology. We have tested this approach in various R&D type projects. So the methodology is not based on one special case but on various experimental R&D projects which have delivered successful results in challenging conditions. Other R&D type projects, where novel methodology has been extensively tested are following pilot studies: (1) The regional strategy process of the Kemi-Tornio region , (2) the strategy process of the Helsinki metropolitan area , (3) the “Arctic Maritime Technology Foresight” study  and (4) the “Regional Business Strategy for North-East Finland 2011–2015” . Thus, this study of the Arctic maritime cluster is not only an experimental pilot study where the methodology has been tested and developed.
In this article, we will describe and elaborate the Arctic Maritime Technology project. This article is not a conventional case study article, because these experimental R&D project cases have been reported extensively in specific research reports. It is not possible to report full case studies in one conceptual article. That is why we focus on one pilot study.
We shall demonstrate the use of the BCG and Delphi integration methodology by one concrete pilot study. We know that the BCG’s analysis tools are primarily intended for business management of corporations and companies. The company’s existence is a precondition that there is something to be sold at a competitive price, and that the company’s products have buyers in markets. We can claim that European regions also should be able to sell their products to replace imports, so that their economic development could be positive, not negative. This approach is called Import Substitution Industrialization (ISI). Another approach is called Export Led Growth (ELG).
Some researchers talk about city states which, by definition, is just indicating the fact that they should be able to pay for their imports [22, 23, 24]. This kind of trade dominated import substitution industrialization (ISI) strategy was typical already in Italian city states. Italian city states like Venice, Milan, Florence, Genoa, Pisa, Siena, Lucca, Cremona and many others became large trading metropolises, able to conquer independence from their formal sovereigns . In Europe, this kind of traditional city state thinking could be helpful in fighting against unemployment and economic hardships. Every city should be interested in their future “portfolio”.
Against this background, it is quite a natural idea that at least a part of the tool package of the BCG is useful for the development planning and strategy formulation of European cities and European regions. In addition, this key idea is supported by the corporate-oriented regional development. There are more and more enterprise networks and other socio-economic networks that support spatial development and management. Actually, our methodology integration approach provides an interesting possibility of deeper public-private partnership in spatial planning processes. To provide a smart integration of different approaches, we first discuss special characteristics of alternative approaches and methods.
The Delphi methodology is a multi-round expert survey. In the Delphi interview process, several iterative rounds of interviews happen. During the Delphi process, statistical and other feedbacks are given to experts by the Delphi manager. The interviews are always carried out anonymously [29, 16, 17, 25]. The Delphi method is not necessarily a computer supported system, although nowadays many Delphi studies are based on the internet and software platforms.
The Delphi method has its birth roots in the Second World War. During the war times, radar observations required the foresight to learn how the two radar observations of the enemy could predict future movements of the enemy. This doctrine gave rise to a strategic anticipation of the U.S. Air Force and in 1945 a new R&D Center, the RAND Corporation, was established. Inside the RAND, the Delphi methodology was invented and developed [p. 10, p. 13, p. 28, pp. 265–268]). The Delphi methodology was applied initially to predict precisely future events, especially focusing on (1) probability, (2) desirability, (3) strategic importance and (4) feasibility of various operations and events. In this kind of foresight activities and related expert processes, a consensus of the experts was emphasized. This consensus orientation and methodological approach of the Delphi was called later the Conventional Delphi .
Later, Harold Sackman (1975)  criticized the conventional Delphi methodology strongly in his book “The Delphi Critique”. He decided to categorize the conventional analysis on the Delphi methodology as an unscientific methodology. Of course, the main critique of Sackman was focused on the ways the Delphi methodology was used. In spite of his critique, Sackman did not criticize the Policy Delphi methodology. His critical analysis was based, on the one hand, on the framework of public opinion research. However, he confessed that the Delphi methodology could be an effective approach, if different interest groups are identified and analyzed in detail. At the same time, Prof. Murray Turoff presented a milestone book entitled “The Policy Delphi”. Later in his book, Sackman noted that this type of Delphi methodology (Policy Delphi type of approach) can have a future as a methodology. Later, Delphi methodology expert Osmo Kuusi in his doctoral dissertation (1999)  came to the conclusion that the Delphi methodology is not a public opinion poll or survey but rather a structured theme interview system. Also, Myllylä  in his doctoral dissertation (2007) agreed with Kuusi (1999) about his methodological insights.
The problem is not a valid approach for a precise analytical method;
Collective and subjective estimates are potentially useful in problem solving;
The problem is extensive or complex, and participating in the review there is no common language or method;
The test problem of determining the need for more people than the efficient working group could be;
The dispute between the experts forced to use intermediaries, and
It is necessary to act anonymously, in order to avoid the majority of strong personalities and opinion leadership of the group.
Finnish methodology expert Jari Metsämuuronen [30, p. 300] also noted the Delphi method has the advantage that it is easy to connect to both qualitative and quantitative research approaches. This example of time-series and expert information joining strong perspective trends (SPT) in the detection part of the Delphi method is based on this very well, according to Delphi methodology expert Myllylä [12, p. 73–74].
In addition, the Delphi method can be used to create a learning and innovation environment, as well as communications and media environment.
When comparing the BCG product portfolio analysis and the Delphi method, it may be noted that beneficial points 1–4 are common to both methods. Instead, beneficial advantages 5–8 are typically the strength of the Delphi methodology, but it can be assessed that these benefits could help to make better BCG product portfolio analyses.
For example, Rowe and Wright , and Myllylä , emphasize the selection of a panel of experts as one of the most critical issues to the success of the methodological application. The literature of expert judgments is broad [32–34]. The key issue is to recognize that each expert has some competences, but also some interests in the premises. The expert’s financial or other vested interest has a powerful impact on expert judgments. This is illustrated by the Finnish proverbs “nobody saw her own pruning” or “the songs you are singing are the ones which give to you the bread you eat”. These Finnish traditional proverbs refer to personal vested interests, which can be meaningful in the analysis of expert judgments. The one actually managing the Delphi study (the Delphi manager) must be very careful in the selection and recruiting process of the Delphi panel experts. This critical attitude will help Delphi manager(s) to select the best possible experts. It is also challenging to identify really knowledgeable people. One can use the so-called “Interest-Competence” Table as a helping tool in recruiting members of the expert panel. This I-C table helps to define what kinds of expertise and interests you want to have in the expert panel [See 12, 35]. Typical interests of knowledge are (1) technical interest of knowledge, (2) pragmatic interest of knowledge and emancipatory interest of knowledge. Technical interest is connected to the concept of work, pragmatic interest is related to the concept of language, and emancipatory interest is related to the concept of power . Of course, a balanced interest-competence pattern is a solid base for successful Delphi studies. Awareness about interests of knowledge is an important aspect of planning a successful Delphi study.
The formation of the questions is one critical phase of the Delphi methodology, and also a very critical phase of a successful Delphi study. Formulation of the issue is worth of time allocation and critical consideration of Delphi manager(s). On the other hand, in the Delphi interview process, several rounds of interviews provide an opportunity to develop good new questions, so that the first few round(s) of expert interview questions are the theme(s) of the interviews for typical open theme questions, or themes. The first interviews can lead to highly structured arguments and statements , which can be tested by other evaluations of the expert panel. The formulation of questions to experts has many qualitative impacts of what kinds of answers you will get.
Although the Delphi method can be regarded as a method, it can be used in co-ordination of various other methods (such as the BCG’s analysis tools, SWOT, Benchmarking techniques etc.), so to ensure the reliability of the results would be better if the results of a Delphi study could be compared also with evidence produced by other methodological tool and management techniques. For example, it may be useful to compare the results of the Delphi study with the results and the evidence produced in other ways.
For example, we want to analyze the future developments of a region in a Delphi study, so it would be good to compare the results of the Delphi study with other available analyses, which confirm that a scenario or a vision of conclusions of a Delphi study are feasible. Another viable and helpful method is to combine statistical time-series data, and the empirical results of the Delphi study. Such control methods have been demonstrated by Petri Tapio  and Yrjö Myllylä  in their doctoral dissertations.
Myllylä has presented in his doctoral dissertation  a verifying application of the Delphi methodology. According to this methodological application, the actual Delphi expert interviews and conclusions (about scenarios, visions, missions, strategies or thematic conclusions) are presented to the subject matter experts (from different scientific fields), which once more discuss the Delphi results and evaluate the relevance of key findings of the study. In this final expert assessment, conclusions and assumptions are submitted to experts’ criticism and reflection. This kind of final expert evaluation probably increases the reliability of results and scientific quality and validity.
Assuming the existence of five critical Delphi methodology factors presented above, and linking them to the BCG’s Product Portfolio Analysis, the Delphi methodology can produce useful outputs to the BCG’s Product Portfolio Analysis. Thus, we propose that the Delphi methodology can be used to verify product/service categorizations of the Matrix of the Boston Consulting Group approach.
The selection of the target group of BCG application is an important issue. In our pilot study we had various target groups, among others industrial leaders, ministry officials, regional development authorities, experts of public agencies and academic scholars. In our industrial pilot study, most experts and ministries liked this approach, because it was concrete and it was linked to business interests of maritime cluster companies. The maritime cluster is one key industrial sector, which provides many employment opportunities. Most ministries prefer to analyse new business and employment opportunities.
A conventional definition of the business portfolio is that is it the collection of businesses and products that make up the firm. The best business portfolio is one that fits the firm’s strengths and helps exploit the most attractive business opportunities. A successful company has a diversified product portfolio , where management is doing things right and leadership is doing the right things . This same idea is relevant for all the stakeholders of regional planning. They must also think about portfolio management and the structure of business fields in the region. This requires a well-defined and well-scoped strategic process. The Delphi methodology can help to build up such well-defined and well-scoped processes. The execution of portfolio management is a typical challenge for many organizations. Building strategies and visions is not the biggest problem for modern leaders and decision makers [37, p. 21–22].
According to Henderson, to be successful one, a company should have a portfolio of products with different growth rates and different market shares. “The portfolio composition is a function of the balance between cash flows. High growth products require cash inputs to grow. Low-growth products generate excess cash. Both kinds are needed simultaneously.“[43, p. 35].
Margins and cash generated are functions of market share. High margins and high market share go together. This is a matter of common observation, explained by the experience curve effect.
Growth requires cash input to finance added assets. The added cash required to hold share is a function of growth rates.
High market share must be earned or bought. Buying market share requires an additional increment of investment.
No product market can grow indefinitely. The payoff from growth must come when the growth slows, or it never will. The payoff is cash that cannot be re-invested in that product.
“Question Mark Product”, it is evolving, and its market share or volume, from the company’s point of view, is small, but growing fast. In the next phase, the product may become known and more popular.
“Star Product”, it is a product whose market share is large. This means that the company or its sub-unit is having a high growth rate. The “Star Product” has not become yet necessarily productive, but requires external funding, cash flow, which will lead to next phase of development.
“Cash Cow Product”. This category includes products or industry clusters that are already very large, so the market share or volume, or from the company’s point of view, the market share is high. The return on investment (ROI) of “Cash Cow Product” may be small, but this product is probably the most important cash flow creator for a company. Cash cows are typically low-growth businesses or products with a relatively high market share. These are mature, successful businesses with relatively little need for new investment. They need to be managed for continued profit so that they continue to generate the strong cash flows that the company needs for its Stars and Question Marks.
The fourth group is “Pets”, “Dogs” or “Stray Dogs”, whose market share or volume is small and growth rate is low as well. Actually, these products should not necessarily be there, this represents a phase of a product’s life cycle. Unsurprisingly, the term “Dog” refers to businesses or products that have low relative share in unattractive, low-growth markets. Dogs may generate enough cash to break-even, but they are rarely, if ever, worth investing in.
Build Share Strategy: here the firm can invest to increase market share (for example turning a “Question Mark” into a “Star”)
Hold Strategy: here the firm or its business unit invests just enough to keep the business unit in its present market position.
Harvest Strategy: here the firm reduces the amount of investment in order to maximise the short-term cash flows and profits from the business unit. This may have the effect of turning “Stars” into “Cash Cows”.
Divest Strategy: the company can divest the business unit by phasing it out or selling it - in order to use the resources elsewhere (e.g. investing in the more promising “Question Marks”).
To sum up, these strategies are basic strategic options for firms but we can also claim that these options are strategically relevant for cities and regions. Hold strategy means very passive spatial policy. More proactive strategies are Build Share Strategy and Divest Strategy. Harvest Strategy is an active strategy. If a city of a region with an industrial or service cluster wants to develop its economy, proactive strategies are needed.
Other key concepts of the BCG framework are (1) strategic and natural competition, (2) experience curve as a key issue in markets, (3) the rule of three or four, (4) market share and (5) pricing paradox [see 42, p. 35]. To use the BCG framework, these concepts should be understood, strategically, as an essential key issue in the portfolio analysis or the BCG matrix. We focus on this key issue in this article.
Characteristically, Cash Cows generate large amounts of cash, in excess of the reinvestment required to maintain share. This excess need not, and should not, be reinvested in those products. In fact, if the rate of return exceeds the growth rate, the cash cannot be reinvested indefinitely, except by depressing returns. [43, p. 35].
“Products with low market share and slow growth are Dogs or Pets. They may show an accounting profit, but the profit must be reinvested to maintain share, leaving no cash throw-off. The product is essentially worthless, except in liquidation.” [43, p. 35]. Thus, from a spatial perspective, if there is too much “dog’s production”, this is not a good policy choice for the future develop and success of a region.
“All products eventually become either Cash Cows or Pets (Dogs). The value of product is completely dependent upon obtaining a leading share of its market before the growth slows.” [43, p. 35]. This is basic logic of economic development according to the BCG portfolio analysis framework. That is why the strategic importance of Cash Cows is so crucial for regions and cities.
Low-market-share, high-growth products are the Question Marks. They almost always require far more cash than they can generate. If cash is not supplied, they fall behind and die. Even when cash is supplied, if they only hold their share, they are still pets when the growth stops. The question marks require large added cash investment for market share to be purchased. The low-market-share, high-growth product is a liability unless it becomes a leader. It requires very large cash inputs which it cannot generate itself. [43, p. 35–37].
To have some future potential, regions need pay serious attention to Question Marks. A lot of ideas and inventions are needed in order to produce real innovations for markets. Typically, this means active science, technology and innovation policy activities. There will not be any Question Marks to observe, if there is not some kind regional innovation systems and efforts of innovation management.
According to the conventional BCG definition, “The high-share, high-growth product is the Star. It nearly always shows reported profits, but it may or may not generate all of its own cash. If it stays a leader, however, it will become a large cash generator when growth slows and its reinvestment requirements diminish. The star eventually becomes the cash throw-off for reinvestment elsewhere” [43, p. 36]. Successful, agile and fast growing start-up companies are typically the Stars in the regional economy. Every region needs such dynamic companies to be competitive and successful in the future.
The logic of the BCG matrix thinking is the following: “The payoff for leadership is very high indeed if it is achieved early and maintained until growth slows. Investment in market share during the growth phase can be very attractive if you have the cash. Growth in market is compounded by growth in share. Increases in share increase the margin. High margin permits higher leverage with equal safety.” [43, p. 35–37]. This statement defines the basic logic of a successful local economy. It indicates that decision makers must pay attention to a balanced portfolio of enterprises and their products and services. For example, if there are not enough Question Marks, we cannot expect to see very many stars in the local economy. If there are too many Dogs, we probably have too little an amount of Cash Cows in the local cluster economy. Some kind of balance in the local portfolio is a desirable target.
“The resulting profitability permits higher payment of earnings after financing normal growth. The return on investment is enormous. The need for a portfolio of business becomes obvious. Every company needs products in which to invest cash. Every company needs products that generate cash flow. And every product should eventually be a cash generator; otherwise it is worthless.” [43, p. 35–37]. Here is the key issue of regional development defined. Regions and cities need a balanced product portfolio, which creates balanced cash flows, and are thus cash generators. Actually, this statement means that regions and cities need to avoid Disaster Sequence of the BCG framework.
We can conclude that only a diversified company or a region with a balanced portfolio can use its strengths to truly capitalize on its growth opportunities. The balanced portfolio has: (1) Stars (indicating many potential start-up companies), whose high share and high growth assure the future, (2) Cash Cows that supply funds for sustainable future growth, (3) Question Marks which are to be converted into Stars with the added funds. We can also say that Pets (or Dogs) are not necessary for a dynamic regional economy. If there are large numbers of Dogs (Pets), they are an evidence of failure either to obtain a leadership position during the growth phase or to get out and cut the losses [43, p. 37].
The Delphi methodology is one of the most well-known and popular of future research methods [see e.g. 10, p. 268). It is an expert interview process which is based on two or more iterative rounds of expert interviews. Typical aspects of the Delphi methodology are anonymity and emphasis of consensus of experts with associated statistical evaluations. In many conventional Delphi studies, consensus view of experts had been emphasized, but the criticism of Sackman (1975)  caused many reformulations of the Delphi methodology. Such reformulations were (1) more detailed stakeholder analyses of experts, (2) deeper statistical analyses, (3) greater samples of interviews/questionnaires and (4) more critical attention to the logic of argumentation of experts. Today, Delphi studies are more valid than in the beginning of Delphi studies in 1960s because of a more critical approach to validity questions.
Of course, the Delphi methodology always includes the element of feedback. In this methodological case we emphasize a feedback process which happens after the Delphi summary report has been reported. According to our experiences, this additional feedback process improves the quality of Delphi studies.
The problem is very complex and there is not a valid approach for a precise analytical method;
Collective and subjective estimates are potentially useful in the problem solving process;
The problem is extensive or complex; and participating in the review, there is no common language or no proper method;
Analyzing research problem, there is a need for more people than the efficient working group could be;
Because of the dispute between the experts there is a forced need to use intermediaries, and
It is necessary to act anonymously, in order to avoid the group thinking problems and biased influence of a majority of strong personalities and opinion leadership of the group.
Critical link A between Delphi and BCG approach is that we must define the sectors of business by the BCG Portfolio Division in the Delphi Questionnaire;
Critical link B between Delphi and BCG approach is that we must define the Success Sequence in the Delphi Questionnaire;
Critical link C between Delphi and BCG approach is that we must define the Disaster Sequence in the Delphi Questionnaire;
Critical link D between Delphi and BCG approach is that we must integrate a feasibility analysis in the Delphi questionnaire: the feasibility analysis of the implementation of Question Marks, Stars, Dogs and Cows?
Critical link E between Delphi and BCG approach is that we must integrate probability analysis in the Delphi questionnaire: the probability analysis of implementation of question marks, stars, dogs and cows?
Critical link F between Delphi and BCG approach is that we must integrate a desirability analysis in the Delphi questionnaire: the desirability analysis of the implementation of Question Marks, Stars, Dogs and Cows?
Critical link G between Delphi and BCG approach is that we must integrate an importance analysis in the Delphi questionnaire: the strategic importance analysis of the implementation of Question Marks, Stars, Dogs and Cows?
Critical link H between Delphi and BCG approach is that we must integrate a risk analysis in the Delphi questionnaire: the risk analysis of the implementation of Question Marks, Stars, Dogs and Cows?
Critical link I between Delphi and BCG approach is that we must integrate an action analysis in the Delphi questionnaire: the action analysis of the implementation of Question Marks, Stars, Dogs and Cows? And;
Critical link J: other BCG characteristics (e.g. broader typology of sectors of business) integrated into the Delphi methodology.
Definitions of the relevant sectors of business,
Definition of the Success Sequence,
Definition of the Disaster Sequence,
The feasibility analysis of the implementation of Question Marks, Stars, Dogs and Cows,
The probability analysis of the implementation of Question Marks, Stars, Dogs and Cows,
The desirability analysis of the implementation of Question Marks, Stars, Dogs and Cows,
The strategic importance analysis of the implementation of Question Marks, Stars, Dogs and Cows,
The risk analysis of the implementation of Question Marks, Stars, Dogs and Cows,
The action analysis of the implementation of Question Marks, Stars, Dogs and Cows,
The analysis of other possible BCG characteristics.
Disaster Sequence is a conventional term of the BCG matrix analysis. It happens when a product which is currently a Cash Cow, due to competitive pressure, might in the future be moved to a Star. This product falls out from the competition and it is moved to a Question Mark and, finally, it may have to be divested because of its low market share and low growth rate. The disaster sequence might happen because of wrong strategic decision making in relation to the decision environment. The disaster sequence, of course, is not the Success Sequence. The Success sequence of the BCG matrix happens when a Question Mark becomes a Star and finally it becomes a Cash Cow. This is the best sequence, which really gives a boost to profitability and growth. The Success Sequence, unlike the Disaster Sequence, is entirely dependent on the right decision making of leaders and managers .
These critical links (1–10) are also typical research questions in the field of futures studies and also in the field of the more pragmatically oriented foresight studies.
In this section, we shall present results of a pilot study: the Finnish maritime cluster analysis with the BCG approach. In this study, we are using 4-product type category (Stars, Question Marks, Cows and Dogs), thus leaving other product types (War Horses, Problem Children and Dodos) out of the detailed Delphi study and analysis. In our pilot study, we decided to keep the analysis understandable, transparent and simple enough for the Delphi expert group.
There were many good reasons why the BCG analysis was useful to perform for the Arctic maritime cluster in Finland. Since 2010, the Maritime cluster has been named to be the so called restructuring industry by the state in Finland. Therefore, the state authorities required that new opportunities would have to be evaluated. The state has supported the evaluations of the maritime cluster with financial resources.
In particular, the Arctic region’s economic potential has been seen as new growing business opportunity. Expert evaluation is a necessary strategic activity in that we want to understand the future options of the maritime cluster in Finland. There must be private firms and public actors integrated into the evaluation process. Public-private partnership is needed in cluster development. Actors in academic research, education, decision makers in the government, politicians and independent experts must participate in the integrated foresight process to perform a participatory foresight process. The goal of the project was not only to develop an individual business strategy but to develop a broader umbrella strategy for the maritime cluster. This kind of evaluation process supports the whole cluster and its future developments.
The Arctic maritime technology foresight project was financed by the Uusimaa Centre for Economy, Transport and Environment (in Finnish ELY-Centre) for Uusimaa county. The aim was (1) to define the Arctic maritime cluster itself, (2) recognize so called mini-clusters inside the cluster, (3) identify training and development needs and (4) identify the cluster’s needs for internationalization and public support [21, p. 7–9].
In the Arctic maritime technology foresight project there were (1) three rounds of expert interviews and (2) so-called Maritime Futures Workshop, as well as (3) a Feedback-round phase. In this case, the Feedback-round phase meant that the project’s interim report was placed on a public display to collect feedback, detailed comments and fast strategic action. Finally, 137 experts participated in the interview rounds or the Futures Workshop during 2012 [21, p. 15–17].
The pilot study began by formulating the main research themes, which were derived from the project’s strategic goals. These themes were used during the pilot interview round. In thematic interviews there were 14 experts who were interviewed. All the interviews were performed face to face. Pilot interviews were summarized and reported in a memo document.
The memo document was the basis for a questionnaire for the first Delphi round. During the first round, the key issue was the BCG matrix analysis with expert evaluation. The interviews were conducted mainly face to face. The second Delphi study round involved 43 experts. The results of the first Delphi round were presented in the Maritime Futures Workshop at the Arctech Helsinki Shipyard in October 2012. In the Maritime Future Workshop, there were 70 experts to participate in the foresight activities and evaluations.
The interviews of the second Delphi panel deepened BCG analyses with insightful reflections. The results of the first Delphi round were presented to the second Delphi round experts. The panelists were asked to evaluate how the future vision 2030 will be reached. The second Delphi round interviews were conducted electronically by using eDelphi-software (www.edelphi.fi). In sum, 39 experts participated in the second Delphi round panel. This process followed the systemic logic of the Feedback Delphi-model (see Fig. 3).
Products with low market share and slow growth are Dogs (or Pets). They may show a small accounting profit, but the profit must be reinvested to maintain market share, leaving no cash throw-off. The product is essentially worthless, except in liquidation [43, p. 35]. If there is too many “dog’s production”, this is not a good starting point for future developments because there are no resources for future potential investments. Dogs can eat in this case “the future potentials”.
All products, eventually, become either Cash Cows or Pets (Dogs). The value of product is completely dependent upon obtaining a leading share of its market before the growth slows [43, p. 35]. This is basic logic of the economic development according to the BCG portfolio analysis framework. That is why the strategic importance of Cash Cows is so important for companies and states. Without “Cash Cows”, no state, region, company or corporation can survive.
Low-market-share, high-growth products are the Question Marks. They almost always require far more cash than they can generate. If cash is not supplied, they fall behind and die. Even when cash is supplied, if they only hold their share, they are still pets when the growth stops. The question marks require large investments to get larger market share. The low-market-share, high-growth product is a liability unless it becomes a leader in the market place. It typically requires very large cash inputs that it cannot generate itself [see 43, p. 35–36].
If regional “states” want to have some future potential, regions need to pay serious attention to Question Marks. Without Question Marks, no region has a prosperous future. Typically, this means active science, technology and innovation (STI) policy activities. It is good to remember that there will not be any Question Marks to observe, if there is not some kind of regional innovation system with R&D activities.
Here we can present a short summary of the key results and findings .
The analysis of stars
The results show that in the year 2030 Star sub-fields are (1) environmental technology – including oil spill response technology and meteorology, (2) weather, measurement, and (3) monitoring systems (more than 50 % of the respondents). Transportation and logistics and ICT, offshore technologies were sub-fields. At least 40 % of the respondents of the questionnaire felt that they are the Star field clusters at the time.
The analysis of question marks
Sunrise sectors of the Question Marks are most noticeable: (1) Oil and gas exploration, (2) subsea technology, and (3) safety and rescue products (more than 50 % of the respondents). In particular, the oil and gas explorations are seen now as a major Question Mark, and as a potential sunrise sector in the Arctic region. This expert view is shared by 65 % of the respondents. The second sub-field, subsea technology, as well as safety and rescue products are seen (more than 50 % thought so) as sunrise sub-fields. In a similar way, respondents evaluated the ship navigational systems and controls.
The analysis of cows
As a Cow sub-field, shipbuilding is seen most clearly among experts (more than 50 % of the respondents evaluated that). Also, transport and logistics systems have a strong role in the sub-fields of dairy cows business. As sources of cash flow, cash cows fields, were seen in 2030 more clearly ship yards. More than 50 % of the respondents analysed the situation in this way. The shipyard strategy could mean the construction of icebreakers and other specialised ice class vessels. The second strategic cow field was transport and logistics products and systems. This might mean to build new cargo handling equipment at sea ports and on ships. For the year 2030, this strategy would mean more cash flow sources, if stakeholders invest in the pre-conditions of the shipbuilding industry. It would be important to utilize the existing infrastructure and industrial capacity in yards in Finland. For the Delphi panel, Finnish and Russian co-production opportunities, and the promotion of these activities are important follow-up task in the coming years. Also co-operation with the Kronstadt ship yard and Finnish Shipyards is important for the strategy of the Finnish maritime cluster. Cities of Kronstadt and St. Petersburg are planned to be production plants for the Arctic ships yards. Finns should be actively monitoring the strategic situation in the Baltic Sea region, because of many other market players.
The analysis of pets
Some shipyards were seen as Dogs (or Pets).
The results of a deeper BCG analysis
If we look at the most important advocated follow-up projects of the maritime industries, the results of the BCG matrix analysis in 2030 and the respondents’ comments about the most important development projects in the years 2013–2017 indicate the following maritime industry strategy: (1) the development of training in offshore-theme, (2) the development of infrastructure of oil spill response laboratory and training, and (3) the development activities to develop ice management activities and a progressive simulation environment.
To create a better ice laboratory is one of the most important strategic development issues in the future. This strategic project helps also indirectly to maintain the competitiveness of shipyards and the maritime cluster in Finland. The expert panel identified as key activity to develop the Arctic marine technology exposition of Finnish international communications and promoting export strategy of the maritime cluster. Physical conditions of the Finnish yards, competitiveness and productivity must be improved with tailored sub-strategies such as training of supervisors, leaders and field managers.
More particularly from the view of shipbuilding for the Arctic region, the economy of Uusimaa county should be focused on one vessel type: research vessels, icebreakers, supply ships, ice management vessels, oil recovery vessels, rescue boats and hybrid-/multifunctional vessels. The Delphi panel results indicate that in 2030, there will be new emerging fields, whose first steps and roots are probably already seen. The development pre-conditions for these emerging new fields need more strategic attention now. The Delphi panel noted important follow-up activities that support the development of these sub-fields in the following way: (1) offshore education and training development, (2) reinforcement of project management skills, (3) marine and mining opportunities identification for joint exploration, (4) the development invention and innovation capacity to develop these issues further, and (5) improvement of competence and training capacity of anticipation/foresight.
In the fields of industrial and corporate foresight, decision makers have to plan their operations some time in advance, and their decisions take effect at some point in the future. This is the case of the maritime industries we analysed in this article. These decisions should not be based on current market circumstances but the expected circumstances when the decisions become effective in markets. Nobody actually knows exactly what will happen in the future, but the best way is to perform foresight and prospective futures studies. In this article, we discussed one promising way to do these kinds of prospective industrial analyses. This study and some other Finnish background studies, which have utilized the Boston Consulting Group framework, clearly show that the Delphi-BCG approach is suitable regional foresight and development tool. The methodology and its applications have wide potential in Europe, which is now facing big economic and industrial transformation challenges. We call this approach the BCG Delphi, which combines two influential futures oriented methodologies.
The Delphi method is one of the best management techniques, which is constantly evolving as expert system. The Delphi method can be used with other “programs”, with analytical decision support tools and with many other key management techniques. Methodological links can be based on both quantitative and qualitative foresight methods. Typical diagnostic, prognosis and prescriptions phases can be linked to the Delphi methodology. The Delphi methodology can be an elementary and integrated part of information technology design and architecture. It is important to understand that the Delphi methodology requires the development of user communities in developing it. Good training and facilitation for the use of the methodology improves the value added and the quality of results.
The key scientific contribution of this article is to connect two generic futures oriented business planning methodologies, the Delphi methodology and the Boston Consulting Group methodology. It is one management innovation which can help the Delphi developer community get more value added from the use of the Delphi methodology. Obvious scientific benefit of our combination approach is that accuracy and objectivity of product positioning is increased. When various experts evaluate product positions of the BCG matrix this produces more reliable results compared to a special case of solo evaluation process.
The pilot study of the marine industries was presented as a concrete example of this new management innovation. The pilot study was focused on analysing future portfolio of the Finnish maritime industries. We can conclude that the combination of these two well-known methodologies worked well in the Finnish case of maritime industries. However, one critical remark is that only a part of the methodological capacity was used in this pilot study. For example, SWOT analyses and STEEPV analyses could be integrated to this framework in a better way in following the BCG Delphi studies. Our study reveals a high potential of BCG Delphi studies in the industrial and corporate foresight fields. Especially, futures of industrial product and service portfolios can be diagnosed and forecast by this novel methodological approach. In this article, we have presented the way how to do it.
A concrete benefit for society in the pilot study was that policy-makers understood how strategically important Cows are for the maritime cluster. According to the evaluation results, yards will be also important cash flow sources for the Arctic maritime cluster in the future. In Finland, after evaluations of our study analysis, the Finnish government woke up to understand the central role of yards. The biggest yard in Finland is Turku yard, it got a new business owner in August 2014. STX Europe sold Turku yard to the state of Finland and to German Mayer Werft. In 2012, when our pilot study was performed, the role of state was not seen very important in yard business. Probably our maritime pilot study results changed the strategic direction of the maritime cluster policy in Finland. This indicates the high policy relevance of this special pilot study.
Of course, an alternative way to strategic studies is the serendipity approach, but planning industrial and service portfolios may require more strategic approaches. Traditional scientific debate is one part of this kind of user-centred development, but the pragmatic development activities are another part of the methodological development. Theoretical and pragmatic approaches should be connected as we have done in this study and article. Current progress in information technology and so called “programmed smartness” provides new promising opportunities to use the Delphi methodology. A concrete example of such activity in the Delphi methodology field is the developer community arena in Finland (see http://edelfoi.ning.com), which has been developed since 2005 to develop Delphi studies from the outset in 2002, created in a Delphi software environment (on the web: www.edelphi.fi). Another good example of current methodological developments is the European Union’s 7th Research Framework Programme 2008–2011, where Delphi 2.0 methodology was used to analyse Wild cards, weak signals (so called WI-WE methodology) in processing and refining empirical foresight results and insight (www.iknowfutures.eu) [see also 44]. There are also many other software Delphi applications for the internet environment. For example, high level Delphi-expert Ted Gordon has developed the Real Time Delphi environment for global projects and applications (see www.realtimedelphi.info).
In the future of Delphi studies, we hope to see more creative accumulation and innovative applications of the Delphi methodology. A promising scientific avenue is, from the perspective of our pilot study experience, industrial and science, technology and innovation (STI) BCG portfolio analyses, which could have a higher validation level by professionally organized expert Delphi panels. This is also one feasible and promising direction for European industrial and corporate foresight studies.
Development Manager Taavetti Mutanen and Project Chief Sasu Pajala at the Centre for Economic Development, Transport and the Environment for Uusimaa are kindly thanked for their financial contribution to the case project Arctic Maritime Technology Foresight from the point of view Uusimaa. Marketing and management expert Ossi Luoma and professor Arto Lahti are thanked for insights and contribution as BCG experts among our above mentioned case project. Colleague and researcher Jon McEwan is thanked for language checking.
Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
- Barksdale HC, Harris CE (1982) Portfolio analysis and the product life cycle. Long Range Plan 15(6):74–83View ArticleGoogle Scholar
- The Boston Consulting Group (1999) The pharmaceutical industry into its second century: from serendipity to strategy. 69 p. The Boston Consulting Group, BostonGoogle Scholar
- Ploetner O (2008) The development of consulting in goods-based companies. Ind Mark Manag 37:329–338View ArticleGoogle Scholar
- Coate MB (1983) Pitfalls in portfolio planning. Long Range Plan 16(3):47–56View ArticleGoogle Scholar
- Stern CW, Deimler MS (2009) The Boston consulting group on strategy. Classic concepts and new perspectives, 2nd edn. Wiley, New JerseyGoogle Scholar
- Lehtinen P, Myllylä Y, Suikkanen A (2001) Osaaminen, Koulutus ja Ennakointi – Kemi-Tornio 2010. (Know-how and Capability Structure, Education and Foresight – Kemi-Tornio 2010.) 192 p.Kemi-Tornion ammattikorkeakoulun julkaisuja Sarja A. Raportteja ja tutkimuksia, KemiGoogle Scholar
- Myllylä Y (2003) Palvelu- ja ihmissuhdetaitoinen Helsinki 2015. (Service and Interpersonal Skills in Helsinki 2015.) 63 p. Helsingin kaupungin opetusvirasto, ammatti- ja aikuiskoulutuslinja, HelsinkiGoogle Scholar
- Myllylä Y (2010) Koillis-Suomen elinkeinostrategiaprosessi 2011–2015. Delfoi-paneelin 1. haastattelukierros. (North-East Finland's Business Strategy Process 2011-2015. The Delphi Panel 1. Round of Interviews.)45 p. Koillis-Suomen kehittämiskeskus Naturpolis Oy, Tulevaisuuden tutkimuskeskus, Turun yliopisto, TurkuGoogle Scholar
- Myllylä Y, Perttunen, M (2011) Koillis-Suomen elinkeinostrategia 2011–2015. (North-East Finland's Economic Strategy for 2011-2015. North-East Finland Cohesion and Competitiveness Programme.)Koillis-Suomen koheesio- ja kilpailukykyohjelma, Koillis-Suomen kehittämiskeskus Naturpolis Oy, Tulevaisuuden tutkimuskeskus, Turun yliopisto http://www.naturpolis.fi/dynamic/Nettiversio_Koillis-Suomen_elinkeinostrat.pdf
- Söderlund S, Kuusi O (2002) Tulevaisuudentutkimuksen historia, nykytila ja tulevaisuus. In: Kamppinen MO, Kuusi O, Söderlund S (eds) Tulevaisuudentutkimus – Perusteet ja sovellukset [Futures Research – Basics and Applications. Suomalaisen Kirjallisuuden Seura, Helsinki, pp. 251–347Google Scholar
- Flechtheim O (1944) Zeitgeschichte und Zukunftspolitik. Hoffmann und Campe, HamburgGoogle Scholar
- Myllylä Y (2007) Murmanskin alueen teollinen, logistinen ja sosiaalinen kehitys vuoteen 2025 (Industrial, Logistic and Social Development of Murmansk Oblast until 2025). 321 p. Dissertation. http://wanda.uef.fi/vaitokset/vaitos2.php?id=936
- Bell W (1997a) Foundations of futures studies. Volume 1: history, purposes and knowledge. Transaction Publishers, New BrunswickGoogle Scholar
- Bell W (1997) Foundations of futures studies. Volume 2: purposes of futures studies. Transaction Publishers, New BrunswickGoogle Scholar
- Turoff M (1975) The Policy Delphi. In: Linstone H, Turoff M (eds) The Delphi Method: Techniques an Applications: 84–101. Addison-Wesley, LondonGoogle Scholar
- Turoff M (2002) The policy Delphi. In: Linstone H and, Turoff M (eds) The Delphi method: techniques an applications. http://is.njt.edu/pubs/delphibook
- Sackman H (1975) Delphi critique, The RAND Corporation. 146 p. Lexington Books, TorontoGoogle Scholar
- Kuusi O (2002) Delfoi-menetelmä. (The Delphi Method.) In: Kamppinen, Matti, Osmo Kuusi and Sari Söderlund (2002): Tulevaisuudentutkimus – Perusteet ja sovellukset. (Futures Research – Basics and Applications.) 204–225. Suomalaisen Kirjallisuuden Seura, HelsinkiGoogle Scholar
- Myllylä Y (2006a) The future of the Murmansk Oblast assessed by three Delphi panels. Fennia 184:1. 53–73. http://ojs.tsv.fi/index.php/fennia/article/view/3732/3523
- Myllylä Y (2006b) Evaluation of the Industrial, Social and Logistical Developments in the Murmansk Oblast by using Delphi Method. In: Rantaniemi, Mare, Kyösti Kurtakko and Katja Norvapalo (2006). Pieces from Peripheries and Centres. University of Lapland Reports in Education 3: 240–258, RovaniemiGoogle Scholar
- Myllylä Y (2013) Arktisen meriteknologian ennakointi – Uudenmaan pk-yritysten näkökulmasta. (Arctic Maritime Technology Foresight from the SME point of view in Uusimaa.) 141 p. Uudenmaan ELY-keskuksen julkaisuja 13/2013. http://www.doria.fi/handle/10024/90791
- Brunsson N, Johannisson B (1983) Lokal mobilisering. Om industriers kommunalpolitik och kommuners industripolitik. Bokförlaget Doxa ABGoogle Scholar
- Harisalo R (1988) Uusi kapitalismi – Paikallinen yrittäjyyskulttuuri ja taloudellinen hyvinvointi. (The New Capitalism - Local Culture of Entrepreneurship and Economic Well-being.)180 p. Yrittäjäin Fennia, HelsinkiGoogle Scholar
- Al-Shafaw M, Guenther C (2012) Citystates. How cities are vital for future of sustainability. SustainAbility, Green-Biz, Ford Motor Company. e-BookGoogle Scholar
- Carruthers BG, Espeland WN (1991) Accounting for rationality: double-entry bookkeeping and the rhetoric of economic rationality. Am J Sociol 97(1):31–69View ArticleGoogle Scholar
- Kuusi O (1999) Expertise in the future use of generic technologies – epistemic and methodological considerations concerning Delphi studies. 268 p. Government Institute for Economic Research, Research Reports 59. J-Paino Oy, HelsinkiGoogle Scholar
- Goldschmidt PG (1975) Scientific inquiry or political critique? Remarks on Delphi assessment, expert opinion, forecasting, and group process by H. Sackman. Technol Forecast Soc Chang 7:195–213View ArticleGoogle Scholar
- Rowe G, Wright G (1999) The Delphi technique as a forecasting tool: issues and analysis. Int J Forecast 15(4):353–375View ArticleGoogle Scholar
- Dinwoodie J, Tuck S, Rigot-Müller P (2013) Maritime oil freight flows to 2050: Delphi perceptions of maritime specialists. Energy Policy 63:553–561View ArticleGoogle Scholar
- Metsämuuronen J (2006) Tutkimuksen tekemisen perusteet ihmistieteissä. (Foundations of Reseach in Social and Humanities Research.)Tutkijalaitos. 1324 p. International Methelp Ky, HelsinkiGoogle Scholar
- Holey EA, Feeley JL, Dixon J, Whittaker VJ (2007) An exploration of the use of simple statistics to measure consensus and stability in Delphi studies. BMC Med Res Methodol 52(7):52–60View ArticleGoogle Scholar
- Bardecki MJ (1984) Participants’ response to the Delphi-method: an attitudinal perspective. Technol Forecast Soc Chang 25:281–292View ArticleGoogle Scholar
- Cuhls K, Kuwahara T (1994) Outlook for Japanese and German future technology: comparing technology forecast surveys. Physica-Verlag, HeidelbergView ArticleGoogle Scholar
- Blind K, Cuhls K, Grupp H (2001) Personal attitudes in the assessment of the future of science and technology: a factor analysis approach. Technol Forecast Soc Chang 68(2):131–149View ArticleGoogle Scholar
- Myllylä Y (2008) Industrial, Logistic and Social Future of the Murmansk Region – Summary of the Doctoral Dissertation by Yrjö Myllylä. 64 p. Publications of the Ministry for Foreign Affairs of Finland 3/2008, Helsinki http://aluekehitys.internetix.fi/fi/sisalto/uutiset/uutiset/ummurmanskkirja
- Habermas J (1987) Knowledge & human interest. Orig. 1968. Beacon, BostonGoogle Scholar
- Tapio P (2002) Disaggregative Policy Delphi: using cluster analysis as a tool for systematic scenario formation. Technol Forecast Soc Chang 70:83–101View ArticleGoogle Scholar
- Markowitz H (1952) Portfolio selection. J Financ 7:77–91Google Scholar
- Drucker PF (2001) Management challenges for the 21st century. Harper Business, New YorkGoogle Scholar
- Ansoff HI, Leontiades JC (1976) Strategic portfolio management. J Gen Manag 4:13–29Google Scholar
- Rink DR, Swan JE (1979) Product lifecycle research: a literature review. J Bus Res 7:219–242View ArticleGoogle Scholar
- Barksdale HC, Harris CE (1982) Portfolio analysis and product life cycle. Long Range Plan 15(6):74–83View ArticleGoogle Scholar
- Hendersson BD (1970) The Product Portfolio. In: Stern CW, Deimler MS (Ed.) (2009) The Boston consulting group on strategy. Classic Concepts and New Perspectives. 35–37. Second Edition. John Wiley & Sons, New JerseyGoogle Scholar
- Myllylä Y, Sajeva M, Kaivo-oja J, Aho J (2011) iKnow Delphi 2.0. Country Report Finland. 126 p. October 2011. iKnow Project, Finland Futures Research Centre, University of Turku. www.iknowfutures.eu http://www.utu.fi/fi/yksikot/ffrc/julkaisut/e-tutu/Documents/eTutu_2011_10.pdf