Open Access

Is the Delphi method valid for business ethics? A survey analysis

European Journal of Futures Research20164:19

https://doi.org/10.1007/s40309-016-0109-x

Received: 29 September 2016

Accepted: 24 November 2016

Published: 3 December 2016

Abstract

Although Delphi has come a long way in the development of the method itself, or even in business organisation, it has not been used at all in business ethics. To fill this gap, we have reviewed the literature on the use of Delphi in business, and particularly in the field of business ethics; we have also evidenced the method’s lack of use in this field, but noted its potential contribution to this research stream. An online survey has been administered to scholars in business ethics that have previously participated in a Delphi survey. The scholars come from nine different countries, and the survey has been held between January 2015 and March-June 2016. The findings show that in the experts’ opinion Delphi is as rigorous, appropriate and useful as any other research method in the field of business ethics, such as focus group, interviews, surveys (online) and case analysis. The Delphi method is assessed anonymously and economically by a group of experts dispersed around the world. Moreover, applying the Delphi method in business ethics could enrich the consensus on limiting the fuzzy area in which ethical business decisions (ethical decision-making) are argued and determined. It is a way of facilitating the search for a solution to the ethical dilemmas delimiting a problem, which is a further advantage of the Delphi technique.

Keywords

ConsensusEthical Decision-Making (EDM)ManagementResearch methodFuzzy

Introduction: is the Delphi method used in ethical decision-making?

Based on the study by Rest [54], Jones [38] developed the concept of moral intensity, in which one of the six components consists of the social consensus that “is defined as the degree of social agreement that a proposed act is evil (or good)” [38: 375]. Many papers have subsequently focused on it; however, and following a review of the empirical literature containing ethical papers and on ethical decision-making conducted by Craft [15], it is confirmed that despite its multiple benefits (predicting technological advances, achieving a better understanding of a subject, and reaching a level of agreement on topics without conclusive information) the Delphi method has not been widely used either to analyse or to identify areas of agreement on ethical business decisions. Nevertheless, the Delphi method could be useful for research in business ethics, as it is now used in the business area, because of its potential in terms of consensus, feedback, and removing group pressure. There are both practical demands, as rigorous methods such as Delphi could be useful for delimiting ethical dilemmas, and Delphi could fill a research lacuna. To support the possibility of adding value in the field of business ethics, it needs to be considered that the Delphi method generates a range of alternative solutions to the issues and problems a researcher faces [43], to avoid ignoring ethical decisions. Quite the opposite in fact, Delphi should be considered a potential way of resolving research questions in business ethics, or at least of helping to demarcate ethical problems in order to reduce the fuzzy space in which ethical decisions are made. Precisely this method could approach the positions of Coates [12] and Mitroff [46], where the former revealed the mistrust that ethics produces in him from a utilitarian perspective, whereby the latter provides him with a way of responding to an unethical point of view, specifically because the appearance of ethical issues in human activities is inevitable. However, to some extent we agree with Coates [12], in his consideration that the extremes are overvalued because “the exceptional should be treated as just that, not as the norm, the mean, the commonplace, or the routine” [12: 348]. As we will argue in this paper, the Delphi method in business ethics could be used to establish the limits of ethical dilemmas, those that are generally accepted (the consensus will permit an overall result accepted by experts participating in the Delphi method). This delimitation is useful, although the results would not be unique; the integrating or diverging positions in a Delphi process will in some way permit demarcating the problem. Consensus will suitably delimit the dilemmas in a narrow area, and if no consensus is achieved it will be an option for establishing new and more specific problems caused by the disruption of positions among experts. In any case, it will be positive for the field of business ethics.

The lack of use of the Delphi method in business ethics is surprising [44, 63] because the Delphi method has several benefits and advantages, which have already been shown in the business area, both for an organisation itself and for forecasting purposes [41, 49, 55]. To fill this gap, this paper seeks (1) to establish the validity and utility of the Delphi method as a research tool in ethical decision-making, determining the fuzzy area in which decisions are applied, and (2) to identify the behaviour of scholars in business ethics in order to assess the contribution this method makes. The research will be based on a literature review of Delphi surveys in the field of business ethics.

Accordingly, we have set out to answer the following research question: “Is the Delphi method valid for resolving business ethics problems?” This research process, although generally ignored by business ethics researchers, has a proven track record in forecasting significant developments in the future, establishing policies and strategic planning, and as a decision-making tool for evaluating business models or concrete business actions [32, 49].

To reinforce and complete these contributions and transfer them to business ethics, an online survey on Delphi methods applied to business ethics has been administered to 21 scholars from nine countries (USA, UK, Spain, Switzerland, Italy, the Netherlands, France, Colombia, and Australia) between January and February 2015 and March-June 2016. This process involves more than 85% of the scholars in business ethics that have taken part in a Delphi method in recent years,1 . 2 As the sample size approaches the population size, the variance decreases, and so our results are only slightly below optimal ones. This survey’s result reflects not only an understanding of the use of the Delphi method in business ethics, but also the behaviour of business ethics scholars as participants in a Delphi method. The opinions of Goldschmidt and Turoff have been taken into account for the pre-test on the survey involving the Delphi method questionnaire, along with those of other practitioners. Their expertise involves both practical and theoretical matters. Goldschmidt and Turoff have reviewed the Delphi survey, and based on their academic expertise in Delphi we have confirmed the validity of the questions, as according to these experts in Delphi “because they are based on the previous literature contributions”. Goldschmidt reviewed Sackman’s comparison of Delphi with physiological standards regarding the evaluation of opinion questionnaires developed by the American Psychological Association [31]. Turoff wrote one of the first papers on this topic in 1970, and he and Professor Linstone went on to write the most cited book about Delphi “The Delphi Method” [43]. Their expertise is therefore beyond doubt.

Our findings suggest that the Delphi method could be on a par with other methods (survey, interview, focus group, and case study) used in business ethics. The experts are extremely honest and highly confident, they do not feel any peer pressure, and feedback is helpful to them; however, they do not give much importance to consensus (one of the potential characteristic of Delphi), with this being referred to as “Consensus Paradox”, probably because argumentation and discussion are key elements in business ethics, but there is a barrier to the possibility that opinions are diluted. Moreover, this paper suggests that using the Delphi method renders it possible to refine the fuzzy application space in which business ethics is embedded, not only to determine the decisions in a value system, but also to establish the value system itself. The aim is therefore to contribute to knowledge in business ethics, both by accepting a new process to resolve, predict or evaluate ethical businesses aspects, and as a tool to improve ethical decision-making.

The paper is structured as follows: We first describe the literature on the use of the Delphi method not only in the business area, but also in business ethics, where the Delphi method can be classified into three groups, namely, Forecast, Decision and Policy. We also provide an overview of the Delphi method’s characteristics. We then present our data-gathering and analysis procedures, as well as a discussion of our results. We end with a pertinent overview of the properties of the Delphi method for its use in applied and fundamental business ethics based on fuzzy logic, with the aim being to establish its benefits and, in particular, refine the boundaries within which ethical business argumentation should take place. The concluding remarks and references are presented.

Overview of the literature on the Delphi method in the business area

The Delphi method was first developed at the beginning of the Cold War to forecast the impact of technology on warfare; however, its consolidation began with the Rand Project in 1950, when the aim was to ask expert opinion to forecast the probability, frequency, and intensity of possible enemy attacks. In a second stage after the 1960s, the technique was used primarily by corporate planners as a forecasting tool for industry and human services, but since then its application has been extended to other purposes, such as exploration, evaluation, or defining previously developed models in a consensual manner.

There are many experts whose contributions increase the quality and understanding of the Delphi method itself. Studies in this subject began before the 1980s [7, 19, 20, 43, 62, 64], continuing over the next decades [34, 55, 67], but it is now in the 21st century that the application has spread to the social sciences, and especially to help resolve business research questions [1, 40, 41, 49].

Global theoretical research into the use of the Delphi method in social sciences has been highlighted by Landeta [40, 41], where it has been considered an appropriate research method in the field; Nielsen & Thangadurai [49], following van Zolingen and Klaassen, have used the classification of the Delphi method based on the research objective in three major areas: Classical, Policy and Decision. It permits classifying the papers published in the business area that have used the Delphi method. Accordingly, the number of papers in the social area, and in particular those with a business focus, have increased since then; for example, to resolve business value issues [58], to increase our understanding of the value chain in retailing [5], to define international business models [18], and to establish the opportunities of business intelligence [11].

With the aim of showing Delphi’s usefulness in the business area, and before analysing its potential use in business ethics, we have used three types of Delphi processes to make the classification. The classical one, whose aim is the forecasting of actions, issues, and factors (e.g., [18, 37]), is called Forecast in the following table (Table 1). The second and third types that could be components of the Goals-Delphi [56, 60], which we will return to in due course, are based more on preferences than on predictions [8]. There are at least two types in this classification level. These Delphi methods are used to determine the strategies carried out to achieve certain future goals, called Policy, and those Delphi processes used to decide what action or actions will be taken to improve the quality of the decision are called Decision. This differentiation is important for this paper’s purpose, namely, to show that it is possible to use the Delphi method for resolving research problems in business ethics, and it also enables us to understand that its applicability in the field of business ethics could be useful in those types as well. In general, the Policy Delphi has been used in business for the medicine curriculum [25], as well as for establishing cultural values [58], the value chain [5], and defining the business intelligence process [11]. The Decision Delphi has commonly been used to evaluate business models [48, 52, 53].
Table 1

Delphi method in business: a purpose classification

Subject area

Study authors

Year

Country / Panel Expert no.

Task & Results

Nature of Delphi

Delphi type

Energy

Pätäri & Sinkkonen

2014

Finland / 11

Benefit: visual and not visual, should review customer relationship

Evaluate a business model

Decision

Medicine/Ethics

DuBois et al.

2014

USA /14

Develop online curricular resources for each one of the highest rated topics.

Delimit Curriculum: Practical vs. Academic

Policy

IT

Huang et al.

2013

Taiwan / 12

By evaluating each CSF in the context of BPCM theories, practitioners can effectively plan, carry out and control the activities of ITSM implementation.

Establish critical success factors (CSFs) related to ITSM implementation.

Forecast

Mobile

Chen & Liu

2016

Asia

Obtain the 10 key elements of mobile business risks according to the SCP model

Review the elements that increase the risk because of the introduction of mobile business

Policy

Management

Schmiedel et al.

2013

13 countries / 36

Four key cultural values supporting BPM: customer orientation, excellence, responsibility, and teamwork.

Review the cultural values of Business Process Management

Policy

Retail

Bhattacharya et al.

2011

Online / 74

Most retailers are still focusing on a small spectrum of Radio Frequency Identification (RFID) possibilities, and not considering a broader perspective.

Review value chain activities. Analyse RFID adoption issues with the potential to maximise usage, and thus benefits.

Policy

International Business

Czinkota & Ronkainen

2005

America, Europe & Asia / 25

Firms will continue their globalisation efforts in two significant and parallel ways: they will pursue economies of scale through standardisation and the ability to leverage resources (such as knowledge) across borders.

Determine the most important and relevant issues in international business and trade over the next decade.

Forecast

Information Service

Chen & Wang

2010

Taiwan / 22

Firms should handle more accurate business information to support their business intelligence (BI) system to make better business decisions.

Establish the characteristics to improve the Business Intelligence System using Hybrid-Fuzzy-Delphi

Policy

Business Intelligence

Müller et al.

2010

Netherlands / 16

39 opportunities and 11 limitations have been identified and rated.

Determine the opportunities and limitations of Service-oriented Architecture applied to business intelligence

Decision

Energy Business

Pätäri

2010

Finland / 36

This paper includes both company-level and industry-level determinants to provide a comprehensive view of the value-creation potential in a mature, basic forestry-energy business.

Evaluate a business model (forestry-energy)

Decision

Sustainability

Caron et al.

2016

Quebec-Canada / 44

This paper focuses on defining the principles and criteria on sustainability in the mineral exploration industry. Business ethics is one of the principles.

Establish the preliminary list of principles and criteria

Decision

All these papers highlight five of the most important characteristics of the Delphi method [51]: feedback, group assessment, communication, anonymity, and a consensus process. It is established “that iteration with feedback allows interchange among the members of the group in a controlled manner” [5: 66]; some assessment of the group increases the responsibility of members trying to do their best [e.g., 5, 48]; a rigorous and continuous communication flow (e.g., [37] place particular attention on communication) is provided in an efficient Delphi process, anonymity is guaranteed between the panel experts at least during the process (special attention has been paid by [5, 48, 52, 53]), and finally, the option to establish a consensus process [4, 16].

Not just these characteristics, but certain other issues are also relevant and necessary to develop a rigorous process. Following Okoli & Pawlowski [51], the choice of experts is one of the most valuable elements for achieving a successful result when applying a Delphi method; the papers reviewed above have covered the selection process and criteria exhaustively (e.g., [5, 58]). Depending on the selection of experts, whatever their expertise and the criterion on which this decision is based, the results will either be useful or simply biased by this process [68]. Secondly, the Delphi process can be continuously iterated until consensus is achieved or disruptions of positions are clear, although a three-round Delphi method is acceptable to the experts [17]. The business papers reviewed have maintained this minimum exigency.

Moreover, the behaviour of experts should also be taken into consideration [55]; for example, their fatigue could compromise the result of the Delphi method [51]. In most of the studies, the number of participants decreases in each round (e.g. [48]), probably because of tiredness. Nevertheless, the participants’ confidentiality and honesty are not in doubt [43] because they are the necessary condition for reliable consensus results, at least in general. We expect business ethics scholars to be aligned with the values of their argument, so it is assumed they will be totally honest, integrated and implicated in the research method’s process and results. Moreover, other situations could affect the results, such as pressure on the group to move the answer in a certain direction [5, 48] or the feedback’s lack of relevance for achieving consensuses (e.g., [36, 55]). The validity and reliability of the Delphi method are evident in these research cases, the former because secure and honest processes are involved, and the latter because the selection of experts, feedback and the anonymity of the participants are guaranteed [59].

Overview of the literature on the Delphi method in the field of business ethics

As mentioned, the Delphi method has not been widely used for research into business ethics in either theoretical or applied studies, but there are some ethical views in investigations in specific fields, such as medicine, the food industry and biotechnology. We have presented them to show Delphi’s potential use in the field of business ethics. Few papers therefore use the Delphi mechanism to resolve ethical business problems. In our analysis of papers in the Web of Science3 we have found only two papers when searching for “business ethics*” and Delphi as topics (including a review of those words not only in the title, but also in the abstract and keywords), namely, “Curricular priorities for business ethics in medical practice and research: recommendations from Delphi consensus panels”, written by DuBois et al. and published in BMC Medical Education in 2014, and “Principles and criteria of sustainable development for the mineral exploration industry”, written by Caron et al. and published in Journal of Cleaner Production in 2016. Both include business ethics aspects, but the first one focuses on medicine and the second one on a specific industry: mineral exploration. These papers are therefore an example of the option of using the Delphi survey technique in the field of business ethics. It is precisely a sample that can be used not only in the business area, but also in business ethics. Those papers are classified as “policy” and “decision” types, and it evidences that the Delphi has been used for these purposes in ethics.

Moreover, we have reviewed the most important journals on business ethics or ethics (see Annex 1) to confirm this situation. There is at least another paper, published in 2008 in Business Ethics: A European Review, and it uses a Delphi method combined with interviews to identify the contexts, mechanisms and outcomes that provide possible explanations of information handling [26]. It is a Decision Delphi that improves the governance of performance measurement systems in public services.

Yet there are a few other papers that seek to adopt an ethical view using the Delphi method. This method has been used with a broader ethical perspective [2, 27] to identify and rank the importance of each aspect of the review conducted by research ethics committees (RECs), as well as to provide input and refine the operational definitions of each one of these factors by polling a sample of institutional researchers, regulatory officials, ethicists, and REC members at the University of Michigan Medical School.

In this line, DuBois & Dueker [24] have applied a Decision Delphi with an ethical component, and they contribute to the responsible conducting of research, using the panellists’ consensus to establish nine overarching objectives that require rethinking common modes of instruction. The results will be used in education and training programmes. It is again a Decision Delphi with ethical components in which forecasting is not the focus, but instead the preference is to improve the quality of ethical and conduct issues, as one of the most valuable aspects of the Delphi process. The method used has been qualified, while the anonymity of the process minimises the limitations of other methods, such as domineering group members, personality conflicts, or groupthink; moreover, on the positive side, it is not expensive and can be adapted to the convenience of participants.

Another paper showing considerable interest in stakeholders, albeit with a methodological view, has been written by Geist [28]. It compares the Delphi method in two ways; on paper and through a real-time computerised model, although the paper format is more highly valued by the experts. The most important contribution the paper makes is that the Delphi process itself increases the interest participants have in the nature of the programme in which they take part in a company. This suggests that the Delphi method is a powerful tool for engaging stakeholders, being yet another reason for its potential use in business ethics. This is consistent with our hypothesis because of its utility not only to the ethical decision itself, but also to the contribution made by the ethical decision-making process or interests.

Other authors, such as Wainwright et al. [66], highlight the importance of ethical aspects, with the Delphi method now being widely used in health and medicine, albeit not in other areas such as business. They do not resolve moral questions, but argue for the use of potential consensus methods to develop a shared understanding of ethical practice. Moreover, they defend the use of vignettes to illustrate the kind of situations that may occur in practice. Support is therefore forthcoming for the arguments on behalf of agreement methods, such as the Delphi method, with the aim being to improve ethical decision-making.

Consequently, although outside the medical, biotechnology or nursing areas, a Decision Delphi has been applied with an ethical perspective to determine ethical leadership characteristics [47]. A panel of leaders with four different views (business, education, political, and religious) listed and assigned values to the characteristics considered to be most important in being an ethical leader. “The analysis of data revealed that there was a great deal of consensus between groups concerning the ethical and leadership traits to which they were asked to assign values” [47: 199]; it increases to 90%, and means an alignment in terms of ethical leadership, although the ethical concept is not exactly the same.

The few papers on business ethics using the Delphi method lead us to compare it with other widely used methods; the review of the Web of Science papers explained previously, and conducted in June 2016 by authors, shows it is the most widely used survey in business ethics (393 papers, 44.91%), followed by case study (283 times, 32.34%) and interview (123 times). The less widely used ones are focus group and the Delphi method, with less than 8.34%4 and 0.1%, respectively. The correlation to the business area in general is quite similar, with the more widely used being survey and case study, with focus group and the Delphi method being the ones least used. However, the percentages are higher for these two last methods (focus group and Delphi) in comparison with other methods.

It is obvious that there are differences across methods, and depending on the research question and objective, one method is better than another; however, the characteristics could also have an influence. It is not the aim of this paper to compare methods, but a quick review of their differences provides a better understanding of the validity and utility of the Delphi method in business ethics.

The next table (see Table 2) compares the methods’ most important characteristics (the first five elements), as well as other variables about the experts, length of the questionnaire, response rate, and cost and speed. It is based on the Delphi method used for business purposes because of the link with business ethics; moreover, the scale and range are based on generalising and making it easier to understand, so it is not universal.
Table 2

Comparing the characteristics and issues of the Delphi method with other methods: a qualitative view

Items/Method

Survey

Case analysis

Interview

Focus group

Delphi method

Classification of the degree of use in Business Ethics Field (1 less, 5 most)

5

4

3

2

1

Feedback

None

Low

Moderate to Low

High

Very High

Group assessment

None

None

None

Very High

Moderate

Communication flow

Low

High

Very High

Very High

High

Anonymity

Yes

None

None

None

Yesa

Consensus process

None

Moderate

None

Low

Very High

Expert range

More than 50

One or more

At least 10

[812]

[10-20b]

Expert (participants) dispersion

Very High

Moderate

Low

Very Low

Very High

Length of questionnaire

Long

Very Long

Medium

Medium

Short

Response rate

Low

High

High

Moderate

Moderate

Cost

Low

Very High

Very High

High

Moderate

Speed

Very Fast

Slow

Moderate

Fast

Very Slow

aThere is anonymity among experts during the Delphi process, but there is no anonymity with the researcher

bWith new technologies and because of the inclusion of computers in the process, with real-time Delphi it is possible to significantly increase the number of experts that could take part in a Delphi process; however, in a highly specific niche of expertise, it is accepted that 10–20 expert participants are enough. Other limits are considered as well: a lower one, for example, with mini-Delphi. In general, we have used the number of experts for the Delphi group according to Schmiedel et al. [58]

The Delphi method is the one that provides more feedback and the highest consensus [51, 65], with moderate assessment of the group and a high communication flow, but with anonymity among participants [5, 47, 52]. There is no overall consensus on the optimal number of participants in each one of the methods, but case study may involve only one, with focus group, interview and Delphi starting with ten [22, 51], and survey needs a higher number of participants [43]. The Delphi method is suitable when the experts’ contributions are highly dispersed (e.g., [22, 58, 65]. Moreover, despite its moderate response rate and slow development speed, the Delphi method has a modest cost [51]. Therefore, the Delphi method will be valid for those research questions that give greater importance to feedback, consensus and the opinion of disperse participants, but without a long pool of questions, and with little time to compile the data, and a modest cash budget [61].

The field of business ethics needs economical methods with short timeframes that provide rigorous and valid research results that can be generalised. Compared to other methods, Delphi is highly specific for those processes requiring feedback; this is not the case for a survey questionnaire. The Delphi method is excellent for those business ethics problems in which the experts are disperse, but the aim is not a first response to questions (a survey does that), as the valuable part is the reflection and group thinking among experts. The advantage of Delphi over a case study is that it initially involves resolving problems, with cases coming second. The iteration of Delphi is a useful advantage compared, for example, to the interview method, in which there is no iteration between different interviewees, or a joint reflection. The iteration is highly valued in business ethics research because the discussion and compilation of thoughts and new views enrich the conclusion [3]. Interview permits feedback, but it is based on a one-to-one application, reducing the global contact and global discussion needed to resolve research questions in business ethics.

Compared with focus group, which is not widely use at all [14], it highlights that “strong personalities can come to dominate and those of a more reticent disposition may be reluctant to speak up” [14: 56]. Focus groups have an objective, the focus content; they are held in a relaxed atmosphere in which experts interact face-to-face and the researcher manages the conversation to ensure it is focused. In business ethics, full participation is positively valued, and Delphi guarantees this in a similar way for all the experts. Moreover, the different views are relevant, and so the dispersion and higher number of participants in the Delphi method compared to focus groups permit the inclusion of more points of view. Business ethics experts are few, but widely dispersed throughout the world, so it is more difficult to gather them together in a focus group, and when this is achieved it is very expensive (money and time costs). The first advantage of using Delphi for business ethics research stems from its principal characteristic that other methods do not develop, namely, it can be used for seeking consensus. This will be developed further here in a next section on the two potential uses of Delphi in the field of business ethics; first, support for ethical decision-making models because of the consistency of their reasoning process, and second, resolving ethical conflicts, or at least reducing their limits of agreement. Neither advantage is achieved by the other methods.

Methodology

An online survey5 has been used to gather information about the business ethics scholars previously involved in a Delphi process, with the aim being to establish not only the validity of the Delphi method for the purpose of business ethics, but also the process and the part experts play in it. This is based on the notion propounded by Linstone & Turoff regarding new technologies, whereby “the ability to communicate, coordinate, collaborate and fully participate with others is the new online order” [42: 1718] that reinforces our choice to use an online survey.

The purpose of the online survey is to add experts’ values to our theoretical arguments. To do so, it has been necessary to establish two conditions: the sample should include only business ethics scholars and those that have previously taken part in a Delphi. This is because only those scholars with both characteristics could integrate and understand both aspects: the needs of business ethics research and the Delphi process and its advantages; firstly, because they have a theoretical understanding of the field, and secondly, because they have taken part in a Delphi process, experiencing and understanding it in practice.

Before sending the survey, and with the aim of validating our short questionnaire, we conducted a pre-test in December 2014 in which five scholars collaborated: two renowned Delphi experts (Goldschmidt and Turoff)6 and two Spanish academics and a British one that have participated in a Delphi in the specifically delimited area of accounting standards [1].7 We have asked them to review the online survey (they are encouraged to improve the quality of the questions), and following their suggestions we have changed a word in the first question, and added a control variable featuring the research area. According to some of the experts “Delphi benefits depend on what the research question is, for some things, Delphi is ideal”, which opens the door to its use for business ethics purposes.

The questionnaire has been constructed as follows: firstly, using the post-Delphi questions developed by Scheibe, Skutsch, & Schofer [57] for the characteristics section; secondly, based on the checklist drawn up by Hasson et al. [34], questions have been formulated on Delphi’s applicability to business ethics, and thirdly, using the variables proposed by Woudenberg [70], the questionnaire has been completed with a review of all the questions (phrasing and wording). The questionnaire’s results only reflect their opinion and valuation of the Delphi process in research questions in business ethics, with descriptive statistics then being used to reinforce and confirm this consensus in quantitative terms.

Ethical experts are defined as “people who are knowledgeable about and skilled at applying moral values in their work context, regardless of their role or rank” [21: 3]; therefore, the ethical experts at academic level are chosen not only because of their knowledge and skills applying ethical values but also because of their publication level [13]. Scholars in business ethics are spread across different countries, so a mailing survey reduces costs and increases data collection, and permits doing so rapidly; moreover, one of the biggest drawbacks of online surveys in this research is the difficulty in locating representative samples. This is overcome here because on the one hand all the scholars have mail, and on the other, apart from being academics with published papers they have previously taken part in a Delphi process (we will explain why in due course). This limits and reduces the sample. This overall drawback of online surveys is not therefore significant, and as we are able to gather the results from almost the whole population (87.50%) there are no statistical limitations arising from the use of a small number of individuals in the sample. No consensus or feedback is required here because the aim is to record the participants’ opinions on Delphi’s use for business ethics research purposes. It has used a simple survey to do so.

A total of 21 completed questionnaires have been returned -a significant number considering the population’s accessible list using an online survey involving experts in nine different countries (USA, UK, Spain, Switzerland, Italy, the Netherlands, France, Colombia, and Australia) between January and February 2015 and March-May 2016. It was initially difficult to obtain information because many of the experts in the field had not taken part in Delphi processes. The list contained 24 business ethics scholars that have previously participated in a Delphi process. The response rate is 87.50%.

Results: characteristics, behaviour and comparison of the Delphi method applied in business ethics

The business ethics experts that have recently participated in a Delphi method have answered a short online questionnaire (see Annex 2) to explain their opinion about its characteristics, their behaviour as Delphi participants, and their opinion about the method compared to other methods.

The next figure (Fig. 1) shows that all the characteristics that define the Delphi method are important for experts (more than three8 points on a five-point Likert scale); however, the most important characteristic is obtaining feedback on the position of other researchers, and the least important one is reaching a consensus. This could explain the lower use of the Delphi method compared to other methods (we will explain why in due course); in other words, consensus is not a major concern in this field, so there is an expected paradox called “Consensus Paradox”.
Fig. 1

Characteristics of the Delphi method in business ethics (a Likert scale of 1–5 has been used)

The behaviour of panellists is important in a Delphi process because of the focus on a narrow group of individuals, and because of the exhaustive feedback with each one of them. In this case, therefore, in which they are participants in a business ethics subject, the Delphi method shows (see Fig. 2) that they considered themselves to be very honest (4.47 on a five-point Likert scale), and they do not feel obliged to move in a certain direction, which means that the researcher has control over the feedback without pressure from the other participants, they have enough time to respond, they are highly confident, and the feedback was helpful to them on a higher-than-average basis.
Fig. 2

Behaviour of business ethics experts in a Delphi method

Figure 3 shows that the selection of experts is the most important step for obtaining qualified and relevant Delphi results that fit the theory (e.g., [51]). As in other areas, in business ethics it seems that the selection of experts and the elimination of extreme positions are more important than researcher bias and the fatigue of experts. Moreover, the importance given to the elimination of extreme positions in business ethics could mean a relative importance for the centralisation of some of the answers to research questions in that field, although consensus was not the most important point for them (see the previous figure).
Fig 3

Descriptive variables that highlight the use of the Delphi method

Value has been given to the use of the Delphi method by comparing it with other methods (see Fig. 4): focus group, interview, survey, and case analysis. The comparison has shown that the Delphi method is no worse than any other method because there are no experts that consider this to be true (no one scored the Delphi method with one or two on the five-point Likert scale), but 50% of respondents consider it much better (scoring the Delphi method compared to other methods with four or five on the five-point Likert scale). The median is three for the first three methods and four for the comparison with case analysis. The Delphi method’s usefulness is therefore similar to focus group, interview and survey, but better (close to four) when compared to case analysis.
Fig. 4

Comparing the Delphi method with other methods

The Delphi method for refining the fuzziness of applied and fundamental business ethics: a proposal

It is clear that little has been published in the field of business ethics using the Delphi method, and as we know, it is not used much either for research into business ethics. Moreover, as we have shown in the previous section, scholars in business ethics that have played an active part in a Delphi process think that, apart from being useful in general, Delphi will contribute to business ethics research questions. This section therefore shows how and why Delphi will improve the ethical-decisions based on the utilities in other specific areas, and the argumentation is based on the fuzzy logic in which there are no single results, but instead a whole range of them. It is because business ethics decisions are not usually individual ones, but instead come in a range; however, there are many casuistic situations, statements and different positions that make difficult or impossible to make decisions in the field of business ethics, not even narrowing down the problem to be solved [14]. In this regard, the Delphi process could be understood using fuzzy logic to clarify the reasons for using this method with business ethics in mind.

As described earlier, there are different types of Delphi. Historically, the Delphi method has been used for different purposes, with perhaps the best known being for obtaining predictions about future scenarios subject to uncertainty [35, 50]; as Godet explains “the most frequent objective of Delphi studies is to bring clarity to a particular decision which may be clouded by a certain amount of uncertainty” [30: 75]. Nevertheless, the Delphi survey technique has also been used for decision-making in relation to the resolution of social issues, such as the use of land, population growth, urban development, pollution, agriculture, or organ transplants [39]. This second perspective originally received the name of Goals Delphi [56], and is used for ethical purposes.

The main difference between the traditional and Goals Delphi is that the former has a prospective basis because it seeks to anticipate the future, while the latter has a constructive nature, as it is used to reach consensus on possible future actions, some of which have a clear ethical component in the decision. More recently, certain authors (e.g., [44, 45]) have proposed the use of an Ethical Delphi, as a specific application of this methodology, with the aim being to resolve ethical questions, or at least establish the dimensions of ethical problems. “An ethical Delphi is an iterative participatory process between experts for exchanging views and arguments on ethical issues. The method is structured around the notion of a virtual committee where the exchange of ideas is conducted remotely through a series of opinion exchanges (in the form of ‘Rounds’)” [44: 5]. This type of Delphi is particularly useful when the issues are controversial, or may have a present or future impact on a large number of stakeholders with different interests, or affect or be affected by sensitive public policy. To date, this type of Delphi has been exclusively applied in relation to biotechnology, an area in which the authors inserted this proposal.

The Delphi method could have two different uses: on the one hand, in the field of applied ethics, where it could be a good technique to support ethical decision-making models, not because of the number of opinions gathered, but because of the consistency of their reasoning and arguments. However, other Delphi methods may be applied to resolve fundamental ethical conflicts, where the goal would not be to reach an immediate agreement, which a priori is impossible, but to establish the limits of disagreement, so as to facilitate the reasoning on the ethical problem involved. We could consider both as specific types within the so-called Goals Delphi, the Ethical Goal Delphi component that includes a significant fuzzy logic.

In this vein, from Kant through to the present day there have been various attempts to formalise ethics [29, 33]. It seems that this could fit a better reasoning in terms of ‘common sense’ that is more vague regarding both its premises and the rules of argumentation. Not only logical aspects, but cognitive, psychological or linguistic ones play an important role in the ethical argument; fundamentally ethical propositions are expressed through a formalised language, not being assimilated in the same way by different people. Since most of the predicates used in human language are vague, ethics could hardly be otherwise. As Wittgenstein affirms “the meaning of a word is its use in language” [69: 43]. In this sense, resorting to fuzzy sets can provide conceptual support to facilitate the discussion. Fuzzy logic can be used both to represent uncertainty and imprecision [23]. In the case of ethics, the imprecision is related to the meaning of ethical propositions, and should reflect the degree of match between the denotative and connotative meanings of the same value, while uncertainty refers to the degree of belief we have in the truth of a certain proposition.

To focus the possible role of the Delphi method regarding the clarification of ethical issues in the field of fuzzy logic, we should consider that there is no opposition, but rather complementarity with classical logic, whereby a classical set (see Fig. 5) is simply a case of a fuzzy set in which the degree of membership is necessarily 0 or 1. For their part, the non-classical sets include all intermediate values between these two extremes [6]. This means that the distance between two antinomian positions in classical logic will always be 1, while fuzzy logic can adopt many intermediate positions, and the disparity between conflicting views can actually be less than one; for example 0.1 or 0.01 (see Fig. 5). It could possibly facilitate agreements, or at least reduce the differences because of the more options or degrees of agreement.
Fig. 5

Classical vs. fuzzy logic

Fuzzy logic finds its centrality in the membership function [μP (x)] (Dubois et al., 2000).
$$ x\underset{r}{\in}\underset{\sim }{P}\iff \mu p(x)=r $$

Returning to applied business ethics, the emphasis would be on the x value (concrete behaviour) that best fits P (the value system), but in the case of fundamental ethics we would be trying to define P. As already noted, the Delphi method can be used in the same way in both applied and fundamental ethics, but they are two different types of resolutions. In applied ethics, being part is key: a solution may or may not be part of a certain system of values (function x belongs to P). For example, to determine a fair price, we would argue and determine whether or not a particular x, the price, is fair, which would correspond to its membership in P, or not. In fundamental ethics, the discussion focuses on the nature of P. For example, staying with the previous example, it involves an attempt to determine the fair price; it could be the price set by the market, or the price that covers production costs with a certain margin; it might be determined as a result of an inter-agreement between the parties, or any other conceptualisation that might be made of it. However, it is nonetheless an interactive process whereby, on the one hand, the definition of P (deductive perspective) contributes to clarifying the possible inclusion or exclusion of x, and on the other, the decision on x helps to define P (inductive perspective).

The following figure (see Fig. 6) shows the combination of the Delphi method and fuzzy logic, which may allow overcoming certain characteristics about the antimony of ethical discussion. In an ethical decision-making process, the Delphi method could be used to establish and clarify the dimensions of the problem.
Fig. 6

Delphi method and fuzzy sets in an ethical decision-making process

In the case of applied ethics, a consensus could be achieved among experts (this area has been called 1: consensus area), but in the worst case, a best option would be achieved (referred to as 2: fuzzy application space). Thus, the Delphi method may include and consider the process of reasoning compared to the determination of the solution using, for example, a decision method such as voting. Thus, following the example of the fair price, a consensus could be reached in which the price is set precisely between 15 and 25% higher than the market price. However, not all experts have to agree, so some experts might think that a 5% increase would suffice, with the lower area in this case ranging between 5 and 15%. Nevertheless, there could be other experts who defend that the fair price is 50% or more regarding the market price; there would then be an upper area between 25 and 50%. These two areas (5–15% and 25–50%) will be those that are designated as area 2, out of total consensus but within acceptable fuzziness. There is also an outlier zone, area 3, which signals other options outside the ranges: outliers with the reasoning completely outside the consensus; for example −1% or 100% regarding market price. Finally, there is a zone, area 4, in which no logical scores are considered.

Following the explanation of the Delphi-fuzzy figure (continuing with Fig. 6), but in the case of fundamental ethics where consensus is normally unattainable, the Delphi method allows delimiting the fuzzy application space as the argument area, and also allows identifying the distance between the participants and the consensus area (called area 1). In this case, the aim is to define the value system (P). For example, continuing with the fair trade case, it will involve whether the overpays will be fair. One might ask, for instance, why the fair price is not the market price. Why is it unethical? What are the criteria for the new price? What criteria are applied? Costs? Benefits? Needs? In this case, it would be optimal to reach consensus (area referred to as 1), but it is complex because it works with different value systems. However, the Delphi method is optimal because the reasoned reflection process improves the options for reaching some sort of consensus on the value system. It is more than likely that no consensus will be reached with the Delphi method, although it may identify areas of disagreement and the distances between experts; furthermore, it allows excluding the areas that fall outside the discussion because of a lack of consistency in the participants’ arguments. Thus, it allows us to limit the discussion of the problem area (fuzzy application space). The aim is to solve a conceptual problem, so it will be complex. It is admittedly a limitation of the Delphi approach, but it will be useful and feasible at least for delimiting the discussion area.

In sum, the Delphi process helps to establish the possibly fuzzy space and determine the distance and arguments with the consensus area. The Delphi method is useful to define different positions through inter-reasoning toward a consensual middle ground.

Concluding remarks

Delphi is a method that has largely been designed to reach consensus in situations of uncertainty. These consensuses have traditionally referred mainly to the future scenarios in which Delphi is used, on the one hand, to complete the information base held by the different experts involved, which is not always symmetrical, and on the other, to compare the different individual mental maps that experts have created in order to integrate the available data into a future scenario. In the case of ethical problems, we are dealing with certain aspects that are similar and others that are different; the former include the degree of uncertainty regarding a possible reference scenario, while the latter involve a scenario of a regulatory nature, instead of being positive. This is not an insignificant difference, as while the traditional Delphi seeks to predict the trend in certain variables, the Delphi applied to issues of business ethics seeks to reach a consensus on what should be done when faced with ethical problems. This difference may well explain why Delphi has been used so little in the field of ethics.

According to the experts, focus group and interview receive a higher appraisal, yet Delphi, in turn, outperforms case analysis and the questionnaire, whereby it may be concluded that, whenever possible, it is better to convene experts or work with them directly, but when this is difficult or even impossible due to the cost in terms of time and travel it seems that Delphi could be a suitable technique for discussing ethical matters.

It is interesting to note that the least valued aspect of the Delphi technique is the possibility of reaching consensuses, when this is precisely the key aspect when it is applied to future scenarios, with the curious paradox that Delphi is valued positively according to secondary aspects while the method’s main contribution is the one least valued by the experts. This phenomenon undoubtedly correlates with the regulatory nature of Delphi’s use in ethics, while in terms of positive aspects, most of the scientists are willing to change their opinion through interactive processes, whereas in the regulatory ambit the experts are much more reluctant to consider ideological modifications that may entail a possible consensus. Given what we might refer to as a “Consensus Paradox”, whereby in ethics all the experts call for consensuses, and it does indeed appear that these are required for their universal application in a firm, they are nonetheless unwilling to advance in consensuses that anticipate ideological risks (consequences that are incompatible with their ideological principles). This leads to a lack of trust in Delphi, precisely where its strength lies, namely, the possibility of advancing toward consensuses.

Nevertheless, in those approaches of a positive nature, experts may not agree on future scenarios, and these may simply be reduced, normally to no more than three, with a different probability of occurrence. With regard to the Delphi applied to business ethics, the perspective shifts from the scenario’s centroid function to its boundaries, generating a fuzzy proposal, which may sometimes be capable of integrating different ethical perspectives. In those cases in which this does not occur, Delphi provides a scenario of dialogue, which allows focusing on the best path toward convergence (fuzzy possibility space), dismissing possible arguments located within the ambit of divergence (marginal space that is external to the set). From this fuzzy perspective, the Delphi in ethics is not only presented as a more suitable technique when meeting costs are overly high, but also that it may overcome the reticence of the “Consensus Paradox” through the use of a fuzzy perspective focusing on the scenarios’ boundaries. This possibility opens a new dimension in Delphi’s use as a way of addressing ethical problems of both a theoretical and a practical nature.

The small number of participants in the survey may be a limitation; however, the scarce use of Delphi in the field of business ethics means there are few business ethics scholars, so the conclusions from this survey could be extrapolated to the entire population at this stage. The second limitation involves the low incentives among experts to participate in Delphi processes with high integration and implications not only for the results, but also for the process itself, which is highly relevant in business ethics research. In future research, once Delphi has become widespread, the analysis will need to be replicated. It would also be useful to conduct a rigorous analysis using Delphi as a complementary method in business ethics research.

Footnotes
1

We have obtained the list of participants from two academic associations: EBEN (European Business Ethics Network) and ISBEE (International Society for Business, Economics and Ethics).

 
2

There are two recent papers with results on the Delphi method. One was presented at the ISBEE conference in Shanghai in July 2016, and called “European Business Ethics Agenda based on a Delphi Analysis”, and the other was presented at IESE (University of Navarre) in 2014, and titled “A Prospective View of Emergent Research Lines in Ethics in Finance”. Both are available to readers of this paper.

 
3

www.webofknowledge.com. Accessed 20. November 2016

 
4

The use of focus group in the field of business ethics has been analysed previously by Cowton & Downs [14]. They analysed the papers published, finding that only 15 of those published on this topic actually focused clearly on research into business ethics. They predict the increase in qualitative methods for resolving business ethics questions because this field is designed to understand the behaviour of the individual and the company, encompassing business conduct.

 
5

Using the platform www.encuestafacil.com

 
6

Their relevance has been explained in a previous section.

 
7

When the pre-test was made, it was the latest paper published using a Delphi in the business area. We randomly chose five of the Delphi expert participants, and three of them helped us review the questionnaire in that stage, which guarantees its validity because they have understood the questions and considered the items used to be coherent.

 
8

The items are independent, so we are not comparing Delphi’s characteristics, simply asking them independently. However, we have conducted a scale reliability analysis in which Cronbach’s Alpha has been 0.627 (lower than both the optimum 0.8 and the accepted 0.7 level). It could reflect that most of the characteristics are included, but not all of them, and so it does not affect the scholars’ opinion about Delphi’s characteristics.

 

Declarations

Acknowledgements

This work is part of the research group ECRI Ethics in Finance and Social Value (GIU 15/10) at UPV/EHU, and it is supported by FESIDE (BOPV 20.01.2014).

We would like to thank Professor Goldschmidt and Professor Turoff and the business ethics academic experts as panellist of Ethics in Finance Delphi method presented at the 18th International Symposium on Ethics, Business and Society (held at the IESE, Barcelona, Spain, 30th June 30 – 1st July 2014) and EBEN (European Business Ethics network) and ISBEE (International Society for Business, Economics and Ethics) members for their helpful comments on earlier versions of this paper.

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
University of the Basque Country UPV/EHU, Bilbao, Spain
(2)
University of Huddersfield, Huddersfield, UK
(3)
Deusto Business School, Madrid, Spain

References

  1. Álvarez I, Calvo JA, Mora A (2014) Involving academics in the accounting standard setting process: an application of the Delphi methodology to the assessment of IASB proposals. J Manag Govern 18(3):765–791View ArticleGoogle Scholar
  2. Armstrong AE, Parsons S, Barker PJ (2000) An inquiry into moral virtues, especially compassion, in psychiatric nurses: findings from a Delphi study. J Psychiatr Ment Health 7(4):297–306View ArticleGoogle Scholar
  3. Bampton R, Cowton CJ (2002) The e-interview. Forum: qualitative social research Sozialforschung 3(2):1–12. http://www.qualitative-research.net/index.php/fqs/article/viewArticle/848/1842. Accessed 29 Sept 2016
  4. Beretta R (1996) A critical view of the Delphi technique. Nurse Res 3(4):79–89View ArticleGoogle Scholar
  5. Bhattacharya M, Petrick I, Mullen T, Kvasny L (2011) A Delphi study of RFID applicable business processes and value chain activities in retail. J Technol Manag Innov 6(3):63–81View ArticleGoogle Scholar
  6. Bilgiç T, Türkşen IB (2000) Measurement of membership functions: Theoretical and empirical work. In: Dubois D, Prade, H (eds) Fundamentals of fuzzy sets. Springer, pp 195–227Google Scholar
  7. Brown BB (1968) Delphi process: a methodology used for the elicitation of opinions of experts. Rand Corporation, Santa MonicaGoogle Scholar
  8. Buckley C (1995) Delphi: a methodology for preferences more than predictions. Libr Manag 16(7):16–19View ArticleGoogle Scholar
  9. Caron J, Durand S, Asselin H (2016) Principles and criteria of sustainable development for the mineral exploration industry. J Clean Prod 119:215–222View ArticleGoogle Scholar
  10. Chen FM, Liu YQ (2016) Research on the risk factors of mobile business: based on the sorting Delphi Method. Int J Eng Res Afr 21:215–230View ArticleGoogle Scholar
  11. Chen MK, Wang SC (2010) The use of a hybrid fuzzy-Delphi-AHP approach to develop global business intelligence for information service firms. Expert Syst Appl 37(11):7394–7407View ArticleGoogle Scholar
  12. Coates JF (1994) Ethics, the new plague on American civilization. Technol Forecast Soc 47(3):345–350View ArticleGoogle Scholar
  13. Cohen JR, Pant LW, Sharp DJ (1996) A methodological note on cross-cultural accounting ethics research. Int J Account 31(1):55–66View ArticleGoogle Scholar
  14. Cowton CJ, Downs Y (2015) Use of focus groups in business ethics research: potential, problems and paths to progress. Bus Ethics Eur Rev 24(1):54–66View ArticleGoogle Scholar
  15. Craft JL (2013) A review of the empirical ethical decision-making literature: 2004–2011. J Bus Ethics 117(2):221–259View ArticleGoogle Scholar
  16. Critcher C, Gladstone B (1998) Utilising the Delphi technique in policy discussion: a case study of a privatised utility in Britain. Public Adm 76(3):431–449View ArticleGoogle Scholar
  17. Cyphert FR, Gant WL (1971) The Delphi technique: a case study. Phi Delta Kappan 52:272–273Google Scholar
  18. Czinkota MR, Ronkainen IA (2005) A forecast of globalization, international business and trade: report from a Delphi study. J World Bus 40(2):111–123View ArticleGoogle Scholar
  19. Dalkey N, Helmer O (1951) The use of experts for the estimation of bombing requirements: a project Delphi experiment. Rand Corporation, Santa MonicaGoogle Scholar
  20. Dalkey N, Helmer O (1963) An experimental application of the Delphi method to the use of experts. Manag Sci 9(3):458–467View ArticleGoogle Scholar
  21. Dane E, Sonenshein S (2015) On the role of experience in ethical decision making at work. An ethical expertise perspective. Organ Psychol Rev 5(1):74–96Google Scholar
  22. Delbecq AL, van de Ven AH, Gustafson DH (1975) Group techniques for program planning: a guide to nominal group technique and Delphi processes. Scott-Foresman, GlenviewGoogle Scholar
  23. Dubois D, Ostasiewicz W, Prade H (2000) Fuzzy sets: history and basic notions. In: Dubois D, Prade H (eds) Fundamentals of fuzzy sets: the handbooks of fuzzy sets. Kluwer, Dordrecht, pp 21–24View ArticleGoogle Scholar
  24. DuBois JM, Dueker JM (2009) Teaching and assessing the responsible conduct of research: a Delphi consensus panel report. J Res Adm 40:49–70Google Scholar
  25. DuBois JM, Kraus EM, Gursahani K, Mikulec A, Bakanas E (2014) Curricular priorities for business ethics in medical practice and research: recommendations from Delphi consensus panels. BMC Med Educ 14(1):235–248View ArticleGoogle Scholar
  26. Fisher C, Downes B (2008) Performance measurement and metric manipulation in the public sector. Bus Ethics Eur Rev 17(3):245–258View ArticleGoogle Scholar
  27. Geisser ME, Alschuler KN, Hutchinson R (2011) A Delphi study to establish important aspects of ethics review. J Empir Res Hum Res 6(1):21–24View ArticleGoogle Scholar
  28. Geist MR (2010) Using the Delphi method to engage stakeholders: a comparison of two studies. Eval Program Plann 33(2):147–154View ArticleGoogle Scholar
  29. Gensler HJ (1996) Formal ethics. Routledge, New YorkView ArticleGoogle Scholar
  30. Godet M (2008) Strategic foresight la prospective. Cahiers du LIPSOR, ParisGoogle Scholar
  31. Goldschmidt PG (1975) Scientific inquiry or political critique? Remarks on Delphi assessment, expert opinion, forecasting, and group process by H. Sackman. Technol Forecasting Soc 7(2):195–213View ArticleGoogle Scholar
  32. Gordon T (2004) The Delphi method. In: Glenn JC, Gordon TJ (eds) Futures research methodology. American Council for The United Nations University Millennium Project, Washington, D.CGoogle Scholar
  33. Hare RM (1981) Moral thinking: its levels, method, and point. Oxford University Press, OxfordView ArticleGoogle Scholar
  34. Hasson F, Keeney S, McKenna H (2000) Research guidelines for the Delphi survey technique. J Adv Nurs 32(4):1008–1015Google Scholar
  35. Helmer O (1967) Analysis of the future: the Delphi method. Rand Corporation, Santa MonicaGoogle Scholar
  36. Hsu CC, Sandford BA (2007) The Delphi technique: making sense of consensus. Pract Assess Res Eval 12(10):1–8Google Scholar
  37. Huang SJ, Wu MS, Chen LW (2013) Critical success factors in aligning IT and business objectives: a Delphi study. Total Qual Manag Bus 24(9–10):1219–1240View ArticleGoogle Scholar
  38. Jones TM (1991) Ethical decision making by individuals in organizations: an issue-contingent model. Acad Manag Rev 16(2):366–395Google Scholar
  39. Kaufman J, Gustafson DH (1973) Multi- county land use policy formation: a delphi analysis technical report of the department of industrial engineering. University of Wisconsin, MadisonGoogle Scholar
  40. Landeta J (1999) El método Delphi. Una técnica de previsión del futuro. Ariel, MadridGoogle Scholar
  41. Landeta J (2006) Current validity of the Delphi method in social sciences. Technol Forecast Soc Chang 73(5):467–482View ArticleGoogle Scholar
  42. Linstone HA, Turoff M (2011) Delphi: a brief look backward and forward. Technol Forecast Soc Chang 78(9):1712–1719View ArticleGoogle Scholar
  43. Linstone H, Turoff M (1975) The Delphi method. Addison-Wesley, ReadingGoogle Scholar
  44. Millar K, Thorstensen E, Tomkins S, Mepham B, Kaiser M (2007) Developing the ethical Delphi. J Agric Environ Ethics 20(1):53–63View ArticleGoogle Scholar
  45. Millar K, Tomkins S, Thorstensen E, Mepham B, Kaiser M (2006) Ethical Delphi manual. LEI, The Hague, Wageningen UR. http://edepot.wur.nl/216590. Accessed 20 Nov 2016
  46. Mitroff II (1994) A reply to Coates. Technol Forecast Soc Chang 47(3):351View ArticleGoogle Scholar
  47. Moorhouse J (2002) Desired characteristics of ethical leaders in business, educational, political and religious organizations from East Tennessee: A Delphi Investigation. Electronic Theses and Dissertations. Paper n° 709. http://dc.etsu.edu/etd/709. Accessed 20 Nov 2016
  48. Müller RM, Linders S, Pires LF (2010) Business intelligence and service-oriented architecture: a Delphi study. Inf Syst Manag 27(2):168–187View ArticleGoogle Scholar
  49. Nielsen C, Thangadurai M (2007) Janus and the Delphi oracle: entering the new world of international business research. J Int Manag 13(2):147–163View ArticleGoogle Scholar
  50. North HQ, Pyke DL (1969) Probes of technological future. Harv Bus Rev 47(3):68–79Google Scholar
  51. Okoli C, Pawlowski SD (2004) The Delphi method as a research tool: an example, design considerations and applications. Inf Manag 42(1):15–29View ArticleGoogle Scholar
  52. Pätäri S (2010) Industry-and company-level factors influencing the development of the forest energy business—insights from a Delphi Study. Technol Forecast Soc Chang 77(1):94–109View ArticleGoogle Scholar
  53. Pätäri S, Sinkkonen K (2014) Energy service companies and energy performance contracting: is there a need to renew the business model? Insights from a Delphi study. J Clean Prod 66:264–271View ArticleGoogle Scholar
  54. Rest JR (1986) Moral development: advances in research and theory. Praeger, New YorkGoogle Scholar
  55. Rowe G, Wright G (1999) The Delphi technique as a forecasting tool: issues and analysis. Int J Forecast 15(4):353–375View ArticleGoogle Scholar
  56. Scheibe M, Skutsch M, Schofer J (1975) Experiments in Delphi methodology. In: Linstone H, Turoff M (eds) The Delphi method: techniques and applications. Addison-Wesley, London, pp 262–287Google Scholar
  57. Scheibe M, Skutsch M, Schofer J (2002). IV. C. Experiments in Delphi methodology. In: In: Linstone H, Turoff M (eds) The Delphi method: techniques and applications, pp 257–281Google Scholar
  58. Schmiedel T, Vom Brocke J, Recker J (2013) Which cultural values matter to business process management? Results from a global Delphi study. Bus Process Manag J 19(2):292–317View ArticleGoogle Scholar
  59. Shaw SM (2008) Leadership development and the characteristics/traits of ethical and effective leaders: the Delphi technique. ProQuest, New YorkGoogle Scholar
  60. Skutsch M, Schofer JL (1973) Goals-Delphis for urban planning: concepts in their design. Socio Econ Plan Sci 7(3):305–313View ArticleGoogle Scholar
  61. Stewart DW, Shamdasani PN (2014) Focus groups: theory and practice (Vol. 20). Sage publications, Beverly HillsGoogle Scholar
  62. Tersine RJ, Riggs WE (1976) The Delphi technique: a long-range planning tool. Bus Horiz 19(2):51–56View ArticleGoogle Scholar
  63. Thorstensen E, Kaiser M (2006) The ethical Delphi. A new tool for ethical assessments. In: Kaiser M, Lien M E (eds) Ethics and the politics of food: preprints of the 6th Congress of the European Society for Agricultural and Food Ethics, EurSAFE 2006, Oslo, Norway, June 22–24, 2006 (Vol. 6). Wageningen Academic Pub. Wageningen, NetherlandsGoogle Scholar
  64. Turoff M (1970) The design of a policy Delphi. Technol Forecast Soc Chang 2(2):149–171View ArticleGoogle Scholar
  65. Van Dijk JA (1990) Delphi questionnaires versus individual and group interviews: a comparison case. Technol Forecast Soc Chang 37(3):293–304View ArticleGoogle Scholar
  66. Wainwright P, Gallagher A, Tompsett H, Atkins C (2010) The use of vignettes within a Delphi exercise: a useful approach in empirical ethics? J Med Ethics 36(11):656–660View ArticleGoogle Scholar
  67. Weaver MO, Connolly SM (1988) From the beginning: using Delphi for curriculum development: integrating evaluation, design, and implementation. Train Dev 42(2):18–23Google Scholar
  68. Winkler J, Moser R (2016) Biases in future-oriented Delphi studies: a cognitive perspective. Technol Forecast Soc Chang 105:63–76View ArticleGoogle Scholar
  69. Wittgenstein L (1958) Philosophical investigations. Blackwell, OxfordGoogle Scholar
  70. Woudenberg F (1991) An evaluation of Delphi. Technol Forecast Soc Chang 40(2):131–150View ArticleGoogle Scholar

Copyright

© The Author(s) 2016