Delphi method history of occurrence. What is the Delphi method and how to use it? Delphi Method in Management Decision Making

The Delphi method is a forecasting method in which direct communication between group members is excluded during the study and an individual survey of experts is conducted using questionnaires to find out their opinion on future hypothetical events.

This method got its name from the name of the oracle of the Delphic Temple (Delphic Oracle), famous in the ancient world.

The main features of the Delphi method are as follows:

Complete rejection of personal contacts between experts interviewed on a specific issue;

Providing experts with the necessary information, including the exchange of opinions between them after each round of the survey;

Ensuring the anonymity of argumentation and criticism of assessments.

The purpose of the Delphi method is to obtain a forecast or list of the potential consequences of solving a problem that is much more reliable than the results of an analysis carried out by a single specialist. The survey of experts is carried out with the help of questionnaires, i.e. special questionnaires, in several rounds with the processing of the results of the survey in each round and informing the experts about these results. This allows the experts to take into account circumstances that they neglected or were not aware of.

Poll procedure. Conduct the first round of the survey. The questionnaires received at the same time analyze and carry out statistical processing of the received answers, taking into account any comments of the experts. Then the second round of the survey is conducted. Survey participants may be asked additional questions and a list of anonymous comments and arguments in favor of a particular position is presented. During the second round of the survey, participants are asked to revise their initial estimates in case they differ significantly from the collective opinion of the group. Then the third and final round of polls is held. At the same time, survey participants are given the opportunity to make additional comments and put forward counterarguments in defense of their position. All this leads to a narrowing of the range of estimates.

The main advantage of the Delphi method is that it avoids the action psychological factors, for example, unwillingness to give up a publicly expressed opinion, personal antipathy towards someone, or, conversely, excessive attention to the opinions of certain individuals.

In the process of applying the Delphi method, a script is also compiled. The scenario describes the general opinion of experts about the background of the problem, its current state and forecasts for the future. The task of the scenario is to identify the key changes taking place in this system. Thanks to this, it becomes possible to identify all the factors influencing the behavior of the system, divide them into primary and secondary ones, formalize the concept of "state of the system" and predict changes in the main and secondary factors of the development of the system as a whole.

The complete description of the system obtained in this way, together with estimates of the probabilities of different ways of its development, allows us to determine the "needs of the system" arising from the desire to provide some specific path for its development. This allows you to get the main result of the scenario - the formulation of the main goal of the system.

The scenario serves as the basis for the second stage - the compilation of the so-called goal tree. It allows you to transform each higher-level goal into a set of sub-goals. At the same time, the goals of each level should be comparable in scope and significance.

"Tree of goals" is an ordered hierarchy of goals, expressing their internal relationships and subordination. Hierarchy (gr. hierarchia, hieros- sacred + arche- power) is the arrangement of parts or elements of the whole in order from highest to lowest. The goal tree is a normative method for identifying system functions, the main content of which is the way to move from a global goal to a set of smaller sub-goals. The logical basis of this method is the conjunctive representation of the goals of the subsystem in terms of a lesser degree of generality. The relationship between the goals, functions and criteria of any system is as follows. In the general case, the goal indicates the direction of the system's actions, i.e., it expresses the desired state of the system. The function reflects the main purpose of the object. The criterion complements the concept of goal, pointing to effective method achievement of goals The criterion allows you to compare different options for achieving goals with each other.

When developing a "tree of goals", the following requirements are taken into account.

The specificity of the formulation of goals.

Comparability of the goals of each level in terms of scope and significance.

Measurability. This means that the formulation of goals should; provide the possibility of a quantitative or ordinal assessment of the degree of its achievement.

Conjunctivity (from lat. conjunctivus- connecting) means that each top-level goal should be represented as sub-goals next level in such a way that the union of the concepts of subgoals completely defines the concept of the original goal.

Continuity, consistency, completeness of decomposition of goals. This means that the “tree of goals” must be built in such a way that intermediate concepts cannot be included between the concept of a goal and its subgoals, and the exclusion of at least one subgoal would change the very concept of the corresponding original goal. The decomposition of goals into the corresponding subgoals is carried out in such a way that the number of elements of the conjunctive "tree of goals" does not include alternative ways achieving goals. This allows the best distribution of resources between the goals, taking into account the relative importance (importance) of each goal, usually determined by an expert.

General scheme the Delphi method is shown in fig. 7.4.

The construction of a "tree of goals" provides linking and harmonization of the goals of the plan not only in qualitative terms, it also makes it possible to conduct some quantitative assessment of their importance (significance) for achieving the corresponding goal of a higher level. This task is solved on the basis of methods of expert assessments, specially developed for the normalization and ranking of goals. Criteria for different levels of the goal tree are formulated based on the scenario. For each level of goals, there may be different criteria for their evaluation.

Rice. 7.4. General scheme of the Delphi method

Experts can discuss scenarios and criteria with each other, but they must put down the coefficients themselves. The assignment of importance coefficients can be carried out in several stages. The number of stages depends on the qualifications and experience of the experts.

Assessment of the state of development and the timing of its completion is carried out by the method of expert assessments. The most commonly used coefficient is "state - term".

The methodology for determining the coefficient "state - term" includes five stages:

theoretical research;

search developments;

technical developments;

construction;

production of the finished product.

The expert must indicate at which stage, in his opinion, is this development and how long it will take to complete each of the subsequent steps.

For the survey, a table is used (Table 7.1, example), where the expert indicates that the subsystem being evaluated is at the stage of search development, which is designed for a three-month period, followed by technical development for two months and production of the finished product for one month. Thus, the total period for obtaining the finished product according to the expert assessment is 8 months (3 + 2 + 2 + 1) .

Table 7.1

Determination of the coefficient "state - term"

The method, which received the name of the ancient Greek city, famous for its predictors of the future, was developed in the early 1950s. in the well-known "think tank" of the United States - the Rand Corporation. Its authors are American scientists O.

Helmer and T. Gordon. Like many developments in the field of political analysis and forecasting, the application of the Delphi method was initially limited to problems of a military-industrial and military-diplomatic nature.

The emergence of Delphi is associated with an objectively urgent need to improve the methods of group decision-making. Before the advent of Delphi, the most common way to agree on various positions and develop a common opinion was the traditional meeting (face-to-face discussion). However, this method has a number of very serious drawbacks, most of which are associated with the negative psychological effects of interpersonal communication, which include:

group pressure. This phenomenon has been studied in social psychology (more specifically, the psychology of small groups) and consists in the fact that the majority in the group seeks to impose its position on the minority. The minority, as a rule, tends to show conformism - to accept the group's opinion, and not to defend their point of view (even if the representatives of the minority retain subjective confidence in its correctness). Thus, the result of the discussion may be the victory of the opinion of the majority only because it is the opinion of the majority;

Personal differences of group members that determine the ability to actively defend their point of view and impose it on others. In a face-to-face discussion, the “competitive advantage” is usually on the side of the more active, assertive, more verbal and persuasive participants. At the same time, the presence of these qualities in a person does not necessarily indicate his deeper understanding of the problem under discussion. Thus, the opinion of not the most competent, but the most “convincing” experts may prevail;

Different formal or informal status of the participants in the discussion. In almost any group, more authoritative and “deserved” experts can be singled out, whose opinion will be listened to in more. So, the opinion of an academician will "weigh" more than the opinion of a graduate student, while a graduate student can study in depth exactly the problem posed, and an academician can only have a superficial idea of it. In groups where there is a certain hierarchy (for example, at meetings in military departments, structures public service etc.), the weight of the point of view of superiors will be higher than the point of view of subordinates (who, moreover, are unlikely to actively defend their points of view);

Psychological difficulty for many participants in the examination to change an already expressed point of view, even if they realized its shortcomings. For many people, especially "deserved" and "authoritative", it can be difficult to take back their words, to admit a mistake, especially if the recognition of a mistake "works" to strengthen the position, for example, of a longtime opponent. Therefore, there are high risks that the expert will defend his point of view, even if he is convinced of its inconsistency;

The vagueness, vagueness of the final assessments, conclusions and conclusions, inherent in many traditional meetings.

It is these problems that the procedure used in the Delphi method can eliminate. It is based on the following basic principles:

Correspondence character of interaction of experts. The specificity of Delphi is that each expert works individually, but the overall assessment is collective (group). This principle is aimed at eliminating the phenomenon of group pressure and the effects of the difference in "public activity" and assertiveness of experts;

Anonymity of expert opinions. Each participant in the examination is given complete freedom to convey his position and arguments to the entire group, but no one will know whose position it is. This principle is aimed at eliminating the “authoritative opinion effect”;

Iterative (repeatable) expertise. The procedure for forming a group expert assessment in the Delphi method takes place in several stages, and each of the experts at each stage can correct their own previous assessment. In Delphi, this is done psychologically painlessly, given the remote and anonymous nature of the procedure;

Controlled feedback. Experts can exchange assessments and arguments, but they do this not directly, but through the organizers of the examination, who provide feedback between experts, systematize assessments and arguments;

Quantitative evaluation and statistical processing of expert assessments. Experts are limited in formulating scores by their numerical format. This is required in order to make the results of the examination as specific as possible.

Let us consider how the above principles find their expression in the procedure of the Delphi method.

At the stage of preparation of the examination, the composition of its organizers is determined, who at the preliminary stage must formulate the problem under study in such a way that it can be worked with within the framework of Delphi. In other words, the problem should be presented as a set of specific questions to the experts, each of which could be answered with a numerical score. For example, it is incorrect to ask experts: “Will the government resign before the expiration of its constitutional term of office?” There are two ways to phrase the question correctly:

When will the government resign? (The deadline is the date of the next presidential election.)

What is the probability of early resignation of the government? (It is clearly stated what time period is meant by early resignation.)

All questions should be worded in such a way that they can be answered on an ordinal or interval scale. The only exception is the sometimes used "unstructured stage", which we will discuss separately.

Of course, such a limitation somewhat narrows the possibilities of using Delphi. However, there remains a very wide range of features that can be measured with it. For example, for a political party, this is:

The level of support in the elections (in percentage or votes - interval level);

Influence (ordinal level);

The timing of joining the alliance on the other political party(time - interval level);

The level of support from certain political groups(ordinal level);

The level of loyalty to the current head of state (ordinal level);

The level of expression in the ideology of certain positions (say, how much the party is committed to liberal values - an ordinal level);

The cost of implementing a particular campaign by a party (interval level), etc.

The Delphi method can also be very useful for achieving purely research goals, for example, when designing a complex instrument for measuring a certain trait. For example, when constructing an index of the governor's political influence, we will introduce into it the sub-indices "support for the head of state", "support from the population of the region", "lobbying opportunities" and a number of others. Each of these sub-indices is directly measurable. But how to understand which of them has more weight when calculating the final index of influence? The weight of each of the components of the index, as a rule, will not be directly measurable. And here we come to the aid of expert assessments, primarily the Delphi method. "Weighting" of the index components is one of the tasks that can be optimally solved with the help of Delphi.

So, the problem should be formulated as a list of questions that require an answer in the form of an assessment on an ordinal or interval scale. The next important task of the organizers of the examination at the preparatory stage is to form the composition of the expert group, i.e. determine its size and composition.

The specificity of the Delphi method, associated with the statistical processing of estimates and the remote nature of the interaction of experts, has a direct impact on the staffing of the expert group in terms of its size. First of all, the number of ratings (and hence the number of experts) must be statistically significant. We cannot involve only three experts in the Delphi procedure, since we will not be able to process their assessments. Accordingly, the lower limit of the size of the expert group is 7-9 people. At the same time, we do not have an upper bound, since there is no need to gather experts in one place. In the real practice of using Delphi, there are examples when several hundred specialists participated in the examination. Their specific number will be determined by the specifics of the problem under consideration, the total number of competent experts, their technical availability and consent to participate in the examination.

Also, at the preparatory stage, a technical channel for communication with experts is determined. At the dawn of the development of the method, it was ordinary mail, at present it is mainly Email and facsimile.

Having prepared the questionnaire and determined the composition of the experts, you can proceed to the first round of the examination. Let's take a predictive problem. Suppose we are interested in the probability of a certain political event occurring, and the only question in the questionnaire will be: “Estimate the probability of the occurrence of the event TV in period M, using estimates in the range from 0 to 1, where 0 is the complete certainty that the event will not occur, 1 - full confidence that the event will happen. Of course, in a real study there would be more questions and explanations for them, but for educational purposes we will limit ourselves to the most simple view questionnaires.

Let's say a question is attended by nine experts. Accordingly, following the results of the first round, we will receive nine estimates of the probability of the event N. Thus, we have an unordered numerical series of nine elements: (1; 0.2; 0.1; 0.1; 0.6; 0.8; 0.3; 0.5; 0.8).
In the Delphi method, the basis of statistical processing of estimates is the calculation of the mean and variation at the ordinal level of measurement, i.e. we are talking about the calculation of the median - the middle of the ranged number series - and quartiles - quarters of the ranged number series. The series ranked in ascending order in our case will look like: (0.1; 0.1; 0.2; 0.3; 0.5; 0.6; 0.8; 0.8; 1).

The median is 0.5, the value of the lower quartile is 0.2; upper - 0.8 (M = Q2 = 0.5; Q1 = 0.2; Q3 = 0.8).

With regard to the Delphi method, the median shows the general group opinion, and the interval between the upper and lower quartiles (or quarterly rank) - the spread of expert opinions, or the degree of consolidation in general: the group score is 0.5 (equiprobable), the interval between the upper and lower quartiles is 0 .8 - 0.2 = 0.6, i.e. is very large. Based on this value of the quartile rank, it can be stated that the opinion of the group has not actually been formed, the estimates are very widely scattered.

For probabilistic estimates, there is an additional tool for interpreting the general group opinion, expressed in the value of the median. In probability theory, there is the concept of uncertainty, and the level of uncertainty is related to the level of probability in the following fundamental way:

The level of uncertainty is zero in two cases: if the probability of an event is 0 and 1. In other words, there is no uncertainty when we are completely sure that either the event will occur or that it will not occur. Accordingly, the maximum level of uncertainty reaches in a situation of equiprobability - 0.5. As you move away from the extreme values (0 and 1) and approach the value of 0.5, the uncertainty increases.
Thus, according to the results of the first round of the examination, we have not only a wide range of estimates, but also a situation of maximum uncertainty regarding the occurrence of this event within the specified time frame. The decision, which is made by the leaders of the examination, in this case is unambiguous: the examination must be continued.

In the second round, experts are introduced to the generalized results of the first round (scatter of estimates, sometimes the average) and are asked to answer the same question about the probability of an event occurring. However, an essential addition arises here: the assessment must be supplemented by a certain set of arguments. Technically, there are two options here:

1. All experts are asked to give reasons for the assessment.

2. Only those experts whose estimates go beyond the interval between the quartiles, i.e. are extreme. In our case, these are two experts who scored 0 and 1, and one expert who scored 1.

The second option is optimal if a relatively large number of experts are involved in the examination and the assessments of a significant part of them are outside the interval between quartiles. Then we will get a complete set of arguments, on the one hand, in favor of a high, on the other hand, in favor of a low probability of the event. In such a situation, it makes little sense to receive the arguments of those experts whose estimates are within the interval: their argumentation will most likely be a combination of the arguments of the "extreme" experts.

However, in our case, when the number of involved experts is small and the estimates of only three of them are outside the quarterly rank, it is advisable to collect the arguments of all experts. Arguments are formulated by experts in the same mode as the entire examination process: in absentia, anonymously and individually. The group of organizers of the Delphi procedure collects, summarizes and systematizes the arguments. The main content of this work: combining similar arguments, removing duplicates, dividing all arguments into two groups: in favor of increasing or decreasing the probability of the event N.

As a result of the second round we have:

New estimates of experts. They may or may not coincide with the scores of the first round. As a rule, the assessments change insignificantly from the first round to the second, since the experts have not yet had time to familiarize themselves with the arguments of their colleagues. Let the following estimates be obtained in our case: (0.1; 0.2; 0.2; 0.3; 0.6; 0.7; 0.8; 0.8; 0.9). Then the statistics of the second round: М= 0.6; Q1 = 0.2; Q3 = 0.8; quarterly rank = 0.6;

Two systematized lists of arguments: in favor of raising and lowering the estimate of the probability of an event. The authorship of the arguments is not indicated.

All the results obtained are communicated to the participants of the examination (a characteristic manifestation of controlled feedback), and the third round of Delphi begins. In the third round, as in the second one, the experts are required to re-evaluate the probability of the event and give a list of arguments. AT explanatory note the questionnaire, as a rule, indicates that the experts are expected either to make new arguments or to strengthen, supplement or specify the arguments used in the second round.

Usually, it is the third round of the examination according to the Delphi method that is a turning point: having received a significant amount of information from their colleagues following the results of the second round, the experts have more reason to correct their own assessments. The overall "shift" in the results of the examination should be much more significant compared to the second round.

Let's assume that the marks of the third round are: (0.1; 0.3; 0.5; 0.5; 0.7; 0.7; 0.8; 0.9; 0.9).

Statistics of the third round, respectively: М= 0.7; Q1 = 0.5; Q3 = 0.8; quarterly rank = 0.3.

Analyzing these statistics, we see two fundamental trends:

The general opinion of the group shifts from an equally probable assessment towards an increase in the probability of the event occurring (0.7). At the same time, the level of uncertainty is reduced in assessing the implementation of the event;

The group's opinion becomes more consolidated. The interval between quartiles is significantly reduced compared to the second round (0.6 and 0.3).

Iterations (new rounds) are held in Delphi on the same principle as the second and first rounds. The decision to complete the examination is made when the biases in the estimates cease to be significant. So, if in the fourth round we have scores: (0.1; 0.5; 0.6; 0.6; 0.7; 0.7; 0.8; 0.8; 0.8) and statistics: M = 0.7; Q1 = 0.6; Q2 = 0.8; quarterly rank = 0.2, - it can be stated that the group opinion has been formed. The shift in estimates compared to the third round is insignificant, the general group opinion has not changed, the interval between quartiles is insignificant. Thus, the experts generally agreed that the probability of the occurrence of the event N in the specified time frame is 0.7; its implementation is "rather likely".

It is useful to visualize the dynamics of expertise development in the Delphi method. The figure below clearly shows the “trajectories” of expert assessments, the formation of a more consolidated opinion and the overall shift towards the median of 0.7. The “isolated position” is also clearly visible: one of the experts never changed his assessment (0.1), despite its strong divergence from the general group opinion.

Aix	Tours	Tour 2	TourZ	Tour 4
1	0,1	one	0,1	0,1
2	0,1	0,2	0,3	0,5
3	0,2	0,2	0,5	0,6
4	0,3	0,3	0,5	0,6
5	0,5	0,6	0,7	0,7
6	0,6	0,7	0,7	0,7
7	0,8	0,8	0,8	0,8
8	0,8	0,8	0,9	0,8
9	1	0,9	0,9	0,8

There are situations when the convergence of estimates either does not occur, or occurs at the extreme poles. Such a case can be seen in the table and figure below.

Tour 1	Tour 2	Tour 3	Tour 4
0,1	0,1	0,1	0,1
0,2	0,1	0,1	0,1
0,3	0,3	0,2	0,1
0,4	0,3	0,2	0,2
0,5	0,5	0,5	0,5
0,6	0,7	0,8	0,9
0,7	0,8	0,8	0,9
0,8	0,8	0,9	1
1	1	1	1

	Tour 1	Tour 2	Tour 3	Tour 4
Q1	0,3	0,3	0,2	0,1
m	0,5	0,5	0,5	0,5
Q3	0,7	0,8	0,8	0,9

id="Figure 29" src="/files/uch_group28/uch_pgroup17/uch_uch327/image/46.jpg">

In this case, the result of the examination - the median of the fourth round of 0.5 - if something reflects, then only the maximum level of uncertainty. Expert estimates are clearly consolidated at the poles of very high and very low probability of the event. The resulting statistics are practically useless for us, but it cannot be said that the work done was completely in vain. During the Delphi procedure, we were able to at least clearly define the polar positions and the arguments accompanying them, which will be needed in the process of further analysis of the situation.

Among the modifications of Delphi in comparison with the traditional procedure described above, the introduction of a structureless stage into the examination should be noted. This technique is used when the study is exploratory in nature, the initiators of the examination are not quite ready to immediately operationalize the problem to the level of specific questions that require a quantitative answer. Then experts are involved in the procedure for formulating the problem and preparing the tools.

For example, we want to get a predictive list of parties that will be able to get into the State Duma, having overcome the barrier. But the study is carried out before the start of the official start of the election campaign, and we are not sure that the list of parties that we have compiled - contenders for seats in parliament is complete, since not all of them have declared their intention to participate in the elections. In this situation, you can use the unstructured stage: in the first round, invite the experts

(to each individually) make a list of all parties that can apply for the passage of the barrier. No quantitative estimates for the structureless stage are expected - that is why it is called structureless. Having received the lists of parties from each of the experts, the organizers of the examination bring them into single list and then proceed to the standard procedure: they ask for an expert assessment of the predicted result of each of the parties in the upcoming elections (in this case, in percentage of votes).

Another modification of Delphi is focused on reducing the time spent on examination. It follows from the above that the Delphi method, with all its advantages, is rather cumbersome and requires significant time resources. The Express-Delphi technique retains all the basic principles of the classical approach, but involves the completion of the entire procedure in a few hours, which requires appropriate technical support. Each expert during the time allotted for examination is at an individual computer terminal; all terminals are united in a common network, closed to the head of the examination. The organizers of the examination are required to be especially efficient in processing assessments and systematizing arguments, since all iterations must fit within a relatively limited period of time.

The disadvantage of express-Delphi compared to the traditional procedure is obvious. The expert is not given time to really think deeply about the proposed problem, to analyze in detail the positions and arguments of other members of the group. In addition, the methodology is complex organizationally and technically.

Send your good work in the knowledge base is simple. Use the form below

Students, graduate students, young scientists who use the knowledge base in their studies and work will be very grateful to you.

Hosted at http://www.allbest.ru/

Introduction

delphi expertise poll

Delphi method - a multi-stage method that provides for the initial isolation of the experts making their judgments and their further multiple adjustments based on the acquaintance of each expert with the judgments of other experts until the spread of estimates is within the predetermined desired interval of variation of estimates.

The essence of the Delphi method consists in the sequential questioning of experts in various fields of science and technology and the formation of an array of information that reflects the individual assessments of experts based on strictly logical experience. This method involves the use of a series of questionnaires, each of which contains information and opinions obtained from the previous questionnaire.

The collection and processing of individual opinions of experts on the forecasts for the development of the object of study is carried out based on the following principles:

Questions in the questionnaires are posed in such a way that it is possible to quantify the answers of experts;

· the survey of experts is carried out in several stages, at each subsequent stage questions and answers are more and more refined;

· after each stage, all interviewed experts are acquainted with the results of the survey;

The expert substantiates assessments and opinions that deviate from the opinion of the majority;

· Static processing of responses is carried out sequentially, from stage to stage, in order to obtain generalizing characteristics.

The estimates obtained by means of these methods are static and one-time, as a result of which it becomes necessary to re-apply to experts when making a market share forecast for subsequent periods. In addition, the method of internal and external expert forecasting is characterized by a certain degree of subjectivity.

The reliability of the Delphi method is considered high when forecasting for a period of 1 to 3 years, as well as for a more distant period of time. Depending on the purpose of the forecast, from 10 to 150 experts can be involved in obtaining expert estimates.

A qualitative approach allows assessing the specifics of each specific situation. In some cases, a careful examination of the various specific elements that define a situation may be more important than a systematic quantitative assessment. A big disadvantage of this method is the excessive subjectivity of the estimates. Old stereotypes of a foreign society can play a fatal role in decision making. J. Simon assessed this approach as "sporadic, based on selective, uncontrolled perception or ideological and personal predilections."

1. Scope of expert methods

Methods of expert assessments are widely used in forecasting and long-term planning, where there are no sufficiently reliable statistical data on the issue under study, where there are several solutions and it is necessary to choose the most preferable of them. Also, these methods are used in the development of new programs in industries that are strongly influenced by new discoveries in the basic sciences.

When analyzing and forecasting the economic situation, a number of difficulties arise:

The impossibility of accurately predicting the consequences of decisions made;

Non-repeatability and impossibility of experimental verification of the proposed course and results of the solution;

The presence of factors that are beyond the control of the decision maker;

Multiple possible ways decisions and the need to choose one of them;

The incompleteness of the initial information, on the basis of which it is necessary to form a problem and make a decision (often the initial information is of a qualitative nature and cannot be quantitatively measured).

The prerequisites for the use of expertise are:

Insufficiency and unreliability of information about the state of certain conditions in which the creation and development of products is carried out;

Stochastic (probabilistic) nature of the information object;

Complexity and novelty of problems.

The organization of the examination is carried out in several stages:

1. Determination of the goals and objectives of the examination.

2. Choice of examination procedure.

3. Selection and formation of a group of experts.

4. Organization of the examination procedure itself;

5. Information processing.

6. Making a decision based on the results of the examination.

2. Determination of the goals and objectives of the examination

First, the problem is posed - the background is determined, the arguments in favor of its solution are considered, and there is a discussion with all interested parties. The main thing here is to recognize imaginary problems. Therefore, when posing a problem, publicity and discussion are necessary.

After the problem is substantiated, the boundaries of its existence are determined, the totality of internal and external factors affecting the problem. For this, the central question is singled out and split into sub-questions. At the same time, they try to limit the field only to those questions, without which it is impossible to get an answer to the central question. Further, the goals and objectives of the implementation of the selected problem are formulated. Thus, the main events, factors, central and secondary issues are selected.

It must be borne in mind - with an increase in detail - the accuracy of the examination increases, BUT the consistency of the opinions of experts decreases.

The organizers of the examination shall choose the procedure for carrying out the examination. There are various approaches to this issue. Can be carried out

- and individual or group survey,

- about full-time or correspondence;

- about open or closed.

Individual survey consists in interviewing an expert and allows you to make the most of the abilities and knowledge of each expert.

Group - With this method, experts can exchange opinions, take into account the missed moment by each of them, and correct their assessment. The disadvantage of group opinion lies in the strong influence of authorities on the opinions of the majority of participants in the examination, in the difficulty of publicly renouncing their point of view, and the psychological incompatibility of some participants in the examination.

From Methods group survey use: various modifications Delphi method.

Delphi Methods are characterized by the following features:

anonymity of expert opinions;

· regulated processing, communication, which is carried out by the analytical group for a number of rounds of the survey, with the results of each round being reported to the experts;

group response, which is obtained using statistical methods and displays the generalized opinion of the participants of the examination

Delphi method is the most formal of all expert forecasting methods and is most often used in technological forecasting, the data of which are then used in production planning and product marketing. This is a group method in which a group of experts is asked individually about their assumptions about future events in various areas where new discoveries or improvements are expected.

The survey is conducted anonymously using special questionnaires, i.e. personal contacts of experts and collective discussions are excluded. The answers received are compared by special workers, and the summarized results are again sent to the members of the group. Based on this information, group members, while still remaining anonymous, make further guesses about the future, and this process can be repeated several times (the so-called multi-round polling procedure). Once consensus begins to emerge, the results are used as a prediction.

The application of the Delphi method can be illustrated by the following example№ 1 : An offshore oil company wants to know when it will be possible to use robots instead of divers to inspect platforms underwater. To start forecasting using this method, a company must get in touch with a number of experts. These experts should come from a wide variety of industries, including divers, engineers from oil companies, ship captains, maintenance engineers and robot designers. They explain the company's challenge, and each expert is asked when he thinks it will be possible to replace divers with robots. The first answers will probably give a very wide spread of data, for example, from 2000 to 2050. These responses are processed and returned by experts. At the same time, each expert is asked to revise their assessment in the light of the answers of other experts. After repeating this procedure several times, opinions can converge, so that about 80% of the answers will give a period from 2005 to 2015, which will be sufficient for the purposes of production planning and implementation of robots.

The Delphi method is named after the Delphic oracle in Ancient Greece. It was developed by Olaf Helmer, a prominent mathematician at the RAND Corporation, and his colleagues, which is probably why, in comparison with other creative approaches, it gives sufficient prediction accuracy.

3. Steps of the Delphi Interrogation Procedure

The Delphi method belongs to the class of quantitative methods of group expert assessments. The survey of experts is conducted in 3-4 rounds, consisting of a series of questionnaires, questions are specified from round to round. To implement this method, it is also necessary to create an analytical group, which, after each round, performs statistical processing of the information received.

First of all, analysts determine the area of preferred quantitative values of objects.

After such a check, the next round is held. The procedure of expert survey according to the "Delphi" method can be divided into several stages.

STAGE 1. FORMATION OF THE WORKING GROUP

The task of the working group is to organize the expert survey procedure.

STAGE 2. FORMATION OF THE EXPERT GROUP.

In accordance with the Delphi method, the expert group should include 10-15 experts in the field. The competence of experts is determined by questioning, analysis of the level of abstracting (the number of references to works this specialist), using self-assessment sheets.

STAGE 3. FORMULATION OF QUESTIONS

The wording of the questions should be clear and unambiguously interpreted, assuming unambiguous answers.

STAGE 4. EXAMINATION

The Delphi method involves repeating several steps of the survey.

STAGE 5. SUMMARY OF THE SURVEY

For the first round, the experts are asked questions. Answers should be presented in the form of quantitative estimates to the question posed. The answer must be substantiated by an expert.

The analytical group conducts statistical processing of the information received from all experts. To do this, the average value of the parameter under study, the weighted average value of the parameter under study are calculated, the median is determined as the average member of the general series of numbers received from experts and the confidence area. It is more expedient to calculate the confidence area through the quartile indicator. The value of the quartile is equal to ¼ of the difference between the maximum and minimum scores of the series. The confidence region itself will be equal to the minimum score minus the quartile value, the maximum score plus the quartile value.

Experts must necessarily get acquainted with the results and conclusions of analysts, after which the second (next) round is held. According to the results of the presented calculations, experts can see how their opinion corresponds with the opinion of the entire group of experts. They can change their opinions or leave them the same, but in this case put forward counterarguments in their favor. The principle of anonymity is strictly observed. Thus, 2-3 rounds are held. As a result, we obtain a fairly accurate group estimate.

When using the Delphi method, consider the following:

1. Panels of experts must be stable and their numbers must be kept within reasonable limits.

2. The time between rounds of surveys should be no more than a month.

3. Questions in questionnaires should be carefully thought out and clearly formulated.

4. The number of rounds should be sufficient to provide all participants with the opportunity to get acquainted with the reason for a particular assessment, as well as to criticize these reasons.

5. There should be a systematic selection of experts.

6. It is necessary to have a self-assessment of the competence of experts on the issues under consideration.

7. We need a formula for the consistency of assessments based on self-assessment data.

The Delphi method is applicable in almost any situation that requires forecasting, including when there is not enough information to make a decision.

There are several modifications of the Delphi method, in which the basic principles of organization of expertise have much in common. The differences are associated with attempts to improve the method through a more reasonable selection of experts, the introduction of schemes for assessing their competence, improved feedback mechanisms, etc. For the convenience of information processing, all modifications, as a rule, imply the possibility of expressing the answer in the form of a number, a quantitative assessment.

4. Initial goals formations and yesfurther development of the Delphi method

The Delphi method was first described in the "Report on the Study of Long-Range Forecasting" by the American Rand Corporation in 1964. The objects of the study were: scientific breakthroughs, population growth, automation, space exploration, the emergence and prevention of wars, future weapons systems. Over the past period, the range of predictable processes using the Delphi method has expanded significantly, but there is no doubt that this method has found the greatest application in areas related to scientific and technological progress.

In particular, in our country this method was used to determine the main directions scientific research in the field of computer technology and forecasting their characteristics, to assess the prospects for the development of industries. In the latter case, with this method the following tasks can be solved:

Determining the timing of work from issuance terms of reference to work before the start of operation of the facility;

Determination of priority directions for the development of enterprises in the industry (according to production technology, the most important economic characteristics- the volume of production, the number of employees, the volume of funds, etc.);

Determining the criteria for assessing the significance of scientific developments, etc. The method called "brainstorming" is fundamentally different from the Delphi method for organizing the work of experts, which is also called the "brainstorming" method, the method of collective generation of ideas. This method involves obtaining a solution as a product of the collective creativity of specialists during a session-session held according to certain rules, and subsequent analysis of its results. Its essence lies in the fact that when substantiating the forecast, two tasks are solved differentially:

Generation of new ideas regarding possible options for the development of the process;

Analysis and evaluation of the ideas put forward.

Usually, during the meeting, all specialists are divided into two groups, consisting of the same or different representatives, so that one group generates ideas, and the second analyzes them. At the same time, during the meeting it is forbidden to express any critical assessment of the value of the idea; as many of them as possible are encouraged, since it is assumed that the probability of a really valuable idea coming up increases with an increase in their total number; free exchange of opinions is encouraged; the expressed thoughts should be picked up and developed, etc. The session is led by an impartial moderator. His task is to direct the development of the discussion in the right direction, towards the achievement of a given goal, without getting lost in conversation, wit competition, etc. At the same time, he should not impose his opinion on the participants in the discussion, orient them to a certain way of thinking.

For Russia, the formulation of goals and the development of methods for selecting the priorities of the state scientific and technological policy is of particular importance. Although the compilation of comprehensive forecasts of the scientific and technological development of the country and the world in the USSR began in the early 1970s, the main guidelines for them were the interests of the defense sector and the party state apparatus. At present, the development goals have certainly expanded, but the procedure for selecting priorities corresponding to them has not been developed, agreed, and does not have regulatory framework and traditions. Under these conditions, when choosing priorities and obtaining appropriate financial and legal support, the displaced and narrow interests of departments, the military-industrial complex, regions, or someone else may prevail, while the interests of the state as a whole will not be taken into account. Under these conditions, the development of the procedure for selecting priorities and the study of the experience of other countries are extremely important.

Most developed countries to determine the priorities of scientific and technological development in the process of forecasting and making decisions on financing large government programs the following methods are used:

o Compilation of a list of critical technologies.

o Expertise

Technology forecast based on Delphi method, is an attempt to predict the development of a particular technology in the long term (20-30 years). Developed for the first time in the 50s by RAND Corp., the Delphi technique was used for the first time for the purposes of national and sectoral technological forecasting by Japan (since 1970, 6 studies have already been carried out), and subsequently, and to a large extent following the Japanese model, by Germany, France , Great Britain, Spain, Austria, South Korea mainly during the last decade (we can talk about the boom of this method in the 90s).

The Delphi method consists in the assessment of technologies by experts (their number varied from 123 people in Spain to 25 thousand at the first stage in South Korea) based on the proposed schemes, which include several positions, including the level of research activity in this area, participation in the creation of national wealth, improving the quality of life and competitiveness, the expected timing of the implementation of new achievements. The two-four-stage evaluation procedure allows experts to clarify or revise their point of view, taking into account the opinions of colleagues, and as a result develop an agreed, truly collective position on the entire range of issues raised, the number of which at the first stage, as a rule, exceeds a thousand.

Forecasting according to the Delphi method is also effective in achieving a number of other results that are fundamentally important for identifying priorities. This is a cognitive effect, training and broadening the horizons of experts - participants in the survey, mapping competencies in certain disciplines, technical fields and countries, developing a consensus among representatives of various sectors of the scientific and technical sphere and, no less important, stimulating a broad discussion by the scientific community of trends in scientific and technological development. their country and the world.

Japan has not only the longest history of predictive estimates of the technological development of its country and the world, but also the most effective practice of using these forecasts for the general orientation of the national scientific and technical sphere, all the more interesting, since the state's share in financing national science has never exceeded 20-25 %. Department of Science and Technology, coordinating through strategic research programs fundamental and applied research other departments, is also responsible for technological forecasting.

The Delphi survey is conducted every five years with a time range of up to 30 years, gradually covering all areas of science and technology. If the first survey, which predicted for the period 1970-2000, was able to cover 5 areas and 644 topics, then the last one, covering the period 1996-2025, already included 14 areas and 1072 topics:

o materials and their processing;

o informatics;

o electronics;

o life sciences;

o health and welfare;

o the study and use of outer space;

o Earth sciences and oceanology;

o energy and natural resources;

o ecology;

o Agriculture, timber industry and fish farming;

o industrial production;

o urbanization and construction;

o transport.

Respondents in the latest survey were asked to evaluate technological topics in terms of their contribution to socio-economic development, improving the quality of life and solving environmental issues, as well as their significance in general. The survey participants had to determine the time range during which the listed technologies will be implemented both in Japan and other leading countries, as well as outline the range of measures that government authorities need to take for this.

In France, at the beginning of 1994, using the Delphi method, a broad survey of the prospects for the development of 15 major scientific and technical areas (electronics, elementary particle physics, environmental problems, urbanization, etc.) was carried out. More than 1,000 specialists from various sectors of the economy were involved in the expert assessments - 45% representatives of industrial science, 30% of state research institutes and 25% university employees, which generally reflected the structure of the scientific sector of the French economy. The same principle was followed in the formation of expert groups by most of the countries starting work on forecasts and priorities.

In 1991, the German Ministry of Research and Technology carried out comparative analysis assessments by Japanese and German experts using a Japanese questionnaire. The results generally showed the similarity of the positions of experts for the two countries regarding the development of promising technologies, although certain differences were revealed, reflecting the national cultural and industrial specifics of these countries.

In the UK, since 1994, the use of the Delphi method has also begun to select national scientific and technical priorities. However, unlike Germany and France, the country did not follow the path of copying the Japanese experience (for example, in France, when polling expert scientists, the question of the prospects for research on rice growing problems, directly borrowed from the Japanese methodology, was posed as a priority).

The new mechanism for setting the priorities of the state science policy in the UK has been called "Foresight" ("Foresight"). The program involves working with industry to identify promising markets and technologies for the next 10-20 years, as well as actions that will take advantage of new opportunities to improve the quality of life and accelerate economic growth. The goals of "Foresight": firstly, to collect the information necessary for decision-making on the state and directions of state-funded R&D, secondly, to create a new culture of interaction between scientists and business, and thirdly, to determine the resources necessary to achieve the set goals.

Distinctive features of the new approach are the definition not of specific technologies, but of development directions, the multivariance of scenarios, the continuity of program stages in time. The Foresight 1 program operated from 1994-1999. and moved to "Foresight II" - 1999-2004. Each program consists of three "mutually flowing" stages - analysis, dissemination of information and application of the results, preparation for the next program. "Foresight" determines state priorities in scientific and technical programs, in personnel training, in methods state regulation. At the same time, Foresight is not a rigid guide for the public sector, but for private industry it serves as an "invitation to action" both in the field of participation in cooperation programs and in the field of strategic planning.

In the first phase, 16 thematic groups, comprising experts from industry and the public sector, analyzed a wide range of markets and technologies. Almost all groups are led by representatives large companies and operate in the following areas: agriculture; natural resources and environment; chemical products; means of communication; construction; defense and aerospace industries; energy; Financial services; food products; health and life sciences; education and leisure; production processes and entrepreneurship; materials; retail; transport; marine technology). Experts using the Delphi method analyzed the points of view of 1000 people. Based on these submissions, the groups produced reports assessing future markets and activities needed to maintain the UK's international competitiveness.

The parent group, led by the Government's Chief Scientific Adviser, based on 360 recommendations formulated by industry groups, identified 6 cross-sectoral strategic themes:

Communications and computers;

New organisms, genetic products and processes;

Achievements in materials science, engineering and technology;

Improving the efficiency of production processes and

The need to conserve the environment and resources;

Improving the understanding and use of social

factors;

Within these 6 strategic directions, the parent group has identified 27 general priority areas for cooperation between the scientific and industrial community.

The parent group also identified 5 major infrastructure priorities:

The need to support a high level of education and vocational training (special importance is attached to the level of training of school teachers in the field of science and technology, on which the qualifications of the next generation of scientists, engineers and technologists depend);

Continued high level maintenance fundamental research(especially in multidisciplinary fields);

Development of a communications infrastructure that will put the UK at the center of information flows;

Support for innovative entrepreneurship (financial institutions and the government should constantly review the policy of long-term financing of small innovative businesses and study the impact of the financial climate on innovative activity);

The need for constant revision of public policy and legislative frameworks (primarily in areas such as the protection of the rights intellectual property in electronic communications, development of new genetic organisms, investments in advanced communication infrastructures).

Almost all R&D entities of the country participate in the development of priorities. Priorities are determined “from below” and, as a result, are not “foreign” to scientific organizations, which, according to the Office of Science and Technology, facilitates and accelerates the very process of reorienting research.

The Delphi method, as an attempt to anticipate the future through a collective procedure, has a number of shortcomings. These are doubts about the reliability of the results obtained by straight-line aggregation of individual opinions as a sample of a group of experts representing the scientific community, as well as the blurring of goals and results, a high probability of developing a deterministic and passive outlook on the future, as well as direct uncritical copying of foreign experience.

At a lower level of aggregation - regional, sectoral or problem - in a number of countries, such as Germany, a study of promising priorities using the Mini-Delphi method is being carried out.

So, although the Delphi method is very popular, its influence on the real structure of priorities in most developed countries should still be considered limited. In many countries, this and other methods of identifying priorities often fall on barren ground, that is, they are either not provided with implementation mechanisms or give way to other priorities chosen in accordance with political or any lobbying interests.

5. AdvantageAdvantages and Disadvantages of the Delphi Method

Benefits of the Delphi method:

Anonymity (experts have the opportunity to change their minds without announcing it publicly, to disagree with the "authorities"). The anonymity of the Delphi method allows you to protect specialists from the emotional pressure of opponents and authorities.

In addition, the processing of the results of an expert survey (questionnaires, questionnaires) is carried out according to special methods with the involvement of a mathematical apparatus, which contributes to the great confidentiality of this method.

Using the results of the previous round;

Possibility of statistical data processing;

Allows you to identify the degree of consistency of opinions of experts;

The use of feedback during the survey, which significantly increases the objectivity of expert assessments.

Disadvantages of the Delphi method:

Subjectivity of the opinions of specialists participating in the survey.

Lack of time allotted to the expert to think about the problem. In this case, the expert may agree with the opinion of the majority in order to avoid the need to explain what is the difference between his decision and the other options.

These shortcomings are eliminated by improving the organization of examinations by creating automated systems processing the survey results. The technical implementation of such a system is based on the use of computers with external terminals (displays). The computer ensures the presentation of questions to experts (communicating with it through their personal displays), the collection and processing of the results of answers, the request and issuance of arguments and other necessary information for preparing answers.

In addition, some experts believe that "the assumption that those who strongly disagree with the opinion of the majority, justify their point of view, may lead to an increase in the effect of accommodation, and not reduce it, as intended." But still, many scientists argue that the Delphi method is superior to "conventional" forecasting methods, at least when developing short-term forecasts.

Conclusion

The Delphi method has undeniable advantages over methods based on the usual statistical processing of the results of individual surveys. It allows you to reduce fluctuations in the entire set of individual responses, limits fluctuations within groups. At the same time, as experiments show, the presence of low-skilled experts has less strong influence on a group assessment than simply averaging the results of the responses, since the situation helps them correct the answers by obtaining new information from their group.

List of used literature

Avdulov P.V., Goizman E.I., Kutuzov V.A. etc. Economic - mathematical methods and models for the leader. M.: Economics. 2008

Agafonov V.A. Analysis of strategies and development of integrated programs. M.: Nauka, 2009

Mathematical methods in planning industries and enterprises / Ed. I.G. Popov. M.: Economics, 2010

4. Vladimirova, L.P. Forecasting and planning in market conditions. M.: 2011

5. Mukhin, V. I. Research of control systems: V.I. Mukhin - M.: Publishing house "Exam", 2010. - 384 p.

6. Popova I.G. Mathematical methods in planning industries and enterprises M.: Economics, 2009

Hosted on Allbest.ru

See what the "Delphi Method" is in other dictionaries:

Economic dictionary

Brainstorming, expert assessments, a method for quickly finding solutions based on their generation in the process of brainstorming conducted by a group of specialists, and selecting the best solution based on expert assessments. The Delphi method is used for ... ... Economic dictionary

Delphi method- (Method "Delfi") - a method of predicting future events (for example, innovation, inflation) by a group of experts during a joint discussion of the problem. It consists in organizing a systematic collection of expert assessments, their mathematical ... ... Economic and Mathematical Dictionary

Delphi method- A method of predicting future events (eg innovation, inflation) by a group of experts in the course of a joint discussion of the problem. It consists in organizing a systematic collection of expert assessments, their mathematical and statistical processing, adjustment ... ... Technical Translator's Handbook

DELPHI METHOD Legal Encyclopedia

Delphi method- 3.1.30 Delphi method: A forecasting method in which direct communication between group members is excluded during the study and an individual survey of experts is conducted using questionnaires to find out their opinion on future ... ... Dictionary-reference book of terms of normative and technical documentation

A method for quickly finding solutions based on their generation in the process of brainstorming conducted by a group of specialists and selecting the best solution based on expert assessments. The method is used for expert forecasting by organizing ... ... Encyclopedic Dictionary of Economics and Law

delphi method- a method of quick search for solutions based on their generation in the process of brainstorming conducted by a group of specialists and the selection of the best solution based on expert assessments. The Delphi method is used for expert forecasting by ... ... Dictionary of economic terms

Delphi method- a method of group expert forecasting, in which an individual survey of experts is conducted using questionnaires to find out their opinion on future hypothetical events (a kind of "brainstorming" on the problem posed). ... ... Dictionary « Innovation activity". Terms of innovation management and related fields

DELPHI METHOD- see METHOD, DELPHI... Big Economic Dictionary

Books

Innovation management in an educational organization. Case of a successful leader. GEF (+CD)
Innovation management in an educational organization. Case of a successful leader: management through design. Project team activities. Effective innovation management technologies. Federal State Educational Standard, Borytko Nikolai Mikhailovich, Ivanova Olga Leonidovna, Fastova Elena Igorevna. The set (book + disk) includes control styles; the analysis of pedagogical logistics as a flexible approach to managing the implementation of an innovative project is proposed; showing the mechanism...

Hello dear readers! The Delphi method is a special tool that allows you to develop optimal solutions of any complexity, as well as conduct a qualitative assessment of these solutions and, of course, predict further events.

A bit of history

The name originated from the Delphic Oracle. This method was originally created by Olaf Halmer, Norman Dalkey, and Nicholas Rescher to predict what impact scientific discoveries of the future will have on the way we wage war. It happened around 1950-1960 in Santa Monica.

Scientists have been working at the RAND Research Center trying to invent a tool to predict accurate events. When it was completed, it was Kaplan (he was an employee and professor at the University of California) who gave it such a name. If you remember, the temple of Delphi was a place where information was systematized and accumulated, it was recorded on special plates and considered true and the only true one.

The work of the modern oracle was built on the same principle. Only information about the future appears not thanks to seers, but from ordinary people competent in the area where the difficulty arose. This happens through a survey, in writing or electronically.

Structure

Initially, a group of experts is selected. The number of participants is not limited, but no more than 20 people are recommended, otherwise the results will be significantly delayed, and the work process will become more complicated.
When the expert staff is ready, each of the specialists receives the same task - having processed the topic of the question, draw up a research plan and questionnaire, or simply break the problem into smaller complexities.
Next, a group of analysts is selected, who consider all the lists created by experts and highlight similar points from them. After that, a new questionnaire is created, based on the received general data, and sent back.
Now experts are making additions and their own adjustments to the updated general list. That is, we get, for example, 20 different opinions and views on the same problem.
The revised questionnaire is sent back to the analysts who try to find similarities in the responses to create updates and send back so that the group can this time develop their own solutions to the problem. During this period, the effectiveness and relevance of the proposed methods are evaluated, and the resources necessary for the implementation of the planned actions are checked.
Analysts are again trying to analyze the answers. If they discover radically different points of view, they will certainly bring them up for discussion so that the previous group, in turn, either gives arguments in defense of their ideas, or reconsiders their adequacy and appropriateness.
Each step will be repeated until all risks and “weaknesses” disappear, and, of course, until all experts come to a consensus. It is desirable that there are no more than 5 such tours, otherwise most of the necessary and valuable information will be lost. And the interval between them is no more than one month.

Variations

The above method is considered a classic, but there are situations when a decision needs to be made as soon as possible, so Express Delphi is very popular. It allows you to analyze the problem and come to a consensus in just a few hours, you only need to prepare a good technical base.

That is, each participant must have a computer connected to one common network. The whole process of analysis and selection of the most the best option is carried out in an accelerated manner, without deep processing of the material provided. Despite the fact that this method is very convenient and effective, it has one drawback that significantly affects the result - due to time constraints, specialists do not have time to carefully consider the task facing them and give an objective assessment of the conclusions of the others.

It’s not even worth giving an example here, you yourself know how haste and inattention to details usually end. Therefore, if someone's life or health depends on this method, then it is better to use the traditional step-by-step instructions in compliance with the time frame.

It is recommended to involve competent specialists who have knowledge regarding the problem that has arisen. Only not of a general orientation, but from different areas, then it will be possible to consider the issue from different angles, agreeing on the conclusions of experts.
Before starting work using the Delphi method, it is necessary to clarify the following information:
- How exactly will the work be done? That is, where the participants will be located, how they will analyze the information and where to send the results.
Who will be responsible for making the final decision? Who will decide that enough is enough and it is time to complete this process?
- Are there other ways by which it would be possible to control and double-check the results?
— Approximately, what options do you expect to get in the end? What exactly are you striving for?
Give experts time to reflect on their findings, as well as gain new knowledge on the topic.

In what cases is it applied?

Delphi allows you to predict by systematizing opinions about a situation various groups of people. Therefore, it is mainly used to evaluate long-term problems, identify a topic and build a judgment on the basis of various statements.

In combination with other techniques, it will be an excellent tool that you can rely on in the future. It is used most often in order to consider both personal prospects in the future, as well as technical, organizational ones.

Advantages and disadvantages

Participants do not intersect with each other, they may not even know each other and have no idea with whom they agree on their opinion in the process. And this is convenient for those who experience excessive stress from the very thought that they will have to communicate with someone and defend their opinion, argue their point of view, and so on. In addition, the type of temperament also has an influence on the quality of work, because no matter how professional a person with an introverted character accentuation is, it will be much easier for him to give his assessment of what is happening without entering into direct contact with the rest of the participants in the process.
+ There is no pressure on experts to reach certain conclusions. Therefore, the study will take place without falsifying the results, which is important for obtaining the most honest and objective judgments.