Expert assessment of quality or qualitative assessment of experts. Methods for expert evaluation of product quality indicators

Expert Methods - evaluation methods carried out by a group of experts under conditions of uncertainty or risk.

Expert Methods are used to determine the nomenclature of quality indicators, their weight coefficients, to measure quality indicators and evaluate them by the organoleptic method. Evaluation of quality indicators by measuring, registration, calculation methods is used to determine the complex quality indicators of various levels of the hierarchy.

Designed for expert evaluation of goods in cases where other previously listed methods are inapplicable or uneconomical.

Expert methods are based on making heuristic decisions, which are based on the knowledge and experience accumulated by experts in a particular field in the past.

Expert methods have certain advantages and disadvantages.

The advantages are that they allow decisions to be made when objective methods are unresponsive. Other advantages include their recoverability. The scope of these methods is not only the assessment of the quality of goods, but also the study of the operations of the technological cycle, decision-making, management, forecasting.

Expert methods, applied by qualified experts, allow for an accurate assessment of goods. The conducted experiments show that with the correct method of expert evaluation, the error of the results is 5-10%, which is allowed for measurement methods. The results of expert assessments obtained experimentally in various groups of experts showed their rather high recoverability.

The disadvantages of expert methods include subjectivity, limited application, high costs for their implementation.

The subjectivism of expert methods is a consequence of the fact that the expert assessment is carried out by each expert individually and, according to E. P. Raikhman and G. G. Azgaldov, is "nothing but his psychological reaction to the physical and chemical characteristics of the product." However, it should be borne in mind that an expert assessment is made up of the opinions of several experts, each of whom is not only a specialist in a particular field of knowledge, but also a consumer. Therefore, the expert assessment to a certain extent reflects the opinions of consumers, which cannot be done with other methods.

Due to a significant amount of subjectivity, expert methods have certain limitations. their use is rational in two cases: firstly, when the goals set for the experts cannot be solved by other methods; secondly, when available alternative methods give less accurate and reliable results or are associated with high costs.

To eliminate this shortcoming, expert methods are combined with other methods when conducting a commodity examination. Most often, expert and organoleptic methods are used together. Moreover, when developing organoleptic scoring scales, choosing a nomenclature of quality indicators, and determining weight coefficients, expert methods are indispensable.

Expert methods are divided into three subgroups: 1) methods of group survey of experts; 2) mathematical and statistical methods for processing expert assessments; 3) methods for expert assessment of quality indicators.

Each group of expert methods, in turn, is divided into types and varieties. The classification of expert methods is shown in fig. 1.3.

Rice. 1.3. in

Methods of group survey of experts - methods based on conducting a survey of a group of experts with subsequent analysis and processing of information received from them.

The purpose of these methods is to obtain group expert opinion for making final decisions.

The rationale for the choice is the need to make complex decisions in a situation of uncertainty or to make a scientifically based forecast that requires the participation of a group of independent and competent specialists in a narrow field or many fields of knowledge (for example, knowledge of a homogeneous group of goods or all food products).

The main advantages of a group expert assessment are the possibility of a comprehensive analysis of the quantitative and qualitative aspects of the problems of determining and / or forecasting individual characteristics of goods or their combination. Interaction between experts makes it possible to significantly increase the amount of total information owned by a group of experts, in comparison with the information of any member of the group. In addition, the number of factors taken into account in the group assessment and affecting the effectiveness of the decision being made is greater than the sum of the factors taken into account by one expert. With a group assessment, there is less error in making basic decisions and indicators that are not essential for solving the problem. Therefore, an important advantage of group evaluation is the possibility of obtaining a generalized result.

Disadvantages of group estimates include: difficulties in obtaining a reliable and consistent estimate; receiving unequal answers to the same question with a large difference of opinion due to the different competence of experts; receipt of unambiguous answers does not guarantee their validity and reliability, and this cannot be verified during the examination; more incorrect information from a group of experts than from an individual expert can lead to significant errors in the final results; the possibility of confrontation, when individual experts for uncertainty or other reasons may agree with the opinion of the majority.

Despite these shortcomings, it has been experimentally established that, subject to certain requirements, a group assessment is more reliable than an individual one. These requirements include: an acceptable distribution of grades; group reliability; examination preparation.

The effectiveness of the examination depends on the accuracy and reliability of the results obtained, that is, on the methods used and on the qualifications of the expert. The choice of an expert is a difficult task, most often several personal characteristics are taken into account: competence - professional and qualimetric, interest of an expert in the results of an examination, attitude to business, objectivity. Almost no account is taken of such traits as risk appetite and other psychological characteristics.

A systematic approach to assessing the quality of an expert has not yet been developed; existing assessment methods are divided into five groups:

heuristic;

Statistical - estimates that are obtained as a result of the analysis of expert estimates with the determination of deviations from the average values;

Test - assessments obtained as a result of the performance of test tasks by experts;

Documentary - assessments of competence obtained in the analysis of individual documentary data of examinations conducted by an expert;

Combined - estimates obtained during the analysis of data obtained by combining the above methods.

Heuristic assessments include self-assessment and assessment made by a team of experts. Self-esteem is most often subjective; to reduce subjectivity, a point scale is used for individual properties of an expert. Self-assessment is carried out according to the types of goods and quality indicators, for example, the aesthetics of products, the expert evaluates himself by questioning. The questionnaire includes the frequency of familiarization with modern domestic and foreign literature, with modern product samples, with the results of sociological surveys.

As a kind of self-assessment, the method of assessment by reasoning and familiarization with the products that are being analyzed is used. The assessment is carried out using a questionnaire with the determination of the coefficient of competence.

Via test grades it is possible to evaluate such important properties of an expert as qualimetric and professional competence, objectivity. Qualimetric competence in carrying out an organoleptic assessment consists in a multiple survey of experts to determine the weight coefficients several times and determine the reliability of the assessments.

Professional competence is tested in the form of control, the ability to use various types of rating scales - order, relationships, intervals, as well as the ability to distinguish a significant number of characteristics, gradations when evaluating the properties of the product being analyzed.

With the participation of experts in a collective assessment and discussion, the phenomenon of conformism is observed, that is, the expert falls under the influence of the conclusions of other experts, which negatively affects the formation of an objective assessment of the expert. Therefore, the ability to adhere to one's opinion and defend one's conclusions is of positive importance during the examination by a collective method and the formation of an objective assessment.

Expert Methods quality assessments are based on the use of expert opinions. They are used when it is impossible or inexpedient, uneconomical to use measuring or calculation methods. This happens when there is a lack of information, the need to use and develop special technical means, when evaluating aesthetic quality indicators, and the like. Expert methods can be combined with other methods or used as an independent type when assessing the quality of regulatory documentation for products and products, determining the range of indicators and their weight coefficients, when choosing basic samples and quality indicators, when determining and measuring quality indicators by the organoleptic method, during the time for evaluating single and complex quality indicators determined by the measuring or calculation method.

The basis of expert evaluation of quality is the objective social utility of products, which reflects its modernity. The opinion about the quality of products expressed by a qualified expert meets public needs and coincides with the opinion of the mass consumer.

However, in choosing a product, the consumer adheres to a conservative position and accepts new products carefully, and sometimes does not perceive them at all. In the opinion of experts on the quality of the goods, a set of opinions of consumers of products is summarized. Studies have shown that the quality assessments of the experts' goods coincide with the assessments of product quality that were obtained during a mass survey of consumers. Determination of the accuracy of expert studies for compliance with the methodology of the examination showed that it is 5-10%.

In relation to product quality, its consumer properties, expert methods are used in such cases:

Determination of the nomenclature of indicators during the examination;

Selection of criteria for evaluating a product or product and placing indicators according to the principle of hierarchy;

Determination of weight coefficients of quality indicators to determine the quality level of a product, goods;

The study of indicators by the organoleptic method using the methods of their quantitative expression;

Evaluation of quality indicators by measuring, registration, calculation methods to determine the complex quality indicators of various levels of the hierarchy.

The most commonly used expert methods are:

Leading expert (single);

commissions;

combined.

The leading expert method makes it possible to quickly conduct an examination, reduce the time for the coordination and discussion procedure, and statistical processing of data by group members. However, the results of an examination conducted by one expert depend on the level of his professional knowledge, personality, and level of competence.

The method of expert commissions involves the participation of a group of specialists who conduct analysis and evaluation. This method allows you to get reliable, objective results, but requires a significant investment of time in the preparation and organization of the examination. The number of experts who form the commission depends on the required accuracy and reliability of the results of the examination. The expert commission consists of two groups - working and expert. The working group prepares, organizes and conducts an expert assessment of product quality, followed by analysis of its results. The working group includes an organizer, a consultant who has professional knowledge about the products, and technical workers evaluate them. The expert group may consist of several subgroups, each of which specializes in solving the relevant tasks - determining the nomenclature of indicators, evaluating individual groups of indicators, and the like.

The combined method, which is based on the consistent use of the work of a leading expert and a small number of expert commissions, is used in some cases.

There are no expert methods in the General Classification. However, classification is used depending on the ratio of the amount of data obtained by the expert or analytical method, the method of obtaining information from the expert, and some other factors.

Depending on the method of obtaining information from an expert, methods are distinguished:

Collective;

Individual.

With a collective method, a technical worker conducts a survey of the entire group of experts at once, with an individual method, each expert separately. To obtain reliable results, it is necessary to clearly and correctly set the goal and objectives for the expert, with the group method it is difficult, but possible with a constantly formed group. With an individual method, interviews, interview questionnaires, questionnaires, mixed questionnaires are used. Surveys can be conducted face-to-face and by correspondence. In the face-to-face method, the expert expresses his judgments to the person conducting the survey. During a remote survey, there is no contact between them and the expert fills out the questionnaire or survey card himself.

During the interview, the technical worker makes a note in the form of a conversation that goes through the program and a specific list of questions. During the interview questionnaire, the list of questions is more specific, directed, the sequence of questions is strictly defined. The questionnaire is filled out in the presence of an expert.

Questioning is different in that the expert independently fills out the questionnaire, having an explanatory note on filling it out. Mixed questioning involves a preliminary explanation of the expert on filling out the questionnaire with a specification of the task.

In terms of informativeness, interview methods, interviews - questionnaires, and questionnaires have the greatest opportunities. The greatest independence of judgments is characteristic of the questionnaire method.

To solve complex situations of uncertainty or during the formation of a scientific and technical forecast, expertise requires the participation of a group of erudite specialists who are well-versed in many fields of knowledge. The main advantage of the collective assessment lies in the possibility of a comprehensive analysis of the quantitative and qualitative aspects of the problems. There are problems that cannot be solved without the participation of specialists. It is assumed that the opinion of a group of experts is more reliable than the opinion of an individual, that is, two groups of equally competent experts are more likely to find an objective solution.

The combination of operations for preparing and conducting surveys of experts, as well as technical operations, made it possible to create several expert methods that have received the greatest recognition and distribution. These include the Delphi, PATTERN and combined methods.

Delphi method (in some sources - Delphi) - a method of interviewing experts based on sequentially implemented procedures that are aimed at forming a group opinion on procedures with insufficient information.

The Delphi method was developed in the American research company RAND Corporation by W. Helmer, N. Dolki and T. Gordon. It was used for military scientific and technical forecasting of the future. The term Delphi comes from the name of the town in Greece, where the oracles lived at the temple of the god Apollo.

The features of the Delphi method are: the refusal of the joint work of experts; anonymity of estimates; adjustable feedback; group response.

Refusal of joint work of experts and anonymity are achieved by the fact that each expert expresses his opinion in the questionnaire, without group discussion. Other techniques of individual survey are also used, for example, answers to questions are entered by experts into a computer. This allows you to reduce the discrepancy between individual estimates and get a group answer, which correctly reflects the opinion of each expert.

The anonymity of the survey allows to reduce the conformity of the authoritarian influence of individual dominant experts, regulated feedback reduces the influence of individual and group interests. The introduction of feedback also increases the criterion of objectivity and reliability of estimates.

When using this method for the purposes of expert evaluation of the quality of consumer goods, the following disadvantages appear: the complexity of interviewing experts and filling out questionnaires, the complexity of the assessment due to the large number of quality indicators (sometimes up to 20-40) and filling out several questionnaires (3-10), cumbersome records of explanations due to the lack of direct contact between the organizer and the experts.

The method is promising for obtaining a group expert assessment and in-depth analysis of events in situations of uncertainty.

PATTERN Method - a method of interviewing experts based on building a hierarchical structure - a tree of goals - and making a decision on these goals after an open discussion.

The name of the method consists of the first letters of English words, meaning "Assisting planning by quantifying technical data."

The method was developed by the American company "Honquell" for evaluating projects of new weapons systems. The method has analogues: PROFILE, in France - the KPI method, PROPLEN etc.

The PATTERN method involves several stages.

Stage I - the formulation of the main problem that needs to be solved, and its division into a number of secondary problems of the first, second, etc. order, which are then divided into narrower tasks. The division continues until simple elements are obtained that can be evaluated by experts.

As a result of this division, a hierarchical structure of main, secondary problems and tasks connected with each other is obtained, called the goal tree.

Stage II - determination with the help of experts of the coefficients of weight (or significance) of each task in relation to the main goal, while the experts make a decision after an open discussion in the expert group.

Such an open discussion, along with a positive factor - the interaction of experts who strive to make a positive decision - also has negative consequences due to conformism, i.e., distortion of the real opinion of experts due to suggestion or adaptation to the opinion of the majority.

Stage III - the use of computers for processing the received data and their analysis. The advantage of the PATTERN method is the simplification of the expert survey procedure. Disadvantages: lack of rationale for the optimal number of members of the expert group, as well as the methodology for selecting competent specialists for the expert group; processing the results of the survey without taking into account differences as individual experts; lack of barriers to the manifestation of expert conformity; insufficient development and uncertainty of the principles of building a tree of goals.

Since the Delphi and PATTERN methods have significant shortcomings and do not fully correspond to the goals of peer review, Oe. L. Raikhman and G. G. Azgaldov proposed a combined method in which the positive features of other expert methods were used and their shortcomings were excluded.

Combined method - a method based on a combination of individual and collective expert assessments.

The advantages of the combined method are sufficient flexibility, which allows eliminating errors in the survey of experts and increasing the reliability of the examination results, a clear definition of the strategy by classifying tasks according to the degree of significance and operations for their implementation, high reproducibility of the results.

The disadvantages of the method include a rich operation, which requires a significant investment of time and money. However, this shortcoming is compensated by increased reliability and recovery of results.

To assess the quality of goods, the combined method has a general algorithm of expert operations:

1. Preparatory stage:

Formation of a working group;

Formation of an expert group;

Classification of products and consumers;

Building a block diagram of quality indicators.

2. The stage of obtaining individual expert assessments:

Selection of the procedure for assigning assessments by experts;

Choosing a method for obtaining information from an expert and preparing documents required for the survey;

Survey of experts.

3. The stage of obtaining collective expert assessments:

Generalizations of individual expert assessments;

Determining the consistency of individual expert assessments;

Determination of the objectivity of collective expert assessments.

There are corresponding tasks for each stage. During the preparatory stage, tasks such as determining the functions and structure of the working group, its quantitative composition, and the responsibilities of individual members are solved. The principles for the formation of an expert group are determined and developed: regarding the number of experts, their professional training, and the like.

At the second stage, the technique of interviewing experts is determined, the presence of contact between experts, the method of transmitting information and the form of expert assessments are assessed. Experts can determine the scores on their own or after discussion with other experts or familiarization with anonymous experts. The scores are justified, quantified and dichotomous, where answers are given in the form of "yes", "no" or 0-1.

Rational use of information received from experts is possible if it is converted into a form convenient for analysis, preparation and decision-making. The possibilities of converting information into appropriate forms depend on the specific features of the object, the completeness of data about it, reliability, the level of decision making, as well as on the accepted criterion, depending on the problem under study.

One of the elements common to many expert methods is the weight coefficient.

The weighting coefficient is a quantitative characteristic of the degree of significance of a particular indicator for assessing quality.

The determination of the weighting coefficients of quality indicators is carried out by the expert method. The weight coefficients are intended to increase the reliability of the expert assessment of the quality of goods.

Each indicator occupies a certain place in the nomenclature of quality indicators in terms of importance. Experts rank indicators according to the degree of significance based on professional knowledge and skills. In addition, any qualified expert seeks to evaluate the quality indicators of the examined goods from the standpoint of the mass consumer.

If an expert has the ability to compare and evaluate possible options for action, giving each of them a certain number, then he has a certain system or scale of preferences. The correct application of the scales is essential to ensure the accuracy of expert judgments. There are the following types of scales: nominal, ordinal, interval, relations. But the order scale has the greatest advantages over others due to the relative simplicity of expert evaluation of quality indicators in terms of significance. The characteristics of the scales are given in Table. 1.1.

Nominal name scale used to distinguish one object from another. Objects should be numbered, however, the numbers indicate the object, and not its quantitative characteristics. This is a simpler type of measurement that only uses numbers or symbols to classify objects. The scale can be used for digital coding of individual properties in questionnaires, to determine weight coefficients.

Ordinal scale (ranks) - such an evaluation method in which the parameters, indicators or objects that are evaluated are arranged in the order of increase or decrease in the indicator of the parameter (indicator) or the properties of the object. A classic example of evaluation using an ordinal scale is the assessment of the hardness of minerals on the Mops scale (the scale of relative hardness consists of 10 hardness standards, with the hardness of talc taken as 1, diamond - 10). This method can be used to determine the intensity of the color of flour, the aroma of fruit juices, the bouquet of wines, the texture of cheeses. The ordinal scale has advantages when used to determine weighting factors, since it simplifies the process of ordering quality indicators in terms of importance for consumers.

Table 1.1. Scale types and their characteristics

Scale type	Scale Definition	Relationships set on the scale
Rated	A simple measurement type that uses numbers or symbols only to classify objects.	Equivalence (=)
Ordinal (ranks)	Objects of one class are in a corresponding relationship with 3 objects of another class (more than, more advantages, stronger, etc.). If [A]>[B] for some objects of classes A and B, then there is a partially ordered scale.	Equivalence (=). Greater than (>).
Interval	The ordinal scale, which is divided into known distances between any two numbers on the scale, the zero point of the scale and the unit of measurement are chosen arbitrarily.	Equivalence (=) Greater Than (>). The ratio of any two intervals is known.
Relations	Interval scale using a true zero point, the ratio of any two points is independent of the unit.	Equivalence (=). Greater than (>). The ratio of any two intervals is defined. The relationship between any two points is defined.

When determining the weight coefficients of quality indicators, experts first evaluate the most important of these indicators (in their opinion) and assign it a certain number, for example 1. All further indicators are evaluated in decreasing or increasing order of importance.

After that, the data of all experts are averaged for each indicator.

In the practice of assessing the quality of goods, a method for determining weight coefficients, called the "fixed sum method", is often used. Its essence lies in the fact that experts assign a weighting coefficient for the indicators included in the upper level indicator, and the sum of these coefficients must be equal to a predetermined number.

The "fixed sum method" is expediently used only for a small number of indicators.

Practical experience shows that it is advisable to apply the following procedure for determining the weight coefficients in the expert assessment of the quality of goods.

1. Preliminary ranking by experts of indicators of a homogeneous group. Rank 1 is assigned to the most important indicator, 2 - to the next most important, and so on. If the indicators are equivalent in importance, then they are assigned the same ranks. The number of indicators in a homogeneous group should be 4 or more. If the number is less, the ranking is not carried out.

2. Determination by experts of coefficients of weighting of indicators. An indicator of the 1st rank is assigned a weighting factor of 10. The weighting factor of the next most important indicator is determined as the proportion of the importance of the first indicator. When determining the third and subsequent indicators, their importance in comparison with the previous ones is taken into account. As a result of these successive actions, the expert determines the weight coefficients of single, and then complex quality indicators.

3. Familiarization of experts with the values ​​of weight coefficients (their justifications) assigned by other experts.

Justification of weight coefficients is a very laborious operation, therefore it is used with a limited number of indicators (about 10-15). Otherwise, the experts are invited to give justification only for some indicators at their discretion.

4. Averaging the values ​​of weight coefficients determined by all experts. Conducted by technical workers by calculating the arithmetic mean or weighted average. In the latter case, a comprehensive assessment of the expert's quality is taken into account.

The disadvantages of the scale are the inaccuracy of ranking estimates due to the absence of an equation for intervals, the impossibility of calculating even the arithmetic mean value.

Interval scale. This is such an estimation method, in which the essential characteristic is the difference between the values ​​of the estimated parameters, which can be expressed by the number of units provided for this scale. With the help of such a scale, objects are ranked, and it is also established in certain units how much one object is larger than the other. An example of an interval scale is the Celsius scale, which is divided into 100 equal intervals and is used to characterize such product properties that are associated with temperature conditions, for example, the frost resistance of synthetic leather, the minimum temperature of the freezer in the refrigerator.

Relationship scale allows you to achieve the highest level of measurement. This is an estimation method that uses a unit of measure, it is used for most parameters that are physical quantities: size, weight, density, force, voltage, frequency, and the like. Scale measurement results have the properties of numbers that can be subjected to statistical processing. An example of such a scale is the Kelvin temperature scale, which starts at absolute zero.

When comparing the scale of intervals and ratios, it was determined that with the help of the latter, more accurate results are obtained. In addition, scores obtained on the interval scale can be used to calculate weighted averages, the calculations of which are typical for experimental surveys. The ratio scale can be the basis for the expert method, the order scale can be used with sufficient justification.

In the practice of peer review, two main types of scales are used - dimensional and dimensionless. The data of dimensionless scales are expressed in fractions of a unit, percent, points.

Mathematical and statistical methods for processing expert assessments are methods designed to increase the reliability of the results of assessing the quality of goods by experts.

They are divided into four subgroups: ranking, direct evaluation, sequential preferences and pairwise comparisons.

Ranging - a method based on the arrangement of objects of expertise in ascending or descending order.

Designed to solve many practical problems, when the objects that determine the final results are not amenable to direct measurement. In addition, individual objects, characterized by different nature, are incompatible, because they do not have a common measure of comparison. The basis for ranking is the need to order any object in time and space, as well as in accordance with the measurable quality without making precise measurements. And finally, in a situation where the quality that is being measured cannot, in principle, be measured for practical and theoretical reasons.

The ranking procedure consists in placing objects by an expert in the most rational order and assigning them a certain rank in the form of a natural number. In this case, the most important object gets rank 1, and the least important one gets rank n. As a result, an order scale was obtained in which the number of ranks is equal to the number of objects. If two objects have the same ranks, then they are assigned the so-called standardized ranks, which are calculated as the average of the sum of the places of objects with the same ranks.

For example, six objects are assigned the following ranks:

Objects 2 and 5 shared 2nd and 3rd places. their standardized rank will be

(2 + 3) /2 = 2,5.

Items 3, 4, and 6 are tied for 4th, 5th, and 6th places, and their standardized rank is 5:

(4 4-5 + b) / 3 = 5. The result is the following ranking:

The ranking method is rarely used in its pure form. Most often, it is combined with the direct assessment method or its modifications (ranking by the sum of assessments, a combined method, etc.).

Direct evaluation method lies in the fact that the range of change of any quantitative variable is divided into several intervals, each of which is assigned a certain score in points, for example, from 0 to 10. The rating scale can be positive and negative, for example, from +3 to -3.Expert should include each object in a certain interval depending on its value. The number of intervals into which the entire range of quality change is divided may be different for different experts. Individual experts are allowed to evaluate qualitatively different factors with the same number.

In some cases, it is more convenient to select the most preferable factor first to evaluate and then to rank.

The total ratings of the ranks can be normalized, this allows you to establish a closer relationship between the ratings that the experts provided to individual objects. For this purpose, the estimates for all objects are summed up and then each of them is divided by the amount received. The normalized scores calculated in this way can be ranked again.

When conducting an examination by several experts, they seek to obtain an average estimate for each object. For this purpose, the normalized estimates of each object are summed up, the resulting sum is divided by the number of experts. The second way to determine the relationship between the ratings of factors is that the most important factor is given an estimate (weight is set) for a certain number of 1 or 10, the following factors are evaluated as a share of the most important factor. The advantage of the method is that it simplifies the process of selecting estimates, because it is not necessary to compare the entire series each time, but only take into account the value of the first and previous estimates in importance. Estimates are averaged by calculating the arithmetic mean.

Successive advantage method - based on comparing an individual object with the sum of subsequent objects to establish its importance. used in measuring the level of quality, evaluating the activities of scientific organizations. The method has such a major advantage over other methods that it makes it possible to compare and measure qualitatively different factors.

The method was developed by V. Churchman and G. Akof and is intended for comparisons within certain tolerances.

The order in which results are presented or grouped does not affect the benefits.

The procedure for successive comparisons is as follows. The expert is presented with a number of objects (indicators, factors, results) that need to be assessed by their relative importance (significance), and he performs the ranking. The most important object is assigned a score equal to 1, the remaining objects - scores below 1 to 0 in order of their relative importance. Then the expert determines whether the object with a score of 1 is more important than the sum of the last factors. If the importance of an object is high, then it increases the score so that it is greater than the sum of all the others.

If the value of an object is lower than the sum of all others, it adjusts the estimates.

Thus, the procedure used is to systematically check the scores by comparing them sequentially.

The method of successive preferences is advisable to apply if the number of compared objects does not exceed 7. With a larger number of objects, they must be divided into subgroups that include 6 objects. Where this is not possible, paired comparisons should be used.

Pairwise Comparison Method - based on comparing the objects of expertise in pairs to establish the most important in each pair.

It is used to identify advantages among a significant number of factors, problems, indicators. Experts can simply make a comparison with a statement of the superiority of one factor over another. It is possible to use a special preference scale, where each degree of advantage has its own specific assessment.

The method of paired comparisons can also be used to establish the total ranks of factors.

To facilitate the procedure, matrices of pair equations are compiled in which all objects (factors) are written in the same order twice: in the top row and the leftmost column. Each expert must put down at the intersection of the row and column an ​​estimate for the two compared factors. Depending on which factor is the most important, this score will be equal to 1 or 0, respectively. Dashes or zeros are entered in the main diagonal of such a matrix (table. 1.2).

Table 1.2.

Each pair of factors can be compared once or twice. There are various options for partial pairwise comparison: choosing the preferred object from pre-grouped pairs; partial pair comparison of one group of objects with all others, while the remaining factors are compared with some others; establishment of total ranks of factors.

The pairwise comparison method is sometimes combined with a preliminary ranking of objects, with the paired comparison being used to clarify the advantage of individual objects. In this case, an additional matrix is ​​constructed, which indicates the proportion of cases where one factor is more significant than the other, in the total number of estimates obtained.

Methods for expert evaluation of product quality indicators- these are methods for determining the actual values ​​of single and complex quality indicators.

Designed to determine the values ​​of the quality index by calculation or heuristics in cases where the use of measurement methods is impossible or uneconomical due to excessive costs for their application or long testing time. For example, when determining the taste and smell of food products, only organoleptic methods are used. Measuring methods do not give an accurate reliable estimate, despite the increased costs.

For differential and complex evaluation of samples that differ significantly in quality, it is recommended to determine the value of a single indicator P as follows:

where P;5 is the base (reference value).

Another, more accurate method is based on the study of indicators in order to determine the types of dependence and, i.e., in order to develop formulas for calculating indicator estimates:

Evaluation of single quality indicators begins with the determination of the permissible intervals for their change (P; - P;) P; - the best value of the indicator, the excess of which is impractical or impossible. The principles for assigning the maximum allowable value of an indicator depend on the purpose of quality assessment, while it is necessary that this principle be the same for all indicators.

Increasing the reliability of expert assessments is achieved by dividing complex operations into simple ones, which make up a multi-stage procedure for assessing the permissible values ​​of the indicator. The transition to each next level is carried out after the adoption of agreed decisions on the previous one.

The expert procedure for determining the acceptable values ​​of quality indicators consists of a number of operations:

issuance of questionnaires and explanatory notes to the experts, which list the quality indicators and describe the principles for choosing the acceptable values ​​of the indicators;

filling out questionnaires by experts and indicating specific models of products, the values ​​of which they consider to be the maximum allowable;

familiarization of each expert with the assessments indicated by other experts, and their discussion;

conducting the second (sometimes third and fourth) round of the survey;

Averaging the evaluation results.

With a significant difference of opinion, an additional round of voting is held. The value of the indicator is taken as the maximum if at least 70% of the votes are cast for it. If this condition is not met, the average of 50% of the largest values ​​of ріmax is taken as the maximum allowable value, the average of 50% of the largest values ​​is taken as the minimum allowable value, the values ​​are used by experts when determining the estimates of quality indicators K.

To determine by experts the type of dependencies (i) between the value of indicators G; and their estimates of K, the "principal method of points" is often used. The need for its use is due to the fact that the division of the estimation procedure into several stages simplifies the work of the expert and allows him to give estimates to some characteristic points, based on which it is possible to build a model of the real value.

"Principal point method" There are several varieties depending on their number.

"Method of the three main points" - based on the separation of the values ​​of indicators G; on the maximum, minimum, average values ​​and determining the values ​​of the estimates P; at these points. The scale interval between the maximum and minimum points is preset (0-1 or 0-10 scales). The expert's task also includes determining the dependence trend in the interval between the main points and plotting. After that, from the graphical dependence, you can go to the analytical formula for calculating the estimates of the quality indicator Ki "Method of the three main points" allows you to develop only an approximate evaluation model.

"Method of the seven main points" - a method for evaluating indicators on a seven-point scale for evaluating indicators, the values ​​of which are determined experimentally or by calculation, as well as by the organoleptic method.

The seven-point scale is uniform, that is, when moving from one quality class to another, the score changes by one point. These scales have found wide application, especially in organoleptic evaluation. To obtain more accurate results, you should proceed to determining the type of relationship between grades and points.

To facilitate the work of the expert, five graphs are given in the explanatory note to the questionnaire. The expert selects the curve (or combination of curves) that, in his opinion, best reflects the nature of the dependence and Then, each quality class is assigned a score in accordance with the nature of the dependence and the values ​​of the quality indicators. In this case, it is advisable to use numbers in the range 0-10, multiples of 0.5, and the "highest quality" class receives a score of 10.

Thus, the graph constructed by the expert characterizes the relationship between the absolute values ​​of Pi indicators and their estimates K;, and for indicators evaluated by organoleptic methods, between quality classes and their estimates.

In conclusion, the obtained results are discussed, processed and analyzed. For indicators that are determined by measuring and calculation methods, it is desirable to give an analytical description of the curves, which makes it possible to calculate an estimate for any values ​​of the indicators.

The use of the "principal points method" makes it possible to group and classify indicators according to the types of dependence.

The definition of complex quality indicators is carried out by two types of methods:

methods of comprehensive assessment of the quality of product samples;

methods of building models of complex quality indicators.

There are two varieties of integrated quality assessment methods - the express method and the methods of moving through levels without training and with training.

Express methods for comprehensive assessment of the quality of product samples are based on the determination of a complex quality indicator by analyzing the values ​​of individual single indicators and appearance without their preliminary assessment and taking into account the weighting coefficients.

When using these methods, it should be taken into account that the maximum number of estimated indicators, even for a highly qualified expert, is 7-9 indicators located at the same level of the hierarchy, which make up a fairly homogeneous group. In addition, experts should take into account the importance of individual indicators with the help of weight coefficients, the relationship between them, and also consider the quality of the product as a system.

The method of moving through the levels without preparation is a set of operations that are carried out sequentially, with a gradual increase in the level. In this case, the analysis starts from the lower level of the indicator tree. Taking into account the value of the indicators of the lower level, the expert evaluates the indicators of the higher level. These operations are repeated with increasing level until the upper level is reached - a comprehensive (generalized) quality assessment.

Method of moving through levels with preparation based on a preliminary determination by experts of the weighting coefficients of quality indicators and their assessments. When assigning complex assessments, the expert knows the average values ​​of the weighting coefficients and estimates of individual indicators. The procedure for determining complex indicators is similar to the procedure for the method of moving through levels without preparation.

Formalization of the peer review process is to find the relationship between the values ​​of quality indicators G; (or their estimates K) and the quality indicator of the higher level, i.e., in determining the type of decisive function that experts use when assigning complex indicators. At the same time, the decisive function, like any model, simplifies the object of study, since not all indicators and the relationships between them are taken into account.

The initial data for determining the type of function can be the results of evaluating the quality of various samples by express methods or methods of moving through levels. Then the estimates assigned by the experts are reduced to a common matrix, each row of which is a set of estimates of individual indicators of the sample and complex expert estimates. Based on this, machine algorithms can be developed and programs can be compiled to find the decisive functions with the help of a computer.

Qualitative examination is an assessment of the qualitative characteristics of the goods by experts to establish their compliance with the requirements of regulatory documents. This examination is carried out in order to determine the quality of the goods in the consignment upon delivery and acceptance, or after long-term storage, or if hidden technological defects are found during storage, when the usual deadlines for filing claims with the supplier have expired. In addition, quality expertise is used when evaluating samples of new products before putting them into mass production. For food products, a qualitative examination only by organoleptic indicators is called tasting.

Depending on the purpose, a qualitative examination can be: acceptance, by completeness, of new types of goods, tasting of food products, under contracts.

Acceptance examination for quality is the assessment of the quality of goods by experts to confirm the reliability of the results of the inspection upon acceptance. The basis for such an examination may be: disagreements between the supplier and the recipient based on the results of the acceptance control carried out by the recipient in the absence of the supplier and the impossibility of his appearance for re-acceptance; detected or alleged discrepancy between the actual quality of the goods specified in the documents; violation of packaging (breakage, deformation, etc.); the presence of significant quality losses during transportation or storage. When conducting a qualitative examination, they use standards, sanitary and veterinary rules and instructions, rules or codes of transport organizations.

When conducting an acceptance examination for quality, experts must adhere to the basic rules, which include the following provisions:

• 1. Before starting the examination, it is necessary to familiarize yourself with all regulatory documents (standards for specific types of products, packaging and labeling, test methods).
• 2. The quality of goods must be determined in accordance with the requirements of existing standards, contracts. If necessary, the accepted goods can be evaluated by comparison with samples or standards. The choice of quality indicators is determined by the goals and conditions of the examination. Most often, experts limit themselves to assessing organoleptic indicators and some physico-chemical indicators using the simplest measuring instruments. Verification of microbiological and complex physico-chemical parameters can be recommended by an expert only in some cases after completion of quality acceptance.
• 3. To assess the quality, a sample or a combined sample must be selected, the size of which must not be less than the established norms.
• 4. In case of heterogeneity of a consignment, including goods of different quality (standard, 1st or other grades, non-standard, marriage, waste), the expert must determine the percentage of each fraction. When defective goods are found, the expert must select samples of goods with the most typical defects and establish the causes of their occurrence. It is advisable to send samples of defective products to the testing laboratory to identify their type and causes. The examination report should reflect the percentage of products with various defects.
• 5. When identifying the reasons for the quality discrepancy, the expert must indicate in the act the condition of the container and packaging materials (their integrity, reliability, sufficiency).
• 6. The expert should not conduct an acceptance examination if the integrity of the consignment is violated or impersonal goods are presented, as well as unpacked or without shipping documents.

Examination of goods by completeness is the assessment by an expert of the availability of the necessary elements of the kit and the establishment of compliance with the data specified in the technical documents. The legal basis for this examination is the Civil Code of the Russian Federation (part 2, art. 478-480). The basis for its implementation are the features of the goods received, the integrity of which is ensured by the presence of all the necessary elements in the kit. All components are divided into three groups:

• * ensuring the functional purpose of the product, its aesthetic properties;
• * intended for the repair of goods during operation;
• * determining the safety of goods during transportation, storage and sale.

Examination of completeness is subjected mainly to non-food products, and food products - only when selling sets of products.

Examination of the quality of new goods is called the assessment of the quality of goods according to the nomenclature of indicators characterizing the degree of novelty, the possibility and expediency of their release into the market. A new product is a product intended for sale and differing from existing products of a similar purpose in modified consumer properties. The purpose of this quality review is to determine the practical usefulness and quality indicators that can create consumer preferences.

There is no legal basis for the examination of the quality of new products, and the regulatory framework is represented by standards, sanitary and other rules, and in this it coincides with the acceptance examination. When examining the quality of new products, various evaluation methods are used: organoleptic, measuring, sociological, expert. Expert assessment of quality is of decisive importance, and other methods play a supporting role.

The purpose of the examination may be to evaluate the entire range of quality indicators or only part of them, for example, the degree of novelty. The choice of quality indicators is determined by the need to comply with mandatory requirements (safety, functionality), as well as determining the degree of novelty, which is the essence of the examination of the quality of new products. The degree of novelty is characterized by quantitative and qualitative changes in consumer properties that form and satisfy new needs. The degree of novelty can be expressed in points or percentages. HER. Zadesenetz offers the following gradations of the degree of novelty: goods of little novelty (degree of novelty up to 20%), goods of a new type (21--70%) and qualitatively new goods (71-100%). Depending on the gradation of a new product, the methods of its study are also chosen. For goods whose degree of novelty does not exceed 70%, comparative methods can be used based on a comparison of the quality indicators of goods of the same name, new and already known. For qualitatively new goods, this method is hardly acceptable, therefore, they use predictive analog, non-analogue or combined, which are varieties of the comparative method, but at a higher level, using forecasting elements. The predictive-analogue method is based on the construction of a prospective-forecast series of basic samples of analogous goods and its use for comparative evaluation of qualitatively new goods. The analogless method is based on the analysis of consumption processes by different groups of consumers or design processes. The combined method is based on combining estimates of previously known and new consumer properties of the studied goods. In this case, the experts widely use all the information available to them about analogue products, about the level of design development. The combined method combines the comparative method for studying qualitative properties and the predictive-analog (non-analogue) method for studying new properties.

Food tasting is the evaluation of organoleptic quality indicators by experts tested for sensory sensitivity. The purpose of the Tasting is to obtain reliable results with organoleptic. quality assessment, reducing its subjectivity due to the high professional competence of experts, which is achieved by their narrow specialization in product groups. The selection of experts for tasting groups is carried out by checking their sensory sensitivity, in addition, the expert must have experience in organoleptic evaluation of the selected product group.

Tasting of food products is carried out during the examination of new products, if necessary, to identify qualitative changes in organoleptic indicators, in determining the commercial grade and other quality gradations. The results of the tasting can be expressed as a description of individual indicators, as well as in points. In the latter case, standard scales (scoring scales) for wines, beer, hard rennet cheeses and butter) are used, or a special scale should be developed for scoring this type of product.

Examination of goods under contracts is an assessment by an expert of compliance with the quality requirements established by contracts of sale, commission and storage. The basis for such an examination is the disagreement between the parties to the contract. The legal basis for such an examination is the Federal Law "On the Protection of Consumer Rights" (Chapters 1 and 2, Articles 1-26), the rules for the sale of goods, the rules for commission trade in non-food products, approved by Decree of the Government of the Russian Federation of June 6, 1998 No. 569.

The need for an examination under storage agreements arises in the event of quantitative losses in excess of the established norms or a decrease in the quality of goods, as well as their complete loss. The decrease in the quality of goods during storage occurs due to microbiological, biological, physical, chemical, physico-chemical and biochemical processes. Often, the deterioration in the quality of goods can be determined by organoleptic indicators, while the expert independently decides on the need to study the goods in testing laboratories by measuring methods. Regardless of the reasons for the occurrence, quality losses are written off according to acts signed by experts, the head and financially responsible persons. There are no norms for writing off goods due to a decrease in quality.

Assortment expertise is an assessment by an expert of the quantitative and qualitative characteristics of a product in order to establish its assortment affiliation. This type of commodity examination as an independent one is used only in case of disagreement between the supplier and the buyer, the seller and the consumer regarding the assortment of goods belonging to a certain group, name, trademark, or when establishing the conformity of the assortment of goods in a batch with previously submitted samples, catalogs or sales contracts .

A documentary examination is an assessment by an expert of the commodity characteristics of goods, based on information from shipping, accompanying, technological and other documents. When conducting a documentary examination of the missing goods, the expert analyzes the available documents: invoices, certificates, quality certificates, write-off certificates, technical documents for monitoring the temperature and humidity conditions, reports of employees of trade organizations. Independently, this type of examination is carried out in the absence of goods due to its sale, theft, damage, death due to unforeseen circumstances. The complexity of documentary examination lies in the fact that in the absence of goods, the examiner has to rely on the submitted documents, which can be falsified. The results of the examination in this case depend on the competence of the expert, since as a result of the examination, the documents submitted by trading organizations can be refuted, and managers and financially responsible persons may be held criminally liable.

A comprehensive examination is an assessment by an expert of all the characteristics of a product based on their testing and analysis of documents. It can include not only commodity, but also cost characteristics. This type of examination is used in cases where it is necessary to comprehensively evaluate the goods, taking into account the positions of the seller, consumer, as well as the prevailing market conditions. A comprehensive examination is widely used in the practice of commission trade, in export-import operations of foreign trade activities, in the conclusion of sales contracts based on samples of large consignments of goods. A comprehensive examination includes other types of examination (quantitative, qualitative, assortment and documentary), and therefore all the means and methods used for their implementation are also applied to this examination.

Page 1 of 2

Quality is quality". Quality, unlike quantity, is not measured, but evaluated. There are two forms of quality assessment: linguistic and quantitative. In the first case, quality is defined in words: for example, "high quality", "medium", "low", etc. e. In the second case, the quality is evaluated on a certain conditional scale by the number k (0< к < м, где м - высший балл в числовой оценке качества). Например, 5 как в российской или 100 как в английской школах.

Experts are involved to assess the quality. We have already resorted to such a quality assessment when we tried to characterize the parameters of a fire bucket (see Fig. 3.31 and 3.32). Goods and services, projects and laws, knowledge and qualifications, performances of figure skaters, gymnasts or vocalists, and much more can be subjected to expert quality assessment. No matter how they scold qualitative rather than quantitative assessments in some sports, while recalling the scandals at the competitions, the expert assessments of the jury remain the only measure of the performance of many athletes. Not to mention people of art and various creative competitions...

In everyday life, when making decisions, we can proceed from a linguistic assessment of quality: we, for example, buy good tea or a high-quality lawn mower, etc. However, there are some tasks in which it is better to be guided by a quantitative assessment of quality. In addition, usually linguistic assessments are based on expert quantitative assessments. So, the quality of a graduate of an English university and the dignity of the diploma that he receives is determined by the following scale, presented in Table. 4.11.

Table 4.11. Diploma Quality Assessment Scale

Regardless of the object, the quality of which is subject to expert evaluation, the methodology for obtaining it is basically the same.

On fig. 4.29 shows the evaluation of the qualities of a certain manuscript of a Mathcad book, made on a five-point scale by different reviewers (an Excel table embedded in a Mathcad document), illustrated by two diagrams, looking at which one can draw only one conclusion that the qualities of the individual seven sides of the manuscript are more consistent each other than the qualities of the individual five reviewers.

But quality assessment can be done not only qualitatively (I apologize for the tautology), but also quantitatively, in numbers. To do this, consider the problem of assessing the business qualities of a leader (manager). The assessment will require a manager quality model, experts and mathematical processing of the results of the experts' work.

The quality model of a specialist can be represented as a certain set of the most important qualitative parameters (qualities) that characterize his professional level, personal, business qualities and predetermine successful work in the field of activity under consideration.

Let's say that this set includes the following qualities:

Level and quality of education, practical work experience;

Erudition, knowledge and skills in their own and related fields of activity;

Learnability and ability to teach;

Cultural level, moral standards;

Physical and psychological health, age;

Character, attitude towards others;

Ability to lead;

Ambitious, striving for leadership; - ability to take risks;

Dynamism, the ability to evaluate and accept changes in the internal and external environment.

Rice. 4.29. Graphical comparison of the qualities of the manuscript and reviewers

Previous -

At the main stage, designed to obtain the results of expert assessments, two subgroups of methods are used: 1) expert assessment of the characteristics of goods and 2) a survey of experts to identify their judgments and opinions.

Methods of expert assessment of the characteristics of goods. Since one of the grounds for conducting an examination is the impossibility or uneconomical application of objective assessment methods (change

negative, registration, calculated), the results of expert assessments, reflecting the judgments and opinions of experts, are usually expressed in the form of descriptions or conventional units (points, ranks, etc.). In this regard, the most common scope of methods for expert evaluation of goods are such subspecies of commodity expertise as identification expertise and quality expertise. Quantitative and documentary examinations can also be carried out by objective methods by using measuring instruments or recalculation, as well as by the method of comparative analysis of documents.

Identification expertise and quality expertise are carried out by determining quality indicators. The determination by experts of the quantitative characteristics of indicators without the use of objective methods is based on a mental study of these indicators using scales of order or names, and in some cases - scales of difference and ratios. At the same time, experts can evaluate single and/or complex quality indicators using differentiated and complex methods.

Methods for expert evaluation of product quality indicators- these are methods for determining the actual values ​​of single and complex quality indicators.

These methods are intended to determine the values ​​of quality indicators by calculation or heuristics in cases where the use of measurement methods is impossible or uneconomical due to excessive costs or long testing times. For example, when determining the taste and smell of food products, only organoleptic methods are used. Measuring methods do not give an accurate and reliable estimate, despite the increased costs.

For differential and complex evaluation of samples that differ significantly in quality, it is recommended to determine the value of a single indicator R differential method.

Differential method for assessing quality indicators- a method based on comparing the actual values ​​of individual indicators with the base values ​​of the same indicators:

§ 5. Expert methods

With the differential method, the quality level is actually determined if the value of the quality indicator regulated by the normative document or the base sample is chosen as the base value. In this case, the expert must give an answer: the quality level for the indicator under study is higher or lower or equal to the base sample.

The scope of the differential method goes far beyond the scope of expert evaluation alone. It is widely used both in scientific research and in assessing the competitiveness of goods by specialists. The actual values ​​of single indicators can be obtained by the measuring method, and then calculated and analyzed by specialists or experts.

However, the use of this method in peer review has certain features.

Evaluation of single quality indicators by experts begins with the determination of acceptable intervals for their measurement (p. max: /> f MHH) p. max - the best value of the indicator, the excess of which is impractical or impossible. The principles for assigning the maximum allowable value of the indicator (r. max) depend on the goals of quality assessment. At the same time, it is necessary that this principle be the same for all indicators.

Increasing the reliability of expert assessments is achieved by dividing complex operations into simple ones, which make up a multi-stage procedure for assessing the acceptable values ​​of the indicator. The transition to each subsequent stage is made after the development of agreed decisions at the previous stage.

The expert procedure for determining the acceptable values ​​of quality indicators consists of the following operations:

Issuance of an explanatory note to the experts, in which the
quality indicators are numerical and the principles of tolerance selection are described
quiet values ​​of indicators;

Completion of questionnaires by experts and indication of specific models of
inductions, the values ​​of which they consider to be the maximum permissible;

Familiarization of each expert with the assessments assigned to the other
by some experts and their discussion;

Conducting the second (sometimes third and fourth) round of the questionnaire
rovaniya;

» averaging the evaluation results.

In case of a significant spread of opinions, an additional round of voting is held. The value of the indicator is taken as the maximum if at least 70% of the votes are cast for it. If this condition is not met, the average of 50% of the largest p values ​​is taken as the maximum allowable value. max, for the minimum allowable value - the average of the 50% of the smallest values ​​of P g min. The obtained values ​​are used by experts in determining the estimates of quality indicators K.

To establish by experts the type of dependencies (/) between the values ​​of indicators P t and their estimates K, - the principal point method is often used. The need for its application is due to the fact that the division of the estimation procedure into several stages simplifies the work of the expert and allows him to give estimates to some characteristic points, based on which it is possible to build a model of the desired value.

Integrated method for assessing quality indicators- a method based on establishing the most significant single indicators for assessing the quality, determining their actual and weighted average values ​​of a complex indicator, taking into account the weighting factors.

Calculation of a complex quality indicator (Q) carried out according to the formula

where P\,Pr, ■■■iPn- actual values ​​of quality indicators;

K in b K „2, ..., K in „ - weight coefficients of these indicators.

The use of an integrated method requires a reasonable choice of the range of quality indicators of goods.

Chapter 4. Methods of Commodity Examination]

§ 5. Expert methods

These indicators are divided into evaluation, identification and restrictive.

Estimated indicators- indicators of consumer properties that have the ability to satisfy basic needs.

These indicators are used, if necessary, to assess the level of quality and competitiveness of goods, as well as to position them in a certain market segment and conduct a commodity examination of quality.

Identification indicators- indicators of consumer properties, intended for systematization of goods and their assignment to certain classification groups.

Such indicators are often used in the identification examination.

Restrictive indicators- indicators designed to limit the established requirements.

The choice of these indicators is determined by the objectives of the commodity examination, and the restrictions are set by the relevant regulatory methods. Quite often, the actual values ​​of the limiting quality indicators are obtained by measuring methods, and the task of the experts is only to compare the test results with the established requirements. In this case, the weight coefficients of the complex of the tested indicators are not taken into account.

Restrictive indicators are widely used in mandatory certification for safety indicators, as well as in sanitary-epidemiological, veterinary, phytosanitary and environmental examinations. In a number of cases, they can also be used in such varieties of commodity expertise as identification and quality. For example, when determining whether a school drink belongs to vodka, the following restrictive indicators are used: the minimum allowable content of ethyl alcohol and the maximum allowable content of fusel oils and megyl alcohol.

The choice of the nomenclature of quality indicators of goods and their expert assessment can be carried out by two methods: the express method i, the method of moving through levels without preparation and with preparation.

Express methods for comprehensive quality assessment of samples of goods-> s are based on the definition of a comprehensive quality indicator by

analysis of the values ​​of individual single indicators and appearance without their preliminary assessment and taking into account the weighting coefficients.

When using these methods, it must be taken into account that the limiting number of estimated indicators, even for a highly qualified expert, is 7-9 indicators located at the same level of the hierarchy and constituting a fairly homogeneous group. In addition, experts should take into account the importance of individual indicators with the help of weight coefficients, the relationship between them, and also consider the quality of the product as a system.

The method of moving through the levels without preparation is a set of operations carried out sequentially, with a gradual increase in the level. In this case, the analysis starts from the lower level of the indicator tree. Taking into account the value of the indicators of the lower level, the expert evaluates the indicators of the higher level. These operations are repeated with increasing level until the upper level is reached - a comprehensive (generalized) quality assessment.

Method of moving through levels with preparation is based on preliminary determination by experts of weight coefficients of quality indicators and their assessments. When assigning complex assessments, the expert knows the average values ​​of the weighting coefficients and estimates of individual indicators. The procedure for determining complex indicators is similar to the procedure for the method of moving through levels without preparation.

Principal point method There are several varieties depending on their number.

Three main method points - a method based on dividing the values ​​of indicators P into maximum, minimum and average values ​​and determining the values ​​of estimates K, - at these points. The scale interval between the maximum and minimum points is preset (0-1 or 0-10 scales). The expert's task also includes determining the dependence trend in the interval between the main points and plotting. After that, from the graphical dependence, you can go to the analytical formula for calculating the estimates of the quality indicator K,. This method allows us to develop only an approximate model for estimating K,.

Chapter 4. Methods of commodity examination

§ 5. Expert methods

Method of seven principal points - a method of evaluation on a seven-point scale for determining indicators, the values ​​of which are established experimentally or by calculation, as well as by the organoleptic method.

The seven-point scale is uniform, i.e. when moving from one quality class to another, the score changes by one point. Such scales are widely used, especially in organoleptic evaluation. To obtain more accurate results, you should proceed to determining the type of relationship between grades and points.

To facilitate the work of the expert, five graphs are given in the explanatory note to the questionnaire. The expert selects the curve (or combination of curves) that, in his opinion, best reflects the nature of the dependence. Then, each quality class is assigned a score in accordance with the nature of the dependence and the values ​​of the quality indicators. In this case, it is advisable to use numbers in the range of 0-10, multiples of 0.5, and the “highest quality” class receives a score of 10.

Thus, the graph constructed by the expert characterizes the relationship between the absolute values ​​of the P indicators and their estimates K, -, and for indicators assessed by organoleptic methods, between quality classes and their estimates.

In conclusion, the results are discussed, processed and analyzed. For indicators determined by measuring and calculation methods, it is desirable to give an analytical description of the curves, which makes it possible to calculate an estimate for any values ​​of the indicators.

The use of the main points method makes it possible to group and classify indicators by types of dependence.

Survey Methods

Survey Methods- methods based on conducting a questionnaire survey of a group of experts with subsequent analysis and processing of the information received from them.

The purpose of these methods is to determine individual and group expertise for making final decisions.

The reason for choosing these methods is the need to make complex decisions in a situation of uncertainty or

making a scientifically based forecast requiring the participation of a group of independent and competent specialists in a narrow field or many fields of knowledge (for example, knowledge of a homogeneous group of goods or all food or non-food products). The most important tools of survey methods are its procedure and questionnaires.

Development of a survey procedure is carried out after the formulation of goals and objectives, the formation of a group of analysts. To do this, questionnaires are compiled (open and closed) and survey methods are determined (individual and group, face-to-face and correspondence). There are certain methods of compiling questionnaires that should be taken into account when developing them.

When developing questionnaires, a group of analysts should provide for the level of detail of the problem, which is determined by the specifics and objectives of the examination. The reliability of the results of the examination depends on the level of detail, and with an increase in the degree of detail, the consistency of expert assessments, as a rule, increases. However, excessive detailing of the problem can also lead to a decrease in the reliability of information received from experts.

Detailing the problem is carried out by including tests (closed questions) with elements of ready-made answers or without answers (open questions) in the questionnaire. Elements of ready-made answers in the future make it possible to facilitate the generalization and formalization of the experts' answers, but also limit the information content of the assessment to one or more alternative answers, while there is a danger of imposing answers on the expert. When including open questions in the questionnaire, the expert can give an answer in any form and with any amount of information. In this case, it becomes possible, with the help of experts, to discover new aspects of the problem, sometimes unexpected for analysts. A specific question with a certain set of ready-made alternative answers is called a feature of the questionnaire. The set of answers can be qualitative and quantitative. With the qualitative nature of the set of answers, the task of the expert is to choose from the proposed elements of the answers, which, in his opinion, most fully reflect the essence of the problem on this issue. With the quantitative nature of the set, the answers to the questions are assigned numerical values.

Chapter 4. Methods of commodity examination

§ 5. Expert methods

When selecting features that it is desirable to include in the questionnaire, the possibility of a double interpretation of the questions should not be allowed. Therefore, it is useful to check the questionnaire by the analysts themselves. If the distribution of judgments on some attribute has a significant standard deviation, we can assume that this attribute is ambiguous.

All questions in the questionnaire can be divided into three groups depending on their content:

Objective personal data about the expert - age, education,
profession, work experience, scientific title, narrow specialization;

Characteristics that allow you to evaluate the motives that guide
an expert was used in assessing the problem under study;

The main questions concerning the essence of the problem under study.
Depending on the breadth of coverage, the methods of interviewing experts subdivided
are divided into individual and group, and on the presence or absence of
way of discussing the results of the assessment in the expert group, its nature
ra, rationale for assessments, personal contacts and interviewed subjects
there are several methods of expert survey (Fig. 17).

The choice of one or another method of conducting an expert survey is determined by the objectives of the examination, the costs of its conduct and the expected results, and the selection criteria are objectivity, awareness and labor intensity.

criterion of objectivity. Although the requirement of the objectivity of an expert is one of the main ones, however, in practice, experts can be influenced to a certain extent. It can be conformal or authoritarian in nature and plays a negative role, since the expert's opinion and assessment are formed under the influence of factors that are not related to the properties of goods or other objects of expertise.

In this regard, when choosing methods for conducting surveys of experts, preference should be given to those in which conformal or authoritarian influence is manifested to a lesser extent. Such methods include, for example, anonymous substantiation of estimates, lack of contacts between experts, closed voting. When making a collective peer review, there is usually no impartial and consistent analysis of problems. Evaluation is based on a compromise between the judgments of individual experts, so the method of collective expert evaluation is rarely used when interviewing experts.

Rice. 17. Classification of expert survey methods

Chapter 4. Methods of commodity examination

§ 5. Expert methods

Awareness criterion refers to the most important factors affecting the quality of an expert, as well as the reliability of his assessments. Experts are more aware of their mutual enrichment during the discussion of assessments. The logical arguments of other experts, while providing additional information previously unknown to other experts, can increase the level of awareness, due to which the expert can correct his assessment.

Based on this, the most successful in terms of awareness are the methods that contain information about the assessments of other experts. There are also personal contacts between experts, there is a debatable discussion of estimates and their personal justification. Therefore, when organizing the work of permanent expert groups, it is expedient to exchange information, since it contributes to the well-coordinated work of experts. When creating temporary expert groups, the exchange of information and collective discussion are recommended only in cases where an assessment and selection of experts for the objectivity of correcting their judgments (lack of conformism) has been carried out. In all cases, when taking into account the criterion of expert knowledge, one should not forget about creating conditions for their objectivity and excluding conformal or authoritarian influence.

Labor intensity criterion depends on the number of operations, their duration and nature. For example, when substantiating assessments in writing, the expert spends more time than when discussing orally. The effectiveness of an expert survey largely depends on the methods and techniques for identifying expert judgments.

Methods and techniques for interviewing experts. In an individual survey, a technical worker conducts a survey of each expert, and in a group survey, the entire group at once. The results of an individual survey, after summarizing and corresponding calculations, can be used to establish group estimates. However, a group assessment can also be obtained as a result of an individual survey of the leader and sub-leader of the expert group.

An individual survey of each expert in the group is associated with increased labor intensity, significant time costs and a spread of judgments, which makes it difficult to identify an objective and reliable expert assessment.

With an individual survey of the leaders and sub-leaders of the expert group, and especially individual experts, the time spent on the survey is less, but the likelihood of conformity among group members and the manifestation of authoritarian opinions of the leader and sub-leader is greater.

Depending on the techniques of questioning, an individual survey is divided into the following types: interviews, interview questionnaires, questionnaires and mixed questionnaires.

The advantages and disadvantages of these methods are shown in table. 5. The individual survey method is divided into varieties depending on the techniques used.

Table 5Advantages and disadvantages of different methods of interviewing experts

Methods and varieties	Advantages	disadvantages
survey data
Individual	Possibility to reveal	High time costs
way	all or most	me and labor criterion
varieties according to technical	unclear questions,	containers. At the primary
nic methods:	clarification is not enough	nom poll expert
interview	formulated;	can be fuzzy
interview questionnaires	use technical	formulated in
mixed survey	sky tricks	proxies that increase
nie		labor costs.
questioning		Possible reduction
		objectivity
Group way	Reducing the cost of time	Impossibility to use
	name of experts and workers	use of technical
	whose group, decline	sky tricks
	labor intensity criterion

Interview- a kind of questioning an expert to identify his assessment in the course of a free, but carried out according to a specific program, conversation with an expert.

Interview Questionnaire- a kind of questioning an expert by putting specific questions in front of him in a certain sequence, asked in the form of a questionnaire.

Chapter 4. Methods of commodity examination

§ 5. Expert methods

Questionnaire- a kind of survey of experts using questionnaire questions, to which the expert gives written answers. Most often, the survey is carried out by correspondence method without contact with the working group. An explanatory note is attached to the questionnaire to increase the expert's awareness.

Mixed survey- a kind of expert survey using questionnaires and additional briefing by a member of the working group. The purpose of the briefing is to clarify the details, to raise additional questions (if necessary).

In terms of informativeness, such methods as interviews, interview-questionnaires and mixed surveys have the greatest opportunities, and in terms of objectivity - questionnaires. Minimal objectivity is inherent in the interview. However, based on the different objectives of the examination, in practice any of these methods can be used. In this case, preference is given to one of these varieties in the following cases:

In the absence of restrictions on the time of the survey, it is advisable
different way to conduct an interview-questionnaire or mixed questioning;

With a significant number of experts and the ability to gather them all
together use a questionnaire;

If it is impossible to ask clearly formulated questions,
which will be understood by all experts unambiguously, as well as
if the expert finds it difficult to express answers to questions asked during
questions in the form of numerical assessments apply mixed questionnaires
ing.

When conducting a survey of experts, it is necessary to comply with general and specific requirements for the conditions of the survey. General requirements to the terms of the survey:

Favorable psychological environment of the survey, contribute to
ensuring the interest of the expert by establishing a benevolent
solid contact between the interviewee and the interviewer;

Exclusion of the possibility of conscious or unconscious
imposing on the expert the opinions of members of the working group or other
experts;

Ensuring the expert's confidence that the assessments made
will not be used for purposes unfavorable to him; here-

in some cases, a condition may be set on the confidentiality of information about the expert's assessments;

Creating and maintaining optimal survey workload by
performing by experts only creative, most complex
valuation transactions and transfer of auxiliary settlement,
graphic, other operations to the technical staff of the working
groups;

Understanding by the expert of the questions asked to him with the help of the necessary
walking briefing, and in some cases, preliminary
th, training survey;

The formulation of clearly formulated questions that do not have two
meaningful nature and not requiring large additional
nyh explanations;

Use of common terms whenever possible
standardized, and in their absence, an explanation of new
or ambiguously applied terms.
Specific requirements:

a) to interviewing techniques

The preferred use of a standardized interview,

Confidence at the beginning of the interview, conducive to frankness,

Fast and economical when coding expert responses
members of the working group;

Provoking experts to argue their assessments by
contesting his answer;

b) to the questioning technique

ensuring a clear and unambiguous understanding by experts of ha
operation procedure by application to each questionnaire
an explanatory note defining the goals and objectives of the examination,
methods and sequence of operations;

Uniformity in the preparation of questionnaires and explanatory notes for
making it easier to work with them;

The random nature of the enumeration of properties that the expert
should give numerical estimates, and the experts should be
warned about this, otherwise it may take place
unconscious overestimation of properties on the list
first.

Chapter 4. Methods of commodity examination

§ 5. Expert methods

The combination of operations for preparing and conducting surveys of experts, as well as technical operations, made it possible to create several expert methods that have received the greatest recognition and distribution. These include the Delphi, PATTERN and combined methods.

Delphi method(in some sources - Delphi) - a method of interviewing experts based on sequentially implemented procedures that are aimed at forming a group opinion on procedures with insufficient information.

The name of the method comes from the ancient Greek city of Delphi, where oracles lived, predicting the future.

The features of the Delphi method are the rejection of the joint work of experts; anonymity of estimates; feedback regulation; group response.

Refusal of joint work of experts and anonymity are achieved by the fact that each expert expresses his opinion in the questionnaire, without group discussion. Other techniques of individual survey are also used, for example, answers to questions are entered by experts into a computer. Regulated feedback is carried out by conducting several rounds of the survey, and the results of each round are processed by statistical methods and brought to the attention of experts. This makes it possible to reduce the spread of individual estimates and obtain a group answer that correctly reflects the opinion of each expert.

The number of rounds (up to four) depends on how quickly it is possible to achieve convergence of expert assessments in the group.

The anonymity of the survey makes it possible to reduce the conformal and authoritarian influence of individual dominant experts, and the regulated feedback reduces the influence of individual and group interests. The introduction of feedback also increases the criterion of objectivity and the reliability of estimates.

Conducting a survey in several rounds with informing the ex-dirts about the results of the previous stages of work, after which each expert must substantiate his opinion, makes it possible to reduce the fluctuations in individual and group responses. This provides indisputable advantages over the usual averaging of individual ratings.

When using the Delphi method, the following requirements are imposed on the polling procedures:

The questions posed should allow for the possibility of expressing
answer in the form of numbers;

Sufficient awareness of experts for nomination
ratings;

Availability of substantiations of the expert for each of the questions asked.
The advantage of the considered method is the increase

the criterion of objectivity due to the anonymity of the answers of experts, due to which the opinion of the expert is not distorted due to suggestion or adaptation to the opinions of other experts. In addition, the awareness of experts is also increasing due to familiarization with the assessments of other experts after the end of the next round, as well as substantiating their own assessments. The identification of prevailing opinions allows bringing together the points of view of different experts, forming a group, but not a single, opinion.

The shortcomings of the method, which manifest themselves at the initial stage of its application - the instability of the composition of expert groups, significant time gaps between rounds, unclear wording of questions, underestimation of the different competence of experts, the unreasonability of the chosen method of averaging expert assessments - were partially eliminated by developing modifications of this method (SIIR method , Delphi - Perth, etc.). As a result, the Delphi method has now become one of the most used, especially for forecasting purposes, as well as in the study of a number of economic and social problems.

When using this method for the purposes of expert assessment of the quality of consumer goods, the following disadvantages are manifested: the complexity of interviewing experts and filling out questionnaires, the complexity of the assessment due to the large number of quality indicators (sometimes up to 20-40) and filling out several questionnaires (3-10), cumbersome explanatory notes due to the lack of direct contact between the organizer and the experts.

Due to these shortcomings, the use of the Delphi method in the expert assessment of the quality of goods is difficult, and in some cases inappropriate. However, it is promising for obtaining groups

Chapter 4. Methods of commodity examination

§ 5. Expert methods

New expert assessment and in-depth analysis of events in situations of uncertainty.

PATTERN Method- a method of interviewing experts based on building a hierarchical structure - a tree of goals and making a decision on these goals after an open discussion.

The name of the method is made up of the first letters of the English words, meaning "Assistance to planning by quantifying technical data." The method was developed in 1962-1964. at the American firm Honnuel to evaluate projects for new weapons systems.

This method includes several stages.

/ stage- setting the main problem that needs to be solved, and dividing it into a number of secondary problems of the first, second, and so on order, which are then divided into narrower tasks. The division continues until simple elements are obtained that can be evaluated by experts.

As a result of this division, a hierarchical structure of main, secondary problems and tasks connected with each other is obtained, called the goal tree.

II stage- Determining with the help of experts the coefficients of weight (or significance) of each task in relation to the main goal, while the experts make decisions after an open discussion in the expert group.

Such an open discussion, along with dignity (the interaction of experts seeking to make a positive decision), also has a disadvantage due to conformism, i.e., distortion of the actual opinion of experts through suggestion or adaptation to the opinion of the majority.

/// stage- the use of computers for processing the received data and their analysis.

The advantage of the PATTERN method in comparison with the Delphi method is the simplification of the expert survey procedure. However, the method also has a number of disadvantages: the lack of justification for the optimal number of members of the expert group, as well as the methodology for selecting competent specialists for the expert group; processing the results of the survey without taking into account differences in the quality of individual experts; lack of bar-

ers for the manifestation of conformity of experts; insufficient development and uncertainty of the principles of building a tree of goals.

Since the Delphi and PATTERN methods have significant shortcomings and do not fully meet the goals of peer review, E.L. Reichman and G.G. Azgaldov proposed a combined method in which the positive features of other expert methods were used and their shortcomings were excluded.

Combined method- a method based on a combination of individual and collective expert assessments.

V1: Quality as a factor in the success of an enterprise in a market economy

V1: ISO 9000 International Standards Guidelines for Quality Assurance

An expert is a specialist (professional) whose assessments and judgments the decision maker considers useful to take into account when making decisions.

Let's give some explanations. Naturally, the opinions of a novice therapist and a world-famous cardiologist are not comparable in terms of the level of assessment of the patient's condition and recommendations for his treatment in a rather difficult situation. However, if the patient needs emergency help, and a highly professional specialist for one reason or another cannot be invited, then the decision will be made by the one who is nearby, even if he is less qualified. In this situation, he is the expert.

Thus, the formation of the composition of the expert commission depends on:

a specific decision-making situation;

· opportunities for organizers of expertise to attract highly qualified specialists for work;

· Opportunities for specialists to take part in the work of the expert commission.

It should also be noted that at present there is no generally accepted unified methodology for assessing the qualities of an expert, so opinions about the professional level of a specialist often differ significantly. In real practice, the decision maker seeks either to form an opinion about the professional level of a candidate for expert himself, or he trusts this to those who are entrusted with organizing the examination. Therefore, in the absence of opportunities and experience in organizing and conducting examinations, it makes sense to turn to the services of independent centers of expertise, information and analytical centers, etc., whose main tasks are to analyze the situation, evaluate objects of expertise, prepare and evaluate alternative solutions.

When evaluating the qualities of an expert, it is necessary to take into account his professional knowledge, experience and work efficiency as part of expert commissions. There are many ways to assess the qualities of an expert, each of which can be successfully used in a particular case. They are divided into three main groups:

a priori;

a posteriori;

test.

A priori are called methods for assessing the qualities of an expert, in which information about the results of his participation in previous examinations is not used.

A posteriori are called methods for evaluating the qualities of an expert based on information about the results of his participation in previous examinations.

Test methods are methods for assessing the qualities of an expert, involving a special test.

A priori methods for evaluating an expert. This group includes, first of all, the most common methods of self-assessment, the essence of which is that the expert himself evaluates his qualities. In this case, the following methods can be used:

self-assessment on one of the point scales (3, 5, 10 or 100 points);

self-assessment using verbal-numerical scales, which, along with the numerical values ​​of the gradations, contain their qualitative description;

self-assessment on a verbal scale, in which the expert gives a verbal assessment of his knowledge and experience, using for this purpose qualitative gradations of a pre-developed scale;

self-assessment using a differential method, in which an expert evaluates his qualities using two main private criteria: a criterion that characterizes his familiarity with the main sources of information in this area (for example, special domestic and foreign periodicals, patent information, in-house information, etc.); etc.) and a criterion that characterizes familiarity with the object of expertise (for example, knowledge of industry specifics, a particular enterprise, familiarity with product samples, etc.).

In this case, the complex self-assessment of the expert is calculated by the formula:

To to = ( To and +ß To h) / 2,

where To k - complex self-assessment of the expert, To and - awareness coefficient (in fractions of a unit), To h - coefficient of familiarity with the problem, ß - weight coefficient.

More details on the measurement of indicators and types of scales will be discussed in the next topic.

The accuracy of self-assessment methods is somewhat controversial. On the one hand, no one better than the expert himself represents the totality of knowledge and experience that he possesses. On the other hand, a person is most mistaken when evaluating his own capabilities.

Another group of a priori methods for evaluating the qualities of an expert includes methods of mutual evaluation, the use of which assumes that experts evaluate each other. The most common of these is the expert list method. The procedure for its application is as follows. Each of the experts draws up a list of specialists whom he considers competent in this area. Then, on the basis of these lists, the expert's competence coefficient is calculated - the ratio of the number of lists in which this specialist is present to the total number of compiled lists.

Among the a priori methods for assessing the qualities of an expert is also the most common documentary or questionnaire method. In order to assess certain qualities of a specialist, it is proposed to use objective characteristics that have documentary evidence, for example, such as: work experience, academic degree, academic title, position held, number of publications, citation index, etc. At the same time, it should be borne in mind that based only on these objective factors it is impossible to adequately assess the suitability and usefulness of an expert's participation in the work of a particular expert group.

A posteriori evaluation methods. These methods involve the use of the results of his participation in previous surveys when assessing the qualities of an expert. With their help, to a certain extent, such qualities as conformity, opportunism, expert competence can be identified. In particular, the method of paired comparisons can be used to assess the level of competence of a specialist. Its essence lies in the fact that the expert is sequentially presented with pairs of compared objects of evaluation and he chooses from each pair the most preferable object from his point of view. After presenting all the pairs, the analysts have information about the comparative preferences of the expert in relation to the objects being evaluated. In this case, a situation may arise when, in a direct comparison, the first object is preferable to the second, the second is preferable to the third, but, at the same time, the third object is preferable to the first, i.e. there is an obvious contradiction. In real practice, such contradictions (of course, in longer chains of comparison) do not happen so rarely. Naturally, the fewer contradictions in the expert's assessments, the more justified his competence.

The a posteriori method also involves assessing the reliability of the expert's judgments. As a criterion for assessing such reliability, the reliability coefficient is used - the relative frequency of cases in which the expert attributed the highest probability to subsequently confirmed events. When using this method, the average value of the reliability coefficient of the expert commission is also calculated and the individual coefficients of experts are compared with it.

Another method of a posteriori estimation is the method of deviation from the resulting group estimate. This method is based on the calculation of the coefficient of deviation - Ko.

Tooi = Doi / D max ,

where To oi- coefficient of deviation of judgments of the i-th expert, Doi- deviation of the individual assessment of the i-th expert from the resulting assessment, D max - the maximum possible deviation of the expert's assessment from the resulting assessment.

Test methods for assessing the qualities of an expert. Their essence is the performance by the subject expert of some pre-prepared task. These methods are well known, because are widely used to determine the professional suitability of specialists in various fields of activity. The advantage of these methods is that they allow not only to establish whether an expert has a certain professional level, but also to identify the skills and experience necessary for productive participation in the work of an expert commission. However, it must be remembered that conducting test experiments requires compliance with the following important rules:

The test should be designed specifically for specific objects of peer review;

· the true values ​​of the estimated parameters (correct answers) should be known to the analytical group conducting the test, but unknown to the expert being tested;

· A scale should be developed to determine the accuracy of assessments given by an expert;

The probability of randomly guessing the true estimate should be very small.

The use of test methods makes it possible to evaluate such an important professional quality of an expert as the reproducibility of expert assessments. To do this, several test experiments are carried out that are close to the conditions of a real examination. At the same time, the time interval between them should be sufficient for the subject to forget the results of the previous one. Then the scores obtained are compared with each other. More stable (reproducible) assessments of an expert indicate both his greater professional competence and his greater suitability for participation in the expert commission.