Shmelev, Alexander G.

The purpose(objective) of the empirical study is the measurement of the accuracy of expert-proctors in detecting cheating in online testing. 

Sample of the study. 35 test takers passed an online test of general knowledge on the basis of 30 multiple choice questions. Half of the subjects (18 persons) were “artificial cheaters” – they used cheat sheets with correct answers. 

Methods. The video recording of the testing process included a “screen capture” so that expert-proctors could observe all cursor movements, see a recording of the subject's facial expressions and a visual focus of attention in a separate window (recording from the front camera), and could listen to the subject pronouncing the task conditions and answers (“oral decision”). 14 experts took part in rating of video recordings, of which 8 experts showed satisfactory results in terms of the level of accuracy in detecting cheating (their accuracy that was measured using the Kappa coefficient was higher than 0.5). 

Conclusions. A high asymmetric validity of expert assessments is revealed. More accurate experts allowed a negligible (about 5 percent) number of errors of the “false alarm” type, but a relatively large number of errors of the “skip” type. Recommendations are made for the practical use of the expert assessment method in combination with automatic chronometric analysis of the degree of atypical protocols and subsequent control of face-to-face offline testing of all suspected subjects (examinees). 

Relevance. The article discusses the possibility of a new technique for testing the professional knowledge. This is a technique of "matrix testing" (or semantic scaling). In the introduction, the author introduces a distinctive system of constructs that allows to localize a new technique in the context of a number of traditional (testing method) and non-traditional methods of assessing educational achievements and professional knowledge.

Objective. The author has set the purpose to reveal potential of "semantic scaling" for a quantitative assessment (measurement) of student competence.

Method. The method of semantic scaling was applied by the author to scale sports with the help of special scales-criteria that denote different types of interaction between athletes and motivational regulation of their activities and thus turn the technique into "psychosemantic scaling". The study was conducted on 150 senior students of the faculty of psychology of MSU, who attended a special course of the author on "Psychology of competition".

Results. The main result (the insignificant correlation coefficient between the success of this technique and the total efficiency of students during the course) does not allow to conclude that the main goal is achieved. More interesting were the additional results from the application of this technique, namely: subset of sport species, obtained by cluster analysis, allow us to talk about the "psychological classification" of these species, or at least on the "psychosemantic classification". This classification is much closer to the classical "Olympic classification", but has a number of specific features.

Conclusion. The positive result of the study can be considered a high similarity of the obtained matrices with the reference matrix (ideal model), from which it can be concluded that the technique of "psychosemantic scaling" contributes to the conceptual learning of students-psychologists – their independent work on mastering the system of concepts in this subject area.

This short article discusses innovations in training and personnel assessment associated with the implementation of gamification of computer tools for learning and assessing (psycho-diagnostics). In the theoretical part of the article the authors give an operational definition of the term “gamification”, and also discussed factors that impede successful gamification — reduces the validity of computer psycho-diagnostic methods based on the use of game elements. In an experimental-methodological part of the article describes a new brief, threeminute computer technique for the estimation of a functional state called “The spoons”. This technique is a game modification of the classic “proof-reading test”. The authors publish data on a sufficiently high reliability and validity of the new method, confirming the possibility of its use as a tool for monitoring (operational security) functional status.

In this article the authors comment and discuss the results of the expert 16 definitions of the term “competence”. The participants were 45 psychologists of higher qualification (candidates and doctors of Sciences) — members of the Internet community of psychologists on the website www.ht.ru. They judged all definitions by means of 7 different rating scales. The resulting data cube 45×16×7 was analyzed by factor and cluster analysis. It was revealed 2 significant factor reflecting the level of popularity of definitions (the first factor), as well as the proximity-distance from the pole of terminological Russian national tradition (the second factor), mainly related to the concepts of “PVK” (PIQ in English “professionally important qualities”) and “ZUN” (KAZ in English — “Knowledge Ability-Skills”). Rating of the definitions by academic specialists was appeared to be close to the national tradition and evaluation of psychologists-practitioners closer to the Western tradition. Using cluster analysis discovered 4 robust clusters: 1) the competence as an area of authority and responsibility, 2) competence is the same as “PVK”, 3) the competence as mobilization the knowledge and skills to solve a specific task, 4) competences are descriptive units (elements) of effective professional activity. The most promising and creative approach, according to the authors, is the third cluster, in which an attempt is made to search a synthesis of the domestic (based on activity theory) and Western (behavioral approach) conceptual-terminological traditions.

The article presents range of problems, connected with computerized procedures of attestation control, such as their elaboration, application, advantages and disadvantages. The experience of working with so called “contextual insertion method” as a technological solution, that allows overcoming the limitations of classical tests and provides complex approach to diagnostics of students’ knowledge, is described. The results of substantive and psychometric testing of this method on example of the students of department of psychology of Lomonosov Moscow State University are discussed. Participants performed tests within the bounds of “The foundations of psychodiagnostics” course. The result of literacy test is analyzed separately. The prospects of method in carrying out current and final control are proved, the limits of its use and possible organizational difficulties are delineated. 

The article describes the results of the several years introduction of computerized testing in the outcome examinations for the final year students of psychological faculty of the Lomonosov Moscow State University. The procedure of testing, the features and the structure of the test item bank, technological and organizational issues are discussed including the procedural rules for students and professors. The main focus of the article is statistical analyses of the obtained correlations between test scores, traditional oral exam scores and academic history scores for all years in the university. As a result of the systematic improvements of test item bank the correlation between test scores and academic history scores increased each year and is equal last year 0.72 (the Pearson’s linear correlation coefficient) and 0.86 (Gulford’s coefficient for extreme groups). At the same time the analogical correlation for oral exams remained at the same level (0.55—0.59 the Pearson’s linear correlation coefficient). 

The article explains the continuity of domestic proforientational methods, DDQ and «Proforientator». The authors described the scientific and organiziational contributions, made by Professor E.A. Klimov in the development of modern proforientational diagnostic methods. The reader of this article can find the illustration of his contributions to the history of emergence of a computerized test «Proforientator» and the logic of its revising. The authors focus on the block «interests» and secondary scales of modern test in the areas of targeted learning. Analysis of massive empirical material using expert assessment and mathematical statistics, allowed to reasonably prove validity of schoolchildren’ test results with the help of the «Proforientator» test real career, with a precision of up to six areas of targeted learning. Also it was given attention to deal with individual bright cases involving real failed choices of higher education and specialty. The materials described the theoretical basis of domestic proforientational approach, which have been tested in practice.