Objective correlates of subjective features of the speech event with pre-and post-speech evaluations across professional, nonprofessional, and experienced professional judge groups
Kitonga, Ellen Crowley
MetadataShow full item record
Purpose: 1) To determine the value of professional training and experience on the pre- and post-speech evaluation of the four subjective features: understandability, fluency, complexity, and accuracy. 2) To determine which of twenty-nine objectives indices were the best predictors of subjective evaluation of speech events. Procedure: Three-minute speech samples were obtained in an interview situation from thirty dialectally homogeneous ninth-grade students divided into three equal groups by stanine scores on the Stanford Language Achievement Test. Fifteen professional judges--graduate students and faculty in English, speech, and Linguistics; fifteen nonprofessional judges--college freshmen; and three experienced judges--the researcher and two transcriber-analysts all from the University of Wisconsin, were selected. These judges, after defining the four subjective features and rating them on a seven-point scale, listened to the speech samples, rating on a seven-point scale the four subjective features plus overall impression for each speaker. Comments on the experiment itself were then added. The speech samples were transcribed and analyzed for the occurrence of twenty-nine objective indices. Analysis of variance determined significant differences in (1) the pre-speech ratings of subjective variables and (2) judge group evaluations of each of the subjective features across all subjects. Stepwise regression analyses determined (1) which of the four subjective variables best predicted overall impression and (2) which of the twenty-nine objective indices were the best predictors of understandability, complexity, accuracy, and overall impression. Results: The only significant difference in pre-speech ratings was that nonprofessional ratings of understandability were significantly lower than professional ratings; each judge group ranked the features in descending order of importance: understandability, accuracy, fluency, complexity. Definitions by the professional groups were more articulate, but did not substantially reflect recent theory or research. Stepwise regression analysis showed that each of the discrete subjective indices had a high positive correlation with the others and with overall impression; the only exception was the low correlation of accuracy with the other discrete subjective features and with overall impression for the experienced professional judges. Experienced professionals offered significantly lower evaluations of accuracy, whereas the nonprofessionals offered significantly higher evaluations of understandability and overall impression. All judge groups agreed on their evaluations of fluency and complexity; thus there were eight judge groups across features. Indices providing significant negative (favorable) correlations with overall impression and ratings of two or more discrete subjective features or judge groups were grammatical error, deletion, the communication ratio, non-simple sentences, sentence transformations, prenominal quantifier, and general adverb. Indices with a significant positive (unfavorable) correlations were vagueness, slang, incomplete maze, phonological error, and codings of repeat and filled pause weighted to account for grammatical structure. Conclusion: Professional experience and training were of some value in formulating definitions of subjective features, but these definitions were of little worth in providing objective measures of subjective evaluations. The ranking of features before speech evaluation was the same for each group. Except for the separate evaluation of accuracy by the experienced professionals, the judges offered a global reaction to the samples, rendering meaningless a discussion of the objective correlates of each feature. The objective correlates of overall impression and global reaction provided a range of traditional, structural, and transformationally oriented indices, measures which had originally been hypothesized as predictors of understandability, fluency, complexity, and accuracy. Future research may seek to determine (1) whether global response is the usual procedure in speech evaluation and (2) if the objective correlates of speech evaluation vary with differing subject and judge groups.