The overwhelming consensus of the majority of literature agrees that the individual characteristics of the examiners in an oral proficiency interview do have a measurable effect on the outcomes of that interview. It seems that the effect of cultural differences is the prevailing concern amongst the literature, as modern academics have found that gender as a defining characteristic does not have any significant effect. This is contrary to previous findings which have concluded that according to one’s gender, there is a conversational style that is adopted and accordingly, participants are likely to respond in a typical or predictable manner. The concern of the literature with regards to the impact of cross-cultural differences can be used to explain the variety of opinion with regards to gender, as well as the impact of other factors of social identity on participants in the interviews. It stands to reason that different cultural contexts will view factors of social identity is different ways, and as a result thereof different candidates will respond to factors of gender, age, ethnicity and religion with different levels of severity with regards to impact. Arguably, the impact of certain characteristics on the candidate’s performance may be categorized into factors of social identity, where there is a potential for variation in performance based on the individual characteristics of the interviewer, as well as the candidates. The second factor relates to the tendency of interviewers to accommodate the candidate and this may be classified as behavioral. Based on the purpose of the Oral Proficiency Interview (OPI), as seeking to objectively assess proficiency, it stands to reason that both behavioral and individual characteristics should be taken into consideration in the design of the interview and assessment of the candidate’s skills. For the sake of pragmatism however, objective assessment and tolerance of behavioral characteristics should be addressed in order to ensure consistency of testing of all candidates and it would be impractical to assess and accommodate the impacts of individual characteristics in OPIs.
BibliographyCross-Cultural Pragmatics
Berwick, Richard & Ross, Steven (1996): Cross-cultural pragmatics in oral proficiency interview strategies. In: Milanovic, Michael & Saville, Nick (Eds.): Performance Testing, Cognition and Assessment: Selected papers from the 15th Language Testing Research Colloquium. Cambridge, UK: Cambridge University Press, 34-54.
The article introduces the problem of reliability and validity in OPIs with regard to the conversational nature of the interview itself allowing for the introduction of accommodating techniques by interviewers presenting a dysfunctional characteristic of the interviewer, as this behavior or language demonstration being accommodated is generally that which displays the weaknesses in the interviewee’s language skills. The tendency of interviewers to accommodate less proficient language speakers is detrimental to the overall purpose of the test. There is a similar cross-cultural impact on those who provide interview ratings and there is a lack of awareness in the rating of OPIs with regards to these cross-cultural differences.
The study aimed to quantify this difference in the context of the interview itself and did so by using twelve language participants (six English as a second language (ESL), six Japanese learners) and one American interviewer. A number of differences in the approach of the interviewer was observed and illustrated, specifically relating to the way in which they accommodated the interviewee and directed the flow of conversation. It was observed that Japanese learners were overall much less fluent than their ESL counterparts and that they simultaneously displayed less morphosyntactic control than those learners. The research raises the question about whether there should be the development of a universal prototype for conducting the oral interview or whether there should be a conscious accounting for cultural differences. The alternative therefore is the tolerance of cultural differences to extent the assessment to pragmatic competence in the second language.
With reference to the above question, this article is highly relevant as it positions the paradigm of cross-cultural differences as it recognizes the unconscious effect of cultural differences on the ability of the interviewer to perform objectively in the test situation. Arguably, this is an innate limitation of the interview and to the extent that cross-cultural differences have an effect on the outcomes of learners in the OPI, the test should tolerate these differences in order to ensure accuracy and reliability of the results across the board.
Lazaraton, Anne (1996): Interlocutor support in oral proficiency interviews: the case of CASE. In: Language Testing 13, 151-172.
This article presents a qualitative analysis of the types of linguistic and interactional support that a native speaking interviewer provides to a non-native speaking candidate in personal interviews. This is one aspect of analysis with regards to the interviewer-candidate interaction and this refers to the unconscious gesture made by interviewers to accommodate their interviewee, with the natural behavior of the interviewer being directed more strongly at candidates with a less proficient skill set.
The results from the test indicated that, in the sample group of fifty-eight transcribed Cambridge Assessment of Spoken English (CASE) interviews, there were eight subtypes of interlocutor support prevalent which were observed, including the use of comprehension checks and clarification requests by the interviewer, grammatical, syntactical or lexical simplification of an utterance to facilitate comprehension and introducing a topic in order to set the scene for the candidate. It was suggested that these subtypes are positive findings for the purposes of the presence of documented conversational practices. It is unclear on the research, whether the presence of these subtypes had any effect on the outcome ratings of proficiency in these cases, however it is clear that there is an effect on these ratings.
As was the case with the Berwick et al. (1996) article, it is clear that there is a tendency of the interviewer to accommodate their own cultural paradigms into the interview setting and that this may have a measurable impact on the candidate themselves. The impact therefore should be tolerated or accommodated in some way in order to ensure the accuracy of the results as a measure of language proficiency, this justifying the inclusion of this article.
Examiner Conduct
Brown, Annie. (2003): Interviewer variation and the co-construction of speaking proficiency. Language Testing, 20(1), 1-25.
The central assertion of this article is that the achievement of consistent ratings in the interview is dependent on the consistency of the examiners conduct during the process of interviewing. The article presents findings in previous research that supports the hypothesis that interviewers generally have distinct and individual styles which they use across a number of interviews and this style is apparent in different forms of interviews, even those with constrained speech. The instrument of the study was an interview by two different interviewers, rated objectively as the most difficult and least difficult interviewer according to quantitative ratings.
Factors associated with different interviewing style include: the level of rapport that they are able to establish with the interviewee, their functional and chosen topics, the ways in which they ask questions and construct prompts, the ways in which they develop and extend the chosen topic, and the ways in which, and the extent to which they accommodate their speech into that of the interviewer. These factors were used to determine the difficulty rating of the interviewer and were applied to the results of the interview.
The outcomes of the interviewer were typical in that the interviewee performed predictably better with the easier rated interviewer and worse with the more difficult rated interviewer. It was found that the interview style of the easier rated interviewer contributed to the quality of the outcome of the interviewee. The conclusions of the research found that standard approaches to training and accreditation is inadequate, as the formal methods of training neglect to consider different interviewing styles that develop post-qualification. This is particularly relevant in second language testing as there is a heavy reliance of these interviews on an unstructured naturalistic interaction, and therefore the style of the interviewer becomes arguably more important. These must be monitored in order to ensure that they do not present different levels of challenge in and of themselves.
It is clear therefore based on this research, that there are a number of factors specific to the interviewer that may have an effect on the performance of the interviewee in the test. The individual characteristics of the examiner therefore may have a significant impact on the results of the test in so far as the individual examiners style may present a challenge independent of the test being undertaken.
Impact of Gender in Oral Proficiency Testing
O’Loughlin, Kieran (2002): The impact of gender in oral proficiency testing. In: Language Testing 19/2, 169-192.
The article presents an introduction to the available literature on gender characteristics having an overall effect on the ability of examiners or interviewers on interviewees. Ultimately, the article looks to understand the differences between interviewers and suggests that the cause of the variability is at least partly due to gender differences. It is suggested that male and female conversational styles are uniquely distinct and as such, men and women constitute different speech communities. Based on this assertion, female conversational style is characterized as collaborative, co-operative, symmetrical and supportive, whereas its male conversational style is characteristically controlling, uncooperative, asymmetrical and unsupportive.
These findings however neglect certain social identity factors in their generalizations as to both the communicative characteristics of the interviewer and those of the interviewee, such as age, ethnicity, occupation and sexual identity. These factors indicate that language use of men and women is flexible and can vary distinctly across cultural, social and situational context, overriding potential gender differences. There is a further suggestion that the behavior of the interviewer may change according to the gender of the interviewee and gendered behavior of the interviewer may strengthen or undermine the performance of the interviewee.
The participants in the study were tested twice in an International English Language Testing System IELTS practice interview, one with a male interviewer and once with a female interviewer. The interviews were transcribed and analyzed in relation to previously identified features of gendered language use, namely overlaps, interruptions and minimal responses. The analyzed scores from a discourse analysis and a test-score analysis indicated that there was no significant effect of gender on the IELTS interview. It was found that there was no significant gender pattern with the use of gendered language features and collaborative language use was experienced by both genders of participants.
It is concluded that gender competes with other elements of social identity in a fluid and dynamic manner, and because of the extensive variety of these factors, the effect of gender on the interviewees performance may be specific to a situation and therefore difficult to predict. This article emphasizes the impact of factors of social identity on the test taker’s performance, as well as the particular impact of gender. Arguably, it may be evidenced that test-takers with particular sensitivity to gender stereotyping may react differently, however the weight of gender as a factor is as a characteristic of social identity.
This article is a relevant consideration as it positions the factor of gender within a paradigm of social identity factors and does not focus specifically on the effect of gender in interviews. In doing so, it becomes clear that the impact of individual characteristics on the interview participants is one which is not easy to predict as it relies heavily on the individual characteristics of the examiner and participant. Therefore mitigating the impact of these characteristics on the participant may not be practical or pragmatic as an approach to improving reliability and validity of these tests.
Meaning Negotiation
Katona, Lucia (1998): Meaning negotiation in Hungarian oral proficiency examination of English. In: Young, Richard & He, Agnes Weiyun (Eds.): Talking and testing. Discourse approaches to the assessment of oral proficiency, Amsterdam: John Benjamins Publishing Company, 239-270.
This article is concerned with the effect of the interlocutor on the test experience or simply stated the effect of the interviewer on the outcome of the test experience. This is based on the premise that any variation in the way in which a task is presented to the test-taker may have a significant or noticeable impact of the performance of the test-taker in that interaction. This echoes the finding of many empirical studies researching the effect of individual characteristics of the examiner on the test participant.
The article proposes meaning negotiation and considers the effect of familiarity on the candidates. The study proposes that an interviewer that is familiar to the candidate may have an effect on the way in which meaning is negotiated between the participants. It does so by studying the effects of meaning negotiation between Hungarian interviewers and interviewees during the English OPIs. The study revealed that candidates who were familiar with the interlocutor interacted more natural in the negotiation sequences and exchanges presented to them in the interview. Conversely, attempts to engage in negotiation discourse with an unfamiliar interlocutor resulted in a more formal and artificial interaction. The article concludes that the frequency and the type of negotiation differ according to whether the interlocutor was known to the participant, and where the interlocutor is unknown, it is more likely to result in misunderstandings between the participants and overall, the discourse is more artificial and formal in nature.
This article is highly relevant for the purposes of the topic at hand, as it presents a potential method for mitigating the impact of certain forms of individual or behavioral characteristics on the interview participant. If one considers that, particularly with regards to individual characteristics, there is no practical manner of eliminating the effect of these characteristics on the participants, meaning negotiation may allow for the mitigation of these characteristics in the interview, allowing for a natural interaction which may allow for the demonstration of the requisite skills.
SummaryBehavioral characteristics of examiners have a significant impact on the outcomes of the OPIs, particularly in their own tendencies to unconsciously accommodate the shortcomings in the proficiency of the participants (Brown, 2003; Lazaraton, 1996). Through training and evaluation of the interviewers after the completion of their training, it may be possible to tolerate these differences and possibly mitigate their effect on the participants (Brown, 2003). This however is problematic due to the conversational nature of the interviews themselves as there is a significant freedom within the interviews to direct questions and subject matter (Berwick, et al., 1996).
Individual characteristics of the interviewer may also have a significant effect on the outcome of the OPIs: however these are not strictly limited to gender. Indeed, previous research indicates that gender does have an impact, however it is noted that this impact is not specific to gender, but rather to features of social identity experienced by both the interviewer and the participant. It stands to reason that mitigating the impact of social identity factors on the participants is a pragmatic difficulty as there is no practical way of accounting for this impact (O’Loughlin, 2002). It was shown that gender as a single factor does not have any significant difference, although in a study which surveys a more diverse social group, this outcome may be markedly different.
Meaning negotiation was shown to have a significant impact on the level of comfort and formality experienced by the participant and it was found that the participants with an interlocutor who was familiar to them had a more natural interaction, whilst those with an unfamiliar interlocutor did not (Katonta, 1998). Arguably, therefore an interlocutor who is familiar may provide necessary relief for the behavioral and individual characteristic impacts noted above. This however is based on the premise that a more natural interaction leads to a better result, which was not commented on in the study.
BiblographyBrown, Annie. (2003): Interviewer variation and the co-construction of speaking proficiency. Language Testing, 20(1), 1-25.
Berwick, Richard & Ross, Steven (1996): Cross-cultural pragmatics in oral proficiency interview strategies. In: Milanovic, Michael & Saville, Nick (Eds.): Performance Testing, Cognition and Assessment: Selected papers from the 15th Language Testing Research Colloquium. Cambridge, UK: Cambridge University Press, 34-54.
Katona, Lucia (1998): Meaning negotiation in Hungarian oral proficiency examination of English. In: Young, Richard & He, Agnes Weiyun (Eds.): Talking and testing. Discourse approaches to the assessment of oral proficiency, Amsterdam: John Benjamins Publishing Company, 239-270.
Lazaraton, Anne (1996): Interlocutor support in oral proficiency interviews: the case of CASE. In: Language Testing 13, 151-172.
O’Loughlin, Kieran (2002): The impact of gender in oral proficiency testing. In: Language Testing 19/2, 169-192.
