A Master of Arts Thesis in Teaching English to Speakers of Other Languages (TESOL) by Mary Anne John Entitled, "Evaluating an Additional Measure for the American University of Sharjah English Placement Test: Using Analytical Scoring," June 2009. Available are both Hard and Soft Copies of the Thesis.
The English Placement Test (EPT) at the American University of Sharjah (AUS) has been accurately placing the majority of newly admitted students into appropriate course levels, but there is a small number of students who seem to be misplaced by the English Placement Test. The issue of misplacements is a common problem with placement tests where one measure does not accurately determine placement for all test takers. It is because of this issue of misplacements that we need additional evidence to determine accurate placement levels. In this research, the first step was to investigate the use of Jacobs, Zingraf, Wormuth, Hartfield, and Hughey's (1981) analytical scale with the AUS EPT. 65 EPTs of students representatively selected in Fall 2007 were analytically double scored by two raters, and Cronbach's alpha was calculated to determine inter-rater reliability. Then, the students' EPT writing samples were evaluated by Criterion SM, a computer-based scoring program developed by the Educational Testing Service (ETS). Criterion SM and the holistic EPT were used as established measures to ascertain criterion-related validity of the analytically scored EPT in the context of this research. After establishing normal distribution and linear relationship, the analytical EPT scores were then correlated with the CriterionSM scores and the holistic EPT scores, using the Pearson correlation coefficient in order to ascertain the appropriateness of use of the analytical scale in this context. These analytical EPT scores were found to be in strong correlation with the holistic EPT scores and the CriterionSM scores. Therefore in the next step of the research it was determined which of the students who had taken the EPT in Spring 2008 had been identified by their teachers as being "borderline/potentially misplaced" through classroom observation and their in-class writing samples. Once these students were identified, their EPT scripts were double scored by two raters, using the analytical scale developed by Jacobs et al. (1981), and were also evaluated by CriterionSM. The placements of the identified students were reevaluated using the analytical scale and CriterionSM. Results indicated that analytical scoring with the Jacobs et al. (1981) scale and CriterionSM provide valuable information complementing the holistic EPT, with a more complete profile for re-evaluating placement. The analytical scale was found to be a useful additional measure in cases of potential misplacement identified by teacher observation.