Research project
Investigating dynamicity of fluency for valid oral language assessment
To what extent do aspects of fluency fluctuate throughout time? And how do raters take the dynamicity of fluency into account when rating performances?
- Duration
- 2024 - 2026
- Contact
- Nivja de Jong
- Funding
- ETS (TOEFL Committee of Examiners Research Grant)
Researchers
- Prof.dr. N.H. de Jong
- Prof.dr. J.J. Goeman
When human raters evaluate speaking performances, they assign a single score. However, speaking is a dynamic process with performance fluctuating through time. The current project evaluates how human raters take fluctuations while rating into account. The focus is on fluctuations of aspects of fluency, such as speech rate and pausing.
Method
The project consists of two main parts, answering the two research questions. For RQ1, existing data of the TOEFL data set is used. With statistical modeling, we will gauge the amount and type of fluctuations of aspects of fluency throughout a speaking performance. For RQ2, we first determine the extent to which information on dynamicity of fluency adds to explaining existing judgements of the speaking performances. Following, we use thinking aloud and stimulated recall techniques to investigate how judges make up their judgement while listening to speaking performances.
Social relevance
The outcomes of the research contribute to more valid assessment of oral language tests.
Scientific relevance
Currently, there is little research on how aspects of fluency fluctuate in speech, let alone in speech by language learners (in a testing situation). In addition, statistical techniques to investigate dynamics aspects in speaking.