25.05.2009

Plenaries


Monday

Charles Alderson: The role and uses in assessment of the Common European Framework of Reference for Languages and some of its current limitations

 

James Milton: What is vocabulary knowledge and how do you measure it?

 

S.C. (Ineke) Vedder: Task complexity, task type and measures of linguistic complexity

 

Gabriele Pallotti: Establishing the acquisition of a linguistic structure: theoretical foundations and methodological problems

 


Tuesday

Gabriele Pallotti: Observing interlanguage development at school: from research to everyday practice

 

S.C. (Ineke) Vedder: Communicative adequacy and linguistic complexity in written L2 performance

 

James Milton: How does vocabulary knowledge fit into overall language level and exam performance?

 

Charles Alderson: Connections between assessment, second language acquisition and second and foreign language education

 


Wednesday

J.J.M. (Rob) Schoonen: Psycholinguistic perspectives on assessment in applied-linguistic research

 




 







Abstracts

 

Charles Alderson: The role and uses in assessment of the Common European Framework of Reference for Languages and some of its current limitations

 

The Council of Europe's Common European Framework of Reference for Languages (CEFR) has had considerable impact on language education in Europe and beyond. It contains a wide-ranging account of aspects of language, language learning and teaching, and assessment, from a social and action-oriented perspective. Particularly influential has been the development of a six-level framework  of second and foreign language proficiency (A1 to C2) and a set of scales which describe these levels in a wide range of macro and micro language skills.  In this presentation, I will describe how the CEFR has been used in a variety of contexts; I will discuss some of the advantages of such a framework, and also describe some of the limitations of the CEFR in its current form. I shall end with proposals for its further development and improvement.

 

James Milton: What is vocabulary knowledge and how do you measure it?

 

In the first lecture the intention is to answer the question of what exactly is being tested in tests of vocabulary knowledge. It turns out that a ‘word’ can be quite tricky to pin down, and therefore to count accurately, and learner knowledge is more often made using some form of word family as the unit of measurement. It turns out too that decisions have to be made as to when a word is ‘known’, this is not always clear cut. There are now some, relatively, standardised tests of vocabulary knowledge and I will introduce and explain some of these. Using results drawn from the use of these tests I can then explain some of the ideas about which words tend to be learned, and when, and how vocabulary knowledge develops over the course of learning.

 

S.C. (Ineke) Vedder: Task complexity, task type and measures of linguistic complexity

 

The investigation of complexity, fluency and accuracy in second language learning requires a thorough analysis of  the type of tasks by means of which language proficiency is assessed. In order to establish how complex, accurate and fluent a learner’s output is, it is necessary to gain insight into the question of which type of tasks, in terms of inherent complexity and task format, are most likely to elicit linguistic performance. More specifically, the questions is thus to what extent specific linguistic features are triggered by the tasks that have been assigned. On the basis of examples of writing tasks administered to L2 learners of different proficiency levels, a number of issues regarding the relationship between task complexity, task type and linguistic performance will be discussed in the lecture, and a study will be presented on the effects of task complexity on linguistic complexity and accuracy in L2 writing. In the study, conducted among 91 Dutch university students of Italian and 76 students of French, both general and specific performance measures were employed. One of the findings of the study was that for describing linguistic performance both general and specific complexity measures, which seem to be complementary to each other, are to be applied.

 

Gabriele Pallotti: Establishing the acquisition of a linguistic structure: theoretical foundations and methodological problems

 

Acquisition criteria are a fundamental issue for SLA research, as they allow to give operationalized and falsifiable answers to the question 'when has structure X been acquired?'. As such, they underlie all research on developmental sequences. Various acquisition criteria have been proposed in the literature, and the talk will begin by giving a brief overview of the field. This will be followed by the illustration of one specific acquisition criterion based on emergence, i.e. the first systematic appearance of a linguistic structure in interlanguage. Methodological problems involved in the criterion's definition will be discussed and various operationalizations of the criterion will be compared.

 

Gabriele Pallotti: Observing interlanguage development at school: from research to everyday practice

 

Second language acquisition research aims at maximizing the validity and accuracy of observations about developmental sequences in interlanguage. This requires the collection of large data corpora, their transcription and a detailed analysis, all of which is extremely time consuming. In order to introduce interlanguage analysis as a part of everyday classroom assessment procedures, it is thus necessary to identify more economical  procedures that still guarantee the validity and reliability of observations. A project  will be presented involving Italian teachers from kindergarten to junior high school. A simplified elicitation protocol has been developed to ensure the highest possible data density for linguistic features with good diagnostic value. Analysis is also simplified by the use of observation grids  guiding teachers in their assessment of interlanguage structures. The talk will discuss the methodological solutions followed and the procedure's strengths and limitations.

 

S.C. (Ineke) Vedder: Communicative adequacy and linguistic complexity in written L2 performance

 

In the lecture the relationship is investigated between the communicative adequacy and the linguistic complexity of the written output, elicited by writing tasks at an intermediate level of L2 proficiency. The aim is to provide evidence of learner performance, both in functional/communicative terms and in linguistic terms (i.e. grammar, lexis, accuracy), at specific levels of the Common European Framework of Reference for Languages (CEFR). The participants in the study are 30 L2 learners of Dutch, with various linguistic backgrounds; 45 L2 learners of Italian, with Dutch as L1; 30 L2 learners of Spanish, with Dutch as L1. To create a baseline comparison, the writing tasks were also administered to a group of 15 native speakers of Dutch, 15 native speakers of Italian, and 15 native speakers of Spanish. The main issue which will be discussed in the talk is that linguistic performance cannot be assessed meaningfully, without taking into account the communicative adequacy, effectiveness and quality of the L2 text.

 

James Milton: How does vocabulary knowledge fit into overall language level and exam performance?

 

It should come as no surprise to discover that vocabulary size grows over the course of learning and that really good speakers of foreign languages know lots of words. What this second lecture will do is to add normalised scores, average vocabulary sizes, for various levels of achievement, for performance in different exams, and for ability in the different skills of language. Hopefully, these will be useful. These figures are thought to be associated with coverage, that is the proportion of words in any text a foreign language user knows. What the second lecture will do therefore, is to review the kind of vocabulary sizes needed for reading and writing, listening and speaking, for passing landmark exams such as Cambridge FCE or CPE, and for achieving the various levels in the CEFR. Each of these will be considered in relation to the kind of coverage that these vocabulary sizes give.

 

Charles Alderson: Connections between assessment, second language acquisition and second and foreign language education

 

Assessment and testing are often seen in a negative light, as causing undue anxiety in learners, as obliging teachers to do things they would not otherwise do, and as being used by policy-makers as a “quick and dirty” lever for change. However, I argue that testing is misunderstood, and is frequently the scapegoat for problems elsewhere in society and in education. Indeed, I argue that assessment is central to applied linguistics generally, as it obliges one to think in concrete terms about theory, about the nature of language and communication, and about language acquisition. Testers are fundamentally concerned with operationalising abstract theories and constructs, and therefore can contribute to an understanding and development of theory. Although there has been little cooperation and communication in the past between language testing specialists and researchers in second language acquisition, this is now hopefully changing with the creation of SLATE - a group of researchers in second language acquisition and testing. In this talk I will describe the aims of this group and illustrate some of the benefits and outcomes of collaboration, particularly with reference to language education.

 

J.J.M. (Rob) Schoonen: Psycholinguistic perspectives on assessment in applied-linguistic research

 

In most empirical research on language learning and language use, whether conducted to test theories of SLA or to investigate educational progress, it is the language user or learner who is at the focus of interest. In order to reach their research goals, researchers have to assess language user’s linguistic skills and language knowledge. Therefore, language testing, including the operationalization of the constructs we seek to understand, is part of almost all research in applied linguistics.

The empirical literature shows that in language testing “knowledge” has been assessed more frequently than “skills”, and end products (of receptive or productive language use) more frequently than the underlying processes. There might be good reasons for this ‘bias’, but I would like to claim that for a good understanding of language proficiency and language learning both knowledge and skill dimensions of language proficiency need our attention and that it would thus be beneficial for research to include more psycholinguistically-oriented measures in our assessments.

In this talk, I will present results we obtained in past and current research in which we used some psycholinguistically-oriented measures, in addition to more traditional knowledge measures. The psycholinguistic measures pertain to the fluency with which language users are able to perform certain linguistic tasks.

The first study (the NELSON project) was in the domain of Dutch L1 and EFL reading and writing. In this study we combined ‘traditional’ knowledge tests with psycholinguistic, time-sensitive tests using RT measures. The second study (the WiSP project) has a similar design, but focuses on (Dutch L2) speaking proficiency. Speaking is more time constraint and thus might be more sensitive to fluent accessibility of linguistic knowledge, or lack thereof. Finally, I will briefly outline an ongoing study that we have recently started, exploring a processing dimension of lexical skills in primary school children, i.e., accessibility of lexical knowledge, in addition to the well-known dimensions of breadth and depth of lexical knowledge.

To conclude we will discuss how this psycholinguistic approach compares to more task-oriented approaches, to what extent these kinds of measures can contribute to diagnostic language testing and, finally, how this approach relates to recent developments in validity theory.