Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.
Oxford University Press Walton Street, Oxford OX2 6DP Oxford New York Athens Auddand Bangkok Bombay Calcutta Cape Town Dar es Salaam Dehi Florence Ho~Kgo ng Istanbul Karachi Kuala Lumpcr Madras Madrid Melbourne Mexico City Nairobi Paris Singapore Taipei Tokyo Toronto and associated companies in Berlin lbadan Oxford and Oxford English are trade marks of Oxford University Press ISBN 0 19 437003 8 @ Lyle F. Bachman 1990 First published 1990 Third impression 1995 No unauthorized photocopying All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of Oxford University Press. This book is sold subject to the condition that it shall not, by way of trade or otherwise, be lent, re-sold, hired out or otherwise circulated without the publisher's prior consent in any form of binding or cover other than that in which it is published and without a similar condition including this condition being imposed on the subsequent purchaser. Typeset in 11 on 12pt Sabon by Pentacor Ltd, High Wycombe, Bucks Printed in Hong Kong For my closest friends and loved ones: Nida, Tina, and Melissa Acknowledgments The author and publishers would like to thank the following for permission to reproduce the material below that falls within their copyright: The American Council on Education for the extracts from R. L. Thorndike (ed.): Educational Measurement (Second edition) and R. L. Linn (ed.): Educational Measurement (Third edition) The American Psychological Association for the extract from the paper by S. A. Messick in American Psychologist 30 . BrookdCole Publishing Company for the table on page 30, adapted from M. J. Allen and W, M. Yen: Introduction to Measurement Theory The Center for Applied Linguistics for the extract from the pager by J. 0. Carroll in Testing the English Proficiency ofForeign Students Educational and Psychological Measurement for the extracts from the paper by C. I. Mosier in Volume 7 Jay Haley for the extract from Strategies of Psychotherapy The John Hopkins University Press for the figure on page 341, adapted from A. J. Nitko, ‘Defining the criterion-referenced test’ in R. A. Berk (ed.): A Guide to Criterion-Referenced Test Construction, and for the extract from the same paper Language Learning, and the authors, for the tabie on page 196, from the paper by J. van Weeren and T. J. J. Theunissen in Volume 37 Newbury House Publishers for three figures (from J. A. Upshur, ‘Context for language testing’) in J. W. Oller and J. C. Richards: Focus on the Learner Pergamon Press PLC for the extract from K. Johnson (ed.): Communzcat- ive Syllabus Design and Methodology Psychometrika, and the authors, for the extract from the paper by K. K. Tatsuoka and M. M. Tatsuoka in Volume 52 Contents . Preface Vlli 1 Introduction The aims of the book 1 The climate for language testing 2 Research and development: needs and problems 8 Research and development: an agenda 12 Overview of the book 13 Notes 15 2 Measurement in Production 18 Definition of terms: measurement, test, evaluation 18 Essential measurement qualities 24 Properties of measurement scales 26 Characteristics that limit measurement 30 Steps in measurement 40 Summary 49 Notes 50 Further reading 52 Discussion questions 52 3 Uses of Language Tests Introducrion 53 Uses of language tests in educational programs 53 Research uses of language tests 67 Features for classifying different types of language test 70 Summary 78 Further reading 79 Discussion questions 79 4 Communicative Language Ability Introduction 81 vi Contents Language proficiency and communicative competence 82 A theoretical framework of communicative language ability 84 Summa 107 Notes 108 Further reading 109 Discussion questions 109 5 TestMetRads Introduction 111. A framework of test method facets 116 Applications of this framework to language resting 152 Summary 156 Notes 157 Further reading 158 Discussion questions 159 6 Reliability Introduction 160 Factors that affect hguage test scores 163 Classical true Score measurement theory 166 GeneralizabiliY theory 187 Standard erdr of measurement: interpreting individual test scwes within classical true score and generalizability theon 197 Item rsponse theory 202 Reltfbility of criterion-referenced test scores 209 F,itOr$ that affect reliability estimates 220 Jystematic measurement error 222 Summary 226 Notes 227 Further reading 232 Discussion questions 233 7 VaIidatign Introduction 23 6 Reliability and validity revisited 23 8 Validity as a unitary concept 24 1 The evidential basis of validity 243 Test bias 271 The consequential or ethicai basis of validity 279 Postmortem: face validity 285 289 Summary 29 1 Notes 284 Further reading 294 Discussioir questions 8 Some Persistent Broblms and Future Directions 296 Introduction 3Q0 Authentic language tests 333 Some future directions A general model for explaining performance on language 348 tests 35 1 Apdogh et prolegomenon 356 Summary 358 Notes 358 Further reading 358 Discussion questions 361 Bibliography 3% Author index 397 Subject index Preface This book has its origins in the close personal and collaborative relationship that Buzz Palmer and I have had for a good many years. We first hatched the idea of writing a book on language testing research somewhere between the 1981 ‘Illinois’ study and the 1982 ‘Utah’ study, at a time when we were both heavily committed to trying our best to incorporate what were then the stili fairly new ideas about ‘communicative competence’ of people like Mike Canale, Merrill Swain, and Sandy Savignon into actual language tests, and to trying to find out if they were different from the kinds of language tests that were then most commonly used. The two studies that Buzz and I conducted together were a lot of hard work (neither of us may ever want to do another multitrait-multimethod study again!), but they provided a wealth of example tests and anecdotes that I have used with my classes of language testing students, and which also hopefully add a touch of both reality and comic relief to this book. More importantly, however, those studies forced us to face head-on some of the issues and problems that are the substance of this book, and to realize that addressing these will require the best ideas and tools that both applied iinguistics and psychometrics have to offer. Buzz has provided me with frequent comments and suggestions as the book has taken form and he must share the credit for the inspiration and many of the ideas herein. Much of what is in this book can also be traced to two individuals whose work has influenced my research interests, and indeed my career, in very fundamental ways. My first introduction to applied linguistics was Robert Eado’s ( 1957) Linguistics Across Cultures, which was required reading for ESL Peace Corps volunteers in the mid-1960s. Even though this book was quite an eye-opener for a medieval English literature major during Peace Corps training, it wasn’t until I was ‘in the field’, teaching ESL in a high school in the Philippines, that I began to appreciate its wisdom. Its real impact on my career, however, came a few years later, when I was drawn back to it, during graduate school, after having read John B. Carroll’s Preface ix (1964) Latqpuge and Thought. It was Carroll’s discussions of language acquisition research and cross-cultural research in psycho- linguistics, along with Lado’s discussion of contrasts across lan- guages, that I found both exciting and challenging, and that piqued ‘ an interest that eventually led me to abandon medieval literary studies for dissertation research in second language acquisition. It was not until after graduate school, when, as a Ford Foundation ‘adviser’, I found myself in charge of the development and adqinistration of language tests at a national tangpage center in Thailand, that my on-the-job learning led me to the library, where I first discovered that either Lado or Carroll had anything to do with language testing! During the next few years I was fortunate to have the opgortunit). to work with John Carroll on several occasions, OR the development of language aptitude tests in Thai, and was always both awed and inspired by his encyclopedic knowledge, his brilliant insights, and his consummate craftsmanship. I continue to read his work with interest and to correspond with him on occasion to ask a question or pose a problem for his consideration. A great deal of whatever is useful in this book is a result of my contact with him and his work. When I was trying to come up with a title for this book, it seemed that all the good titles had already been taken. There have been titles in language testing with ‘issues’ (for example, Qller 1983b; Alderson and Hughes 1981; Lowe and Stansfield 19881, ‘current develop- ments’ (Hughes and Porter 1983)’ ‘problems’ (Upshur and Fara 1968; Interuniversidre Sprachtestgruppe Symposium Proceedings: Culhane et al. 1981, 1984; Klein-Braley and Stevenson 1981; Kohonen et al. 1985; Lutjeharms and Culhane 1982), ‘approaches’ (Spolsky 1978a; Brindley 1986), ‘directions’ (Read 1981; Lee et a!. 1985), ‘concepts’ (hike and Hinofotis 1975a) and ‘research’ (Olier and Perkins 1980; Oller 1983b; Bailey et al. 1987). And while I’m not aware of any ‘principles’ or ‘essentials’ titles in language testing, I’m not convinced that what I have to offer is quite as certain as these terns would imply. The title I’ve chosen turns out to be a portmanteau of the titles of two seminal works in language testing that happen to have been published in the same year: ‘Fundamental considerations in the testing for English language proficiency of foreign students’ (Carroll 196la) and Language Testing (Lado 1961). Thus, in solving my title problem, I also echo my debt to Lado and Carroll; hopefully what I’ve taken from them is returned in some small measure in the pages that follow. Throughout the travail of writing this book, I have (sometimes)