The Evaluation and Selection of Basal Readers

Author(s): Roger Farr, Michael A. Tulley and Deborah Powell

Source: The Elementary School Journal , Jan., 1987, Vol. 87, No. 3, Special Issue: The Basal Reader in American Reading Instruction (Jan., 1987), pp. 266-281

Published by: The University of Chicago Press

Stable URL: http://www.jstor.com/stable/1001176

REFERENCES Linked references are available on JSTOR for this article: http://www.jstor.com/stable/1001176?seq=1&cid=pdf- reference#references_tab_contents You may need to log in to JSTOR to access the linked references.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at https://about.jstor.org/terms

The University of Chicago Press is collaborating with JSTOR to digitize, preserve and extend access to The Elementary School Journal

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

The Evaluation and

Selection of Basal Readers

Roger Farr Indiana University, Bloomington

Michael A. Tulley Indiana University, Kokomo

Deborah Powell University of Northern Colorado

The Elementary School journal Volume 87, Number 3 o 1987 by The University of Chicago. All rights reserved. 001 3-5984/87/8703-0003$0 1.00

Abstract

Basal readers dominate reading instruction in most classrooms across the country. In fact, considering the central role of basal readers in the classroom, the selection of a basal reader represents the selection of a reading curricu- lum in most American schools. And, because textbook companies publish what school dis- tricts and states choose to buy, textbook selec- tion directly influences the quality of basal readers. Operating properly, textbook selec- tion should contribute to the improvement of reading instruction. However, major weak- nesses are evident in the process of basal reader selection by states and school districts: Deci- sions on basal reader selection are often based

on peripheral or insignificant criteria; members of selection committees frequently are handi- capped by lack of time, training, and direction; selection decisions often do not reflect philo- sophies about the teaching of reading; publish- ers and influential committee members often

wield disproportionate power to influence de- cisions; and, statewide selection, conducted in 22 states, yields few benefits to justify the ad- ditional investment of money and time. For basal reader selection to operate as it should, major changes are needed in assumptions regarding the adoption process, selection of reviewers, es- tablishment of criteria, and in procedures for evaluating materials.

This article describes what we know about

reading-textbook evaluation and selection and, based on this information, sets forth several recommendations for improving that process. We believe that an improved textbook adoption process-that is, one that is more reliable and valid-may lead to the selection of better basal readers and

ultimately to better reading instruction. There are two basic facts regarding

basal readers that relate to their evaluation

and selection. Perhaps the most important

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

. ,::.,.:: 'I ? 1- 1- .;..?- ' ? . . .. ......... .... .... ? ?- ?i':'?.= .'? .... . . :;. .. I . .::;:. ? I I - ..'.'..' ,..111 ..1 . ...... ..... I -',:??sn;::- :.'. . .:'- ::%?. ???::? .. ?:?: : : ? m h v ? ? .... : - . . . , .. ....... ?:? ? y . ????! ::::i: I -.? I ??? t!::::::::?? .... ..... .z man THANIRYM '? .-.: ?-.'??.'..A jf SHE ..... ..?t ? ...? ......... 1? . , ?: ,:.1 '... i ??' ?: ????' ?: f:::??..,.,.!:??:.::???:????????::?????.?.?:?: ?:? ; ???

?!?.'..': ?????."., .?:?:::?:??:?!?::.?.?,'.:.. .,.:"., -:?:....?.. ... q:.: :?:s': v ... I. ..... ... ... !:.?. .. :?':! ,??:? .. . " IF TV ! WN

.?? '..' . ?. ? I ,- ,- ? ::?:.! ??.::

?r ?::-?,?I':-R ?'..i'?.. -...? .t ?.' W h o , ?.'.? :..::::: ...... j ?.:::: . '... . . .... ... ?.'..'. :?n..?.i:..?: " :":. . ls??a. ... ' ' "'?%]'i,?":s. - ?i' ? 1.9 ..:i.: !?ses: i:i?i?: ?:?.' ' ffi'.'-","?Uzi?i::: . ' .:.i: , :.::z? i::,:i ?? 1, W . ? ?; a ? . , ..

P't:?i-q'iii?'?'m C -: :.. .":i?,'?."iji-?Ixr I ? . ?.' ? , : '. ., y v.. 1: -, ??,. "'.'? ? . . . .,...e.,. . . ?- , - ,

:??Nl??111?..!..vle.'. ?i?: ??.-i--.'Fls I -,'. ? ?? i, ?:? ? i!:! ..li..$ .?l??!!?i!? ?:: " ?? ? ? ? I - I ? ? 0 ??':: I . .: .. v:1.j W . I I ? ? I ?i??.. ?... ? M e" T':? ..' I ? ? I ? - I .

i?: ' .v ,- sllil? ::.. -?????????...? ,,?-?il 1!.".!'.?::? .??. ...... Ii? ? ::. .... . ... , ',? ...:?-.tl:?'..??:. I ? I ?'.'.' :::?-- ? ??a s& .: I ? ? ? ? I ? :

........... 's? ... . . ..?? 1. I .? .. ..... .:?i . SM& -.vl: M . 11: 1 . . ?.?:.:?..: - .. . ??:':. . .. ?-.?...i...U n : ? ? I ? ': ?? ., , I .1 '..?:..??:? t??:";?:??;'i"fc ,? I I 1, I " . :!?I?:. ? :? .i: ..... ... .... . . ?i?..":.?:...?..::. .....?:'ok .:,.: ... I ..... . yq v q l f"? :?? ? ?::? J?:? ? ,"g . '?'. . I 10 : a ., . il?.. - I .:?..........%.- - ..... ;?. I I - ..':??- -?.- ?' . ?'.?..; f ? ...... I I - '.

I I %:- ... ? '. * .-V N "Y . A . t??.:': ?i ,.. .... :.:::. % .. i . .. . .'... .. .i2..?.... . . : .... I ... . .,

.i? -.. " - m m " .....`x . ???:. ..: . ... .. ei?.. ......... . . . . ?. . .i?ry ... . ? ? .' ?::! ?' -. . I ? . n?-.?.;;...- - :--- : .. . , ... ?:n- ,"s. ?'." ., ? I . . ..... 2 .::: ....:..:.: 'i .??::?.:::.,... :1 1 . .... ..' ... . .. ?::.:ms:..:? . . . 7 ?? ::?!:?!?:.!o ??::?:i:j ?:??:i...:::: :- ' . . .. .. ??::: '..il- . . . ? . . - - '.- nC'.'..' .... ??..: ?'? "' :.z ,::: :R:.. -.! .:.::.., , ??-?? '? I I ' : -1?? " .1 ?';!'i'. ... .. ;?? '??!?.'? :: .!::?!?:.??.: , ?? iiiiiii , : :

, - - .. .. ,-;??; .. ....ig!"!'... i;??;??.. I ,- . '-' ',-,??;?.. - .... ... . ... ?% '. z o l ??. . - ??:??v???. !!.: 1 - - '..... '. %. , - - , '.. - ... ? ... .... .... , ?.. .? .. ?:?:::: f ? .: - - .... .: :::.. `?.??r?: ..... .:?. ? I `..!siii? ..:: . : ... I V

.i? % :%:..S ,...... ... ? . ? - ? .A ..' , - - I I I ,9 1 - I I

I ......... , 1 i .... .. I .?.:??. :i. : .. 111'?:: 161 j p ...... I,.::. . ? I ?zz- ..:.,:..::! .? I ? :; :::

: I ? . I ? ? ? : :??::??????..?!???????????i???????????? 1 ? , " I . ?:::.... . ? ??? ......... 1 'i'.'.-.P .:z :::- ? -e : .... ? i? 1

.?P?? "" I 11

I I . - ? ? I , i

. I I ? ?",??i .......... ?v:??. ??M ... . , : ? : .. . ;;;;; .. .

ii?.n?: ? I ..... ??::::::: 66 ?!? . '!.? ? -?? ? ,.i 1 1 ? ......... 11 . . :. : I ? I .. . ?:.'- I ' I , ? .

. . ? - . ?:O .f' I I ? ? :'-, . ...!. . ..... ..... I ? I ? ? I . . ..... ::. : ?,

? ? I : : : .

? , . , .:i? I ? I I i.'f:??:?!??:?.?. ' - ?;.?? ... ... I .? I ? I I I ? ? ? . . ? I , , 1 .i::: ? I ? ? ? ;....'?.,.:i:?. ". - ?' `.'? ..: I I I I . :-.? ..... I ..'..: . ..... .. .. ?-.5`i'?"'-..?.aE '. . . ... . ".. ........- . 1 :.:::'?.!.z!:.. .: ?:?. ' ?. '? ..i? .. 'i' - i:.:.s . z I.: i ':..' ?.?.?.. . ...:. . 7: I I I - ..:?: , .. ...... : n.??!'.;:?p'?';,? - .'. - ' 1? I I ? I ' f .? I .. ... . ??n'r:. i.% ?. ? ?:?k. - ?i...-'.??. .?:??..??..???'..?:..?"..,.,,?? '? ..

. . " :::?i: s , ? . . . . . . .. . ........'...... .... . . ...

:- .-..:?i; .... 1.1?;?!i . ... . ... . ... . '...' ..... ... . L :'O .:% :.?'.? ... ?" ' . .. . . ? . . . . ..

R-- - . . ... .... .. . ...... . ...'-- - ... ..... I ? :?? '. , ?` .' ! . ?: ?:i-, ......... ? ...... . . . . . . . . .. . . X.. !. I ?- .'.' :,.???.,;!.%,...!?....!,...,,....'...- . .......... .... .... . '. . ...... ... ,: .:? '. . . .. .'.. ..' .. , .'.. .. ?...? '..i-. .... - .... .......... ? .. . .. . ,,, ?. !'.."...'...?jjj'... . . . . .. ? . .... . : .... ? ?' zk ?!?? .... % . ... . ?. .. ',. . .. '.

I - ? . & s . .. . . .. ..' . . .. VW K Y .. ! : ? ? .... . . . . .'.. . ,.. . ,?'- V:' - .. .

..... .. . V . . . . .. .. ... - . . . ...? .....

.. . .. .. .. ... . ........ .? . . ..... .. .. .. . . . . . . . 1-

.%?J I 11 .. i.. ?:? :? ?i .. . . . ... . ... ..' . .. n. . ? ... ... ... . . v p q = q ?::X: . . . . . . . ... :: . . . . . .. ? - , '. ., .. .. ?' ?. : . ??:? . . . .. . . . . '.?...'. , . . . . ... . .. , . . . .. .. '.? ? : p: ;..;., ::j '.i!!!. 4, ' . ... .... ... . ... . ........ . ? . ......... ?! !? . . . .. . , . ..'. e.. .. ... ..... .. .. .... ? . ...... .. Z. . , ... . .

: . . . . .. ....:...;. ? . . . . , ? ... ..... ? - e ... .. ..

... . ..! ,..-;.. .?? ? . . . , , , . . . ... .. ... . ... . ?: ? ... . . . . . . .... . . . . . . . .. . . . ..... % ... - ... .. ::?: ? . . . . . . .

' ' .... . .. .. .% ..? .. . . . .. . .. . ......... - ..... . ..

0 9 1 11 .?. ?::: , " , , . . .............. .. ......... . .. 1..?:? ? .:..'.:'.k', n .. ., . .. . . . . ... .. . . ... . . . . : ?:::??:::: f l ? .. .: .`!::: ?.??I.'. -?;? I . . . ... . . . . . ,'.",'??' ."P A U 4 1 ly , ' " :' , . ::: .:::::: -, ?.. ,.? .....

.... ... ., . - .......'.... ..'.1.- ... ,..: ...

........ ' ? ' a?- .:. . . .. . ...1.1 ?. R R 1 .... - . . . - .?. " .E 1 E f A . .. -':.? ... !:?:?..::!?'Z.,?.!?. ?:.:?'?::: , . -- % . Q ., I , in ? . ,' ? sy ," I "N.E.- . ..... ...... ? i-'- ...' ... 1 -i??.:?: .. . ? ? x5fol", ?:!? %"F'. - gtitv4 1 W T H ., ?:::???.. . ;;:k - ir?-'.?.'.:!?;A , sii':ii :?. , ' ? ':Iiiii I . ? . :?? I , W.1 ....... .... . .?'; ... .?!::!?:??::.!:?N : '.. T'. . . . : . - . ?r'?::!-- '. % Z ... . '. . . . . ft i?. . : ::?' % A t ?"'- j y m W l . ? . 7 . , , :?r : '.' ..' =?i"'= f ? ?: ' :.. .. 'A : %% : . %.?:?? ? .... :. " ?i'6 .

:.. .. . M n ?..' I ,.? ..... % . . ... : " % ' ? " ?: sk I " ? .. ... ? ' ? ni - ? ? : ' ?E . ?11.? . . ............. I , -'--??.f lij ..... , . . .. ..... . . .... . :: iz. .' I ."? ? , -1 -1 %-?". . . . A ? 'A , . . ., . . : " .

-" lip :? ?? : .'. -- . ? 11 ??? : ' ' ?- ... . " ??.:?.? .-?:-f .. . . ?.?? ... - - -': '. .??-' ?.- . . ?: ? .:??.::: "'???? ,, ??: 114 V -11 X. - ..... iii, .. ::.,, 'f ?: :'i' : ?.,.%' ' .. . ? '?? . ?F . . . . .1 ??"'-? ?f:'.': " " ?? ... %i ::`-"'? : : i: ... '. - ? - 7::.?.%.?- :. n :: ... .?. :? ?::" ..?.%?..? ... . ... -'- . - %... ., ? ?: : , . .. -0 :: :? : :: .? 1 ? . ... wt??: ::' ' . . . . . ? 4 1 1 "E S ..' ...:,. ..... ...... !.. . . :: .. . ..... ., .

. ? : '. 1 ...... M ON.~. -' . . ?-:-: . ??? .M . W '--..'. : .." - i: ?' ? .' ? ? . :?::: .::.: : . . . . . . . . . . . . . f ... 1. -'. :. ? ,.;? , ... :?.:.: . ?? . . . % . S ????? ? .? :::::: ;:? i.E??. - .. . '?f.?n?* :? . ::?..:::.'.-.. % ,". ... , -- '? ?': ?-.? ?... ?:'? -''

%': i-?i?? ? . . . . . . M R-M .. ! - '- ,M . .:: ::.:.::?.V sr.: D?"'.? .. -V ? : . :; i .?? ?? ? .. . :.. '? i- ?? '?'..' ?i? : , -- .',??.? 0 1 .'?? ...: ? . '. T W :? ','.. ? ?n? , . - -.-:. ?- .. ..' - - - , F1 Q = -. - :?? %-;??i' , ? ; ... ......... .. ...VA IM S g I "k......"..?"r,- . -111, - -l-'r ON A?? ? ? ", % ?. ? :?!: :.: :?R"!Iff ?A? . ?,';;..' . I.. -!?? A . . - - - Iv- ' ? ,. * i n ! . -? ?-,, ... ??:; . . -.;? ::-. ?'.-r ,-,: 4 - .

.. . ?. ' ?'i4r'?..??:i?"'.' :.? . . 1 " -?%::?::%! f of . ..? 1 1 - - .- F A 2 .. ?.: ?. . -- - . . ..' .? : :. . 11 ?iE .-- I...... ? .... :::? .... ?.... , ? . - . "... ? M. X-- - - - . .-.= . I : - ? g .f?':?'rr :??.t'- .? ... : ?.- . I . . . . :::::: . ' -? ? ...' ....- - ... , .. ... n:;?%.T ':!s?i ':!:.... : PAN "?N?'?"' ?:? F?'.-: :. : ?: : ?, v'.: "... ?.. ':? ' 10 H . 1 .:. .. I F I a. As ? : . .. . ? i??. , ,.I .? -':;? %. .... ,X. ? .... .. , - .. V : % ? ?: . . . 'J A ?' "'? . . - :? ::: ?* . . nz: '? i?:i.%'??.i:l?:_ ?%.-x . - . . . ?

a .:. ? . :e.Z . ??'?: :e,:...7. ..

- ? k I . . . :.. .. 0C ?; ? ".';j??::....- " ' ,I- ,.::... .. . %?n?_" ?"?`i%':! 0 0 Q , . ...? ?:: - . X . ?' .. :: . ....... .. ...: ... ?.' .. ??.".-"?: . ? 'p r . ...... , , ' '?:::: ??%" ?? ???::??:??:.:.. ?:lt??%%,.,.:,???:::???::..?:??. ?- . A A I W s , % ::P. .. . . ! ? . "?j ? ?, ? ,X, ?' ;'. , ?? ?. ? ... n..r%:.?::?. ? . . . . . . ', "n I ?,"."-".??.i : ?. .:. .:: ..?: ... : . ...! 1 . '. , ',? . ---

N X .. ---M - %- . M=,v--- ...2. ?-- It ? :?,-...x ..:%.. n? ? 0 --- F- W.g. ' %%%%%... W ARM " . , -I ??.. il . ? 4. , I - ': z:.::.'cft %? .. ..: % i: : % ^ R& .:?:.! .. ... .... :- .,

- : ... :..?-. :- .. ;.M ? ?:': .- . '. .. ", . ? i;.

.....:?..: . . , . '.' % : i., - ..... :: ... :.. ? ? ?.:?..?? ... :: ::::??:.?::. ?" ? , ..,. : . 1A ' .?:? - 4. j w ?!Z .. - . .. .::..: ?.. .- -5 . . . ?..? ? ...?:. . ? ? 1?r I.. .. .?' .?'.?? i. ... ? R . ..::: . ?? ! ? ? '?.?? : : s??s?"' ... - -i?:::.', ? .... 1:1 - , - . - .: ,',. :;. ?: ? - , . ...? . W - . : v 0 M GM . o t . ,?? !?i . I Q yj ?: % :?? . -;.,..% N " ".- ... I ... -.....- 1 r M -g : M s . . I - 4 10 1: ?i:':' i 'x ''!;ti'? ? .. . V - - . - . ... ?.. T:1 . .? ,?'- :.... :.. ... 1 ; ?:?? ? , I 10 - - - ' A P , . ....6 .. .. I ...?;: 4':'. i ?"--!-':?: ' ' - -- K : y W .... . .. :0 : - %.".' .: L i???.::: . .? ??:?i ,:::?: .... ?4!1n'?. . k?.?::'?:. -'t. .. :: % U - ...Iq A s ? '? .! . . . . , I ? . : ' .* .- %. ...- , i:,:: : : ... .. " I - % . ... F?. .. ,? ?. I 1 11 i: .. ... . - Y ?n . . I ': I ? 1 ?7n sly n :".. . '% I . I , , ? ?.. ? .. .... . . %. , z. . : ?? 1 .?- . ... -?L:W ' . .. .: .?. W ?- l . I ? i:: . y W? X..% .. .. W - - . ?, T v ? ?.' ? :? x ..., ..... .?......... .sn I ? "F 1. . ?. ow - - ?: ... . . . . . '? A== 1- .. Us 11 .. I MV.,.! . ? 4 :,:.:?.. '. ..... . I .. , :v : ? :.....? ... - - I :- I , .. ..::.- , ..:::..% ? '..'.:., ? i:10 1 :11 . .. g a Q ,.I.If. , S NEF . ,.;;", ... .... ; "?:T. ' ' ? t ? :,-%-ni?. ?: i, , .: 44 X: - j , ,. . . 1. -::: 111 0 ?4 ..... .. - - - ...... . ...'..? --- ? ... ? 1. I AW . .. ....... . I n.. . ' 1 iii: -a ..? . - % . '1 11.1...' - : . - . . . I 0 .?? '-K7.;'.i'.'??": .... ':???:??-. '? .." I ':.?: ' ? . ., " I . -!;?? ? . ..... . . . . . ?F: . 1 AM 549 ' ....V. . . . ?11%;;:"U:??:...'.".?: : . . - . ... ... ? ..... .. -;i ? . 11 . '?. . -T. , . - - -

..!! ..... ... - .....?:::r- .:' : - ?:.:?ii:??: ?' .,.,. . ........ .. 0" W , 'i;?... 1:.. . ?:?--:-'? ..... . - :. I .::. I , ? ...... , ? ..?.:.-, ? '1 '?w i n - ..: U?.'.?if.. I . -'. . ...... I ... ' : ? - ; " . . . ... . 0 ,I i -, i'. .? ? ??- , - ?: , . :?: '?,. .. , ??-. ,... .. ;? : , I - . . ::%. - ,1.1?4?.Y ??'%':?n?::: - I ?- .. ... .. . . ,1 :. I ... -- i - :? i . :: - :?..

.. ... I. . .. '-'. ? ? ? ??' i .. 11.1. : . .. I '. ' ' : ?: . - . ?.'I . .% . . .?.'?'?? - .::.. ? ':.?? . 0. ? ?. 'R??. '.?', ft , . . . . .::"'? " ?.-'.'.'., : ... . : ;.. ::... X f : ... . ... . ?A - .Z?t??..??.-?%':: 9 1 A .: .. ?f.?. ?"!;'.%::?Z.?.:..? - .1 '....':':.... . . - '. : ??.. ...? ., . .], .i . 'K - . . . . . .: &-:?'.. :: . .. : : "' 1-ivW -'.n?.n.' . . - . :. ?: ... "....' ..... ?' ... ...... . ... I 7..... .. . . : ..... .. ? ... ... - : ',: ?' 11 1 '; . . . ....'..... : ';n? .... :?... .. .- . . ?....: .. , . : , ?. -

. . ..... . ?- ??':.?- - .? . 7' li.%:.:? -1 :?I- . ? -.::?.-?. ..X-.." ..... " .. 1. n.

:??' .?. ..: la .. ? . :? . % ....1 ? . .; ??-. ')E?. - . :?? . %?f !?%'Ak ........ '1..- ..-- ?;?. :?:?!?! , . . - , I A ? ??.. - -... -'.. :? 1 :? ? M , ? . ..z ? : M ?:?.A .. : ..i;i?... .'? ....' ' `z ?'? % .... .. . -- - - 'I- ? .. " . , '?' .;, . ? ?:: .:.:,.?;? .... . . .??.- .. . ann I .. . . . "- ? ? I . eis?. ? ? 7f , ?"-? r?;". .,?" - .- - r?. , ..' I ...- ? T. I I A W W " J '?!'!. '?' ? ? .??:? ..::: %:? . v .... , ,%-!

. . - ?...: i.: ... : f .;?; in ho y l M ?.:- ' . :::.!:: 1!1??::::?%'??'??'.- %..."....? '. . .0 . . 5- :1 , I I .?. .;;;...?? P? " ? ..! :%::.,.:. ...'?.'?. ' -? ... .. ...: :-:..'..?::;r?.' ... .. .. .. ... ... S .. ? - -1 K ? - % % ... t o , 1 , p s ...... ? ' , . . .. ? ... .. ,.,v - ..... . ..... ,".5 ??..?!?.- ' .. i s f.' " :. :. .... /. .&?T K, ? : 1. .?' :. I . ...... ... ... .. .. -. .::................'-111.1. - '. .. ? ::' ' . .: ":. A . - .J . . 1, .-? -An n .. ilk -'-" '' ?? " !:?. ':?is?::n. !?!.' '?;: ..'A :?' . . ".1 . ..: ..?? . . .c - : : = . . . - . '...:...? ... .ff ?. s. ?2?%'- . ? im n -..n.? . . . . . - ... -W A - A . 1. - . ...... 1. -. ?W . .!:? ?..' , . % I ..... ?-..."....... .... ? ":: . ::j . I !?. I ? '- . ir . - -;- '. ? %"?

: : - ?

... q . ? :.. ? ,:: ....' : ?:..? ?? : ,; ..... .. . %, %': ?.: , ..... . , ?? - ?i: , ....:. ; j!i: .- . % ="??'L? , :: z 0 -., "?, A 1 ?' - 4.. .- - . ? , " - - ..... . . .:ii ?, E ..? :4 ..... . .. ... ::. '? -on a M? ' .1.1.1 . . . . . ? '..' . ? I . . . : . 1: ..

?.?-??.?:L . .? .... ; . .. '. . k " ? -.?4,? ? ., -,it - .. ' "'. . " . " - ., .:.::. ..... M - 11 0 1 110 ??' T:K"%n?: ?,?::: :'! ... ?.".. .' ? I ? " ? I ... ? .. .. 'I .: - . I ....... . . K, .:!:?;. ... - 1.1'.-N i;"?", I llk %.??- ?'T "'t'.?' ?::: ? ??' . .?..::%. . ...... ? T .:i -:?:.:.f.:....;?% . e, . :?: ..: r % '. '. ..1. .% T . ! ... ????: :: . . .. ". '. . Z yl.-?"si] ? ... .... .... . ... 'I' : . :..%.x , - . . ;"-?. ??:??!:?', .....?.:!.??::?. .z 4 h y I ?. ., . . F -- A "s ? I ;?... .1. . . . . . . . . . . : ? 1. - ? ' ? ?? -? --- , - .: f%-?? - , -? ?" ? ,: ; , , I I e - , ! ::'%'.-if;!"i?'? --?."q r ? - ??'N' !!!!!!! ? f . . ??'z

I r% .. . . ? 'i ,

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

268 THE ELEMENTARY SCHOOL JOURNAL

of these is that basal readers determine the

content and structure of reading instruc- tion in elementary classrooms (Dixon, 1979). Thus the selection of a basal reader is tantamount to selecting the reading cur- riculum. The importance of the textbook in American schools cannot be minimized.

Since the advent of public education, it has been recognized that the majority of direct instructional activities in the classroom in

all subjects involve textbooks. Most esti- mates indicate that those interactions ac- count for as much as 75% and more of

classroom time (Goldstein, 1978). Reading instruction in particular appears to involve the use of textbooks (Mason & Osborn, 1982). Research repeatedly documents the fact that basal reading textbooks are pres- ent in more than 90% of all elementary classrooms, and that those texts are the basis for reading instruction in the over- whelming majority of those classrooms in which they are available (Davis, Frymier, & Clinefelter, 1977; Durkin, 1984; Edu- cational Products Information Exchange Institute, 1977).

The second fact about basal readers that

seems inexorably tied to textbook adop- tion is the similarity among those pro- grams presently available. Basal readers, regardless of the publisher, include ma- terials for students from kindergarten through the eighth grade. The materials at each grade include a student book, a teacher's manual, a workbook, a set of ditto masters, and a collection of tests. Some programs sell other supplemental mate- rials with their programs. However, unless a basal program has these materials, it is almost certain to be a commercial failure

because of the observed tendency of local districts to avoid the "new," the "unique," or the "experimental" in reading mate- rials. Perhaps this reflects the importance attached to reading and the reluctance to "take chances" with reading. Regardless of the explanation, however, one funda- mental aspect of the present basal reader

marketplace is that less risk results in greater homogeneity.

The similarities, however, go much far- ther than just individual components of programs. The structure of the lesson plans, the specific stories that are included, skills that are taught, length of teaching units, and even the artwork are more alike than different when one basal reading pro- gram is compared with another. Indeed, in some of our interviews with textbook

adoption committee members, many in- dividuals could not tell one program from another when they examined instructional materials without seeing the name of the program, even though these interviews took place immediately after the commit- tee members had completed a several- month review of the materials and had se-

lected the program that they believed was best for their school or district.

Beck (1985), who also documents the similarity among basal readers, suggests that the selection of a basal reader may be analogous to the purchase of a family car. He contends that the decision in both cases

is made on the basis of the reputation of the manufacturer/publisher, what your neighbor is driving or what is being used to teach reading in a nearby school, the advertising slogans of the auto dealer/ publisher, the various companies' records for service, and other such issues. Beck

suggests that a thorough and critical ex- amination of the vital differences among both automobiles and basal readers is sel-

dom made. If this is the case, it seems rea- sonable to conclude that both automobile

manufacturers and textbook publishers will spend as much money on advertising and promotion as they will on improving the quality of their products because periph- eral rather than substantive considerations often determine commercial success in both industries. The lack of critical ex- amination of basal readers seems to be a

second contributing factor in the homo- geneity of basal readers.

JANUARY 1987

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

EVALUATION AND SELECTION 269

Publishers of basal textbooks are often

blamed for perceived problems in the con- tent and structure of basal readers. Many critics contend that schools would use bet-

ter books if publishers took the time to produce better books. Indeed, many critics of education suggest that the ills of the profession are partially caused by inade- quate textbooks. One of the most quoted of these critics was former Secretary of Ed- ucation Terrel Bell, who claimed in 1984 that the textbooks had been "dumbed down."

Publishers of basal readers, however, see the situation quite differently. Al- though publishers agree that textbooks provide the structure and content for what is taught, they nevertheless believe that the books they publish are based on curricu- lum outlines and instructional guidelines that state departments of education and local school districts have developed. In other words, they argue that the textbooks are responsive to the demands of the mar- ketplace. Robert Follett (1985), chairman of a major educational publishing com- pany, emphasizes the importance of the textbook adoption committee, the mar- ketplace for textbooks: "Successful pub- lishers produce textbooks that appeal to adoption committees."

The homogeneity of basal readers is certainly related to the fact that basal reader publishers attempt to publish what sells. Follett (1985, p. 19) further empha- sizes the importance of this point when he states that "from the publisher's stand- point, the textbook adoption process is the system that produces sales. Publishers are dependent on sales for survival. Because of sales their work has an impact on the education of children, confirming the value and worth of their efforts. Sales keep pay- checks coming; they are the ultimate mea- surement of publishing success or failure."

Among the most significant factors that publishers respond to is the timing of their revisions so that they correspond with the adoption cycles of major states or the larg-

est school districts. Squire (1985, p. 16), an executive with a major educational pub- lisher, makes the point quite specifically: "Where large state adoptions do influence publishing even more directly is the timing of revisions or new programs. Because of the magnitude of revenues available dur- ing the first year of adoptions in a Cali- fornia or Texas and the marketing advan- tage of having a late copyright, many publishers will time introductions to co- incide with the call for new programs in two or three of the larger states, or a com- bination of calls in states and large cities."

The similarity among basal readers is the result of publishers' attempts to pro- duce what sells. We believe that what sells

is determined more by marketing tech- niques than by the quality of the materials. Unless textbook adoption committees be- gin to analyze the textbooks more care- fully and thoroughly and communicate their evaluations to textbook publishers, textbooks will continue to look more and

more alike. However, if committees will begin to examine textbooks critically and let publishers know that they are con- cerned with the way lessons are planned and the way reading skills are taught in basals, some publishers will put more ef- fort into those aspects. We believe that better textbook evaluation procedures will result in a greater diversity among basal readers. Moreover, we believe that this greater diversity will have to do with fac- tors that make a real difference in teaching reading.

State textbook adoptions Considering the importance of the basal textbook to reading instruction, it is sur- prising that we know so little about the manner in which educators go about the task of selecting reading textbooks. Fur- thermore, that which is known is less than encouraging. Describing the factors that influence reading textbook evaluation and selection is difficult because most adoption processes are complex. However, a hand-

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

270 THE ELEMENTARY SCHOOL JOURNAL

ful of recent studies shed some light on the dynamics of these processes. The tendency among those who de-

scribe textbook adoption policies and pro- cedures is to divide the 50 states into two

distinct groups: (1) 22 "adoption" states that maintain some form of centralized, state-level textbook evaluation and selec-

tion process, and (2) 28 "open" or nona- doption states in which local school dis- tricts evaluate and select textbooks with little or no state control or intervention.

It has now been adequately demonstrated that the adoption process of neither of these two groups can be considered mon- olithic. Specific adoption practices differ sufficiently to dispel any notion that gen- eralizations about the nature of adoption processes can be easily formed and validly applied across either group of states. Nevertheless, enough is known about adoption practices to identify the major similarities and differences that exist be-

tween the adoption and the nonadoption states.

Chief among the differences, ob- viously, is the two-tiered evaluation and se- lection process in the adoption states. In these 22 states, statutes require state au- thorities or commissions to review all sub-

mitted textbooks in each subject area and to approve a list (varying in number from state to state) from which school districts must choose. In the other 22 states, school districts are essentially free to adopt whichever books they choose.

A second major difference between adoption and nonadoption states is the in- fluence on textbook development and textbook content that state adoption pro- cesses make possible. The "California" or "Texas effect" on textbook content, as well as on the adoption process of other states and school districts, is now widely accepted as valid (Crane, 1975; Keith, 1981). To reiterate, this effect refers to the situation in which publishers coordinate revisions or updates of basal series with the adoption cycles of these large states. Being among

those basals adopted by a large state gen- erally means increased sales and restricted competition from other publishers. Thus, the publishing companies attend closely to the adoption criteria, guidelines, and cy- cles of these states. The adoption processes of other states and districts are thought to be influenced by these large states as well in that the basal texts available to the rest of the nation are the same texts that have

been developed first and foremost with the interests of the large "adoption" states in mind.

What is it that the adoption states have in mind when they conduct textbook adop- tions? What do they hope to accomplish? Tulley (1985) attempted to answer these questions by ascertaining the presumed advantages and disadvantages of statewide adoption. In his literature review, Tulley found that from 1890 to 1950 three major advantages of statewide adoption were em- phasized: (1) a reduction in textbook costs through volume purchasing, (2) the selec- tion of better textbooks, and (3) some as- surance of a uniform statewide curricu-

lum. Tulley's literature review also identified counterarguments to each of these supposed advantages. It was argued that most school districts possessed the necessary resources to select their own textbooks, that the effects of a competitive marketplace would be sufficient to keep prices down, and that states could employ other and more reasonable procedures to ensure compliance with a state-mandated curriculum. In his own study, Tulley (1985) also found that those who currently fa- vored statewide adoption believed that these centralized processes would: (1) en- sure a periodic review of textbooks, (2) save time and work for local personnel, (3) al- low public participation in the adoption process, and (4) provide structure and or- ganization for the local review process. Farr and Tulley (1985, p. 469) point out that "the debate as to whether these purposes can best be achieved by statewide or local adoption has been going on for a number

JANUARY 1987

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

EVALUATION AND SELECTION 271

of years. Nor is the debate likely to be eas- ily resolved, because there are strong pro- ponents on each side of the issue."

Farr, Tulley, and Rayford (in press) conducted a study to determine whether or not there were, in fact, any differences among school districts in adoption states as contrasted with school districts in non-

adoption states in terms of textbook adop- tion practices and reading textbooks se- lected. That study examined the textbook adoption laws in all 50 states, and also in- cluded personal interviews with state ad- ministrators in each state as well as a ques- tionnaire survey of school district administrators in more than 300 districts

throughout the country. The results of that study indicated that

except for slight variations in cost, there were no significant differences in reading textbook adoptions between school dis- tricts in adoption and nonadoption states. The authors concluded that any advan- tages that accrue as a result of state-level adoption could be achieved through state legislation without the expense and energy necessary to conduct a state-level adoption of textbooks.

How adoption committees function

The overwhelming single commonality in textbook adoption in both adoption and nonadoption states is the influence of the textbook adoption committee. Decisions about which reading textbooks will be used in classrooms are basically made by these committees, which are usually composed of teachers, administrators, and, often, parents. Considering the importance of the textbook adoption committee, it may be presumed that much is known about how these committees are organized and how they go about evaluating basal readers. Such, however, is not the case. Relatively little research has been conducted, and the majority of that which does exist consists primarily of surveys that have attempted to document committee structure and or- ganization (Educational Research Service,

Inc., 1976; Institute for Educational De- velopment, 1969; Kreiner, 1979; Stewart, 1980).

Much is known from these studies about

how committees are organized, but little is known about how the committees actually function. For example, the surveys indi- cate that most reading adoption commit- tees are organized by grade level, and that reviewers are assigned materials to review that correspond to the grade level they teach or in which they are most interested. The typical basal reader adoption com- mittee considers an average of eight to 10 textbook series of the 15 to 18 basal reader

programs that are presently published. As we have studied the reading text-

book adoption processes we have found that committee members review the text-

books using predetermined criteria, usu- ally in the form of a checklist or rating sheet. The criteria sheets are a part of al- most every adoption process; however, the reviewers often do not actually use the cri- teria sheets-a point we will discuss later in this article when we review the func-

tioning of textbook adoption committees. Many of the checklists that we have read were developed by or are adaptations of instruments available from publishing companies, state departments of educa- tion, or professional organizations. Other districts develop their own instruments based on tradition (previously used check- lists) and what the district believes is im- portant. It is interesting to note that the items on many of the checklists that we have examined require the reviewers to determine only the presence of particular features rather than to evaluate the quality of these features.

Probably the most important common feature we found as we examined the func-

tioning of textbook adoption committees is the brief amount of time that districts

assign to the selection process. Most dis- tricts complete their textbook selection within 1 school year; the actual amount of time spent examining textbooks is usually

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

272 THE ELEMENTARY SCHOOL JOURNAL

much less than this-usually about 2 months. It is estimated that teachers spend 1 hour to slightly over 2 hours per basal reading series to complete an evaluation (this includes examining all of the text- books and ancillary materials, discussing the choices, and making the final selec- tion). Follett (1985) estimates that each reading series, including textbooks and an- cillary materials, contains the equivalent of 5,000 pages of text. One hour spent re- viewing one series would represent less than 1 second per page. It is no wonder that Powell (1986) found that the predom- inant review technique was the "flip test," during which the pages of the pupil's read- ing text are quickly examined for their general appearance: color, white space, size of type, variety of selections, and variety of illustrations.

Meetings with representatives of each publisher whose books are being consid- ered are another common activity of most textbook evaluation committees. Repre- sentatives are usually invited to make a for- mal presentation of their textbook pro- grams to the adoption committee or to the entire school district faculty. This may oc- cur early in the process or after the com- mittee has narrowed the textbook choices.

Most of the members of adoption com- mittees with whom we have talked feel ill- trained to evaluate textbooks. This is not

surprising, since teachers, parents, and ad- ministrators who serve on these commit- tees receive little or no in-service educa-

tion in the evaluation of reading textbooks. An exception to this is the state of Florida, which has recently passed legislation re- quiring textbook reviewers to receive in- service training before they serve on a state textbook adoption committee.

The criteria sheets that are used in the

evaluations do not seem to provide much help to reviewers, primarily because the selection criteria are not ranked by prior- ity, leaving committee members to set their own priorities. Rarely does the district or the committee set standards that help re-

viewers agree on good and poor examples of various text features. Thus the process is one in which reviewers are generally left to develop their own procedures and stan- dards for distinguishing among basals.

A trend that seems to be widespread and growing is the "all-teacher vote." In this situation, all of the teachers in a school district are asked to vote for the program they believe should be adopted, even though a school district committee has carefully examined all textbooks. Our ex- perience with all-teacher votes is that they produce less careful textbook evaluations. The more persons involved in the final de- cision, the less commitment individuals feel

to review the textbooks carefully. When all teachers vote, most committee mem- bers seem to develop the feeling that the committee review is superfluous, since the final adoption decision will be based on the teachers' vote. We have found that in most

"all-teacher vote" adoptions, very few teachers have carefully examined each of the texts that is available for adoption. In two case studies of district basal reading textbook adoption processes, Powell (1986) found that in a district that allowed all teachers to have an equal vote, teachers spent considerably less time reviewing textbooks than did teachers in a district that allowed the committee to make the final recommendation to the school board.

Although the overall organization and activities of adoption committees are im- portant, the actual work of evaluation is accomplished by individual committee members. What does a committee member

actually do? What factors influence a com- mittee member's evaluation? How do com-

mittee members feel about the task they have been given? Questions such as these have been examined in studies by Court- land, Farr, Harris, Tarr, and Treece (1983) and by Powell (1986). Although there is little basis for generalizing the findings of either study to committee actions in other districts and states, these two studies none- theless provide interesting insights into the

JANUARY 1987

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

EVALUATION AND SELECTION 273

types of procedures that might character- ize adoption practices throughout the na- tion. We believe that these insights merit discussion here not only because of the in- triguing issues they raise, but also because our experiences with professional groups and with conducting in-service sessions throughout the nation lead us to suspect that these practices may, in fact, be more widespread than is immediately apparent.

Courtland et al. (1983) conducted a case study of a statewide reading textbook adoption in one of the 22 "adoption" states. The study focused on the criteria and procedures used by those who actually reviewed reading textbooks. The review- ers were appointed by members of the state textbook commission (the body granted statutory authority for textbook adop- tion), and they were assigned to review the basal readers and report their evaluations to the textbook commissioners.

Although most of these reviewers were experienced classroom teachers who seemed to have strong theoretical and practical beliefs regarding the teaching of reading, many expressed confusion and lack of focus regarding the evaluation pro- cess. Several did not believe that the eval-

uation guidelines they were given pro- vided them with adequate direction for conducting the evaluation.

Another major problem that many of the reviewers expressed was the lack of time to conduct the reviews. The review- ers had a total of 6-8 weeks to conduct

their evaluations, which they conducted in addition to their regular teaching and ad- ministrative responsibilities; most felt overwhelmed in trying to get theirjob done in this limited amount of time.

Because of the lack of time, many re- viewers commented that they first "browsed through" a program to decide whether or not they liked it. After making their evaluation decisions, they filled out the specific items on the criteria sheets so the sheets would agree with their overall evaluations. These reviewers therefore

based their specific evaluations on a global evaluation of a program rather than form- ing (as logic might suggest) a global opin- ion as a result of more detailed analyses. Not too surprisingly, when asked to iden- tify specific factors that caused them to se- lect as "best" one basal reading program, the reviewers responded with a wide va- riety of general responses and very little specificity. Courtland et al. also report that the reviewers often seemed to be looking for reasons to exclude a set of texts rather

than for significant reasons to choose one textbook over another. This conclusion

seems to be supported by the fact that many of the reviewers stated that there were few

significant differences between the pro- grams.

These researchers also attempted to determine if the reviewers followed any patterns in going about their task. They were not able to classify all of the reviewers because many (33%) did not seem to follow any discernible pattern. Although the re- searchers identified four patterns, two pat- terns accounted for almost all of the cases that were classifiable. Of the 67% of the

cases that were classified, the largest num- ber of reviewers (50%) followed a pattern in which they began with the teacher's manuals, since their primary interests were with the skills sequence and lesson plans. After they had assessed the teacher's man- ual, they examined the pupils' books and finally the supplemental materials. These reviewers completely reviewed one pro- gram before starting a second program.

Reviewers who used the second pattern (approximately 13%) began with the pup- il's book. They typically looked at the sto- ries and illustrations and considered the

overall appeal that they thought these fac- tors might have for pupils. They then re- viewed the teacher's manual and con-

cluded their review with the supplemental materials. These reviewers also completely reviewed one program before starting a second.

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

274 THE ELEMENTARY SCHOOL JOURNAL

Only 2% of the reviewers were in- cluded in the third pattern. These review- ers seemed to be very conscious of the cri- teria on the review sheets. They started with the first item on the review sheet and

progressed through a program, examining it in regard to the first criterion. They would then go to the second item on the criteria sheet, and so on, until they were finished with the sheet. Then they would follow the same pattern with a second pro- gram. The order in which they examined the various materials in the program was generally determined by the order of the items on the criteria sheet.

The fourth pattern included only about 1% of the reviewers. This group examined all of the programs that were being con- sidered on one factor at a time-such as

the quality of the illustrations. After eval- uating this factor across all the programs, they would go to a second factor. They believed that the evaluation process was more efficient if they made comparisons across programs from the very beginning of their review. The four patterns depict the variety of practices-some logical and seemingly efficient, and some obviously wasteful-that committees and individuals use to evaluate basals in the absence of ad-

equately defined evaluation procedures. Courtland and her colleagues also asked

the reviewers to identify the most impor- tant factors in their individual evaluations of the basal readers. The areas mentioned

most prominently were: (1) the quality of the stories, (2) the quality of the illustra- tions in the pupils' books, (3) the ease of use and organization of the teacher's man- uals, (4) the relationship of the workbooks to the skills being taught, and (5) the re- lationship of accompanying tests to what was taught in the program. It is important to point out that the researchers found ma- jor discrepancies among the reviewers' def- initions of each of these five factors. For

example, to one reviewer, "quality stories" meant traditional stories that the teacher

had read when she was an elementary stu-

dent, while to another reviewer "quality stories" meant those that related to the

backgrounds and experiences of the pupils in her class.

In the second study, Powell (1986) con- ducted extensive examinations of the ac- tivities of several individuals who had

served on local reading adoption commit- tees. She identified a number of factors

that reviewers perceived to be important influences in choosing a basal program. Powell reports these influences in five ma- jor categories. The first of these cate- gories, physical appearance, was likely as in- fluential as, if not more influential, than any other factor. Committee members' concern with appearance is illustrated in these examples: (1) reviewers disliked the outside cover of one program because both third-grade teachers' editions were the same color, and (2) one reviewer com- plained about the shape of the teacher's edition because it "flopped in half" when she carried it. Reviewers also thought that the teacher's manuals and the pupils' edi- tions needed to look similar to those that were familiar to the teachers. Most teach-

ers wanted manuals to include vocabulary and enrichment sections, but did not men- tion any criteria for the content of these sections. In other words, looks and labels could sell a textbook.

The second category was labeled pilot tryouts. When conducted, pilot tryouts seemed to be one of the most influential factors in reviewers' decisions. Powell

found that committee members generally listened to teachers who had tested read-

ing programs in their classrooms and highly valued their opinions. It is interesting that almost all of the teachers who had piloted programs became proponents of the pro- grams they piloted.

The third category, pedagogy, was char- acterized by the belief among committee members that they selected textbooks based on the district's philosophy or on their personal philosophy of teaching reading. However, when asked to explain

JANUARY 1987

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

EVALUATION AND SELECTION 275

their beliefs or the district's philosophy, committee members often discussed past instructional practices rather than articu- lating a succinct view about an ap- proach(es) to reading instruction. Atti- tudes about previously used textbooks were often mentioned as affecting the review process as well. Reviewers often attached importance to primary-grade teachers' be- liefs about the early levels of a particular program, probably because of the impor- tance generally assigned to reading in- struction in the beginning grades.

The fourth category, publishers, refers to reviewers' perceptions of what influ- enced not only the final decision but also what determined how carefully a reviewer examined a particular textbook program. Most of that influence, it was discovered, was determined by the reputation and im- age of each publishing company and the market strategies that the company em- ployed. Typical comments by reviewers often mirrored the marketing emphases of the publishers: "That was the really diffi- cult series"; "That program gets good test results"; "That was the book that had those ugly covers"; "We wanted a writing pro- gram and it was the one program that had the kids write in complete sentences." In- fluential marketing strategies included providing free items that the district might not otherwise purchase, hosting dinners or wine and cheese receptions, trying to in- fluence key people, and providing in-ser- vice sessions for added exposure to the company. Some companies host summer institutes for extended exposure for their companies and textbook programs to which key school personnel are invited, with all expenses paid. Reviewers reported that publishers' "special events" can influ- ence persons to examine textbooks more closely or to look no further. Yet another perceived influence was the personalities of the publishers' representatives. Sales representatives who knew how to talk to teachers without either "talking down" to them or appearing to be "totally unin-

formed" about reading seemed to have the most success. There was no question that the images of a basal reader program and its publisher are determined to a large ex- tent by the personalities of the sales rep- resentatives.

The fifth category, politics, refers to the effect on the selection process of persons who were highly respected or well liked, as well as the effect of interpersonal rela- tionships among committee members. For example, in one school district, the adop- tion committee chairperson's search for a job in educational publishing may have in- fluenced the committee selection and cri-

teria development. It is interesting that the chairperson was hired by the company whose books were adopted.

With the exception of physical appear- ance, it is not clear from Powell's study which of the five general factors had the greatest influence, not did she intend to determine the relative influence of each of

these factors. The results of Powell's study do, however, further emphasize the com- plexity of the textbook adoption process.

Improving the adoption process

Our suggestions for improving the text- book adoption process start with the prem- ise that most teachers have strong beliefs about reading development and that they know how to teach reading. There was, of course, a range of beliefs about teaching reading among the teachers we inter- viewed. When asked about their teaching procedures, many teachers described the teaching steps they usually followed in us- ing a basal reader.

It is certainly true that many teachers have not kept up to date on new strategies and concepts regarding the teaching of reading. It seems that the time available for committee members to conduct their reviews and textbook evaluations is too

limited to add an in-service program on the teaching of reading. Such information is essential; however, such knowledge needs

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

276 THE ELEMENTARY SCHOOL JOURNAL

to be developed prior to the time text- books are reviewed.

One of the most illuminating findings from our extensive discussions with teach-

ers on adoption committees was the ab- sence of a connection between their strong beliefs about the teaching of reading and their textbook adoption decisions. There was a significant discrepancy between what committee members said and what they ac- tually did.

When we talked with textbook adop- tion committee members, the reasons they gave for the differences between their be- liefs and practices included such comments as, "The evaluation form was given to me to use and what I know about the teaching of reading didn't seem to be included on the sheet"; or, "There wasn't enough time to really look at what I thought was im- portant"; or, "We were told to pick the books that matched our school district ob-

jectives, and that didn't seem to leave any room for what I believe about the teaching of reading."

We believe that teachers need a process that allows them to develop a feeling of "ownership" of the textbooks by gaining control over the adoption process. Those who use the textbooks must establish the criteria that are used to evaluate the books. The review and selection of a basal reader involve evaluation, and evaluation is sim- ply the collection of information on which to base a decision. In the case of textbook

adoption, the decision to be made is quite clear: Which of these books should we se-

lect for use with our pupils? What is not clear is what information reviewers should

collect and how they should evaluate that information.

Our response to this question is ex- pressed in the form of specific recommen- dations, divided into four sections: (1) basic assumptions regarding the adoption pro- cess, (2) selection of those who are to do the reviewing, (3) establishing criteria, and (4) procedures for evaluating and review- ing materials. It is not possible for us to

present all the guidelines for reviewing basal readers because of space limitations. We believe, however, that the recommen- dations that follow include most of the is- sues that need to be considered in text- book evaluation and selection.

Guidelines for basal reader adoption process

Basic assumptions

1. The selection of a reading textbook series should not be considered the same as the adop- tion of the total reading curriculum. Basal reader textbooks should be considered

merely as an aid to instruction. When basal readers are viewed as the total curriculum, they are expected to do too many things, many of which may be unrealistic. When this happens, basal reader publishers in- clude in their series as many labels of as many different things as they can-and good reading instruction gets lost.

2. Basal reading adoptions should be con- ducted by school districts rather than by states. We believe that teachers with complete freedom to choose from all available pro- grams will feel much more in control of their own reading programs. Further- more, research does not indicate any sig- nificant benefits for school districts in state

adoption over nonadoption states. 3. The final decision regarding textbook se-

lection should reside with the committee that

spends the time and energy reviewing the books. All-teacher votes lead to situations in which

teachers who spend no time reviewing bas- als have as much influence on the final de-

cision as those who spend many hours en- gaged in careful study. Furthermore, we believe that administrators and boards of education should follow the recommen-

dations of the adoption committee. Some- times committee recommendations are ov-

erridden because the program selected is more expensive than others. If cost is to be a factor in the final selection of the pro- grams, the adoption committee members should be given this information when they begin their task. The entire adoption pro-

JANUARY 1987

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

EVALUATION AND SELECTION 277

cess is significantly weakened when com- mittee recommendations are not followed.

Selection of reviewers 1. Reviewers should have the respect of other

teachers in the school system. Although this guideline often leads to the selection of more experienced teachers, the critical criterion for selection should be compe- tence in reading instruction. Committee members should be teachers who under-

stand issues and research in the teaching of reading and curriculum development. The onset of a basal reader evaluation pro- cess is not the time to conduct in-service

training for teachers who have not been keeping up with current information on the teaching of reading. Waiting until the adoption process is about to get under way may not leave enough time to affect the selection.

2. We do not recommend in-service training in the teaching of reading, but we do strongly recommend training for reviewers in the review and evaluation of reading textbooks. Unfor- tunately, classroom teachers typically do not receive adequate training in textbook evaluation and adoption. Virtually no preservice preparation in this area is in- cluded in present education programs; much of what reading teachers do know in this area appears to be the result of pre- vious experience on adoption committees. The lack of systematic training not only casts considerable doubt on the reliability and validity of procedures that teachers use, but also likely limits the extent to which they are able to participate effec- tively in the entire textbook adoption pro- cess. Our experiences have demonstrated, however, that, with adequate (i.e., system- atic, well-planned, specific) training, most teachers can evaluate basal reader text- books.

Establishing criteria

1. The adoption committee's most important task is the determination of the basal reading series factors to be used in evaluating the pro-

grams. That is, the committee members have to decide, based on the school dis- trict's philosophy and goals, their own be- liefs about the teaching of reading, their knowledge about effective textbooks and reading instruction, and their classroom experience, which factors constitute a "good" reading text for their district. Be- cause most committees have developed lists that are certainly too long to yield an ad- equate review, the committee should con- sider dividing the list into three categories: (1) factors that are most crucial to having a top-quality program, (2) factors that are desirable but not essential, and (3) factors that are important to consider if the fac- tors in the first two categories are equiv- alent for several programs. Ranking the factors allows committees to function more

efficiently by eliminating any programs that do not meet the essential criteria. The

committee can then spend more time with a smaller number of programs-those that include the essential factors.

2. As the selection criteria are established,

the committee must agree on the meaning of each factor. One of the best ways to accomplish this is to share examples of the criteria from various reading textbooks. These exam- ples should be selected from books other than those being considered in the evalu- ation. Bad examples as well as good ex- amples are useful. Committee discussions of these examples are almost certain to in- crease the reliability and validity of the evaluation when the criteria are actually used in the review process. The discussions should focus on why particular examples are good or poor. These examples can then be used as standards against which each reading series is evaluated.

Procedures in reviewing and evaluating basal readers

1. Committees must be provided an ade- quate amount of time to conduct thorough eval- uations of reading textbooks. The lack of time to conduct adequate reviews of the texts was the most often expressed concern of

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

278 THE ELEMENTARY SCHOOL JOURNAL

textbook evaluation committees. Text- book evaluation must not be rushed. We believe that it is essential that teachers be

given release time to review textbooks. The recommendations that we have presented in this article will certainly take more time than is typical of the adoption committee reviews we have observed, and increased time will almost certainly increase the cost of the adoption process. Perhaps these in- creased costs could be met if a certain per- centage (such as 2%) of a school's or a state's total textbook budget could be set aside for the review process. This might also provide the needed funds to release teachers to evaluate textbooks.

All too often we found teachers who

spent their entire weekends reviewing books, who reviewed books during lunch- room duty, or who spent late evenings trying to get through the books they were to review. Those conditions can only lead to hurried reviews and an attitude on the

part of committee members that the school district does not view textbook evaluation as an important task. Committee members also need time to develop carefully and rank the criteria for their evaluations; to discuss examples of each criterion; and to develop, test, and revise evaluation pro- cedures.

2. Committees should be organized in ways other than by grade level. When committees are organized by grade levels, there is little opportunity to review the development of reading across program levels, and the overall evaluation of a basal program gets lost in a series of grade-by-grade evalua- tions.

3. Procedures used to evaluate basal pro- grams should be tested before the actual eval- uation takes place. Even when these evalu- ation procedures are nothing more than criteria sheets with a number rating sys- tem, each committee member should test

them with a "try-out" program that the school district is not considering, or with some previously published program that the district has not used. All committee

members should evaluate the same try-out program and discuss the evaluations and any problems they encountered. Revisions in the evaluation procedure will almost certainly be needed. Even when such re- visions are not made, and even when com- mittee members have used the same eval-

uation procedure in previous adoptions, the trial evaluation and discussion will en- able committee members to use the same

procedure and will increase the validity and reliability of the ensuing evaluation.

4. Whatever evaluation procedures are used, committee members must do more than make a

check mark. Evaluation procedures should require committee members to provide written examples of the features they like and dislike in the basal series. Such ex- amples need not be extensive. Even simple page numbers, single examples, brief out- lines of effective or ineffective practices, and other such comments can increase the validity of the reviews. Furthermore, spe- cific strengths and weaknesses should be identified and discussed with peers at com- mittee meetings in which the books being considered are available for further ex- amination.

5. Any person who wishes to address the entire adoption committee or any individual committee members should be allowed to do so.

During the evaluation, those who wish to speak about a particular program should have an opportunity to do so-either orally or in writing. The goal is to allow all per- sons concerned with the adoption an op- portunity to make their views known in a comfortable situation. The committee

should urge those presenting their views to discuss specific examples of the factors of basals that they are discussing.

It is also important that all of the pro- grams that are being considered are avail- able to anyone who wishes to review them. This will usually necessitate some sort of public display and multiple copies of all the programs.

6. Reading adoption committees need to consider carefully how much and what contact

JANUARY 1987

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

EVALUATION AND SELECTION 279

to have with publishers' representatives. We do not believe that an adversary relationship must necessarily exist between publishing companies and educators. Nor do we ad- vocate strict guidelines governing inter- actions between representatives of pub- lishers and committee members. We see

no reason why the educational publishing industry should be subjected to any greater-or any less stringent-regulation than all other commercial enterprises with which public schools are involved.

It would be naive, however, to discount the potentially powerful influence that representatives of publishing companies can and do exert on committee members.

We have seen many adoptions determined by a publisher's marketing efforts rather than by the quality of the basal program.

We believe, however, that this influ- ence is often exaggerated by the manner in which districts typically evaluate text- books. It is easy to understand how com- mittee members can be influenced by dy- namic salespersons and presentations, particularly if those contacts occur during a time-consuming, pressure-filled adoption process characterized more by a lack of time and expertise than by adequate re- sources. It appears that educators are often too willing-because of apathy or perhaps merely through default-to allow publish- ers to "save them the trouble" of thor-

oughly examining all textbooks. Thus, we believe that undue influence can be- re-

duced by implementing many of the rec- ommendations discussed here. More time

and a greater degree of competency in evaluating texts will allow committees to make decisions based primarily on the characteristics of textbooks.

Publishing companies and their rep- resentatives can be a valuable resource to

adoption committees. Well-focused, timely sales presentations not only save time for reviewers but also clarify questions that arise before, during, or after review. The key to effective use of publishers is the abil- ity of committee members to make certain

that the presentations respond to the needs of the committee.

7. Pilot studies are useful if they are care- fully controlled. Pilot studies should be the responsibility of the committee, not the publishers. If any program under consid- eration is to be piloted, then all programs should be piloted. Pilot studies need not last for an entire year; teachers could be asked to try out only one or several units from a particular series. A focused test of several units is often adequate for a teacher to determine the strengths and weaknesses of a particular program. It is usually best if a teacher tries more than one of the pro- grams that are being considered for adop- tion, enabling the teacher to compare the programs with the same children in the same year. Try-out teachers should be se- lected carefully. They should be experi- enced teachers from a range of schools in the district. The pilot teachers should not select the programs they are to pilot; rather, programs should be assigned ran- domly to teachers. It is not uncommon for pilot teachers to ask to try out programs with which they are familiar and about which they have already formed favorable impressions. Finally, pilot teachers should evaluate the programs using the same cri- teria, so their evaluations will be compa- rable. This does not, of course, mean that pilot teachers cannot also provide addi- tional information and insights about the programs or try out programs they like.

8. When the committee has completed its work, a report of the committee's evaluation pro- cedures and findings should be made public. The report should include a description of how the committee went about its task and

the specific reasons that it ranked the pro- grams as it did. Examples of features of each program that were strong and weak should be included. The report should be sent to each publisher. Publishers will make changes in their books only if they rec- ognize that adoption committees find spe- cific features of their programs unattrac- tive.

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

280 THE ELEMENTARY SCHOOL JOURNAL

Conclusion

We believe that basal reader evaluation and

selection can be an important means of im- proving reading instruction. The recom- mendations described in this article are in-

tended to guide educators toward that type of influence. Admittedly, it is speculative to assert that improved basals will result from improved adoption processes and that improved reading instruction will result, in turn, from improved basals. The logic be- hind these assertions seems compelling and attractive, but the evidence necessary to support this view is, nevertheless, absent. Relatively little is known about the way textbook adoption is presently conducted throughout the United States, and unfor- tunately what is known is derived from re- search findings limited in both quantity and scope and findings that may not be appli- cable or generalizable to many adoption processes.

What we do feel certain about, how- ever, is that improved basal readers are likely to be developed if textbook adoption procedures improve. The major improve- ment in basal readers will be the devel-

opment of programs that are more con- sistent with individual teachers' beliefs

about the teaching of reading and about the manner in which instructional mate-

rials should be employed during reading instruction. To achieve this consistency, it seems imperative that basal reader evalu- ation criteria and procedures be developed and applied by those professionals most di- rectly responsible for the teaching of read- ing.

In closing, we would like to note that we are not among those who subscribe to the belief suggested by English (1980) that textbooks are "mediocre" products cre- ated by the interactions, compromises, competing forces, and interests that seem to characterize many adoption processes. It is true that much of what we are begin- ning to discover and to understand about textbook adoption is less than encourag- ing. Yet, we prefer to assume a more pos-

itive posture. There is a need for improve- ment in the way educators presently go about the task of selecting basal readers, and there is much about basals that can be

improved. By and large, however, Amer- ican educators are fortunate to have avail-

able such a large quantity of high-quality textbooks and other materials to support reading curricula. Aside from any prob- lems that may beset existing state and local processes, textbook adoption represents choice. As long as teachers continue to have a choice, the promise of and potential for effective reading instruction will exist. Our hope is that these suggestions will contrib- ute to those choices being more rational, valid, and informed.

References

Beck, M. (1985). Overview and other views. Book Research Quarterly, 1, 36-46.

Courtland, M. C., Farr, R., Harris, P., Tarr, J., & Treece, L. (1983). A case study of the In- diana state reading textbook adoption process. Bloomington, IN: Center for Reading and Language Studies.

Crane, B. (1975). The "California effect" on textbook adoptions. Educational Leadership, 32, 283-285.

Davis, O. L., Frymier, J. R., & Clinefelter, D. (1977, April). Curriculum materials used by eleven-year-old pupils: An analysis using the An- nehurst Curriculum Classification System. Pa- per presented at the annual meeting of the American Educational Research Associa- tion, San Francisco.

Dixon, C. N. (1979). Selection and use of in- structional materials. In R. C. Calfee & P. A.

Drum (Eds.), Teaching reading in compensa- tory classes (pp. 104-113). Newark, DE: In- ternational Reading Association.

Durkin, D. (1984). Is there a match between what elementary teachers do and what el- ementary manuals recommend? Reading Teacher, 37, 734-744.

Educational Products Information Exchange (EPIE) Institute. (1977). Report on a national study of the nature and the quality of instruc- tional materials most used by teachers and learn- ers (Rep. No. 76). New York: EPIE.

Educational Research Service, Inc. L. H. Kun- der (researcher). (1976). Procedures for text-

JANUARY 1987

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

EVALUATION AND SELECTION 281

book and instructional materials selection. Ar-

lington, VA: Educational Research Service. English, R. (1980). The politics of textbook

adoption. Phi Delta Kappan, 62, 275-278. Farr, R., & Tulley, M. A. (1985). Do adoption

committees perpetuate mediocre text- books? Phi Delta Kappan, 66, 467-471.

Farr, R., Tulley, M. A., & Rayford, L. (in press). Selecting basal readers: A comparison of school districts in adoption and nonadop- tion states. Journal of Research and Develop- ment in Education.

Follett, R. (1985). The school textbook adop- tion process. Book Research Quarterly, 1, 19- 23.

Goldstein, P. (1978). Changing the American schoolbook. Lexington, MA: Heath.

Institute for Educational Development. (1969). Selection of educational materials in the United States public schools. New York: Institute for Educational Development.

Keith, S. (1981). Politics of textbook selection (Re- search Rep. No. 81-AT). Stanford, CA: Stanford University School of Education, Institute for Research on School Finance and Governance.

Kreiner, R. P. (1979). A status study of existing procedures for textbook selection in Ohio. Un- published doctoral dissertation, University of Akron. (University Microfilms No. 80- 19115).

Mason, J., & Osborn, J. (1982). Is there a match between what elementary teachers do and what basal reader manuals recommend? (Tech. Rep. No. 44). Urbana, IL: Center for the Study of Reading.

Powell, D. A. (1986). Retrospective case studies of individual and group decision making in dis- trict-level elementary reading textbook selection. Unpublished doctoral dissertation, Indiana University, Bloomington, IN.

Squire, J. R. (1985). Textbooks to the fore- front. Book Research Quarterly, 1, 12-18.

Stewart, P. L. (1980). The selection of basal reading textbooks: A study of procedures and evaluative criteria (Doctoral disserta- tion, University of Iowa). Dissertation Ab- stracts International, 41, 1393-A.

Tulley, M. A. (1985). A descriptive study of the intents of selected state level textbook adop- tion processes. Educational Evaluation and Policy Analysis, 7, 289-308.

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:17:56 UTC�������������

All use subject to https://about.jstor.org/terms

How to Evaluate a Reading Program

Author(s): Sidney J. Rauch

Source: The Reading Teacher , Dec., 1970, Vol. 24, No. 3 (Dec., 1970), pp. 244-250

Published by: International Literacy Association and Wiley

Stable URL: https://www.jstor.org/stable/20196483

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at https://about.jstor.org/terms

Wiley and International Literacy Association are collaborating with JSTOR to digitize, preserve and extend access to The Reading Teacher

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:18:54 UTC�������������

All use subject to https://about.jstor.org/terms

Sidney Rauch is a Professor of Reading and Education at Hofstra University, Hempstead, Long Island, New York.

How to evaluate a reading program SIDNEY J. RAUCH

the purpose of evaluation is to take a comprehensive, unbiased and cooperative look at the reading program and to decide what modifications or changes, if any, should be made to improve the program. Evaluation involves value judgments. It is not a care fully controlled research study. Recommendations for improvement of the program must consider not only what should be done, but what can be done. Hemphill (1969) places evaluation studies within the framework of decision making rather than research. The administrator, after taking all facts from the evaluation study into consideration, then must weigh their relationship to its effect upon the community, staff, and tax structure. Hemphill (1969) lists six characteristics of school evaluations:

1. The problem is almost completely determined by the situation in which the study is conducted. Many people may be involved in its definition and, because of its complexity, the problem initially is difficult to define.

2. Precise hypotheses usually cannot be generated; rather, the task becomes one of testing generalizations from a variety of research studies, some of which are basically contradictory. There are many gaps which, in the absence of verified knowl edge, must be filled by reliance on judgment and experience.

3. Value judgments are made explicit in the selection and the definition of the problem as well as in the development and implementation of the procedures of the study.

4. The study is unique to a situation and seldom can be replicated, even approximately.

5. The data to be collected are heavily influenced if not deter mined by feasibility. Choices, when possible, reflect value judg ments of decision makers or of those who set policy. There are often large differences between data for which the collection is feasible and data which are of most value to the decision mak ers.

6. Only superficial control of a multitude of variables important to interpretation of results is possible. Randomization to eliminate the systematic effects of these variables is extremely difficult or impractical to accomplish.

Four major steps are necessary in the evaluation of school 244

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:18:54 UTC�������������

All use subject to https://about.jstor.org/terms

rauch: How to evaluate a reading program 245

reading programs. These are clarification of the roles of the evalu ator, collection of data, analysis of data, and reporting of data. "A Checklist for the Evaluation of Reading Programs," (Rauch, 1968) is used as an overall guide. The five major categories cov ered in this checklist are the reading program, the administrative and supervisory staff, the teaching staff, the pupils, and the parents.

CLARIFICATION OF THE ROLES OF THE EVALUATORS

Since value judgments are involved, it is recommended that the evaluation be conducted by a team of two to four reading specialists from different parts of the country so that the various backgrounds and points of view can be brought to the program. This type of representative team is preferable to the selection of a number of specialists from a single university or college whose philosophy or viewpoints may be too similar.

It is strongly recommended that the team meet with admin istrators and teacher representatives prior to the actual evaluation to explain the purposes and procedures, and to answer any ques tions. It is most important that the anxieties of teachers (partic ularly as to classroom observations) be allayed. The evaluation concentrates on the reading program; it is not an evaluation of individual teachers. No names or ratings of teachers are to appear in the final report. The purpose of the evaluation is a constructive one, i.e., to make recommendations for improvement. It is not a criticism of individual teachers, though strengths and weaknesses of reading techniques will be listed. If the evaluation is to be successful, it must have the confidence and cooperation of all concerned. Any team approaching the program with the intent to downgrade it or pick it apart is doomed to failure. Thus, the first and probably the most important step for the evaluation team is to gain the confidence and support of the teachers. This can only be done by clarifying all objectives and procedures before the evaluation actually begins.

COLLECTION OF DATA

Despite the various criticisms directed at standardized tests, test scores still remain an important part of reading evaluation. Robinson and Rauch (1965) describe the merits of standardized tests as follows:

Like other tools of teaching, standardized tests can be appraised in terms of both their form and their results. In their form?that is, their structure and operation?these tests have very important ad vantages: 1] their content is usually determined by careful design; 2] there are often parallel forms for comparison; 3] they permit

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:18:54 UTC�������������

All use subject to https://about.jstor.org/terms

246 THE READING TEACHER Volume 24, No. 3 December 1970

many children to be treated simultaneously; and 4] they are ob jective in administering and scoring. In many schools, standard ized tests are the first step in identifying those students who are below grade level and who are in need of further diagnosis. They are particularly useful in measuring the wide range of reading levels in a class, school or school system. They also provide stand ards for comparing students on a nationwide basis. Standardized tests make a valuable contribution to modern education by dem onstrating rather clearly that children differ. They provide stand ards for making improvements in school programs in the areas of curriculum, school and classroom organization, and methods and materials of instruction.

A summary of cautions to be exercised in the use of stand ardized tests has been listed by Harmer (1967):

1. Test users should keep in mind that the test score is simply the result of performance on a particular day, at a particular time, and in a particular testing environment.

2. The diagnosis of reading achievement through the use of stand ardized tests may be fallacious unless carelessness and atti tudes toward taking tests on the part of the students are controlled.

3. The grade score on a standardized reading achievement test should not be thought of as precise indication of overall reading achievement; rather, it should be thought of as a measure of reading ability on that test at a particular point in time.

4. Teachers should keep in mind that grade scores on standard ized tests are derived by interpolating scores between grade levels or by extrapolating scores from one grade level to an other; thus, grade scores cannot be treated as empirically ob tained indications of month-to-month progress.

If these cautions are observed, and if schools begin develop ing their own local norms as the tests are used over a period of time, then the advantages of standardized tests will far outweigh their limitations. This point is reinforced by the American Psy chological Association in its Standards for Educational and Psycho logical Tests and Manuals: "Local norms are more important for

many uses of tests than are published norms." In addition to the evaluation of the results of standardized

reading tests, the results of individual and group IQ tests must be taken into consideration. Reading is primarily a mental process, and reading test results must be evaluated in terms of intellectual potential or capacity. For example, a different interpretation must be applied to the performance of two sixth grade classes whose median reading score places them both in the 60th percentile on a statewide basis, but one class has a median IQ score of 106 and the other a median IQ score of 118. Obviously, the latter group is not performing up to potential and the reasons for this discrepancy must be examined.

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:18:54 UTC�������������

All use subject to https://about.jstor.org/terms

rauch: How to evaluate a reading program 247

Observation of teachers is the heart of reading evaluation. To evaluate, without observing teacher performance and pupil behav ior, is like reviewing a book without reading it. A representative daily schedule of the evaluator follows:

9:00 A.M.? 9:30 A.M. Observation of a first grade teacher. 9:30 A.M.?10:00 A.M. Conference with teacher following ob

servation. (Class is covered with by another teacher, administrator or substitute. )

10:00 A.M.?10:30 A.M. Observation of a third grade teacher. 10:30 A.M.?11:00 A.M. Conference with teacher following ob

servation. 11:00 A.M.?12 NOON Conference with one or two of the

following : administrator, reading specialist, librarian, school psycholo gist, school nurse, social worker, etc., (anyone connected with reading pro gram).

1:00 P.M.? 1:30 P.M. Observation of a sixth grade teacher. 1:30 P.M.? 2:00 P.M. Conference with teacher following ob

servation. 2:00 P.M.? 3:00 P.M. Evaluators confer, comparing notes

on observations and results of indi vidual conferences.

During the conference period following classroom observa tion, the topics covered should include: information about the class (e.g., reading range, potential, socio-economic background, interests), teacher's background and experience, her philosophy about reading instruction, and opinion about materials. It has been found most helpful to save the last five to ten minutes for the teacher's response to this type of question: "Assuming that budget problems were secondary, what recommendations would you make to improve the reading program in this school?"

Interviews with administrative personnel give them an op portunity to express their opinions about the reading program to the evaluator. In addition, each administrator is requested to sub

mit a brief statement in response to these three questions: 1] What are the strengths of the program? 2] What are the weak nesses of the program? and 3] What recommendations would you make to improve the program?

The author has used five opinion surveys as part of his eval uation of elementary reading programs. These are:

An Opinion Survey of Parents of Kindergarten Children, An Opinion Survey of Parents of Children, Grades 1-6, An Opinion Survey of Kindergarten Teachers,

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:18:54 UTC�������������

All use subject to https://about.jstor.org/terms

248 THE READING TEACHER Volume 24, No. 3 December 1970

An Opinion Survey of Teachers, Grades 1-6, and Grade Level Observation Form (to be used by reading personnel).

The survey for teachers is included as an example.

An Opinion Survey of Teachers (Grades 1-6) Needs To

Strong Be Unable Program in General Component Good Fair Improved To Answer Basal Reading Program - -:? - - _ Word Recognition - - - - _ Comprehension - - - - - Critical Reading - -~ - - - Literature Program - - - - - Individualized Reading - - - - - Content Area Reading - - - - - Work and Study Habits - -- - - Supplementary Materials ?- - - - -

Meeting Needs of Children Meeting Individual Needs - - - - - Superior Reader - - - - - Average Reader - - - - - Disadvantaged Reader - - - - - Remedial Reader- - - Present Organizational Pattern - - - - - Diagnostic Services - - - - - Corrective Reading Program - - - - - Summer School Program - - - - - Supplementary Programs - - - - -

Helps to Classroom Teachers Administration - - - - - Consultants - - - - - In-Service Courses - - - - - School Meetings and Workshops - - - - - Materials - - - - -

Atmosphere Conducive to Learning Freedom to Develop Own Program - - - - - Development of Children's Love

of Reading - - - - - Competent Reading Leadership - - - - - Parent-Teacher Relationship - - - - -

It is important that all teachers be urged to complete their "Opinion Surveys" and turn them in to the evaluators. Names are not required. One reading program was harshly criticized by some members of the school board because only thirty-two out of sixty teachers had submitted their forms. The board's reaction, and rightly so, was "How can we have confidence in these teachers if they are not concerned enough to fill out a form which will help us in our evaluation?"

In summarizing the results of the parent opinion survey, make sure that two or more parents are involved in tabulation. Unless this is done, it is always possible that some parent (who is not too happy with the program) will question the accuracy of the findings.

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:18:54 UTC�������������

All use subject to https://about.jstor.org/terms

rauch: How to evaluate a reading program 249

ANALYSIS OF DATA

It was previously noted that one hour at the end of each eval uation day (2:00 P.M.?3:00 P.M.) should be set aside for the evaluators to compare notes and opinions. These sessions also enable the evaluators to gather and classify data for the final report.

Each evaluator prepares his own individual report based on interviews, classroom observations, and analysis of opinion sur veys. The measurement specialist on the evaluation team is re sponsible for the analysis of standardized test scores. The director of the study analyzes the reports of each member of the evaluation team and prepares the final report.

In evaluating classroom performance, the following "Char acteristics of a Good Reading Lesson" are used as a guide: 1. Teacher has a definite goal or purpose for lesson and that

purpose is evident to students. 2. Lesson is planned, systematic, yet flexible according to dy

namics of classroom situation. 3. Classroom atmosphere is a pleasant, attractive and optimistic

one.

4. Attention is paid to individual differences. 5. Rapport between teachers and students is evident. 6. Teacher is diagnosing as she is teaching. 7. There is readiness for the lesson. 8. Pupils are motivated. 9. Materials are varied (basais, library books, workbooks, kits,

mimeographed materials, etc.). 10. Full use is made of audio-visual aids. 11. Questions are varied to check different levels of compre

hension. 12. Material is at appropriate level for students. 13. Teacher is obviously aware of such levels as "instructional,"

"independent," and "frustration." 14. Meaningful oral reading activities are used to check compre

hension. 15. Pupils have been trained in self-direction (i.e., go from one

activity to another without disturbing teacher). 16. All children are productively involved with some aspect of

reading. 17. Use is made of classroom and school libraries. 18. There is application of basic reading skills to content areas. 19. Efficient record keeping is done by teacher and students. 20. Teacher has sense of perspective and humor. 21. There is evidence of review and relationship to previously

learned material. 22. There are follow-up or enrichment activities.

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:18:54 UTC�������������

All use subject to https://about.jstor.org/terms

250 THE READING TEACHER Volume 24, No. 3 December 1970

REPORTING OF DATA

Both an oral and written report of the findings of the evalu ation team are presented to the administrators' council, teacher representatives, and the school board. The school board, if it wishes, can then hold an open meeting for all parents and ques tions can be asked of the evaluators. Copies of the written report should be made available to all teachers. Three or four copies can be placed in the teachers' professional library or in a special sec tion of the school library.

A representative "Table of Contents" of the final written evaluation might follow this format:

A. Introduction (How the study was initiated) B. Description of the School-Community C. Characteristics of a Good Reading Program D. Evaluation Procedures (How the study was conducted) E. Strengths of the Program F. Weaknesses of the Program G. Recommendations (The heart of the report) H. Analysis of standardized test scores I. Results of Opinion Surveys (Appendix)

SUMMARY STATEMENT

There is a need for constant evaluation of reading programs. However, all concerned must participate. Teachers must have confidence in the evaluators, and the evaluators must recognize the many day-by-day problems faced by the average teacher. De spite the importance of standardized test results, the heart of the evaluation is classroom performance. Recommendations must be realistic. They must consider not only what should be done, but what can be done within a specific school-community environment.

In most instances, evaluation has a positive effect on the read ing program. It compels administrators and teachers to take a closer look at their methods, their materials, and their children? and this close examination generally results in progress.

REFERENCES Harmer, W. R. The selection and use of survey reading achievement tests. In

T. C. Barrett (Ed.) The evaluation of children's reading achievement. Perspectives in Reading, 1967, 8, 53-64.

Hemphill, J. K. The relationships between research and evaluation studies. In R. W. Tyler (Ed.) Educational evaluation: new roles, new means. The Sixty-eighth Yearbook of the National Society for the Study of Education, Part II, 1969, 189-220.

Rauch, S. J. A checklist for the evaluation of reading programs. The Reading Teacher, 1968, 21, 519-522.

Robinson, H. A., and Rauch, S. J. Guiding the reading program. Chicago: Science Research Associates, 1965.

This content downloaded from �������������131.95.109.31 on Mon, 21 Dec 2020 14:18:54 UTC�������������

All use subject to https://about.jstor.org/terms