Reading (Gr. 1-3)
|Disaggregated Reliability and Validity Data|
|Administration & Scoring Time||23 - 43 Minutes|
|Scoring Key||Computer Scored|
|Benchmarks / Norms||Yes|
|Cost||Technology, Human Resources, and Accommodations for Special Needs||Service and Support||Purpose and Other Implementation Information||Usage and Reporting|
Cost per student for 1-3: $3.80 (materials only); $6.04 materials plus Online Score Entry and Reporting System (based on a class of 25 students).
PALS 1-3 Teacher Set includes everything a new teacher needs to screen 25 students in fall and spring and an Assessment Training CD. Includes the following: an Administration and Scoring Guide, fall and spring Class Summary Sheets, a Student Packet, a Teacher Packet, fall and spring Student Spelling Sheets (masters), and 25 fall and spring Student Summary Sheets.
$6.10 per Online PALS Teacher Account.
$2.00 per Online PALS Student Account.
Internet access is required for full use of product services.
Testers will require less than 1 hour of training.
Developer strongly recommends that teachers screen their own students, though others (e.g., paraprofessionals) may as well administer the test.
Training manuals should provide all implementation information.
The Phonological Awareness Literacy Screening (PALS) is a criterion-referenced screening, diagnostic, and progress monitoring tool. PALS consists of two instruments (PALS-K and PALS 1-3) that measure young children’s knowledge of important literacy fundamentals: phonological awareness, alphabet awareness, letter sound knowledge, spelling, concept of word, word recognition in isolation, and oral passage reading. The major purpose of PALS is to identify students who may be at risk for reading difficulties and who need additional reading instruction beyond what is provided to typically developing readers. The second purpose of PALS is to provide teachers with explicit diagnostic information about what students know and need to know about the fundamental components of literacy that may be used to target instruction to meet students’ needs. The third purpose of PALS is to monitor students’ progress and determine the effectiveness of instruction or intervention.
PALS 1-3 uses a gated three-tiered approach that differentiates the administration of the assessment based on students’ scores. The first tier (or Entry Level) measures a student’s general level of word knowledge measured by a word recognition and spelling task. If a student does not meet the Entry Level benchmark for his or her grade level further assessment is required. Level A assesses the accuracy, fluency, rate, and comprehension of a student’s oral reading in context. Level B assesses emergent and beginning reading essentials in alphabetic knowledge and concept of word. If Level B benchmarks are not met, students are routed to Level C for a more in-depth evaluation of phonemic awareness skills, including blending and segmenting speech sounds.
Raw scores from each PALS 1-3 Entry Level task are summed to produce the Entry Level Summed Score; this consists of Word Recognition in Isolation at the appropriate grade level (20 points max), and Spelling (maximum varies across grades 1, 2, and 3). To compute AUC using ROC analysis, we converted Entry Level Summed Scores to simple standard scores (z-scores).
Predicting performance on the Stanford-9 Achievement Test (Reading)
Predicting Risk on Spring 3rd Grade Standards of Learning (SOL) Reading
|False Positive Rate||0.298||0.092|
|False Negative Rate||0.078||0.592|
|Positive Predictive Power||0.336||0.802|
|Negative Predictive Power||0.982||0.628|
|Overall Classification Rate||0.733||0.670|
|Cut Point:||The 23rd percentile on the Stanford-9 Achievement Test (Reading) was used as the cut score as it represents the top band in the 3rd stanine, which can be used as a gross indicator of relative risk.||Virginia’s SOL for Reading results in scaled scores ranging from 0 – 600, with 400 representing pass, or ‘proficient’ (used in this classification analysis), while a score of 500 represents ‘advanced pass’.|
Description of study sample:
- Number of States: 1
- Size: 1,088
- 52.7% Male
- 47.3% Female
- 50.1% White, Non-Hispanic
- 45.5% Black, Non-Hispanic
- 1.0% Hispanic
- 0.1% American Indian/Alaska Native
- 1.6% Asian/Pacific Islander
- 1.2% Other
- 5.2% speech/language impairment
- 1.2% learning disability
- Language proficiency status: 2.1% received ESL services
Cross Validation Study Description of study sample:
- Number of States: 1 (Virginia)
- Size: 63,626
- 51.0% Male
- 49.0% Female
- SES: 37.2% (median) eligible for free or reduced-price lunch
- 56.8% White, Non-Hispanic
- 26.9% Black, Non-Hispanic
- 8.8% Hispanic
- 0.3% American Indian/Alaska Native
- 4.1% Asian/Pacific Islander
- 3.1% Other
- 5.0% Speech/Language Impairments
- 2.3% Learning Disability
- 1.8% Developmental Delay
- 1.3% Other Health Impairment
- <1.0% Other Disability
- Language proficiency status: 6.0% received ESL services
|Type of Reliability||Age or Grade||n (range)||Coefficient|
|Cronbach’s alpha: entry level tasks||1||6,525 – 14,741||0.66 – 0.96||0.81|
|Cronbach’s alpha: Word recognition in isolation||K – 3||25 – 617||0.81 – 0.96||0.91|
|Cronbach’s alpha: Word recognition in isolation||K – 3||4,668, 4,541, and 4,387||0.92 – 0.93||0.92|
|Cronbach’s alpha: Spelling||K – 3||0.86 – 0.92||0.90|
|Inter-rater: Word Recognition||K – 3||45 – 63||0.98 – 0.99||0.98|
|Inter-rater: Spelling||K – 3||130 – 375||0.99 – 0.99||0.99|
|Inter-rater: Oral Reading in Context||K – 3||478||0.94 – 0.98||0.96|
|Type of Validity||Grade||Test or Criterion||n (range)||
|Predictive||Grade 1, 2||Stanford-9 Total Reading||739 – 766||0.73, 0.63|
|Predictive||Grade 3||Virginia SOL||277||0.60|
|Concurrent||Grade 1||Qualitative Reading Inventory (QRI-II)||146||0.73|
|Concurrent||Grades 1 – 3||QRI-II||679||0.73-0.90|
|Concurrent||Grades 1 – 3||Developmental Reading Assessment (DRA||179 (instructional) 96 (independent)||0.82 0.81|
|Concurrent||Grade 1||California Achievement Test (CAT/5):Total Reading Scaled Score||195||0.75 (entry level) 0.70 (spelling)|
|Predictive||Grade 1||Fall 2007 2nd grade Entry Level Summed Score||0.792|
|Predictive||Grade 1||Fall 2007 2nd grade Entry Level Summed Score||0.822|
|Predictive||Grade 2||Fall 2007 3rd grade Entry Level Summed Score||0.810|
|Predictive||Grade 2||Fall 2007 3rd grade Entry Level Summed Score||0.846|
|Construct||As one piece of evidence in support of the construct validity of PALS 1-3, we conduct Principal Component Analyses on the tasks that make up the PALS 1-3 Entry Level Summed Score each year, based on statewide data. These analyses consistently yield a single factor (eigenvalue greater than 1.0) that accounts for more than three quarters of the variance in these scores.|
|Content||The content validity is supported by the process of item development (described in the Technical Reference, Section III, pp. 11 – 19). The guiding principles underlying this process were that (a) tasks and items were a representative sample of tasks from other early literacy screening instruments, (b) items had a history of use in phonological awareness and early literacy research, and (c) items were aligned with Virginia’s Standards of Learning for English (Reading). Each of these principles was further supported by the process of review by an Advisory Panel of literacy professionals from across Virginia, and by an External Review Panel, which consisted of nationally recognized experts in the field of reading, communication sciences, or psychology.|
Disaggregated Reliability, Validity, and Classification Data for Diverse Populations
|Classification Accuracy in Predicting Proficiency on the Stanford 9: By Race/Ethnicity|
n = 498
|False Positive Rate||0.367||0.235|
|False Negative Rate||0.054||0.132|
|Positive Predictive Power||0.369||0.284|
|Negative Predictive Power||0.981||0.982|
|Overall Classification Rate||0.691||0.774|
|Cut Points||23rd percentile||23rd percentile|
|Type of Reliability||Grade||Coefficient|
|Summed Score Reliability (internal consistency)||Across 4 levels (quartiles) of school-level SES||K, 1||0.73 – 0.85||0.78|
|Summed Score Reliability (internal consistency)||Across race/ethnicity groups||K, 1||0.73 – 0.88||0.82|