Acuity
Mathematics
Rating Summary
Classification Accuracy  

Generalizability  Moderate High  
Reliability  
Validity  
Disaggregated Reliability and Validity Data  
Efficiency  
Administration  Group  
Administration & Scoring Time  50 Minutes  
Scoring Key  Computer Scored  
Benchmarks / Norms  Yes 
Cost  Technology, Human Resources, and Accommodations for Special Needs  Service and Support  Purpose and Other Implementation Information  Usage and Reporting 

Annual cost per student: In additional to the subscription price, a onetime set up fee and one day of professional development are required in the first year only, for a total of $5,200.

Internet access is required for full use of product services. Testers will require less than one hour of training. Paraprofessionals can administer the test. Accommodations: 
Tom Moellering Phone: 6307894574 A local point of contact, high quality professional development staff, and tiered technical support for all system users is included in the subscription price. Ongoing technical support is provided though a call center, email, a 24x7 online knowledge database, and a 24x7 online community of other users to share tips and best practices. 
Acuity is an online assessment solution designed to improve student achievement and learning. Available in Math and ELA for students in grades 38, Acuity provides three predictive screening assessments within each student and grade. 
Acuity assessments can be administered within a single 45 minute classroom period in a variety of modes, including online, paper/pencil, or using clicker devices. Scoring is done automatically when scores are submitted online through scan and score or online administration. The following scores are available: raw, standard, percentile, IRTbased, equated, developmental benchmarks, subscale/subtest, error analysis, and predicted performance level on state summative test. 
Classification Accuracy
Classification Accuracy in Predicting Proficiency Level on State Summative NCLB Assessment  

MO, Form A 
Grade 3 n = 3,576 
Grade 4 n = 3,958 
Grade 5 n = 3,510 
Grade 6 n = 3,295 
Grade 7 n = 2,481 
Grade 8 n = 2,439 
False Positive Rate  0.07  0.05  0.07  0.06  0.08  0.14 
False Negative Rate  0.42  0.38  0.44  0.38  0.36  0.35 
Sensitivity  0.58  0.62  0.57  0.62  0.64  0.65 
Specificity  0.93  0.95  0.93  0.94  0.92  0.86 
Positive Predictive Power  0.17  0.44  0.10  0.48  0.53  0.45 
Negative Predictive Power  0.99  0.98  0.99  0.97  0.95  0.93 
Overall Classification Rate  0.92  0.93  0.92  0.92  0.88  0.83 
AUC (ROC)  0.91  0.94  0.90  0.92  0.91  0.87 
Base Rate  0.02  0.06  0.01  0.08  0.13  0.15 
Cut Points:  293*  315*  327*  390*  455*  492* 
At 90% Sensitivity, Specificity equals  0.80  0.87  0.76  0.76  0.77  0.70 
At 80% Sensitivity, Specificity equals  0.86  0.92  0.76  0.88  0.86  0.76 
At 70% Sensitivity, Specificity equals  0.90  0.94  0.88  0.90  0.90  0.85 
*The Acuity cut point is determined through a prediction study of matched student data for Acuity and the associated state summative NCLB test. The cut score is the Acuity score that best predicted the cut score for the second lowest performance level on the state summative. Students with scores below that Acuity cut score were identified as at risk of being in the lowest performance level on the state summative.
Generalizability
Description of study sample:
 Number of States: 1 (Missouri)
 Size: Approximately 2,500 – 4,000 per grade and content area

Gender:
 50.9% Male
 49.1% Female
 SES: 24.4% Eligible for free or reducedprice lunch

Race/Ethnicity:
 53.1% White, NonHispanic
 30.3% Black, NonHispanic
 14.1% Hispanic
 0.7% American Indian/Alaska Native
 1.8% Asian
 0.0% Native Hawaiian/Pacific Islander
Note: Acuity is developed specifically for each state and therefore it is not appropriate to have a representative national sample but rather a representative statespecific sample for each statespecific version of Acuity. That is, in Colorado there is a representative sample of Colorado students. In Missouri there is a representative sample of Missouri students. This provides generalizability to students in each state. Each state has its own state content standards, and each statespecific Acuity form measures the content standard for that state. The Missouri Acuity form data is provided as an example that is typical of Acuity in different states.
Reliability
Type of Reliability  Age or Grade  n 
Coefficient median 
Information (including normative data)/Subjects 

Feldt Raju 
Mathematics Grade 3 
7,144  0.89  State representative sample of more than 2500 students. 
Feldt Raju 
Mathematics Grade 4 
7,366  0.88  State representative sample of more than 2500 students. 
Feldt Raju 
Mathematics Grade 5 
7,201  0.82  State representative sample of more than 2500 students. 
Feldt Raju 
Mathematics Grade 6 
7,264  0.86  State representative sample of more than 2500 students. 
Feldt Raju 
Mathematics Grade 7 
5,918  0.86  State representative sample of more than 2500 students. 
Feldt Raju 
Mathematics Grade 8 
5,607  0.82  State representative sample of more than 2500 students. 
Validity
Type of Validity  Age or Grade  Test or Criterion  n (range)  Coefficient (if applicable)  Information (including normative data)/Subjects  

Range  Median  
Construct  3  Criterion: Grade 3 MAP (Math)  6,863  NA  0.75  Construct validity is supported by the correlation of the Acuity form C test and the state NCLB test, which are typically administered within six to eight weeks of each other. 
Content  3  Criterion: Grade 3 MAP (Math)  NA (supported by test design)  NA  NA  Content validity is supported by (a) the Acuity test design which reflects the content structure of the associated and subsequent state NCLB assessment in terms of the percent of items measuring each standard, (b) the alignment of the items with the state content standards, and (c) the alignment of the Acuity item formats (selectedresponse and constructedresponse) with that of the criterion assessment—the state summative assessments. Thus, the content structure of Acuity and the associated and subsequent state summative assessments are congruent, as described in the Acuity technical report. 
Predictive  3  Criterion: Grade 3 MAP (Math)  2,004  0.670.72 
Acuity/MAP Correlation: 0.70
Classification Consistency: 93.72% 
The correlation is the Pearson correlation for matched data between Acuity and the criterion MAP test.
The classification accuracy is the percent of students whose predicted and observed performance levels matched exactly when predicting to “Level 1: Below Basic” versus “Level 2 or Higher: Basic, Proficient, or Advanced.” This information is provided in Tables 3 and 18 of the Acuity Prediction Study Technical Report provided. 
Construct  4  Criterion: Grade 4 MAP (Math)  6,861  NA  0.80  Construct validity is supported by the correlation of the Acuity form C test and the state NCLB test, which are typically administered within six to eight weeks of each other. 
Content  4  Criterion: Grade 4 MAP (Math)  NA (supported by test design)  NA  NA  Content validity is supported by (a) the Acuity test design which reflects the content structure of the associated and subsequent state NCLB assessment in terms of the percent of items measuring each standard, (b) the alignment of the items with the state content standards, and (c) the alignment of the Acuity item formats (selectedresponse and constructedresponse) with that of the criterion assessment—the state summative assessments. Thus, the content structure of Acuity and the associated and subsequent state summative assessments are congruent, as described in the Acuity technical report. 
Predictive  4  Criterion: Grade 4 MAP (Math)  2,313  0.760.78 
Acuity/MAP Correlation: 0.77
Classification Consistency: 93.77% 
The correlation is the Pearson correlation for matched data between Acuity and the criterion MAP test.
The classification accuracy is the percent of students whose predicted and observed performance levels matched exactly when predicting to “Level 1: Below Basic” versus “Level 2 or Higher: Basic, Proficient, or Advanced.” This information is provided in Tables 3 and 18 of the Acuity Prediction Study Technical Report provided. 
Construct  5  Criterion: Grade 5 MAP (Math)  6,741  NA  0.77  Construct validity is supported by the correlation of the Acuity form C test and the state NCLB test, which are typically administered within six to eight weeks of each other. 
Content  5  Criterion: Grade 5 MAP (Math)  NA (supported by test design)  NA  NA  Content validity is supported by (a) the Acuity test design which reflects the content structure of the associated and subsequent state NCLB assessment in terms of the percent of items measuring each standard, (b) the alignment of the items with the state content standards, and (c) the alignment of the Acuity item formats (selectedresponse and constructedresponse) with that of the criterion assessment—the state summative assessments. Thus, the content structure of Acuity and the associated and subsequent state summative assessments are congruent, as described in the Acuity technical report. 
Predictive  5  Criterion: Grade 5 MAP (Math)  1,787  0.700.74  Acuity/MAP Correlation: 0.72 Classification Consistency: 92.84% 
The correlation is the Pearson correlation for matched data between Acuity and the criterion MAP test.
The classification accuracy is the percent of students whose predicted and observed performance levels matched exactly when predicting to “Level 1: Below Basic” versus “Level 2 or Higher: Basic, Proficient, or Advanced.” This information is provided in Tables 3 and 18 of the Acuity Prediction Study Technical Report provided. 
Construct  6  Criterion: Grade 6 MAP (Math)  5,826  NA  0.76  Construct validity is supported by the correlation of the Acuity form C test and the state NCLB test, which are typically administered within six to eight weeks of each other. 
Content  6  Criterion: Grade 6 MAP (Math)  NA (supported by test design)  NA  NA  Content validity is supported by (a) the Acuity test design which reflects the content structure of the associated and subsequent state NCLB assessment in terms of the percent of items measuring each standard, (b) the alignment of the items with the state content standards, and (c) the alignment of the Acuity item formats (selectedresponse and constructedresponse) with that of the criterion assessment—the state summative assessments. Thus, the content structure of Acuity and the associated and subsequent state summative assessments are congruent, as described in the Acuity technical report. 
Predictive  6  Test: Acuity Grade 6 Math. Criterion: Grade 6 MAP (Math)  1,552  0.730.77 
Acuity/MAP Correlation: 0.75
Classification Consistency: 93.45% 
The correlation is the Pearson correlation for matched data between Acuity and the criterion MAP test.
The classification accuracy is the percent of students whose predicted and observed performance levels matched exactly when predicting to “Level 1: Below Basic” versus “Level 2 or Higher: Basic, Proficient, or Advanced.” This information is provided in Tables 3 and 18 of the Acuity Prediction Study Technical Report provided. 
Construct  7  Criterion: Grade 7 MAP (Math)  4,493  NA  0.73  Construct validity is supported by the correlation of the Acuity form C test and the state NCLB test, which are typically administered within six to eight weeks of each other. 
Content  7  Criterion: Grade 7 MAP (Math)  NA (supported by test design)  NA  NA  Content validity is supported by (a) the Acuity test design which reflects the content structure of the associated and subsequent state NCLB assessment in terms of the percent of items measuring each standard, (b) the alignment of the items with the state content standards, and (c) the alignment of the Acuity item formats (selectedresponse and constructedresponse) with that of the criterion assessment—the state summative assessments. Thus, the content structure of Acuity and the associated and subsequent state summative assessments are congruent, as described in the Acuity technical report. 
Predictive  7  Criterion: Grade 7 MAP (Math)  1,101  0.750.77 
Acuity/MAP Correlation: 0.76
Classification Consistency: 90.19% 
The correlation is the Pearson correlation for matched data between Acuity and the criterion MAP test.
The classification accuracy is the percent of students whose predicted and observed performance levels matched exactly when predicting to “Level 1: Below Basic” versus “Level 2 or Higher: Basic, Proficient, or Advanced.” This information is provided in Tables 3 and 18 of the Acuity Prediction Study Technical Report provided. 
Construct  8  Criterion: Grade 8 MAP (Math)  4,282  NA  0.73  Construct validity is supported by the correlation of the Acuity form C test and the state NCLB test, which are typically administered within six to eight weeks of each other. 
Content  8  Criterion: Grade 8 MAP (Math)  NA (supported by test design)  NA  NA  Content validity is supported by (a) the Acuity test design which reflects the content structure of the associated and subsequent state NCLB assessment in terms of the percent of items measuring each standard, (b) the alignment of the items with the state content standards, and (c) the alignment of the Acuity item formats (selectedresponse and constructedresponse) with that of the criterion assessment—the state summative assessments. Thus, the content structure of Acuity and the associated and subsequent state summative assessments are congruent, as described in the Acuity technical report. 
Predictive  8  Criterion: Grade 8 MAP (Math)  1,033  0.700.73 
Acuity/MAP Correlation: 0.72
Classification Consistency: 85.69% 
The correlation is the Pearson correlation for matched data between Acuity and the criterion MAP test.
The classification accuracy is the percent of students whose predicted and observed performance levels matched exactly when predicting to “Level 1: Below Basic” versus “Level 2 or Higher: Basic, Proficient, or Advanced.” This information is provided in Tables 3 and 18 of the Acuity Prediction Study Technical Report provided. 