FAST: Adaptive Math

aMath

Rating Summary

Efficiency
Classification Accuracy
Generalizability		Moderate Low
Reliability
Validity
Disaggregated Reliability and Validity Data
	Administration	Individual
	Administration & Scoring Time	10-45 Minutes
	Scoring Key	Computer Scored
	Benchmarks / Norms	Yes

Cost	Technology, Human Resources, and Accommodations for Special Needs	Service and Support	Purpose and Other Implementation Information	Usage and Reporting
The Formative Assessment System for Teachers™ (FAST) is online software that requires no hardware or special add-ons. FAST is supported by an extensive set of materials to support teachers and students, including self-directed training modules that allow teachers to become certified to administer each of the FAST assessments. The entire FAST assessment package (i.e., reading, math, behavior, and on-line training) is provided at an annual flat rate of $6 per student.	Testers will require less than 4 hours of training. Paraprofessionals can administer the test.	Where to Obtain: www.fastforteachers.org Address: FastBridge Learning 520 Nicollet Mall, Suite 910 Minneapolis, MN 55402-1057 Phone: 612-254-2534 Website: www.fastforteachers.org Training materials are included in the cost of the tool. Additional, optional on-site and webinar-based training services are available for a fee. Ongoing technical support is available by calling 612-424-3710 or emailing fast1@umn.edu	The ultimate goal of aMath is instructional efficacy. aMath directly addresses issues of instructional relevance – it is designed to optimize each assessment for each student based on their response pattern. Grade level references are just a convenience to identify the most relevant skills across the developmental range for the typically developing child. As such, a student in third grade with substantial math deficits might receive an assessment that is similar to that of a typically developing kindergarten student. Likewise, a student in the third grade with substantial math ability might receive an assessment that is similar to that of a typically developing 5^th grade student. Both assessments and instruction must be individualized for students. aMath optimizes the efficiency, precision, and utility of assessment with individualized (adaptive) assessment.	The tool is intended for use in grades K-6 or with ages 5-10+. Administration is computerized and is 20-30 minutes per typical student. Scoring is done automatically within the software and does not require any additional time. Available scores include: raw scores, local through national percentile scores, local through national growth norms, developmental benchmarks and cut points, accuracy rates and error analysis.

The Formative Assessment System for Teachers™ (FAST) is online software that requires no hardware or special add-ons. FAST is supported by an extensive set of materials to support teachers and students, including self-directed training modules that allow teachers to become certified to administer each of the FAST assessments. The entire FAST assessment package (i.e., reading, math, behavior, and on-line training) is provided at an annual flat rate of $6 per student.

Testers will require less than 4 hours of training.

Paraprofessionals can administer the test.

Where to Obtain: www.fastforteachers.org

Address:

FastBridge Learning
520 Nicollet Mall, Suite 910
Minneapolis, MN 55402-1057

Phone: 612-254-2534

Website: www.fastforteachers.org

Training materials are included in the cost of the tool. Additional, optional on-site and webinar-based training services are available for a fee.

Ongoing technical support is available by calling 612-424-3710 or emailing fast1@umn.edu

The ultimate goal of aMath is instructional efficacy. aMath directly addresses issues of instructional relevance – it is designed to optimize each assessment for each student based on their response pattern. Grade level references are just a convenience to identify the most relevant skills across the developmental range for the typically developing child. As such, a student in third grade with substantial math deficits might receive an assessment that is similar to that of a typically developing kindergarten student. Likewise, a student in the third grade with substantial math ability might receive an assessment that is similar to that of a typically developing 5^th grade student. Both assessments and instruction must be individualized for students. aMath optimizes the efficiency, precision, and utility of assessment with individualized (adaptive) assessment.

The tool is intended for use in grades K-6 or with ages 5-10+.

Administration is computerized and is 20-30 minutes per typical student. Scoring is done automatically within the software and does not require any additional time.

Available scores include: raw scores, local through national percentile scores, local through national growth norms, developmental benchmarks and cut points, accuracy rates and error analysis.

Classification Accuracy

GMADE Results: Summary

Grade	n	aMath Cut Score	Sensitivity	Specificity	PPP	NPP	AUC
Some Risk – 40^th Percentile GMADE
K	81	187	0.72	0.80	0.65	0.98	0.81
1	72	196	0.79	0.90	0.67	0.98	0.88
2	67	206	0.80	0.92	0.67	0.98	0.82
3	86	207	0.96	1.00	0.43	1.00	0.89
4	60	215	0.67	0.80	0.64	0.97	0.74
5	42	225	0.85	0.88	0.81	0.97	0.97
High Risk: 10^th Percentile
K	81	187	0.71	0.75	0.12	0.98	0.75
1	72	190	0.95	0.83	0.62	0.98	0.83
2	67	202	0.88	0.80	0.53	0.96	0.92
3	86	201	0.95	1.00	0.33	1.00	0.98
4	60	206	0.95	0.75	0.50	0.98	0.88
5	42	218	0.84	0.80	0.40	0.97	0.88
Criterion			0.70	0.70	-	-	0.85

MAP Results: Summary

Grade	n	aMath Cut Score	Sensitivity	Specificity	PPP	NPP	AUC
Some Risk (MAP specified cut score)
K	89	188	0.84	0.91	0.77	0.94	0.91
1	77	193	0.84	1.00	0.20	1.00	0.93
2	91	196	0.97	1.00	0.62	1.00	0.98
3	89	208	0.77	1.00	0.21	1.00	0.91
4	74	206	0.91	1.00	0.50	1.00	0.98
5	53	220	0.86	1.00	0.62	1.00	0.94
Criterion			0.70	0.70	-	-	0.85

Classification Accuracy in Predicting Proficiency on GMADE
	Kindergarten n = 81	1^st Grade n = 72	2^nd Grade n = 67	3^rd Grade n = 86	4^thGrade n = 60	5^th Grade n = 64
False Positive Rate	0.20	0.18	0.20	0.16	0.21	0.50
False Negative Rate	0.23	0.18	0.22	0.10	0.33	0.07
Sensitivity	0.77	0.82	0.78	0.90	0.67	0.93
Specificity	0.80	0.82	0.80	0.84	0.79	0.50
Positive Predictive Power	0.65	0.67	0.67	0.43	0.64	0.34
Negative Predictive Power	0.88	0.91	0.88	0.98	0.82	0.96
Overall Classification Rate	0.79	0.82	0.79	0.85	0.75	0.59
Observed	0.79	0.82	0.79	0.85	0.75	0.59
Chance	0.54	0.55	0.53	0.70	0.54	0.45
Base Rate	0.32	0.31	0.34	0.12	0.35	0.22
Cut Points:	40^th percentile	40^th percentile	40^th percentile	40^th percentile	40^th percentile	40^th percentile

Classification Accuracy in Predicting Proficiency on MAP
	Kindergarten n = 89	1^st Grade n = 77	2^nd Grade n = 91	3^rd Grade n = 89	4^thGrade n = 74	5^th Grade n = 53
False Positive Rate	0.15	0.28	0.26	0.15	0.28	0.18
False Negative Rate	0.21	0.08	0.00	0.07	0.00	0.00
Sensitivity	0.79	0.92	1.00	0.93	1.00	1.00
Specificity	0.85	0.72	0.74	0.85	0.72	0.82
Positive Predictive Power	0.86	0.38	0.43	0.54	0.52	0.76
Negative Predictive Power	0.78	0.98	1.00	0.98	1.00	1.00
Overall Classification Rate	0.82	0.75	0.78	0.87	0.78	0.89
Observed	0.82	0.75	0.78	0.87	0.78	0.89
Chance	0.50	0.58	0.58	0.66	0.53	0.51
Base Rate	0.54	0.16	0.16	0.16	0.23	0.36
Cut Points:	40^th percentile	40^th percentile	40^th percentile	40^th percentile	40^th percentile	40^th percentile

Generalizability

Description of study sample:

· Number of States: 1 (Minnesota)

· Size: 496

· Grades K-5

· Regions: 1

· Gender

o 49% Male

o 51% Female

· SES: 8% Eligible for free or reduced-price lunch

· Race/Ethnicity:

o 88% White, Non-Hispanic

o 1% American Indian/Alaska Native

o 6% Black, Non-Hispanic

o 2% Asian, Pacific Islander

o 3% Hispanic

· Disability status: approximately 15% students with disabilities

· First language: It was estimated that more than 95% of participants had English as their first language.

Reliability

Type of Reliability	Age or Grade	n (range)	Coefficient		SEM	Information (including normative data)/Subjects
Type of Reliability	Age or Grade	n (range)	range	median	SEM	Information (including normative data)/Subjects
Test-Retest Reliability						Pending; however, all tests are derived from a fully calibrated item bank with IRT parameters, which supports the assumption of form equivalence.
Internal Consistency		Approximately 2,000 students		0.95
Alternate Forms Reliability		Approximately 2,000 students		0.95
Inter-rater				1.0		Computer scored so there is no “inter-rater”

Validity

Type of Validity	Age or Grade	Test or Criterion	n (range)	Coefficient (if applicable)		Information (including normative data)/Subjects
Type of Validity	Age or Grade	Test or Criterion	n (range)	Range	Median	Information (including normative data)/Subjects
Concurrent	K	GMADE	81		0.62	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	1	GMADE	72		0.66	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	2	GMADE	67		0.67	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	3	GMADE	86		0.76	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	4	GMADE	60		0.67	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	5	GMADE	65		0.84	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	K	MAP	89		0.76	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	1	MAP	77		0.71	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	2	MAP	91		0.81	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	3	MAP	89		0.76	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	4	MAP	74		0.84	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	5	MAP	76		0.88	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	3	MCA	88		0.80	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	4	MCA	70		0.84	See table below; 2-4 week delay between aMath and criterion administration
Concurrent	5	MCA	73		0.85	See table below; 2-4 week delay between aMath and criterion administration
Predictive	3	MCA-III	483		0.86	2013-2014 Fall aMath to Spring prediction of the MCA.
Predictive	4	MCA-III	486		0.85	2013-2014 Fall aMath to Spring prediction of the MCA.
Predictive	5	MCA-III	493		0.85	2013-2014 Fall aMath to Spring prediction of the MCA.
Concurrent	3	MCA-III	155		0.87	2014 May aMath to Spring MCA
Concurrent	4	MCA-III	90		0.87	2014 May aMath to Spring MCA
Concurrent	5	MCA-III	67		0.88	2014 May aMath to Spring MCA

Demographic Data for Spring 2013 Validity Study

Sample Demographics
	K	1	2	3	4	5	All
n	89	77	91	89	74	76	496
Gender (Male)	40%	47%	48%	51%	53%	54%	49%
White	89%	90%	89%	85%	88%	90%	88%
Black	6%	3%	7%	6%	7%	8%	6%
Hispanic	2%	3%	2%	5%	3%	3%	3%
Asian	1%	4%	1%	5%	0%	0%	2%
American Indian	0%	0%	1%	0%	2%	0%	1%
F/R Lunch	2%	1%	0%	20%	18%	11%	8%
Special Education	5%	10%	11%	24%	22%	18%	15%

Disaggregated Reliability, Validity, and Classification Data for Diverse Populations

Disaggregated Classification Accuracy in Predicting Proficiency on Minnesota Comprehensive Assessment III (MCA-III)

	3^rd Grade (Asian) n = 110	3^rd Grade (Hispanic) n = 65	3^rd Grade (Black) n = 54	3^rd Grade (White) n = 243
False Positive Rate	0.20	0.20	0.50	0.06
False Negative Rate	0.07	0.21	0.07	0.35
Sensitivity	0.93	0.79	0.93	0.65
Specificity	0.80	0.80	0.50	0.94
Positive Predictive Power	0.63	0.63	0.70	0.46
Negative Predictive Power	0.97	0.90	0.86	0.97
Overall Classification Rate	0.84	0.80	0.74	0.92
AUC (ROC)	0.95	0.91	0.90	0.94
Base Rate	0.26	0.29	0.56	0.07

Disaggregated Classification Accuracy in Predicting Proficiency on Minnesota Comprehensive Assessment III (MCA-III)

	4^th Grade (Asian) n = 97	4^th Grade (Hispanic) n = 58	4^th Grade (Black) n = 67	4^th Grade (White) n = 254
False Positive Rate	0.24	0.23	0.14	0.06
False Negative Rate	0.10	0.13	0.22	0.32
Sensitivity	0.90	0.87	0.78	0.68
Specificity	0.76	0.77	0.86	0.94
Positive Predictive Power	0.62	0.57	0.75	0.46
Negative Predictive Power	0.95	0.94	0.88	0.97
Overall Classification Rate	0.80	0.79	0.84	0.92
AUC (ROC)	0.95	0.89	0.93	0.93
Base Rate	0.30	0.26	0.34	0.07

Disaggregated Classification Accuracy in Predicting Proficiency on Minnesota Comprehensive Assessment III (MCA-III)

	5^th Grade (Asian) n = 103	5^th Grade (Hispanic) n = 62	5^th Grade (Black) n = 72	5^th Grade (White) n = 243
False Positive Rate	0.14	0.18	0.17	0.04
False Negative Rate	0.22	0.22	0.21	0.43
Sensitivity	0.78	0.78	0.79	0.57
Specificity	0.86	0.82	0.83	0.96
Positive Predictive Power	0.62	0.64	0.70	0.57
Negative Predictive Power	0.93	0.90	0.89	0.96
Overall Classification Rate	0.84	0.81	0.82	0.93
AUC (ROC)	0.94	0.89	0.90	0.94
Base Rate	0.22	0.29	0.33	0.09