About the Data
Cautions | Definitions | Calculating Means and Mean Change | Sources of Data | More about MDAT
Cautions
- Results for the MDAT topic "One-year change in WKCE scale scores" provide estimates of student academic progress, but are not official DPI growth measures.
- All scores and change scores are estimates of academic status and progress. These estimates are subject to variability due to measurement error. Results for small groups of students tend to be less statistically reliable than results for larger groups of students.
- While scale scores are equated over time and across grades, occasional issues exist for a specific subject/grade/year at the floor (bottom of the minimal range) or ceiling (top of the advanced range) of the test. For specific information about the floor and ceiling of each test, see scale scores for reading and scale scores for mathematics.
- Equating scale scores across tests assumes comparable content across tests. This assumption is generally reasonable when comparing WKCE scale scores in a subject across adjacent grades or consecutive years. This assumption may become less reasonable with increasing differences between the grade levels of the test scores being compared.
- MDAT data are confidential and unredacted. Even aggregated data can lead to personally identifiable information.
What is WKCE?
The WKCE (Wisconsin Knowledge and Concepts Examination) is a statewide standardized exam given each year. Reading and mathematics are tested in grades 3 through 8 and 10. Language arts, science, and social studies are tested in grades 4, 8, and 10. Students also provide a rough draft writing sample. WKCE is part of the Wisconsin Student Assessment System (WSAS). The Wisconsin Student Assessment System includes WKCE and the Wisconsin Alternate Assessment (WAA). [MORE]
Note that only a subset of the WKCE subject and grade levels are available for the topic "One-Year Change in WKCE Scale Scores." Scores for adjacent grade levels are required for this topic. WKCE scores are only available in adjacent grades for reading and mathematics and only if the selected grade is 4, 5, 6, 7, or 8.
What is a scale score?
A scale score is a score on a numeric scale with intervals of equal size. For WKCE, the scale is applied to all students taking the WKCE in a particular subject, making it possible to compare, add, subtract, and calculate mean scores for different groups of students or individuals from year to year and grade to grade. Higher scores suggest higher achievement of the knowledge and skills measured by the test.
A scale score is not a raw score. For WKCE, the raw score is the number of items answered correctly on the test. Raw scores are converted to scale scores using statistical procedures designed to facilitate evaluation of trends over time. Each subject area is scaled separately; therefore, the scale scores for one subject area cannot be compared to another subject area.
What is the proficient cutscore?
Cutscores are scale scores used to assign student WKCE performance to one of four performance levels: Advanced, Proficient, Basic, and Minimal Performance. The proficient cutscore is the lowest scale score associated with the Proficient level at the beginning of the school year for the grade level of the test.
What is the standard error of measurement?
The standard error of measurement (SEM) estimates how repeated measures of a student's achievement on a test tend to be distributed around his or her “true” score. The true score is always an unknown because no test can be constructed that provides a perfect reflection of the true score. The larger the SEM, the lower the reliability of the student's obtained score. Given the student’s obtained score, there are two out of three chances that the individual’s true score would fall within a 2 SEM confidence band around the student's obtained score. The very low or very high scores within a scale score range tend to have higher SEMs (lower reliability) than scores around the proficient cutscore.
How is the one-year change in WKCE scale scores calculated for a student or student group?
For an individual student, the one-year change in scale scores is the selected year scale score minus the prior year scale score. A student must have scores in both years to have a change score or to be included in means or change scores for a student group. Generally the scores are from consecutive grade level tests, so if the student's selected year score is for grade 4, then the student's prior year score would be for grade 3. For a student group, the change would be an average for the students with scores in both years rounded to the nearest tenth of a scale score point. Note that the rounded change might not be exactly equal to the difference of the rounded means because the change is not rounded until the last step.
Why might the table on the Results page include two different numbers of students for the same student group?
The table contains two counts for each student group. The first count, # of Students in Group, provides the total number of students in the group in the selected year regardless of whether they had WKCE scores in the selected year or prior year. The second count, # of Students with Scores in Selected Year and Prior Year, includes all or a subset of the students in the first count and matches the number of students in the graph. If the second number is far fewer than the first, then the means and change score for the group should not be used to reach conclusions about the performance of the total group. Students missing one or more WKCE scores are unlikely to be representative of the group as a whole. Note that the Student Detail pages include all students in the group as a whole along with their scores, if any.
What if a student does not have a valid Wisconsin Student Number (WSN) attached to either the selected year score, or the prior year score, or both?
Any student without a WSN attached to either WKCE score is excluded from change scores and means because WSNs are needed to connect student scores across years. Students without WSNs are labeled "not identified" in "Individual Student" reports.
Are mobile students included in means and change scores?
Any mobile student with scores in both relevant years is included unless a mobility filter is selected. In other words, by default, a student in District/School B in the selected year and District/School A in the prior year is included in the results for District/School B. However, if a District/School B user selects one of the filter options that restricts results to students in a particular district or school for at least a full year or more, then the student would be excluded. Since the student is not in District/School A in the selected year, then the student would not show up in results for District/School A users under any circumstances.
How are the arrows used to display data about mean scores and mean changes in scores?
The tip of the arrow represents the (mean) score for the selected year, and the tail of the arrow represents the (mean) score in the prior year. Students must have scores in both relevant years to be included in the means. The length of each arrow indicates the difference between these two scores which is an estimate of progress in the content area measured by the test. Green arrows (pointing to the right) indicate an increase in scores; yellow arrows (pointing to the left) indicate a decrease in scores. Longer arrows indicate greater change.
How does this method of calculating change compare to other commonly reported methods?
The key difference between MDAT "One-year change in WKCE scale scores" and other reports of change currently available through DPI and the WI testing contractor is that the former method provides evidence of change at the individual student level and the latter provides evidence of change in the performance of successive groups of students. "One-year change in WKCE scale scores" is longitudinal and addresses problems stemming from measuring changes in schools or districts with mobility. Other reports are cohort to cohort.
Where do the data come from?
Data about one-year change in WKCE scale scores come from the WKCE results. Other data about individual students come from one of the following: WSAS and/or the ISES Third Friday September, Year End, or Discipline record sets.
On the Report Options page, data record source for each variable can be determined by a code in parenthesis at the end of the name of the variable. "(WSAS)" is Wisconsin Student Assessment System. "(CD)" is ISES Third Friday September records. "(YE)" is ISES Year End records. "(DISC)" is ISES Discipline records. Some variables do not include information about the source because DPI has only one source of these data.
One variable "Primary Disability (WSAS or CD)" uses WSAS as the primary data source but fills in with ISES Third Friday September CD records for students identified as having a disability in WSAS but for whom no specific primary disability was identified in WSAS. If ISES data conflicts (i.e. student was submitted in ISES as not having a disability) or if an ISES CD record is not found for the student, then the student is included in "one-year change in WKCE scale scores" reports as a Y with no specific primary disability assigned.
WSAS data in MDAT come from the WSAS data used for AYP determinations and may vary slightly from data included in EDISA reports. EDISA reports are generated by the testing contractor based on an earlier version of the student level data. Prior to determining AYP, data are cleaned by DPI.
Where can I get more information about MDAT?
For information about the Multi-Dimensional Analytic Tool topic One-Year Change in WKCE Scale Scores, see Using MDAT, MDAT Glossary, and/or MDAT Frequently Asked Questions.
Questions or comments about MDAT should be directed to
ldshelp@dpi.wi.gov .
Last updated on 1/25/2010 2:29:18 PM
Home
About Us