You are here

Development of the WKCE

grapic of test development cyclegraphic that goes to information about Quality analysis graphic that goes to information about operational formsgraphic that goes to information about Item statisticsgraphic that goes to information about Field Testinggraphic that goes to information about Item Developementgraphic that goes to information about Reading passage reviewgraphic that goes to information about the blueprintgraphic that goes to information about the frameworksgraphic that goes to information about the standardsgraphic that goes to information about Alignment of the testgraphic that goes to information about scoring and reportinggraphic that goes to information about Standard Settinggraphic that goes to information about test administrationgraphic that goes to information about Finalization of formsgraphic that goes to information about educator involvementgraphic representation of test development process

Involvement of Wisconsin Educators in WKCE:
Wisconsin educators have played an integral role at all stages in the development of the state’s academic content standards, achievement standards, and assessments. Educators with diverse geographic, demographic, racial, ethic, and cultural backgrounds across the state participate in WKCE development activities. Special-education and English-language learner representation is also ensured at all stages of the development. Selected examples of this participation include the following: Back to diagram
 
  • Development of the Wisconsin Model Academic Standards (WMAS);
  • Development of the Assessment Frameworks in Reading, Mathematics, and Science;
  • Participation in the February 2003 bookmark standard-setting process that established performance descriptors and cut scores for the WKCE, and in the spring 2006 descriptor-writing workshop for the WKCE;
  • Annual review and selection of passages and prompts for WKCE Reading assessments;
  • Annual review and editing of WKCE assessment items in Reading, Mathematics, and Science for content appropriateness, depth of knowledge, and difficulty and fairness prior to field testing;
  • Annual range-finding activities that establish parameters and anchor papers for the scoring of WKCE short-answer questions and writing prompts;
  • Refining of the WKCE test blueprint so that it reflects what students should know and should be able to do at the beginning of the school year.
Wisconsin Model Academic Standards: The Wisconsin Model Academic Standards (WMAS) are fourth-grade, eighth-grade, and twelfth-grade goals in reading, mathematics, science, language arts, and social studies. They were written by Wisconsin educators, parents, school-board members, and business owners. The standards are not designed to be a curriculum; rather they serve as guides to help districts to create their curricula. Back to diagram
 
Assessment Frameworks:
The assessment frameworks contain elements from the WMAS that are appropriate for state testing. The frameworks are use to develop test questions (items) for the Wisconsin Knowledge and Concepts Examination (WKCE). Frameworks were created for reading and mathematics for grades 3 through 8, and 10. Science frameworks were created for grades 4, 8, and 10. Back to diagram
 
Blueprint:
The test blueprint outlines the number and types of items on the WKCE. It also specifies the difficulty of the questions and passages, the percentage of multiple-choice or written-response items, and outlines how the emphasis on various skills differs from grade to grade. The blueprint helps to keep the structure and format of the WKCE consistent from year to year. Back to diagram
 
Reading Passage-and-Bias Reviews:
The Department of Public Instruction – Office of Educational Accountability (DPI-OEA), Wisconsin educators, and the test vendor review the reading passages to make sure that all students have an equal-and-fair opportunity to demonstrate their knowledge and skills. They also review passages for potential bias against ethnic, cultural, religious, or geographically-represented groups. Back to diagram
 
Item Development:
The test vendor develops annually both multiple-choice and written-response items for reading and mathematics. Each year, some items are removed from the assessment, and new items are added. Once DPI reviews all new items, Wisconsin educators are brought together to review and to approve them.

Item Review:
After items are developed, the DPI-OEA staff and a team of Wisconsin educators are brought together to review them and to decide whether they should be included on the WKCE. The educators review the items to make sure that they are appropriate for the intended grade level and that all students would have an equal-and-fair opportunity to demonstrate their knowledge and skills. The team of educators may modify the items as necessary to achieve these goals. Items unable to be modified appropriately are rejected from use on the WKCE. Back to diagram
 
Field Testing:
After items have been approved by DPI and the team of Wisconsin educators, they are embedded in the WKCE for field testing. How students perform on the field test items does not affect students’ score on the WKCE, but it does allow the DPI-OEA staff to collect data to ensure that the items provide all students with an equal-and-fair opportunity to demonstrate their skills.Back to diagram
 
Range Finding: (Written response items require an extra step for scoring purposes):
The DPI-OEA staff and the test vendor conduct a three-day Range Finding process with Wisconsin educators in which they review and approve scoring guides used to score the WKCE field-test items.
 
  • Scoring Guides are documents that are used to familiarize evaluators with the content areas, criteria, and objectives of the testing program. Scoring guides provide detailed elaborations of the goals of assessment. Scoring guides provide specific criteria to describe a range of possible student responses and a consistent set of guidelines to rate student work. In other words, these guides provide information about how the student is expected to respond to the short-answer questions for earning different score points.
     
Item Selection:
Using data from field testing, an item is rejected from future use on the WKCE if a large number of students are confused by the item or if a specific group of students do not perform as well on the item. Acceptable items are kept for possible use on future tests. Back to diagram
 
Operational Forms:
An operational form is the portion of the test that is scored. This portion contains only items from the item bank. The blueprint is used to determine which items to include.

Item Bank:
If a question is determined to be appropriate after field testing, then it is placed in the item bank. The bank of items is used to create the test forms.Back to diagram
 
Multiple Reviews, Statistical Analysis:
The test vendor uses statistics to make sure that operational forms are consistent from year to year. The DPI-OEA staff then conducts multiple reviews of the operational form to make sure that the items are appropriate for the grades and the subjects. Back to diagram
 
Final Forms:
This is a copy of the full WKCE, including items from the item bank and from field-test questions. An actual printed version of the final form is reviewed by the staff at DPI-OEA and the test vendor for approval before the forms are released for mass printing. Back to diagram
 
WKCE Test Administration:
The reading and mathematics tests are administered to students in grades 3 – 8 and 10 in all Wisconsin public schools. Students in grades 4, 8, and 10 are also tested in science, language arts, writing, and social studies during the test window. The test window is a one month period from late October to late November. (October 23- November 22, 2007) Back to diagram
 
Standard Setting:
Standard Setting is a process of determining cut scores that match performance levels. Wisconsin educators set these cut scores by engaging in a structured conversation that includes content standards, performance levels, the test, and expectations for students. The cut scores that are determined during the standard-setting procedure distinguish one performance level from another. Back to diagram
 
Scoring:
The tests are scored using a pattern scoring technique based on item response theory (IRT). A student’s score is dependent not only upon the number correct, but on each item’s difficulty, discrimination, and guessing parameters in conjunction with the student’s response on the item.

Reporting:
Reports are generated annually for parents, districts, the staff at the DPI, and the public. Back to diagram
 
  • Parents receive a reports of their students’ performances on the WKCE
  • Districts can access a secure site to find reports on their students and their schools
  • The DPI receives many reports from the test vendor to ensure the validity of the WKCE
Alignment:
After testing and scoring are complete, an independent group reviews the items on the test to see if they align to the WMAS. The group makes sure there are enough items, coverages, and ranges of difficulty for each objective.Back to diagram