Guidelines for Appropriate Testing Procedures and Test Security

INTRODUCTION

Appropriate and ethical testing practices are not always universally understood or followed. Good testing practices are sometimes violated because the person involved is not informed about what is appropriate in testing. To help school staff who administer the Wisconsin Student Assessment System (WSAS) Knowledge and Concepts Examinations (WKCEs) at grades 3-8, and 10, have a common understanding of what practices are appropriate, the Department of Public Instruction has prepared these guidelines.

This paper covers general principles and standards as presented by a number of organizations and studies and is applicable to all types of assessments, including standardized multiple-choice tests as well as performance assessments. Topics covered in this paper include:

A. Test Security
B. Ethics in Testing
C. Testing Conditions
D. Post-Test Activities/Procedures

Related Web Pages

A. TEST SECURITY

1. What Is Meant by Test Security?

Tests used in the WKCE statewide testing program are secure, proprietary instruments published and copyrighted by a testing company, not the state. Any disclosure or dissemination of actual test items to any person may be considered a copyright violation and may severely undermine the value of the test and adversely affect the validity of test results. The confidentiality of test questions and answers is paramount in maintaining the integrity and validity of the test. Therefore, the Department of Public Instruction (DPI) and all Wisconsin educators must take every step to assure the security of the test instruments.

2. Why Is Test Security Important?

Test security is important to:

• make valid inferences on student and school performances as required by law;
• guard against limiting the curriculum to content covered on the test;
• give accurate measures of students’ abilities; and
• keep the integrity of the test and testing situation intact.

3. Who is Responsible for Test Security?

Everyone who works with assessment, communicates test results to others, and/or receives testing information is responsible for test security. This includes:

• Staff of the Department of Public Instruction
• District Administrators and both certified and non-certified school staff
• District Assessment Coordinators (DACs)
• School Assessment Coordinators (SACs)
• Students, parents, and the community at large
• Staff at the Cooperative Educational Service Agencies (CESA)

4. Can District Staff Review the Test?

School and district staff should be familiar with the testing procedures and schedule before testing. However, review of the test instrument by district and school staff may be arranged only after the test administration has concluded.

Test review by staff will help familiarize them with the test content and format and assist them in understanding and using test results as well as in their curriculum development efforts. When reviewing the test content with the staff, the DAC/SAC must make sure that all test books are numbered before distribution. At the end of the review session, all test books must be collected and accounted for. No staff member should be allowed to leave the review session until all test books are counted and secured.

The following precautionary steps must be taken by the DAC/SAC at the time of test review with the staff:

• Use actual test books. DO NOT use any reproduced copies of the test books.
• Number all test books before entering the review session.
• Ask staff to sign a Confidentiality agreement acknowledging ethical practices, copyright, and proprietary restrictions before beginning the test review session.
• Distribute the numbered copies of the test books to staff at the review session.
• Work with staff as a group; do not allow individuals to retain a copy of the test.
• DO NOT allow any staff member to make copies of the test or any test items, take notes, or otherwise reproduce the test or test items.
• Concentrate on the review of the Objectives Performance Report and the Item Analysis Summary Report when discussing test content.
• DO NOT allow any individual to leave the test review session before all numbered test books are collected and accounted for.

5. Is the Public Allowed to Review the Tests?

Review of the test by parents and other private citizens must be a guarded matter. It must follow a formal security procedure. Any interested person may request to review the test but only after the test administration has concluded. Wisconsin law stipulates that “The state superintendent shall make available upon request, within 90 days after the date of administration, any examination required to be administered under this subsection [s. 118.30, Wis. Stats.].”

Tests administered under WSAS are copyrighted, secured instruments. They are the product of a costly contract with the state. The DAC must coordinate the public review of the test instrument in a way that does not compromise the integrity and security of the test, test items, and test results.

The following are some of the safeguards that should be taken by the DAC/SAC when a reviewer reviews the tests:

• Have the reviewer sign a confidentiality agreement (sample attached) prior to reviewing the test.
• DO NOT permit the reviewer to make any photocopies or other reproductions of the test, take notes, or copy test items.
• Allow the reviewer to review only materials that DO NOT identify individual students. DO NOT allow the reviewer to review material that identifies individual students.
• Be prepared, or secure the availability of a qualified staff member, to explain the purpose of the test, answer questions about test content, and explain the meaning of test results to the reviewer.
• The reviewer must be accompanied at all times and should not be left alone with the test.

6. Pre-Test Security

It is essential that all test materials remain secure. That is, when the tests are not being used for testing, testing materials should be kept in a LOCKED STORAGE area. Access to these materials should only be with the knowledge and expressed permission of the DAC.

District Assessment Coordinators should work closely with school assessment coordinators, and together they direct the management of WKCEs. Their number one responsibility is to ensure test security throughout the testing process in order to protect not only the integrity of the test, but also to protect principals and teachers from any appearance of impropriety. On a daily basis, DACs and SACs should make sure that all test materials are placed in locked storage when not in use in a testing session. They also must make sure that students do not share information about test content when the test is administered to same grade-level students at different times. If any deviation is noted, they should take immediate action to correct it. Depending on the severity of a deviation in security, it may be necessary to advise the Department’s Office of Educational Accountability.

7. What Are Some Examples of Test Security Violations?

Test security violations can be committed by educators, students, or others.

Some examples of test security violations by educators include, but are not limited, to:

• leaving students unsupervised during testing;
• leaving test materials in an unsecured place;
• photocopying or keeping a personal copy of the test;
• taking notes about test questions and using them or a close paraphrase to prepare students for testing;
• offering “hints” that indicate an answer or help eliminate answer choices;
• rephrasing the test questions;
• editing (changing) student answers after completion of the test by erasing any wrong answers and writing in the correct ones;
• extending testing time beyond regulations for students other than those with documented disabilities per their IEP or for students covered by Sec. 504 per their IAP or for certain limited-English proficiency (LEP) students;
• providing test accommodations for students with disabilities that are not included in the student’s IEP or IAP that are not specified in accommodations offered to an LEP student;
• allowing students to go back to previous sections in the test booklet to check their work; and
• allowing students to go back to the current section in the test booklet to change their answers after allowed testing time has expired.

Some examples of test security violations by students include, but are not limited to:

• illegally obtaining a test booklet to study or to let others study;
• securing a marked test booklet or “crib sheet” from a teacher or another student;
• copying or “stealing” answers from another student during testing;
• sharing specific test information with other students in the same grade who are scheduled to take the same test at a later time; and
• taking a test during the make-up period and asking a student who has already completed the test to disclose test questions and/or answers.

8. Possible Consequences/Sanctions for Compromising Test Security

Administrators, certified and non-certified school staff, students, and parents must adhere to ethical procedures in testing. Violation of these procedures can be investigated and appropriate sanctions may be taken by the local school board, the Department, and/or the court system.

Test security violations may be reported by the school faculty, conscientious students, their parents or other family members, and persons in the community. They also can be detected by erasure analysis, unusual score gains, or other irregularities.

Potential sanctions for educator violation of security measures may include:

• Suspension or acceptance of voluntary surrender of license.
• Suspension without pay or a written reprimand.
• Termination of contract, acceptance of resignation, or retirement.
• Civil legal liability for copyright violations.
• Legal prosecution.
• Public embarrassment.
• Others as determined by local school boards.

Potential sanctions for student violation of security measures may include:

• Invalidation of test results.
• Invalidation of specific test questions or subtests, or invalidation of pass/fail proficiency results.
• Suspension or expulsion from school or other disciplinary actions according to the local code of conduct.
• Suspension or exclusion from participating in school extra-curricular activities, such as sports, plays, school-sponsored social functions, etc., as dictated by the local school board policies.
• Denial of appointment to a school membership team, such as mathematics or debate team.
• Removal from an elected office, such as president of the student council, etc., as dictated by the local school board policies.
• Others as determined by local school boards.

B. ETHICS IN TESTING

Aside from security issues, the most significant consideration for appropriate and ethical testing practices in pre-test activities relates to preparing students for the test in ways that allow for a valid interpretation of the test results. A WKCE test score is an estimate of the student achievement in the content areas of mathematics, science, social studies, English language arts, and reading.

It is important to be reasonably certain that if the student has done well on the WKCEs, that she or he understands the content sufficiently to perform well on similar tests and to apply that understanding (knowledge) in real life. If a student is coached or taught only the content specific to a given test, his or her scores may not be valid indicators of what the student knows and can do. The result will simply be measure of how well the student has been taught the specific content on the test.

1. Wisconsin Model Academic Standards and Test Alignment

The WKCEs allow students to demonstrate their knowledge and skills using selected-response and constructed-response items in one test instrument. They also include performance assessment in the form of a writing essay. In April 1998, CTB/McGraw-Hill, the Department’s contractor for WKCEs, conducted a workshop for the Department to match items on the TerraNova (WKCEs) to the Wisconsin Model Academic Standards in Reading, Language Arts, Mathematics, Science, and Social Studies, adopted in January 1998 by an Executive Order issued by the Governor. The purpose of conducting the match was to determine whether individual TerraNova test items assess the Model Academic Standards and the extent to which the standards are addressed by TerraNova items. A copy of the alignment findings may be obtained by contacting the Department’s Office of Educational Accountability.

It is important to note that no single assessment can measure domains as large as those identified in the Wisconsin Model Academic Standards or in locally adopted standards. The Knowledge and Concepts Examinations measure only part of what students need to know. The Department supports district teaching efforts that focus on the breadth and depth of materials encompassed by state and local content standards rather than efforts that are solely and narrowly focused on the items or content of the Knowledge and Concepts Examinations.

2. Classroom Instruction and the WKCE Content

Students may be given instruction, experience, and practice in the objectives that the WKCEs sample. These objectives have been widely distributed to district and school staff in Wisconsin. However, this does not mean that a school or district’s curriculum should be narrowed to fit the objectives covered by the examinations or that teachers should focus mainly on these objectives. These objectives should be covered along with many other objectives that are in the curriculum but not measured on the test. The WKCEs span the content taught over several academic years and are not the sole responsibility of a particular grade-level teacher.

3. Preparing Teachers to Administer the Test

Test administration procedures must be carried out in a way that is consistent with prescribed, standardized procedures in order to give every student an equal opportunity to succeed and to allow for making valid inferences and interpretation of test results. When conducting test administration training sessions, the DAC/SAC should rely entirely on information found in the Directions for Test Administration manual, provided as part of the WKCEs test materials. No actual test books should be used when training the staff.

The responsibilities of the test administrator are to:

a) be familiar with test administration directions before entering any testing session;
b) plan for the distribution and collection of materials;
c) plan student seating arrangement, making sure that spacing between students prevents them from sharing answers;
d) adhere strictly to standardized testing procedures;
e) for students with disabilities under IDEA or students covered by Sec. 504, follow the accommodation provisions outlined in the student’s IEP or IAP and provide accommodations for students with Limited English Proficiency (LEP) consistent with their LEP status, as described in the DPI Guidelines to Facilitate the Participation of Students with Special Needs in State Assessments;
f) ensure that adequate and complete sets of materials are available to all students;
g) provide an adequate testing environment, free from interruption and public address announcements;
h) schedule make-up sessions for absentees; and
i) ensure all security procedures are followed at all times.

4. Preparing Students to Take the Test

THE GOAL OF TEACHING IS TO INCREASE LEARNING RATHER THAN TO INCREASE TEST SCORES. Therefore, students’ attention and effort should be directed to learning the entire scope of the curriculum, not just the limited knowledge and skills measured by the WKCEs.

The Department does not encourage school staff to buy, develop, or promote the use of extensive test practice materials that closely parallel the WKCE’s items or tasks. The Department of Public Instruction released some of the items from the test administered in fall 2005 and will not be used again on future tests. These items may be used as practice tests. Staff must adhere to the following ethical test preparation procedures:

• Student learning should cover the entire scope of the curriculum. Teaching students the entire subject domain is ethical, teaching to the test is not.
• One or two short practice sessions may be given to students to familiarize them with the test format one or two days before the administration of the actual test. Sample WKCE test items are also included in our web site at [http://dpi.wi.gov/oea/releaseitems.html].
• Instruction, experience, and practice should not be limited to the content that the examinations will sample.
• A reasonable notice of the upcoming examination schedule should be given to students, teachers, and parents.
• All students must be encouraged and prepared to participate in the examination. All eligible students must participate in the testing.

5. Reasonable Notice and Full Participation

Reasonable notice of the WKCEs should be provided to all concerned, including teachers, students, and parents. However, this notice should not be used to discourage students from participating in the assessment, particularly if these students are members of groups whose test scores have been historically low. Adequate make-up sessions should be planned so that all students have an opportunity to fully participate in the assessment.

Participation plans in WKCEs for students with disabilities under IDEA must be made on an individual basis and must be specified in the Individualized Educational Program (IEP) of the child.

Although all students with disabilities under IDEA are also covered by Section 504 of the Vocational Rehabilitation Act, there are a limited number of students who are not considered students with disabilities under IDEA but who are covered by Section 504. Students qualifying only under Sec. 504 must be provided with the necessary individual accommodations in testing as specified in the student’s Sec. 504 Individualized Accommodation Plan (IAP). It is possible, although extremely rare, that some of these Sec. 504 students may not be tested.

Decisions regarding inclusion of limited-English proficient (LEP) students must be based on s. PI 16.01, Wis. Admin. Code. Qualified school staff shall determine, on an individual basis, whether an LEP student will participate in the assessment and will specify the type of accommodations that will be provided, if necessary. Students who fall in categories 4 and 5, based on s. PI 13.03 (3), Wis. Admin. Code criteria, may be included in the assessment. The inclusion of these students in the assessment is encouraged by DPI and their exclusion should be justified in writing.

Students with disabilities under IDEA who are appropriately excluded from the WKCEs should complete alternate assessments. Guidance concerning these alternate assessments were published in October 1998 by the Department of Public Instruction, Division for Learning Support: Equity and Advocacy Information Update Bulletin No. 98.14.

Information concerning testing of all students with special needs including students with disabilities,LEP students, and students covered by Sec. 504 or the Vocational Rehabilitation Act of 1973, can be found in the DPI Guidelines to Facilitate the Participation of Students with Special Needs in State Assessments. Additional guidance concerning alternate assessments will be provided by the Department in 1999. Students whose parents request their school board to excuse their student from taking the WKCEs must be excused from testing.

Reasonable notice prior to the administration of WKCEs can be accomplished by providing all students and their parents/guardians with a copy of the Student/Parent Pre-Test Guide.

6. Effect of Ethics on Test Results

Although not acceptable, it is very tempting for some school staff to teach too closely to the actual test questions in order to achieve high test scores. Temptations increase in a testing situation where the stakes are high and where sanctions may be attached to test results. However, a test, no matter how well designed, can only measure a small part of the overall curriculum.

The inferences made about a student who does well on the WKCEs indicate that the student has learned the larger domain from which that test content has been sampled, not solely the content knowledge included in the test. If teachers limit their classroom instruction to skills measured on the test, this assumption has been violated and; therefore, students can only be considered proficient in the particular skills covered on the specific test. These students may not do well on other questions or tasks covering the larger domain.

Similarly, it is very important to precisely follow standardized test administration procedures. If the WKCEs are administered in inappropriate, non-standardized ways, the results will not be comparable to those produced under standardized testing conditions. For example, if a teacher helps students by paraphrasing and explaining the test items and another teacher adheres strictly to the guidelines by repeating the initial instructions, the scores of students in the two classrooms cannot be interpreted in the same manner. In either case, this would result in an inaccurate representation of student learning in Wisconsin.

Strict adherence by school staff to the test standardization procedures and to the guidelines presented in this paper will ensure that the test results are accurate and reflect student learning in our state.

C. TESTING CONDITIONS
I. Testing Procedures

Test administrators must strictly follow the written test administration procedures included in the Directions for Test Administration, which is provided to districts as part of the WKCEs materials. These procedures include planning for the test, organizing the classroom, preparing students to take the test, completing student-identification information, timing of testing sessions, reading instructions to students, and collecting test booklets after each testing session. Failure to follow the specified procedures jeopardizes the validity and integrity of the test results.

2. Testing Environment

Testing conditions should be comfortable and similar for all students. To the extent possible, the conditions should reflect the school’s instructional environment. School Assessment Coordinators and test administrators must ensure that announcements are not made on the public address system during testing sessions, lighting is adequate, chairs and desks are available, and “QUIET” signs are posted. This will permit students to do their best work.

It is recommended that the testing session be conducted in small groups of classroom size, rather than in a large group and auditorium-type hall. This will help students in their concentration since instruction is normally given in smaller, class-size groups.

3. Testing Materials

Before students begin taking the test, test administrators must ensure that adequate and complete sets of test materials are available to all students, including test booklets, pencils, calculators, and manipulatives, such as rulers, protractors, punch-out tools, and geometric shapes, as required.

4. Test Administration

• Test Directions

  Test administrators must be completely prepared and familiar with the test directions before entering any testing session. Administrators should anticipate and be ready to answer questions about the test. When reading test directions aloud, test administrators must ensure that all students understand what is expected of them. Students must be given the opportunity to ask questions and understand how to mark their answers before they begin taking the test. However, test administrators MUST NOT answer questions about specific test items. They may only repeat the initial instructions about item format, scoring rules, and timing. They may also help students with test-taking mechanics but be careful not to inadvertently give clues that indicate the correct answer or help eliminate some answer choices.

• Special Populations

  The Department is committed to including ALL students in testing. Special population students must be included in the WKCEs and, when necessary, be provided with any necessary accommodations to ensure their participation. The majority of these students require minor or no accommodations. Accommodations in assessment for students with disabilities under IDEA, LEP students, and students covered by Sec. 504 of the Vocational Rehabilitation Act of 1973 should reflect the accommodations used in classroom instruction.

  
  

  In those cases where a student with disabilities under IDEA, even with accommodations, would be unable to demonstrate at least some of the knowledge and skills tested in WKCEs, an alternate assessment must be provided to measure the student’s performance. Guidance concerning these alternate assessments was published in October 1998, by the Department of Public Instruction, Division for Learning Support: Equity and Advocacy Information Update Bulletin No. 98.14. Additional guidance concerning alternate assessments will be provided by the Department in 1999.

  
  

  The Individuals with Disabilities Education Act (IDEA) requires that children with disabilities be “included in general State and district-wide assessment programs, with appropriate accommodations, where necessary.” It also requires that the State report “to the public with the same frequency and in the same detail as it reports on the assessment of non-disabled children” specific information about the participation of children with disabilities in assessment and their performance on the assessment. The federal government will monitor the extent to which the state complies with these requirements.

  
  

  Federal and state law require students with disabilities under IDEA to participate in the assessment program. Students with disabilities must be assigned a grade level. Generally, it would be appropriate for the district to use age-based guidelines. These guidelines should allow for some flexibility, recognizing that there is a range of student ages within any grade. There should not be so much flexibility, however, that it is possible to defeat the purpose of requiring student participation in the assessment program.

  
  

  The Department recognizes that there will be some instances where the nature or severity of the disability of a student under IDEA or the English proficiency of a LEP student may necessitate his/her participation in the statewide assessment system through an alternate assessment. Since it is the intent of the Department to include ALL students in the WKCEs, decisions to exclude any student from testing should be made only after careful evaluation of each student’s ability and with written justification.

  
  

  Students covered by Sec. 504 of the Vocational Rehabilitation Act of 1973, must be provided with the necessary accommodations in testing as specified in the student’s Sec. 504 Individualized Accommodation Plan (IAP) .

  
  

  Title I students and students who receive free or reduced-price lunches are not, by such definitions alone, students with disabilities. No accommodations are to be provided when testing students served by Title I unless they also are identified as students with disabilities under IDEA, students covered by Sec. 504, or LEP students. In such cases, the IEP, IAP, or LEP status will determine the type of accommodations needed.

5. Monitoring/Proctoring the Test Session

Test administrators must carefully monitor (proctor) the testing session to ensure that all students have the opportunity to succeed. It is not acceptable for test administrators to leave the room, visit with another person, read, or ignore what is happening in the testing session. Test administrators and proctors must be trained to follow the testing procedures and to understand the significance of their responsibilities. Test administrators must:

a) Study the Directions for Test Administration manual thoroughly and be prepared to answer questions.
b) Follow standardized test administration instructions. Adhere strictly to standardized procedures and follow the written script, verbatim, without adding or deleting information.
c) Ensure that all students understand the directions.
d) Be sure that students know how to mark their answers. Help students in their test-taking mechanics without inadvertently giving them hints that indicate an answer or help eliminate answer choices.
e) Encourage students not to spend too much time on any test item. Be careful not to imply that they should guess randomly. However, tell them that if they do not know the correct answer they can eliminate some of the choices to help them find the correct answer.
f) Encourage students to attempt to answer every item on the test. Monitor their work ensuring that they do not skip or overlook any of the test questions.
g) Ensure that students respond in the appropriate places in the test booklet.
h) Direct students to mark only one response for each selected-response item and ask them to erase completely any responses they do not want.
i) Ensure that students do not exchange or copy answers from each other.
j) Encourage students who complete the test before regulation time to review their answers.
k) Ensure that students are not disruptive and do not interfere with or distract each other.
l) Ensure that students use only permitted test materials and devices.
m) Follow the provisions of the IEP for students with disabilities under IDEA.
n) Follow the provisions of the IAP for students covered under Sec. 504.
o) Follow Department’s guidelines for testing of LEP students.
p) Encourage students to do their best on the test and to check their work.
q) Do not engage in conversation with other staff while the testing session is in progress.
r) Collect and check all materials when the testing session has concluded.
s) Write a report about all deviations, irregularities, and anomalies that may have compromised the testing situation and give to your school principal or school assessment coordinator.
t) Keep track of absent students and plan make-up sessions.

D. POST-TEST ACTIVITIES/PROCEDURES
1. Collecting Test Materials and Completing the Report

[Confirmation of Destruction /Confidentiality Agreement WKCE]

When the testing session has concluded, the test administrator will collect and check all materials and follow test security procedures. The test administrator must account for all test materials and deliver them to the SAC immediately.

The test administrator must write a report of all incidents and events that may have compromised the testing situation and could have the potential of invalidating test scores. This includes disruptions, illnesses, cheating, refusal of students to complete the test, etc. The report must be submitted to the school principal and/or school assessment coordinator who, in turn, will take it to the district assessment coordinator and/or the district administrator.

2. Use of Test Information

School and district staff must follow strict confidentiality measures to protect individual student test scores and maintain student privacy, as required by federal, state, and local laws. Students' scores should be made available only to authorized personnel, i.e., the student, the student’s parents or legal guardians, and the specific staff responsible for the student’s education.

Schools must make sure that test interpretation guides are provided to students and their parents and should use student scores in context with other relevant information about that student.

3. Making the Test Instruments Available to the Public

The Department will comply with the requirement of s. 118.30(3), Wis. Stats., by making the WKCEs available to the public upon request, within 90 days after the date of administration. However, such availability will be conducted under limited and controlled conditions and will be based on the ability of school staff to assure the security of the test contents and answers, confidentiality of individual students’ test results, and capability of explaining the purpose of the test and the meaning of test results. These conditions include that the:

a) School district has filed the “Confidentiality Agreement” form, signed by the district administrator, with the Department prior to meeting with any member from the public for the purpose of viewing the WKCEs.
b) School district will have the viewer sign a confidentiality agreement form, provided by the Department and printed on a district letterhead, prior to viewing the test.
c) The test should be disclosed to a member of the public only upon request.
d) Viewer will be permitted to view the test and test results without any violation of student privacy or jeopardizing student confidentiality issues.
e) Viewer will be accompanied by a qualified school staff member at all times.
f) Viewer will NOT be allowed to copy or take notes on any portion of the test.
g) School district will have a qualified staff member available to explain the proper use of the test, the purposes of the WSAS program, and the meaning of test results.

Refer to Section A, Item 5, page 2, of this paper for more information on the public review of the WSAS Knowledge and Concepts Examinations.

---

Related Web Pages

• Test Security Training (A mediasite presentation)
• Confidentiality Agreement
• Printable Version - Testing Security Guidelines
• Confirmation of Destruction

---

Certain documents linked to this page are in .pdf format and require Adobe Acrobat Reader. A free Acrobat Reader and instructions on configuring your browser are available from Adobe.

September  2004

For questions about this information, contact oeamail@dpi.wi.gov

Last updated on 2/26/2008 10:43:09 AM