Building Assessment Initiatives for Schools: Guidelines to Support the Contract Development Process Between Test Publishers and States

Braille Downloads

• Download .BRF version of guidelines for contract development
• Download .DXB version of guidelines for contract development

19th Annual Josephine L. Taylor Leadership Institute

Boston, Massachusetts

Friday, March 11, 2005

Introduction

Contracts and Requests for Proposals (RFPs) negotiated between state assessment agencies and test publishers carefully outline the responsibilities and expectations for the state assessment development and implementation process. While these documents have specific points for consideration, often language does not include the assurance of accessible test development and implementation. Accessible test items enable all students to participate in the assessment process in a way that allows abilities rather than disabilities to be assessed. Accessible formats of tests, including the practice tests, must be available for students with visual impairments at the same time as their sighted peers. The checklists provided below outline considerations for inclusion in each state's RFP or test contract. The usual contractual language found in state contracts should be employed, with these special considerations added.

Universal Design Principles

The following guidelines are general considerations for contract and RFP development that ensure test development and use for all students, including those with disabilities:

• The same assessment system is used to measure the achievement of all public school students in the state. Groups to be included in the state assessment need to be clearly defined.
• The student assessment system provides coherent information on attainment of state standards across grades and subjects.
• The tests are designed to be valid and accessible for all students. This includes students with disabilities and students with limited English proficiency.
• The tests are aligned with the state's challenging academic content and student achievement standards.
• The tests are valid, reliable, technically sound, and consistent with nationally recognized professional and technical standards such as national test publisher standards and guidelines of the American Psychological Association (APA) and American Educational Research Association (AERA).
• The reporting system allows results to be disaggregated (according to the No Child Left Behind Act (NCLB) and Individuals with Disabilities Improvement Act of 2004 (IDEA) guidelines) within each state and local education agency and school by gender, racial and ethnic group, migrant status, disability, socioeconomic status, and limited English proficiency.
• The tests involve multiple up-to-date measures of student academic achievement, including measures that assess higher-order thinking skills and understanding.
• The reporting system allows production of individual student reports.

Roeber, E. (2003). Assessment models for No Child Left Behind.

Education Commission of the States. http://www.ecs.org/html/Document.asp?chouseid=4009

Item Development and Review Process with Publisher

The following guidelines are provided for consideration as language to include in contracts and RFPs that ensure the development and implementation of accessible test formats, specifically for students with visual impairments.

• Test publishers must maintain access to experts, i.e. individuals who know and have either taught or are knowledgeable about braille, tactile graphics, large print, and audio. These individuals can provide information during each phase of test development.
• Experts in visual impairment must be included on Item Writing Committees and Bias Review Committees.
• The use of accommodations must be considered during test item development to ensure appropriateness to test purpose and test access.
• The test item pool must be large enough for Bias and Item Review Committees to replace items determined to be inaccessible when presented in braille, large print, audio formats, or as tactile graphics.
• A representative sample of students with visual impairments needs to be included in any field-testing of the assessments, as prescribed in Standard 10.3 (p. 106) of the Standards for Educational and Psychological Testing (1999).
• An adequate amount of time for tests and practice tests to proceed through a subcontractor's processes needs to be built into contracts so that accessible media as required by each student's Individualized Education Program (IEP) are delivered at the same time as the original test materials.
• All test administrators' manuals, supplemental manuals which accompany the accessible media versions of tests, and local test administrators'/proctors' instructions and training manuals must be provided in accessible formats for visually impaired staff. These accessible materials must be requested far enough in advance to allow for delivery at the same time as the original test materials.
• At the end of each testing season, both students and teachers should give input regarding the testing experience.
• Item analyses for accessible format test items will be carried out at the end of each school year (or testing season) as part of a continuous improvement plan.

Allman, C.B. (2004). Test Access: Making tests accessible for students with visual impairments:

A guide for test publishers, test developers, and state assessment personnel. Second Edition.

Louisville, KY: American Printing House for the Blind. http://www.aph.org/tests/access2/index.html

Accessible Media Development with Subcontractors

This section provides guidelines for consideration when contracts are developed with subcontractors such as agencies or individuals who will provide tests in one or more accessible formats (braille, tactile graphics, large print, and audio). The process may include steps for editing, transcribing, designing tactile graphics, proofing, producing and quality checking the accessible media. It is essential that the timeline allow adequate time for each of these steps. Additional time may need to be built into contracts depending on specific requirements of the state such as an independent proofreading by another person or agency, or aligning various media for multimedia presentations.

• The subcontractor must agree to work closely with the test publisher, the state department of education, and the test editor.
• The construct to be measured must be documented by the test publisher in test item specifications and made available to test editors and accessible media producers.
• Proofreading by a qualified individual, i.e. a person who knows the needed codes and formats and is experienced or certified (if applicable), in braille, tactile graphics, large print, and audio versions of the test must occur before multiple copies are made. High-stakes tests should be proofed a minimum of two times.
• Accessible versions of the test must be aligned so that a multimedia presentation (as approved by state assessment programs) is possible if specified by a student's IEP.
• Allowable test format changes, accommodations, and general assistance to test takers by the test administrator or proctor must be stated in the test administration manual or supplemental materials produced by the subcontractor.
• Subcontractors must be able to meet their deadlines so that high quality accessible media are delivered to school systems at the same time as the original test materials.
• Test security and confidentiality standards must be upheld by testing subcontractors.

Allman, C.B. (2004). Test Access: Making tests accessible for students with visual impairments:

A guide for test publishers, test developers, and state assessment personnel. Second Edition.

Louisville, KY: American Printing House for the Blind. http://www.aph.org/tests/access2/index.html

Resources

Assessment Models for No Child Left Behind,

from Education Commission of the States (ECS) http://www.ecs.org/clearinghouse/40/09/4009.doc

Building Tests to Support Instruction And Accountability: A Guide for Policymakers,

from National Education Association (NEA) http://www.nea.org/accountability/buildingtests.html

Designing School Accountability Systems,

from Council of Chief State School Officers (CCSSO) http://www.nciea.org/publications/desigSchAccSyst_Gong02.pdf

Illustrative Language for an RFP to Build Tests to Support Instruction and Accountability,

from American Association of School Administrators (AASA) http://www.aasa.org/issues_and_insights/assessment/Illustrative_Language_for_an_RFP.pdf

Information on Writing a Request for Proposal (RFP)

http://www.arches.uga.edu/~ninaaug/ITclasses/7550/

Model Contractor Standards and State Responsibilities for State Testing Programs,

from Education Leader's Council (ELC) http://www.accountabilityworks.org/publications

National Federation of the Blind (NFB/New Hampshire Resolution on Accountability)

http://www.education-rights.org/brailletwomey11399.html

Tennessee RFP for Development of Online Tests

http://www.state.tn.us/finance/rds/ocr/rfp/rfp33104001.pdf

Test Access: Making Tests Accessible for Students with Visual Impairments:

A Guide for Test Publishers, Test Developers, and State Assessment Personnel. Second Edition.

American Printing House for the Blind. http://www.aph.org/tests/access2/index.html

The Standards for Educational and Psychological Testing (1999),

American Psychological Association http://www.apa.org/science/standards.html

Thompson, S., & Thurlow, M. (2002). Universally designed assessments: Better tests for everyone! (Policy Directions No.14).

Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Retrieved 1-28-05 from the World Wide Web: http://education.umn.edu/NCEO/OnlinePubs/Policy14.htm

Contributors

Dr. Carol Allman, Accessible Tests Department with the American Printing House for the Blind, allmanc@prodigy.net

Barbara Henderson, Accessible Tests Department with the American Printing House for the Blind, bhenderson@aph.org

Debra Sewell, Texas School for the Blind and Visually Impaired, debrasewell@tsbvi.edu

Mary Ann Siller, American Foundation for the Blind, siller@afb.net

Debbie Willis, Accessible Tests Department with the American Printing House for the Blind, dwillis@aph.org

Permission is given to distribute copies with appropriate credit: American Foundation for the Blind, American Printing House for the Blind and Texas School for the Blind and Visually Impaired from the Josephine L. Taylor Leadership Institute, March 11, 2005.