The Reliability of the CVI Range: A Functional Vision Assessment for Children with Cortical Visual Impairment
Under the Individuals with Disabilities Education Act (IDEA, 2004), children who are identified as having a visual impairment are required to have additional assessments to determine how the disability affects their educational performance. Medical reports and visual acuities may help determine eligibility for educational services, but they do not provide useful information about the impact of a particular child's vision loss on that child's education (Teplin, 1995). Instead, educational impact is determined through a functional vision assessment or a learning media assessment or both that helps delineate how a child uses vision to perform activities in the classroom or activities of daily living (Burnett & Sanford, 2008; Langley, 1998). On the basis of this information, appropriate developmental and educational modifications and accommodations can be determined that will give the child access to the educational curriculum.
Most current functional vision assessments, such as ISAVE, the Individualized Systematic Assessment of Visual Efficiency (Langley, 1998), and the Functional Vision and Learning Media Assessment (Burnett & Sanford, 2008), have been based on the needs of children with ocular visual impairments. Children with visual impairments that are due to brain damage, or cortical visual impairment (CVI), have unique visual characteristics that are often different from those of children with ocular visual impairments (Jan & Groenveld, 1993). Despite these documented differences in the visual skills of children with ocular versus cortical visual impairment, the functional vision assessments that are currently used in the field rely heavily on the needs of children with ocular visual impairment and do not take into account the unique characteristics of children with CVI.
In response to this need, Roman-Lantzy (2007) developed a functional vision assessment that appropriately addresses the visual characteristics of children with CVI, the CVI Range. However, to be considered a worthwhile instrument, an assessment needs to have sound psychometric properties, such as reliability and validity. This article describes a study that assessed the reliability of the CVI Range. It first offers a brief review of studies that have described the assessment of children with CVI. Next, it examines the reliability of the CVI Range, including internal consistency, test-retest, and interrater reliability. Finally, it discusses the implications of these findings.
Review of the literature
Visual characteristics of children with CVI
Jan and his colleagues were among the first researchers to document the unique visual characteristics of children with CVI. In a series of studies (Good & Hoyt, 1989; Jan, Good, & Hoyt, 2006; Jan, Groenveld, & Anderson, 1993; Jan, Groenveld, & Sykanda, 1990; Jan, Groenveld, Sykanda, & Hoyt, 1987) over 20 years, these researchers systematically documented the characteristics of children with CVI. They found a unique set of visual and behavioral traits that applied to this population, including such behaviors as visual inattention, light gazing, color preferences, field losses, difficulty with distance viewing, looking away when reaching, and difficulty viewing novel objects. Dutton (2003; see also Dutton et al., 1996) also conducted extensive research on the characteristics of children with CVI. He developed a model of how the visual system works and the effects of damage to the different visual pathways. He noted specific visual behaviors that were associated with damage to the ventral and dorsal streams in the brain, including difficulty handling complex visual scenes, difficulty moving around the environment, and lower field loss.
Measurement of vision in children with CVI
The measurement of vision in children with CVI is often a challenge (Morse, 1992). Two major types of information are of interest to professionals who work with children with visual impairments. First, professionals need testing that confirms a diagnosis of CVI and the relative degree of visual impairment. Appropriate medical documentation of a visual disability is usually needed to gain access to appropriate services from personnel who are trained in visual impairment. Second, beyond a diagnosis, educational professionals are interested in functional measures of vision, such as how well a child uses vision in daily tasks.
In the assessment of visual skills, Colenbrander (2006) suggested a distinction between visual functions and functional vision. Visual functions describe how the eye functions and describes skills, such as fixation, tracking, visual fields, and various acuity measures. Visual functioning describes how the child functions in vision-related tasks and includes measures such as using vision to orient in the environment and using vision to recognize and manipulate objects. The following sections describe how various researchers have attempted to measure visual functions as well as functional vision.
Assessment of visual functions
The visual functions that are measured in children with CVI are typically acuity, visual fields, and general visual awareness (fix and follow and response to objects, for example). The most common visual function that is measured is visual acuity, or the ability to discriminate and recognize detail (Teplin, 1995). One test that is used frequently to measure visual acuity is the visual evoked potential (Good, 2001; Good & Huo, 2006; Watson, Orel-Bixler, & Haegerstrom-Portnoy, 2007). Another measure of visual acuity that is often used with children with CVI is the Teller Acuity Card (TAC) procedure (Birch & Bane, 1991; Weiss, Kelly, & Phillips, 2001).
Many researchers have also examined field losses in children with CVI. Two methods of testing visual fields have been described in research on children with CVI: an arc perimetry (Cioni et al., 1997; Jan et al., 1993) and confrontation testing. With confrontation testing, when the child is fixated on a target, other targets are presented in the upper, lower, left, and right fields, and the examiner notes if the child notices the object presented in the periphery and in which fields. Dutton et al. (1996) and Jacobson, Ek, Fernell, Flodmark, and Broberger (1996) used the confrontation method to determine possible field loss.
In addition to the more formal assessment of visual functions, most researchers have used informal observations of various other skills. Dutton et al. (1996) made note of each child's ability to fix and follow, reach near objects, and show facial awareness. Jan et al. (1993) made note of each child's response to bright objects. Cioni et al. (1997) rated the following: eye contact, nystagmus, fix and follow, convergence, eye alignment, and blink-to-threat reflex. Brodsky, Fray, and Glasier (2002) used behavioral responses to light and moving objects when children could not respond to TAC acuity procedures.
Assessment of functional vision
For educators, it is more critical to measure functional vision, or how a child uses vision in the context of daily tasks. In the literature, prior to the development of the CVI Range only two methods of determining functional vision for children with CVI were described. Jan et al. (1987) used a five-point classification of functional vision, from "no apparent vision" to "vision beyond 10 feet." Huo, Burden, Hoyt, and Good (1999) developed a six-level measure, which ranged from Level 1 (light perception only) to Level 6 (completely normal vision). Neither system of quantifying functional vision included any information about how a child actually used vision in tasks of everyday life or in an environment other than a medical clinic.
The CVI Range
Based primarily on the findings of researchers in British Columbia, Roman-Lantzy (2007) developed an instrument, the CVI Range, that specifically addresses the unique visual characteristics of children with CVI. This assessment built on her earlier work to validate an interview instrument that could differentiate children with CVI from children with ocular visual impairment (Roman, 1996). Characteristics of children with CVI that had been identified in the literature were used to design an interview and to develop a behavioral observation protocol. Experts in the field of CVI established the content validity of the interview questions. Roman found that the interview could successfully differentiate children with CVI from children with ocular visual impairment and that there was concurrent validity with the visual behavioral observations. In subsequent work, Roman-Lantzy developed the CVI Range for children who have been diagnosed with CVI to determine how the various visual and behavioral characteristics that are unique to children with CVI were affecting a child's functional vision.
Administration of the CVI Range involves a combination of an interview with the parent, observation of the child, and direct assessment. Specific directions for the assessment and scoring of each item on the CVI Range are presented in Cortical Visual Impairment: An Approach to Assessment and Intervention (Roman-Lantzy, 2007). The following characteristics are rated in the CVI Range: color preference, need for movement, visual latency, visual field preference, difficulties with visual complexities, light-gazing-nonpurposeful gaze, difficulty with distance viewing, atypical visual reflexes, difficulty with visual novelty, and the absence of visually guided reach. Roman-Lantzy selected these characteristics on the basis of a review of the literature, her previous interview research, and her personal work with children with CVI.
The CVI Range is scored in two ways. The first section of the CVI Range is the Across-CVI-Characteristics method. This method provides a "snapshot" of the child's visual abilities at different levels of visual functioning (Roman-Lantzy, 2007, p. 54). Each statement describes a behavior that pertains to one or more of the CVI characteristics. The second section is the Within-CVI-Characteristics method, in which each characteristic is scored separately to describe the degree to which each is interfering with the child's visual functioning. The child's final score is a number between 0 and 10, with 0 indicating no visual responses and 10 representing typical or near-typical vision.
The overall scores on the CVI Range are then divided into three broad categories or phases. Phase I includes children whose scores are 0-3; Phase II includes children whose scores are 3.25-7.0; and Phase III includes children whose scores are 7.25-10. At each phase, there are broad overarching visual goals. The major goal of Phase I is to build consistent visual behaviors, the main goal of Phase II is to integrate vision into all functional routines, and the main goal of Phase III is to demonstrate visual curiosity and to use vision in all tasks consistently and spontaneously.
Reliability involves the ability to measure consistently any behavior of interest (Salvia & Ysseldyke, 2007). The extent to which a test yields the same results on repeated trials is the extent to which it is considered to be reliable (Carmines & Zeller, 1979). The purpose of this study was to examine the reliability of the CVI Range. The following types of reliability were examined: internal consistency, test-retest, and interrater reliability.
Children with CVI were assessed with the CVI Range by professionals who had been trained during a five-year, multistate CVI Mentorship Project. Twelve professionals, including this author and Roman-Lantzy, conducted the assessments for this study. The evaluators included three special education teachers-early interventionists, six teachers of students with visual impairment, two occupational therapists, and one neonatologist. These assessments were analyzed to determine the reliability of the CVI Range. Before the data were collected, approval was obtained from the University of Maryland's Institutional Review Board.
A data analysis to determine the reliability of the CVI Range was conducted on assessments from 104 children. For the internal consistency analyses, one assessment from each of the 104 children was used. From this total sample of 104 children, 27 children were assessed by two examiners for interrater reliability, and 20 children were assessed twice for test-retest reliability. The average age of the sample was 46.5 months, with a range of 6-144 months. The children were recruited as part of a multistate project in which children who were diagnosed with CVI could be referred to a CVI mentor. All the children had additional disabilities, including developmental delay, intellectual disability, cerebral palsy, health impairment, hearing impairment, and other ocular conditions. The children were representative of the population of children with CVI that has been described in the literature.
All the children received early intervention or special education services. Additional services included physical therapy, occupational therapy, speech therapy, vision services, and hearing services. For children who were eligible for early intervention services (those from birth to 36 months), their homes were the most frequent placement. For children who were eligible for special education services (those aged 36 months and older), the placements represented a continuum from their homes to special centers.
One of the assumptions of reliability is that all the items on an assessment measure one underlying construct. Internal consistency is a measure of how well the items in a test correlate with each other. High correlations are suggestive of the consistent measurement of a single construct (Carmines & Zeller, 1979). It was important to look at the internal consistency of the CVI Range because the total score is used to describe the severity of CVI and is used in the other reliability analyses. A high alpha would indicate that the scale was likely measuring a single underlying construct. In addition, the total score could then be used to determine test-retest and interrater reliability. For this study, 104 assessments were analyzed using Chronbach's alpha as a measure of internal consistency. As a general rule, reliabilities are acceptable at .80 for most scales. At that level, the measurement represents mostly a true score and little random error. The CVI Range had an internal consistency measure of .96, which represents a very high reliability. The higher the alpha level, the more likely that the scale is measuring a single underlying construct--in this case, the severity of CVI.
Test-retest reliability is an index of stability across time (Nunnally, 1978). For this study, 20 children were tested on two different occasions by the same examiner, in the same setting, at the same time of day, with the same materials. The same examiner was used to avoid confounding the results with interrater issues. The time between the two tests ranged from 1 to 14 days, with an average of 6.7 days. The two sets of scores were correlated using the Pearson's correlation coefficient yielding a test-retest reliability of .99. Test-retest reliability was also computed using Cohen's kappa, which is appropriate for calculating agreement with categorical data (Fleiss, Levin, & Paik, 2003). There was perfect agreement in phase placement from the first test to the second for all 20 children. On the basis on the correlation coefficient (.99) and kappa (1.0), the CVI Range has very high test-retest reliability.
For interrater reliability, two similarly qualified examiners must derive the same or similar results on a given assessment protocol (Nunnally, 1978). For this study, 27 children were assessed by two trained examiners who then independently scored the CVI Range. The two sets of scores were correlated using the Pearson's correlation coefficient, which yielded an interrater reliability of .98. In addition to the correlation, interrater reliability was computed using Cohen's kappa. A kappa greater than .75 represents excellent agreement beyond chance, and the kappa was .83 for the interrater reliability of the CVI Range. A further analysis of the absolute difference in the scores of the two examiners revealed a mean difference in scores of .31 points (range 0-2 points). Differences in phase placement were affected by cutoff points (3.0 is considered Phase I, and 3.25 is considered Phase II) more than by large differences in total scores.
Comparison of the Across- and Within-CVI Characteristics methods
As was described earlier, the CVI Range is scored by two different methods. The final question that was examined was whether the two parts of the CVI Range placed the child in the same phase. When a child had more than one assessment, the researcher randomly chose one of the assessments to be used in the analysis. In the total sample, 34 children (33%) scored in Phase I, 45 children (43%) scored in Phase II, and 25 children (24%) scored in Phase III. The kappa was .88 for the CVI Range. The average difference in the scores in Part I and Part II was .55 (range 0-1.5 points). There was a very high agreement in the scoring of Part I and Part II of the CVI Range.
The purpose of the study was to determine the reliability of the CVI Range, and the results of the study indicate that this assessment has high interrater, test-retest, and internal consistency reliability. In a comprehensive review of the literature, no researcher who assessed or designed interventions for children with CVI used this assessment as a measure of functional vision, perhaps because this is a recent assessment. However, no one described a functional vision assessment that addressed any of the known characteristics of children with CVI. This research demonstrates that the CVI Range has good psychometric properties and is therefore a high-quality instrument to use as a functional vision assessment for children with CVI. One of the major implications of this study is that the CVI Range needs to be consistently used to determine a child's visual needs when the child is diagnosed with CVI.
The results of the CVI Range can help determine which of the characteristics of CVI are interfering with the child's use of vision and can give valuable information about how materials and the environment need to be adapted for the child to use his or her vision. For example, if the examiner discovers that the child responds best to certain colors or that the child needs to be in a supported position to use vision, this information is critical in designing appropriate interventions. Another major implication of the study is that each characteristic that is affecting functional vision needs to be addressed in designing interventions.
Training and the CVI Range
The interrater reliability in the study was very high (r = .98); however, the assessments were conducted by highly trained CVI mentors. Each mentor had participated in five years of training with Roman-Lantzy and conducted numerous CVI assessments. Although this is one of the strengths of the study, it can also be seen as a limitation. With regard to whom these results can be generalized, there are few individuals who may use this instrument who have such a high level of training and supervision. With confidence, the conclusion can be made that well-trained evaluators can score the CVI Range reliably. The question then is this: How much training is enough? Does the training have to be conducted by Roman-Lantzy, or can independently trained evaluators be as reliable? The results of an assessment given by someone who only read Roman-Lantzy's (2007) book may be different from the results of an assessment given by someone who has trained extensively. This does not make the instrument unreliable; it only means that the question of training must be addressed in future research.
Another training need that emerged from the study was related not to the administration of the CVI Range, but to demographic data that were collected on the children. The children in the study, who reflect most children with CVI, had additional disabilities and were served by multiple providers. This point highlights the need for training about CVI across disciplines. Although not everyone may need to be qualified to administer the CVI Range, all professionals as well as family members need to understand the characteristics of CVI that affect a child's vision and how interventions are designed to meet the child's visual needs. For example, a speech therapist would benefit from understanding how the child's vision affects the development of a communication system. Can the child look at pictures? Do object symbols need to be presented on a plain background? The physical therapist would benefit from knowing that the child cannot use vision when he or she is in a challenging position, so that the therapist may use music, rather than a visual target, as a motivator during therapy sessions.
Many of the children in this study also had medical needs and were involved with various medical professionals. Families often commented that medical professionals had not given them any information about CVI or interventions that might work. Again, medical professionals may not need to know how to administer the CVI Range, but they do need to understand the information that is covered by the assessment and that there are resources available to families.
The stability of visual functioning in children with CVI
A significant finding of the study was the test-retest reliability (r = .99). For a long time, the conventional wisdom in the field has been that children with CVI have vision that fluctuates from day to day or even hour to hour. The test-retest reliability finding in the study is in direct opposition to previous findings. Given stability in examiners, settings, materials, and time of day, the children with CVI in this study did not have functional vision that fluctuated. This was true for children across all three phases of CVI.
The test-retest reliability finding has several implications. First, the finding needs to be replicated. Another study of test-retest reliability needs to be conducted with more children at each level of CVI. The current study had children at all levels, but included only 20 children. The results need to be replicated with a larger sample of children.
Another implication of this finding is that there may be another plausible explanation for previous findings of variable visual functioning. Looking at the results of an assessment using the CVI Range, one can see that any number of factors can affect a child's visual functioning. In earlier observations, researchers did not take into account any reason for the child's variable responses. In addition, they did not use a systematic measure of functional vision, such as the CVI Range. They used informal observations without a measure to quantify their findings. The child may have looked at a toy at home, but in the physician's office with a different toy, novelty was affecting visual functioning. The child may have looked at a cup when his or her mother was wearing a plain shirt and did not look when presented with the same cup when the mother was wearing a flower print shirt, thus increasing the complexity of the background. In both cases, the conclusion could be drawn that the child looks sometimes, but not at other times, while it was actually something in the environment that changed more than the child's vision.
The test-retest finding also has implications for assessment and intervention. When assessment is used for pre- and posttest measures, conditions need to be similar in both situations. If the results of an assessment are used to make decisions about the effectiveness of an intervention, there needs to be stability in the assessment conditions. Understanding that the child's vision is stable also has an effect on how families, teachers, and therapists view the child. If significant adults believe that the child has stable vision, they are more likely to take responsibility for engineering the environment for the child's success. I have observed that when a child is not looking at something, adults who believe the child's vision is variable are likely to conclude, "Well, that's what kids with CVI are like. Sometimes they look, sometimes they don't." If they do not believe that vision is variable, adults are more likely to examine other factors in the environment and adapt them so the child can be successful. Understanding that vision is stable, but can be influenced by many things in the environment, puts the responsibility for success on the adult, not the child.
Conclusion and directions for future research
CVI is the leading cause of visual impairment in young children (Hatton, Schwietz, Boyer, & Rychwalski, 2007). Early researchers primarily conducted studies that described the causes and traits of children with CVI. Early assessment studies focused on the medical diagnosis and typical measures of visual functions, such as acuity. As more children with this condition have been identified in early intervention and special education, the need to understand functional vision and to develop appropriate interventions for children with CVI has increased. This study represents one step forward in identifying the needs of children with CVI. The CVI Range has high interrater, test-retest, and internal consistency reliability and, as such, can be used with confidence as a high-quality functional vision assessment for children with CVI. A good functional vision assessment is the bridge to the next research focus in the field: the development of evidenced-based interventions.
Children with CVI can and often do make progress in functional vision (Huo et al., 1999; Khetpal & Donahue, 2007; Lueck, Dornbusch, & Hart, 1999; Matsuba & Jan, 2006). In the future, systematic studies that match children's characteristics to interventions and that document progress across time will lead to a body of literature about practices that are effective. Only with detailed assessments to design interventions and a way to monitor change in children can any intervention be called evidenced based. The results of this study indicate that the CVI Range is a stable instrument that can be used to design interventions and to monitor change in children with CVI.
Previous Article | Next Article |
Table of Contents
JVIB, Copyright © 2010 American Foundation for the Blind. All Rights Reserved.