Reliability of trachoma clinical grading--assessing grading of marginal cases

BACKGROUND

Clinical examination of trachoma is used to justify intervention in trachoma-endemic regions. Currently, field graders are certified by determining their concordance with experienced graders using the kappa statistic. Unfortunately, trachoma grading can be highly variable and there are cases where even expert graders disagree (borderline/marginal cases). Prior work has shown that inclusion of borderline cases tends to reduce apparent agreement, as measured by kappa. Here, we confirm those results and assess performance of trainees on these borderline cases by calculating their reliability error, a measure derived from the decomposition of the Brier score.

METHODS AND FINDINGS

We trained 18 field graders using 200 conjunctival photographs from a community-randomized trial in Niger and assessed inter-grader agreement using kappa as well as reliability error. Three experienced graders scored each case for the presence or absence of trachomatous inflammation-follicular (TF) and trachomatous inflammation-intense (TI). A consensus grade for each case was defined as the one given by a majority of experienced graders. We classified cases into a unanimous subset if all 3 experienced graders gave the same grade. For both TF and TI grades, the mean kappa for trainees was higher on the unanimous subset; inclusion of borderline cases reduced apparent agreement by 15.7% for TF and 12.4% for TI. When we assessed the breakdown of the reliability error, we found that our trainees tended to over-call TF grades and under-call TI grades, especially in borderline cases.

CONCLUSIONS

The kappa statistic is widely used for certifying trachoma field graders. Exclusion of borderline cases, which even experienced graders disagree on, increases apparent agreement with the kappa statistic. Graders may agree less when exposed to the full spectrum of disease. Reliability error allows for the assessment of these borderline cases and can be used to refine an individual trainee's grading.

Clinical examination and laboratory tests for estimation of trachoma prevalence in a remote setting: what are they really telling us?

Worldwide, an estimated 84 million people have active trachoma and 7.6 million people have trachomatous trichiasis. WHO's SAFE strategy is an effective tool in the worldwide effort to eliminate blinding trachoma, but its institution and monitoring requires a simple, reliable, and cost-effective method to detect disease. To date, clinical examination has provided the main method of diagnosis. Detection of Chlamydia trachomatis with nucleic acid amplification tests does not always correlate well with clinical findings, which has prompted the suggestion that these methods should replace clinical examination. However, a review of the research carried out in animals and human beings suggests the relation between laboratory tests and clinical examination is due to the kinetics of trachoma and not to an inherent problem in either detection system. Given the increased difficulties of using laboratory tests in parts of the world where trachoma is endemic, we should not abandon clinical grading as a tool to assess the need for, and the effectiveness of, trachoma intervention programmes.

Modified grading system for upper eyelid trachomatous trichiasis

AIM

This study was conducted to determine the level of inter-observer agreement in the modified grading scheme for trichiasis of the upper eyelid.

METHODS

A grading system that covers the entire spectrum of trachomatous trichiasis, extending from the simple to the severe forms, was initially developed by the first and second authors and field-tested. The grading scheme recognizes three levels of severity, based on deviation of the lid margin (TT1, TT2 and TT3); and four anatomical locations indicating where the eyelashes touch the globe in TT1 (nasal, central, temporal, or combined). The grading of trichiasis patients was then conducted on the basis of the modified grading scheme.

RESULTS

The inter-observer agreement between the grades of simple trichiasis (TT1) and entropion (TT2) using Cronbach's alpha was 0.84, and the agreement within each sub-grade was 0.86. The weighted Kappa value was 0.68 (95% CI 0.57-0.79) for the main grades and 0.49 (95% CI 0.23-0.79) for the sub-grades.

CONCLUSIONS

The inter-observer agreement was very reliable and could easily be attained by all levels of ophthalmic professionals. As the grading follows the natural progression of trichiasis from the simple to the severe forms in relation to the anatomical site involved, it helps to determine the degree of correction and how far to extend the surgical incision.

Risk factors for trachoma: 6-year follow-up of children aged 1 and 2 years

The authors investigated the long-term stability of risk factors in predicting the presence of active trachoma and severe inflammatory trachoma in 176 children in Kongwa, Tanzania, who were aged 1 and 2 years in 1989 and were available for follow-up in 1995. Familial cattle ownership, living more than 2 hours away from a water source, and facial cleanliness at both time points were associated with the presence of active trachoma at both time points (odds ratio (OR) = 2.58, 95% confidence interval (CI): 1.15, 5.79; OR = 3.07, 95% CI: 1.23, 7.64; and OR = 0.52, 95% CI: 0.26, 1.03, respectively). An association of familial cattle ownership with facial cleanliness and water accessibility was observed. Having a clean face at both time points was associated with lower odds of active trachoma at both time points for children in non-cattle-herding families (OR = 0.40, 95% CI: 0.18, 0.87). Living more than 2 hours away from a water source at both time points increased the odds of active trachoma at both time points in children of cattle-herding families (OR = 8.00, 95% CI: 1.99, 32.10). Noticeably, severe inflammatory trachoma at baseline predicted mortality in children from villages in which trachoma was less common (OR = 3.75, 95% CI: 1.09, 12.98). The results suggest that risk factor reduction could diminish persistent disease.

Trachoma in Jimma zone, south western Ethiopia

A population-based survey on trachoma was carried out in Jimma zone. The survey showed that trachoma is hyperendemic: of 7,423 people examined, 33.02% had signs of trachoma, 28.9% of males and 37.01% of females; 26.13% of the urban and 34.09% of the rural population had clinical signs of trachoma. Active trachoma (TF/TI) was seen in 24.5% of the study population (an estimated half million people). The prevalence of signs of active trachoma among children 0-10 years of age was 35.7%. Blinding trachoma (CO/TT) was seen in 3.81% of the population: 6.86% of the female population aged 15 years had trichiasis and/or entropion. Both active and critical trachoma were significantly associated with the female gender (P < 0.0000001 for each), living in rural areas (P < 0.0001 for each type), parental illiteracy (P < 0.0000001 for each) and absence of a latrine (P < 0.01 for each). Shorter distance of the household water supply was associated with a higher prevalence of both active and cicatricial trachoma. This was significant only for cases of active trachoma living in households within 16 to 30 minutes walking distance from the water source (P < 0.03). An estimated 17000 people in the zone are blind; 3500 of these from trachoma. About 52,000 people are in danger of blindness from trichiasis. Primary eye care activities such as promoting health education (face washing) among community members, especially women and children, mass chemotherapy for trachoma, training of health workers and establishment of community-based surgical services are recommended. These are to be executed by the Zonal Health Department in close collaboration with the community, governmental and non-governmental organisations.

Reduction of trachoma in a sub-Saharan village in absence of a disease control programme

BACKGROUND

Trachoma is a leading cause of blindness in the developing world and is most prevalent among people who live in poor rural communities in arid locations.

METHODS

We analysed the results of surveys of trachoma prevalence in Marakissa, a rural village in The Gambia. These surveys were undertaken in 1959, by the Medical Research Council, and in 1987 and 1996 by the Gambian National Eye Care Programme.

FINDINGS

During this 37-year period, the prevalence of active inflammatory trachoma among children aged 0-9 years fell from 65.7 cases per 100 children in 1959 to 2.4 cases per 100 children in 1996. The prevalence also fell dramatically among people of 10-19 years (52.5 to 1.4 per 100) and among people of 20 years and older (36.7 to 0 cases per 100).

INTERPRETATION

The dramatic fall in disease occurrence was paralleled by improvements in sanitation, water supply, education, and access to health care in the village. Of particular importance is that the decline in trachoma occurred without any trachoma-specific intervention.

Prevalence of trachoma in eastern Turkey

BACKGROUND

This study aims to define the prevalence of trachoma in Erzurum, a town in Eastern Turkey where all types of conjunctivitis are known to be endemic.

METHODS

To determine the epidemiological characteristics of the active and residual disease in the area, we chose 20% of the households in 17 residential areas using simple random sampling.

RESULTS

We examined 6386 individuals aged < 30 and 2220 aged > 30 years. The active and the residual prevalence of the disease was found to be 3.2% in those aged < 30, and 1.9% for the group aged > or = 30. Risk factors included crowded households with five or more members, infrequent washing of the face, visiting thermal springs and the mutual use of towels by household members. According to the laboratory findings, trachoma with conjunctival smear was found positive in the first and second stages, but seropositivity was quite high in the third and fourth stages. The validity of the methods used for the diagnosis and the positive predictive values showed the sensitivity of the eyelid examinations to be 55.3%, specificity 95.4% and positive predictive value 23.9% which were the highest values in the field survey.

CONCLUSIONS

Trachoma continues to be a serious public health threat in Erzurum, and eyelid examination is the most suitable method of diagnosis for surveillance of trachoma.

Trachoma, an imported disease

The authors review the incidence of trachoma in the Czech Republic during the last 20 years. Trachoma has become almost exclusively an imported disease. In 1989-1990 the authors treated 34 foreign nationals with imported trachoma at the Infectious Diseases Clinic in Prague. Even after many years' stay in the Czech Republic foreigners may develop exacerbations of the trachomatous process.

[Feasibility of the new WHO Trachoma Grading System in China]

The New WHO Trachoma Grading System is found feasible in China by an epidemiological survey of trachoma using the two trachoma grading systems, the new WHO system and the Chinese conventional system, especially for large scale examination and treatment and for epidemiological survey. However, the New System regards the proportion of active trachoma amongst children under 10 years as the index of the scope and severity of the disease in a community, while the authors opine that the trachoma prevalence in teenagers, especially in middle school students, better reflects the situation of trachoma infection in a Chinese community.

[Application of the new trachoma grading system of WHO]

The new trachoma grading system of WHO was followed in an epidemiologic survey of a rural sample population of 2,728 in Pingyin County, Shandong Province. The prevalence of trachoma was found to be 9.60%, and in children under 10 years of age, 3.69%. Trachoma was related with age; the rates of trachomatous inflammation-intense (TI), trachomatous scarring (TS), trachomatous trichiasis (TT) and corneal opacity (CO) all increased with aging. The epidemiologic significance of the grading system was discussed.

[Trachoma in the province of Ouarzazate, Morocco]

A survey on the prevalence and severity of trachoma was carried out in the province of Ouarzazate, Morocco. In conformity with the guidelines proposed by the WHO Programme for the Prevention of Blindness, a random sample of 30 clusters was extracted from the general population of the province, according to probability proportional to size. Thus, the sample comprised 1200 individuals, of whom 1185 were examined. Participation in the survey was 98.8% and, overall, the sample is considered representative of the province. The simplified grading system proposed by WHO was used to register the data on trachoma and its complications. The global prevalence of trachoma was estimated at 40.8% (95% confidence interval (95% CI) = 30.2-51.4%) and that of active trachoma (follicular (TF), intense (TI), and mixed (TF + TI)) at 18% (95% CI = 12.8-23.2%). The trachomatous intensity indicator (presence of TI) for children under 10 years of age was 12.8% (95% CI = 6.8-18.8%). The severity of the infection is confirmed by prevalences of trichiasis-entropion of 2.2% (95% CI = 1.4-3.0%) and central corneal opacity of 3.3%. Corneal blindness is estimated at 1.6%. The epidemiological pattern of trachoma merits particular attention in the field of public health, particularly in the valley of Oued Drâa, where all the indicators are consistently higher than those elsewhere in the province.

Reliability of photographs for grading trachoma in field studies

The validity and reliability of the use of photographs to assess active trachoma and scarring was studied in a village where the disease was hyperendemic. One hundred and thirty six subjects were graded clinically and had photographs taken of the upper tarsal plate. Agreement between the clinical grade in the field and the grade on the photograph was very good for follicular trachoma, severe trachoma, and scarring (kappa statistic values were 0.71, 0.74, and 0.73 respectively). In field studies photographic documentation of trachoma appears to be a valid and reliable approach.

Trachoma grading: observer trials conducted in southern Malawi

A variety of grading schemes have been proposed for the clinical classification of inflammatory trachoma. During a population based study of ocular disease conducted in southern Malawi we tested a simplified version of the current WHO grading scheme. Intraobserver agreement statistics were less than satisfactory for three of four graders. Interobserver agreement when compared against either a well experienced standard ophthalmologist or a consensus grade improved over time for two of the three graders. However, initial agreement for all three graders was only fair to moderate. Previous studies of trachoma grading schemes support these unsatisfactory results. A new system of classification is needed that is both accurate and reliable in a field setting.

A simple system for the assessment of trachoma and its complications

A simple grading system for trachoma, based on the presence or absence of five selected "key" signs, has been developed. The method was tested in the field and showed good observer agreement, the most critical point being the identification of severe cases of the disease. It is expected that the system will facilitate the assessment of trachoma and its complications by non-specialist health personnel working at the community level.

The epidemiology and geographical distribution of trachoma in Lebowa

The prevalence of active trachoma and its potentially blinding sequelae have been estimated in ten settlements scattered throughout rural Lebowa. The disease represents a major public health problem only in northern Lebowa where 25% of subjects over 60 years of age were found to be suffering form visual disability as a result of the infection. Although the disease is endemic in most other parts of lebowa it rarely causes blindness and impaired vision. A rational approach to the control of trachoma in the area which takes into account both local considerations and recent advances in the knowledge of the epidemiology of the disease is recommended.

A community approach to trachoma control in the Northern Transvaal

Trachoma represents the largest single cause of preventable blindness in the Black population of the Northern Transvaal. Epidemiological and cultural considerations indicate the necessity of community involvement in trachoma control at the household level. Involvement of the community by means of groups of voluntary health workers (care groups) has proved to be effective and acceptable to the local inhabitants. These groups, who form part of the comprehensive health system, integrate trachoma control into the primary health care activities of the area.