Validation of the reporting bases of the orthopaedic and paediatric surveillance schemes

How to measure the need for transition to adult services among young people with Attention Deficit Hyperactivity Disorder (ADHD): a comparison of surveillance versus case note review methods

BACKGROUND

Health services have not provided adequate support for young people with long term health conditions to transfer from child to adult services. National Institute of Health and Care (NICE) guidance on transition has been issued to address these gaps. However, data are often sparse about the number of young adults who might need to transition. Using Attention Deficit Hyperactivity Disorder (ADHD) as an exemplar, this study used an existing surveillance system and a case note review to capture the incidence of the transition process, and compared and contrasted the findings.

METHODS

The Child and Adolescent Psychiatry Surveillance System (CAPSS) was used to estimate the incident transition of young people with Attention Deficit Hyperactivity Disorder (ADHD) from child to adult services. This involves consultant child and adolescent psychiatrists from the United Kingdom (UK) and Republic of Ireland (ROI) reporting relevant young people as they are seen in clinics. In parallel, a case note review was conducted using the Maudsley Biomedical Research Centre (BRC) Clinical Records Interactive Search (CRIS). The study period ran for twelve months with a nine month follow up to see how the transition proceeded.

RESULTS

CRIS identified 76 cases in the study period, compared to 18 identified using surveillance via CAPSS. Methodological issues were experienced using both methods. Surveillance issues; eligibility criteria confusion, reporting errors, incomplete questionnaires, difficulties contacting clinicians, and surveillance systems do not cover non-doctors and psychiatrists who are not consultants. Case note review issues using CRIS included the need for researchers to interpret clinical notes, the availability and completeness of data in the notes, and data limited to the catchment of one particular mental health trust.

CONCLUSIONS

Both methods demonstrate strengths and weaknesses; the combination of both methods in the absence of strong routinely collected data, allowed a more robust estimate of the level of need for service planning and commissioning.

The impact of the National Institute for Health Research Health Technology Assessment programme, 2003-13: a multimethod evaluation

BACKGROUND

The National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme supports research tailored to the needs of NHS decision-makers, patients and clinicians. This study reviewed the impact of the programme, from 2003 to 2013, on health, clinical practice, health policy, the economy and academia. It also considered how HTA could maintain and increase its impact.

METHODS

Interviews (n = 20): senior stakeholders from academia, policy-making organisations and the HTA programme. Bibliometric analysis: citation analysis of publications arising from HTA programme-funded research. Researchfish survey: electronic survey of all HTA grant holders. Payback case studies (n = 12): in-depth case studies of HTA programme-funded research.

RESULTS

We make the following observations about the impact, and routes to impact, of the HTA programme: it has had an impact on patients, primarily through changes in guidelines, but also directly (e.g. changing clinical practice); it has had an impact on UK health policy, through providing high-quality scientific evidence - its close relationships with the National Institute for Health and Care Excellence (NICE) and the National Screening Committee (NSC) contributed to the observed impact on health policy, although in some instances other organisations may better facilitate impact; HTA research is used outside the UK by other HTA organisations and systematic reviewers - the programme has an impact on HTA practice internationally as a leader in HTA research methods and the funding of HTA research; the work of the programme is of high academic quality - the Health Technology Assessment journal ensures that the vast majority of HTA programme-funded research is published in full, while the HTA programme still encourages publication in other peer-reviewed journals; academics agree that the programme has played an important role in building and retaining HTA research capacity in the UK; the HTA programme has played a role in increasing the focus on effectiveness and cost-effectiveness in medicine - it has also contributed to increasingly positive attitudes towards HTA research both within the research community and the NHS; and the HTA focuses resources on research that is of value to patients and the UK NHS, which would not otherwise be funded (e.g. where there is no commercial incentive to undertake research). The programme should consider the following to maintain and increase its impact: providing targeted support for dissemination, focusing resources when important results are unlikely to be implemented by other stakeholders, particularly when findings challenge vested interests; maintaining close relationships with NICE and the NSC, but also considering other potential users of HTA research; maintaining flexibility and good relationships with researchers, giving particular consideration to the Technology Assessment Report (TAR) programme and the potential for learning between TAR centres; maintaining the academic quality of the work and the focus on NHS need; considering funding research on the short-term costs of the implementation of new health technologies; improving the monitoring and evaluation of whether or not patient and public involvement influences research; improve the transparency of the priority-setting process; and continuing to monitor the impact and value of the programme to inform its future scientific and administrative development.

Surveillance of rare diseases: a public health evaluation of the British Paediatric Surveillance Unit

BACKGROUND

The British Paediatric Surveillance Unit (BPSU), a joint undertaking between the Royal College of Paediatrics and Child Health, Institute of Child Health and Health Protection Agency, undertakes nationwide surveillance of rare paediatric disorders. In 2007-09, formal evaluation to examine its effectiveness commenced.

METHODS

Centres of Disease Control guidelines for appraising public health surveillance systems were applied. Data sources included BPSU databases, published and unpublished reports. Questionnaires were sent to 600 participating paediatricians and 27 researchers. Half of the questionnaires were administered online to assess the feasibility of electronic reporting.

RESULTS

Three thousand UK paediatricians report monthly to the BPSU (94% return) and eighty BPSU studies have been published. These studies have influenced immunization and screening policy, altered clinical practice and informed health service configuration. Surveillance operations are simple, stable, representative and responsive to changing demands. Returns from the paediatricians' survey were 75%; investigators 89%. Paediatricians valued the BPSU and did not find participation burdensome. Most supported online questionnaires (56%) but not monthly electronic reporting (35%).

CONCLUSIONS

Evaluation demonstrated the effectiveness of the BPSU as a valuable resource for clinicians and policy-makers. Opportunities identified for future development include secure online reporting, improved responsiveness to urgent health threats and promoting public involvement.

Using multiple sources to improve and measure case ascertainment in surveillance studies: 20 years of the British Paediatric Surveillance Unit

BACKGROUND

The British Paediatric Surveillance Unit (BPSU) was established in 1986 to facilitate national surveillance of uncommon paediatric disorders. This study investigated the effectiveness of using multiple source reporting and capture-recapture analysis to maximize case ascertainment in studies undertaken through the BPSU.

METHODS

Structured review of all surveillance studies completed through the BPSU. Quantitative and qualitative analysis of the effectiveness of multiple reporting sources and capture-recapture methods was made.

RESULTS

Of 71 studies undertaken through the BPSU, 59 were included in this review and 38 used additional data sources. Established national sources were most readily adapted for use as secondary data sources, including routine health data, communicable disease and specialty-specific surveillance units, whilst the involvement of parent groups and the media was less successful. Six studies employed capture-recapture techniques to estimate the completeness of case ascertainment.

CONCLUSIONS

Active surveillance through the BPSU remains a timely and reliable primary source of cases, but employing additional reporting sources is effective in enhancing case ascertainment. When the assumptions for its valid use are met, capture-recapture analysis allows the estimation of completeness of ascertainment. It is essential to define the purpose of an additional source at the outset of a study and to ensure that subsequent analysis is appropriate.

Rare disease surveillance: An international perspective

BACKGROUND

The International Network of Paediatric Surveillance Units (INoPSU) was established in 1998 and met formally for the first time in Ottawa, Ontario in June 2000.

OBJECTIVES

To document the methodology and activities of existing national paediatric surveillance units; the formation of INoPSU; the diseases studied by INoPSU members; and the impact of such studies on education, public health and paediatric practice.

METHODS

Directors of paediatric surveillance units in Australia, Britain, Canada, Germany, the Netherlands, Latvia, Malaysia, Papua New Guinea, New Zealand and Switzerland were asked to provide information on each unit's affiliations, funding and staffing; the method of case ascertainment, the mailing list and response rates; and diseases studied. Original articles that reported data derived from units were identified by a search of an electronic database (MEDLINE), and additional information was obtained from units' annual reports.

RESULTS

Worldwide, 10 units (established from 1986 to 1997), use active national surveillance of more than 8500 clinicians each month to identify cases of rare or uncommon diseases in a childhood population (younger than 15 years of age) of over 47 million (monthly response rate 73% to 98%). By January 1999, units had initiated 147 studies on 103 different conditions, and 63 studies were completed.

CONCLUSION

INoPSU enhances collaboration among units from four continents, providing a unique opportunity for simultaneous cross-sectional studies of rare diseases in populations with diverse geographical and ethnic characteristics. It facilitates the sharing of ideas regarding current methodology, ethics, the most appropriate means of evaluating units and their potential application.

Neonatal meningitis in England and Wales: 10 years on

OBJECTIVES

To determine the incidence of neonatal meningitis in England and Wales.

DESIGN

A national postal survey using the British Paediatric Surveillance Unit (BPSU) card scheme supplemented by information from other sources.

SETTING

England and Wales 1996-1997.

SUBJECTS

A total of 274 babies less than 28 days of age who were treated for meningitis.

RESULTS

The incidence of neonatal meningitis in England and Wales has not changed since our previous study in 1985-1987. However, the acute phase mortality has fallen from 19.8% in 1985-1987 to 6.6% in this study. Group B streptococci (42%) and Escherichia coli (16%) remain the most common infecting microorganisms. Eight of 69 (12%) babies with group B streptococci and 4/26 (15%) with E coli died. Antibiotic regimens based on the third generation cephalosporins, notably cefotaxime, were most commonly used (84%). The BPSU scheme identified 72% of cases during the study period. Most cases of viral meningitis were not reported through the BPSU. Less than a third of samples from aseptic meningitis were examined for viruses; 56% of these were positive.

CONCLUSIONS

Although the incidence of neonatal meningitis remains unchanged, mortality from this infection has fallen significantly. If this improvement is maintained as reflected in the level of sequelae at 5 years of age, then the fear surrounding meningitis during the neonatal period will have been dramatically reduced.

Evaluation of a national surveillance unit

AIM

The Australian Paediatric Surveillance Unit (APSU) facilitates national active surveillance of uncommon childhood conditions. This study assessed whether it fulfilled its objectives and satisfied criteria established by the Centers for Disease Control and Prevention (CDC) for evaluating surveillance systems.

METHODS

Anonymous questionnaires were sent to users of the system, individual studies were reviewed, and data were collected from independent sources.

RESULTS

Seven hundred and sixty six clinicians, 48 investigators, and 15 public health professionals responded to the questionnaires. Clinicians reported that the APSU was useful, 33% saying information provided by the APSU informed or changed their clinical practice. Most (88%) reported that completing monthly report cards was not a burden. Impact on policy development was limited by suboptimal dissemination of information to public health professionals. Flexibility and timeliness were limited by design. Estimated sensitivity of APSU studies ranged from 92% (congenital rubella) to 31% (drowning/near drowning). Positive predictive value of notified cases was over 70% for most studies.

CONCLUSION

The APSU fulfils most of its objectives and meets CDC criteria salient to these. Ways in which the APSU could be improved have been identified, as have methodological challenges and limitations in applying CDC guidelines to this type of unit.

Surgery for congenital dislocation of the hip in the UK as a measure of outcome of screening. MRC Working Party on Congenital Dislocation of the Hip. Medical Research Council

BACKGROUND

Universal clinical screening for congenital dislocation of the hip to detect hip instability in neonates was introduced in the UK as a national policy in 1969, but its effectiveness is not known. We aimed to assess the extent to which surgery for congenital dislocation of the hip is the result of a failure of detection through screening or follows non-surgical treatment after detection by screening.

METHODS

We established a national orthopaedic surveillance scheme and used routine hospital data for inpatients for 20% of births in the UK (Scotland and the Northern and Wessex regions) to ascertain the number of children aged under 5 years per 1000 livebirths who had received at least one operative procedure for congenital dislocation of the hip from April, 1993, to April, 1994. Estimates of the incidence of operative procedures were adjusted for under-ascertainment by capture-recapture techniques.

FINDINGS

The ascertainment-adjusted incidence of a first operative procedure for congenital dislocation of the hip in the UK was 0.78 per 1000 livebirths (95% CI 0.72-0-84). Congenital dislocation of the hip had not been detected by routine screening in 222 (70%) of 318 children reported to the national orthopaedic surveillance scheme. In 112 (35%) children the diagnosis was made primarily as a result of parental concern. 67 (21%) children had previously received non-surgical treatment. In Scotland and the Northern and Wessex regions, 81 cases were notified to the national orthopaedic surveillance scheme, 62 cases were identified only through routine hospital data on inpatients, and an estimated 20 cases were not identified by either source, making a total of 163 cases. Thus, 81 (50%) of these 163 cases were identified by surveillance, 125 (77%) by routine data, and 143 (88%) by both sources.

INTERPRETATION

The incidence of a first operative procedure for congenital dislocation of the hip in the UK was similar to that reported before screening was introduced. In most children who received surgery, congenital dislocation of the hip was not detected by screening. Formal evaluation of current and alternative screening policies, including universal primary ultrasound imaging, is needed.