MAPPinfo, mapping quality of health information: study protocol for a validation study of an assessment instrument

[Advancing standards for the development of evidence-based health information for consumers: Needs and priorities among members of the Network for Evidence-based Medicine]

BACKGROUND

The Good Practice Guidelines for health information, Good Health Information Austria and Guideline Evidence-based Health Information are established resources for developing evidence-based health information in the German-speaking regions. The aim of this project was to capture challenges in applying these standards in practice, identify gaps and development needs and gain insights for their further development.

METHODS

In December 2020, members of the working group for patient information and involvement of the Network for Evidence-based Medicine were invited to share their experiences and needs in applying German standards for evidence-based health information through an online survey focussing on open questions (part 1: needs assessment). The feedback was analysed using qualitative content analysis and presented in a workshop at the EbM Congress 2021 with the goal of specifying the feedback from the needs assessment and discussing ideas for the further development of the standards (part 2: specification). In the final step, a second survey was conducted in February 2023 to prioritize the identified topics by the working group members (part 3: prioritization). The results were analysed descriptively.

RESULTS

Among the 41 participants, only 23% considered the standards to be sufficient, and only 55% found their application in the development of information to be easy or rather easy. The needs assessment and workshop (n=46) helped to identify various areas of action. With regard to the application of the standards, the following challenges were identified: lack of user orientation, content gaps, methods and risk communication. Gaps in the standards were identified regarding formats, content, and the connection to healthcare provision. For the advancement of the standards ideas for additional content, stakeholder involvement, and improvement of the usability of the standards were identified. In the prioritization survey, the topic areas, "presenting benefits and harms" and "content beyond treatments (in particular, diagnostics and prognosis)" were considered to be the most important (n=36).

DISCUSSION

Among members of the working group for patient information and involvement of the German Network for Evidence-based Medicine, a high demand has been identified for the further development of standards for creating evidence-based health information. In addition to content development, the integration of existing documents and tools should also be considered, including products issued by other institutions. The success of advancing the standards also depends on improving their applicability - for example through an attractive online platform. The results are limited by the sample which only included members of the EbM Network's patient information and participation working group and a limited response rate.

CONCLUSIONS

The needs assessment showed that the currently established standards and recommendations for the development of evidence-based health information in the German-speaking regions represent important cornerstones but need to be expanded to answer more practice-oriented questions. The challenges and proposed solutions stated by the participants can help further develop the standards. The prioritization can be used to set priorities for the development of the standards, guide the order of possible work packages and allocate resources.

MAPPinfo - mapping quality of health information: Validation study of an assessment instrument

BACKGROUND

Health information is a prerequisite for informed choices-decisions, made by individuals about their own health based on knowledge and in congruence with own preferences. Criteria for development, content and design have been defined in a corresponding guideline. However, no instruments exist that provide reasonably operationalised measurement items. Therefore, we drafted the checklist, MAPPinfo, addressing the existing criteria with 19 items.

OBJECTIVES

The current study aimed to validate MAPPinfo.

METHODS

Five substudies were conducted subsequently at the Martin Luther University Halle-Wittenberg, Germany and the Medical University of Graz, Austria: (1) to determine content validity through expert reviews of the first draft, (2) to determine feasibility using 'think aloud' in piloting with untrained users, (3) to determine inter-rater reliability and criterion validity through a pretest on 50 health information materials, (4) to determine construct validity using 50 developers' self-declarations about development methods as a reference standard, (5) to determine divergent validity in comparison with the Ensuring Quality Information for Patients (EQIP) (expanded) Scale. The analyses used were qualitative methods and correlation-based methods for determining both inter-rater reliability and validity.

RESULTS

The instrument was considered by experts to operationalise the existing guidelines convincingly. Health and nursing science students found it easy to understand and use. It also had good interrater reliability (mean of T coefficients = .79) and provided a very good estimate of the reference standard (Spearman's rho = .89), implying sound construct validity. Finally, comparison with the EQIP instrument revealed important and distinct areas of similarities and differences.

CONCLUSIONS

The new instrument is ready for use as a screening instrument without the need for training. According to its underpinning concept the instrument exclusively comprises items which are justified by either ethics or research evidence, implying negligence of not yet evidence based, however, potentially important criteria. Further research is needed to complete the body of evidence-based criteria, aiming at an extension of the guideline and MAPPinfo.

TRIAL REGISTRATION NUMBER

AsPredicted22546; date of registration: 24 July 2019.

New status report from Norway: Implementation of patient involvement in Norwegian health care

In Norway, shared decision-making (SDM) is on the top of the priorities announced by the health authorities. Accountability for implementing this priority has been delegated to the four health regions, and from there into particular departments, hospital trusts, working groups or SDM coordinators. Using abundant public funding, different approaches to producing and implementing patient decision aids have been developed. However, none of these is implemented by any Norwegian services on a regular basis, while the accessible decision aids on the national health platform are not quality approved and in very little use. An ongoing new project is aimed to resolve this dilemma. Health professional training has been developed as a meta-curriculum with multiple modules, adaptive to setting and user group, and designed as an open-source learning platform, based upon the experience of "SDM Ambassadors" delivering the training. Most of the modules are already implemented on a regular basis in the South-Eastern Health Region, and 5000 health care professionals have already graduated from the training modules. However, in the standard patient pathways, and in most of the national medical guidelines, the patient is still considered to be a passive receiver of directives or recommendations, rather than an active participant in their own health decisions. Considerable structural implementation has been done in the education guidelines of all health professions on all levels to embed SDM principles. Teaching in SDM skills, quality of information and evidence-based nursing has become standard in most of the programs at Norwegian universities. Nevertheless, we currently still have no reliable estimate on the degree to which patients are actually involved in their own health decision-making. Further efforts in the process of implementing the patient's choice in the health system should involve the municipal services, follow a research-based strategy, include monitoring and consider the quality level of the informational environment of health-related decisions.