1
|
Booker QS, Austin JD, Balasubramanian BA. Survey strategies to increase participant response rates in primary care research studies. Fam Pract 2021; 38:699-702. [PMID: 34213547 DOI: 10.1093/fampra/cmab070] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/24/2022] Open
Affiliation(s)
- Quiera S Booker
- Department of Epidemiology, Human Genetics, and Environmental Sciences, University of Texas Health Science Center at Houston, School of Public Health, 6011 Harry Hines Blvd, Dallas, TX, USA
| | - Jessica D Austin
- Department of Epidemiology and Sociomedical Sciences, Columbia Mailman School of Public Health, 722 W 168th St, New York, NY, USA
| | - Bijal A Balasubramanian
- Department of Epidemiology, Human Genetics, and Environmental Sciences, University of Texas Health Science Center at Houston, School of Public Health, 6011 Harry Hines Blvd, Dallas, TX, USA
| |
Collapse
|
2
|
Tassiopoulos K, Roberts-Toler C, Fichtenbaum CJ, Koletar SL. Web-Based Data Collection for Older Adults Living With HIV in a Clinical Research Setting: Pilot Observational Study. J Med Internet Res 2020; 22:e18588. [PMID: 33174854 PMCID: PMC7688395 DOI: 10.2196/18588] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2020] [Revised: 06/26/2020] [Accepted: 08/26/2020] [Indexed: 01/25/2023] Open
Abstract
Background Longitudinal follow-up of older persons living with HIV is essential for the ascertainment of aging-related clinical and behavioral outcomes, and self-administered questionnaires are necessary for collecting behavioral information in research involving persons living with HIV. Web-based self-reported data collection results in higher data quality than paper-and-pencil questionnaires in a wide range of populations. The option of remote web-based surveys may also increase retention in long-term research studies. However, the acceptability and feasibility of web-based data collection in clinical research involving older persons living with HIV have never been studied. Objective This study aims to assess the acceptability and feasibility of a web-based survey to collect information on sexual, substance use, and physical activity behaviors; compare the data quality of the web-based survey with that of a paper-and-pencil questionnaire; and summarize web-based survey metrics. Methods This pilot study took place within the AIDS Clinical Trials Group A5322 study, a longitudinal cohort of men and women living with HIV (aged ≥40 years), followed at 32 clinical sites in the United States and Puerto Rico. A total of 4 sites participated in this study. A web-based survey was created using self-administered questionnaires typically completed in A5322 via paper and pencil. Pilot study participants completed these questionnaires via web-based survey at one research visit in lieu of paper-and-pencil administration. Two questions were added to assess feasibility, defined as participants’ perception of the ease of web-based survey completion (very hard, hard, easy, very easy), and their preferred format (computer or tablet, paper and pencil, no preference) for completing the questions in the future (acceptability). Feasibility and acceptability were summarized overall and by demographic and clinical characteristics; the proportion of evaluable data by web-based survey versus previously administered paper-and-pencil questionnaires (data quality) was compared for each question. Results Acceptability and feasibility were high overall: 50.0% (79/158) preferred computer or tablet, 38.0% (60/158) reported no preference, and 12.0% (19/158) preferred paper and pencil; 93.0% (147/158) reported survey completion easy or very easy. Older age was associated with lower odds of preferring computer or tablet to paper and pencil (odds ratio per 1-year increase in age: 0.91, 95% CI 0.85-0.98). Individuals who found the survey hard or very hard had a lower median neurocognitive test score than those who found it easy or very easy. Data quality with web-based survey administration was similar to or higher than that with paper-and-pencil administration for most questions. Conclusions Web-based survey administration was acceptable and feasible in this cohort of older adults living with HIV, and data quality was high. Web-based surveys can be a useful tool for valid data collection and can potentially improve retention in long-term follow-up studies.
Collapse
Affiliation(s)
- Katherine Tassiopoulos
- Department of Epidemiology, Harvard TH Chan School of Public Health, Boston, MA, United States
| | - Carla Roberts-Toler
- Center for Biostatistics in AIDS Research, Harvard TH Chan School of Public Health, Boston, MA, United States
| | - Carl J Fichtenbaum
- Division of Infectious Diseases, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, OH, United States
| | - Susan L Koletar
- Division of Infectious Diseases, Department of Medicine, Ohio State University, Columbus, OH, United States
| |
Collapse
|
3
|
Couper MP, Gremel G, Axinn W, Guyer H, Wagner J, West BT. New options for national population surveys: The implications of internet and smartphone coverage. SOCIAL SCIENCE RESEARCH 2018; 73:221-235. [PMID: 29793688 PMCID: PMC10799630 DOI: 10.1016/j.ssresearch.2018.03.008] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/21/2017] [Revised: 03/05/2018] [Accepted: 03/19/2018] [Indexed: 06/08/2023]
Abstract
Challenges to survey data collection have increased the costs of social research via face-to-face surveys so much that it may become extremely difficult for social scientists to continue using these methods. A key drawback to less expensive Internet-based alternatives is the threat of biased results from coverage errors in survey data. The rise of Internet-enabled smartphones presents an opportunity to re-examine the issue of Internet coverage for surveys and its implications for coverage bias. Two questions (on Internet access and smartphone ownership) were added to the National Survey of Family Growth (NSFG), a U.S. national probability survey of women and men age 15-44, using a continuous sample design. We examine 16 quarters (4 years) of data, from September 2012 to August 2016. Overall, we estimate that 82.9% of the target NSFG population has Internet access, and 81.6% has a smartphone. Combined, this means that about 90.7% of U.S. residents age 15-44 have Internet access, via either traditional devices or a smartphone. We find some evidence of compensatory coverage when looking at key race/ethnicity and age subgroups. For instance, while Black teens (15-18) have the lowest estimated rate of Internet access (81.9%) and the lowest rate of smartphone usage (72.6%), an estimated 88.0% of this subgroup has some form of Internet access. We also examine the socio-demographic correlates of Internet and smartphone coverage, separately and combined, as indicators of technology access in this population. In addition, we look at the effect of differential coverage on key estimates produced by the NSFG, related to fertility, family formation, and sexual activity. While this does not address nonresponse or measurement biases that may differ for alternative modes, our paper has implications for possible coverage biases that may arise when switching to a Web-based mode of data collection, either for follow-up surveys or to replace the main face-to-face data collection.
Collapse
Affiliation(s)
- Mick P Couper
- Institute for Social Research, University of Michigan, United States; Joint Program in Survey Methodology, United States.
| | - Garret Gremel
- Institute for Social Research, University of Michigan, United States
| | - William Axinn
- Institute for Social Research, University of Michigan, United States
| | - Heidi Guyer
- Institute for Social Research, University of Michigan, United States
| | - James Wagner
- Institute for Social Research, University of Michigan, United States; Joint Program in Survey Methodology, United States
| | - Brady T West
- Institute for Social Research, University of Michigan, United States; Joint Program in Survey Methodology, United States
| |
Collapse
|
4
|
Mauz E, von der Lippe E, Allen J, Schilling R, Müters S, Hoebel J, Schmich P, Wetzstein M, Kamtsiuris P, Lange C. Mixing modes in a population-based interview survey: comparison of a sequential and a concurrent mixed-mode design for public health research. ACTA ACUST UNITED AC 2018; 76:8. [PMID: 29423220 PMCID: PMC5791202 DOI: 10.1186/s13690-017-0237-1] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2016] [Accepted: 10/20/2017] [Indexed: 11/10/2022]
Abstract
Background Population-based surveys currently face the problem of decreasing response rates. Mixed-mode designs are now being implemented more often to account for this, to improve sample composition and to reduce overall costs. This study examines whether a concurrent or sequential mixed-mode design achieves better results on a number of indicators of survey quality. Methods Data were obtained from a population-based health interview survey of adults in Germany that was conducted as a methodological pilot study as part of the German Health Update (GEDA). Participants were randomly allocated to one of two surveys; each of the surveys had a different design. In the concurrent mixed-mode design (n = 617) two types of self-administered questionnaires (SAQ-Web and SAQ-Paper) and computer-assisted telephone interviewing were offered simultaneously to the respondents along with the invitation to participate. In the sequential mixed-mode design (n = 561), SAQ-Web was initially provided, followed by SAQ-Paper, with an option for a telephone interview being sent out together with the reminders at a later date. Finally, this study compared the response rates, sample composition, health indicators, item non-response, the scope of fieldwork and the costs of both designs. Results No systematic differences were identified between the two mixed-mode designs in terms of response rates, the socio-demographic characteristics of the achieved samples, or the prevalence rates of the health indicators under study. The sequential design gained a higher rate of online respondents. Very few telephone interviews were conducted for either design. With regard to data quality, the sequential design (which had more online respondents) showed less item non-response. There were minor differences between the designs in terms of their costs. Postage and printing costs were lower in the concurrent design, but labour costs were lower in the sequential design. No differences in health indicators were found between the two designs. Modelling these results for higher response rates and larger net sample sizes indicated that the sequential design was more cost and time-effective. Conclusions This study contributes to the research available on implementing mixed-mode designs as part of public health surveys. Our findings show that SAQ-Paper and SAQ-Web questionnaires can be combined effectively. Sequential mixed-mode designs with higher rates of online respondents may be of greater benefit to studies with larger net sample sizes than concurrent mixed-mode designs.
Collapse
Affiliation(s)
- Elvira Mauz
- 1Department of Epidemiology and Health Monitoring, Robert Koch Institute, Berlin, Germany.,3FG 23 Monitoring Studies and Survey Methods, Robert Koch Institute, P.O. Box 650261, G-13302 Berlin, Germany
| | - Elena von der Lippe
- 1Department of Epidemiology and Health Monitoring, Robert Koch Institute, Berlin, Germany
| | - Jennifer Allen
- 1Department of Epidemiology and Health Monitoring, Robert Koch Institute, Berlin, Germany
| | | | - Stephan Müters
- 1Department of Epidemiology and Health Monitoring, Robert Koch Institute, Berlin, Germany
| | - Jens Hoebel
- 1Department of Epidemiology and Health Monitoring, Robert Koch Institute, Berlin, Germany
| | - Patrick Schmich
- 1Department of Epidemiology and Health Monitoring, Robert Koch Institute, Berlin, Germany
| | - Matthias Wetzstein
- 1Department of Epidemiology and Health Monitoring, Robert Koch Institute, Berlin, Germany
| | - Panagiotis Kamtsiuris
- 1Department of Epidemiology and Health Monitoring, Robert Koch Institute, Berlin, Germany
| | - Cornelia Lange
- 1Department of Epidemiology and Health Monitoring, Robert Koch Institute, Berlin, Germany
| |
Collapse
|
5
|
Berlin NL, Hamill JB, Qi J, Kim HM, Pusic AL, Wilkins EG. Nonresponse bias in survey research: lessons from a prospective study of breast reconstruction. J Surg Res 2017; 224:112-120. [PMID: 29506826 DOI: 10.1016/j.jss.2017.11.058] [Citation(s) in RCA: 31] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2017] [Revised: 10/05/2017] [Accepted: 11/21/2017] [Indexed: 01/26/2023]
Abstract
BACKGROUND Survey-based research is essential for evaluating the outcomes of health care in an era of patient-centered care. However, many such studies are hampered by poor response rates in completion of study questionnaires, thus limiting the generalizability of any findings. The objectives of this analysis were to identify independent variables associated with nonresponse to surveys following breast reconstruction to improve future patient-reported outcomes research. MATERIALS AND METHODS The Mastectomy Reconstruction Outcomes Consortium is a prospective cohort study involving 11 leading medical centers from the United States and Canada. Nonresponse rates for surveys assessing satisfaction with breast, satisfaction with care (BREAST-Q), depression (Patient Health Questionnaire-9), and anxiety (Generalized Anxiety Disorder-7) were measured at 1 y and 2 y postoperatively. Clinical complication rates were compared between responders and nonresponders, and multivariable models were used to assess predictors of nonresponse. RESULTS Among 2856 women in the analytic cohort, 1882 (65.9%) underwent implant-based, 817 (28.6%) received autologous, and 157 (5.5%) underwent latissimus dorsi myocutaneous flap breast reconstructions. Nonresponse rates to surveys at 1 y and 2 y were 27.8% and 34.4%, respectively. Race, ethnicity, and annual household income were associated with nonresponse to surveys. Women who underwent implant-based procedures were less likely to complete long-term surveys. CONCLUSIONS As survey-based research plays an increasingly prominent role in evaluating the outcomes of breast reconstruction, we found socioeconomic and procedure-related differences in survey response rates. Investigators must consider systematic differences in response rates among particular groups of women on the generalizability and validity of findings and perform rigorous nonresponse bias analyses.
Collapse
Affiliation(s)
- Nicholas L Berlin
- Section of Plastic Surgery, University of Michigan, Ann Arbor, Michigan
| | - Jennifer B Hamill
- Section of Plastic Surgery, University of Michigan, Ann Arbor, Michigan
| | - Ji Qi
- Section of Plastic Surgery, University of Michigan, Ann Arbor, Michigan
| | - Hyungjin M Kim
- Center for Statistical Consultation and Research, University of Michigan, Ann Arbor, Michigan; Department of Biostatistics, University of Michigan, Ann Arbor, Michigan
| | - Andrea L Pusic
- Division of Plastic and Reconstructive Surgery, Memorial Sloan-Kettering Cancer Center, New York, New York
| | - Edwin G Wilkins
- Section of Plastic Surgery, University of Michigan, Ann Arbor, Michigan.
| |
Collapse
|
6
|
Schmidt JI, Kellie KA, Chapin FS. Detecting, estimating, and correcting for biases in harvest data. J Wildl Manage 2015. [DOI: 10.1002/jwmg.928] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Affiliation(s)
- Jennifer I. Schmidt
- Institute of Social and Economic Research; University of Alaska Anchorage; Anchorage AK 99508 USA
- Institute of Arctic and Marine Biology; University of Tromsø/The Arctic University of Norway; Naturfagbygget Naturf 2418 9037 Tromsø Norway
| | - Kalin A. Kellie
- Alaska Department of Fish and Game; 1300 College Road Fairbanks AK 99701-1551 USA
| | - F. Stuart Chapin
- Institute of Arctic Biology and Department of Biology and Wildlife; University of Alaska Fairbanks; Fairbanks AK 99775-6100 USA
| |
Collapse
|
7
|
Lewis EF, Hardy M, Snaith B. Estimating the Effect of Nonresponse Bias in a Survey of Hospital Organizations. Eval Health Prof 2013; 36:330-51. [DOI: 10.1177/0163278713496565] [Citation(s) in RCA: 48] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Nonresponse bias in survey research can result in misleading or inaccurate findings and assessment of nonresponse bias is advocated to determine response sample representativeness. Four methods of assessing nonresponse bias (analysis of known characteristics of a population, subsampling of nonresponders, wave analysis, and linear extrapolation) were applied to the results of a postal survey of U.K. hospital organizations. The purpose was to establish whether validated methods for assessing nonresponse bias at the individual level can be successfully applied to an organizational level survey. The aim of the initial survey was to investigate trends in the implementation of radiographer abnormality detection schemes, and a response rate of 63.7% (325/510) was achieved. This study identified conflicting trends in the outcomes of analysis of nonresponse bias between the different methods applied and we were unable to validate the continuum of resistance theory as applied to organizational survey data. Further work is required to ensure established nonresponse bias analysis approaches can be successfully applied to organizational survey data. Until then, it is suggested that a combination of methods should be used to enhance the rigor of survey analysis.
Collapse
Affiliation(s)
- Emily F. Lewis
- University of Bradford, Bradford, UK
- Mid Yorkshire Hospitals NHS Trust, Wakefield, UK
| | | | | |
Collapse
|
8
|
Halbesleben JRB, Whitman MV. Evaluating survey quality in health services research: a decision framework for assessing nonresponse bias. Health Serv Res 2012; 48:913-30. [PMID: 23046097 DOI: 10.1111/1475-6773.12002] [Citation(s) in RCA: 191] [Impact Index Per Article: 15.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023] Open
Abstract
OBJECTIVE To address the issue of nonresponse as problematic and offer appropriate strategies for assessing nonresponse bias. STUDY DESIGN A review of current strategies used to assess the quality of survey data and the challenges associated with these strategies is provided along with appropriate post-data collection techniques that researchers should consider. PRINCIPAL FINDINGS Response rates are an incomplete assessment of survey data quality, and quick reactions to response rate should be avoided. Based on a five-question decision making framework, we offer potential ways to assess nonresponse bias, along with a description of the advantages and disadvantages to each. CONCLUSIONS It is important that the quality of survey data be considered to assess the relative contribution to the literature of a given study. Authors and funding agencies should consider the potential effects of nonresponse bias both before and after survey administration and report the results of assessments of nonresponse bias in addition to response rates.
Collapse
Affiliation(s)
- Jonathon R B Halbesleben
- Department of Management & Marketing, Culverhouse College of Commerce & Business Administration, University of Alabama, Tuscaloosa, AL, USA
| | | |
Collapse
|
9
|
Nederhof E, Jörg F, Raven D, Veenstra R, Verhulst FC, Ormel J, Oldehinkel AJ. Benefits of extensive recruitment effort persist during follow-ups and are consistent across age group and survey method. The TRAILS study. BMC Med Res Methodol 2012; 12:93. [PMID: 22747967 PMCID: PMC3585928 DOI: 10.1186/1471-2288-12-93] [Citation(s) in RCA: 50] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2011] [Accepted: 05/22/2012] [Indexed: 12/03/2022] Open
Abstract
Background Extensive recruitment effort at baseline increases representativeness of study populations by decreasing non-response and associated bias. First, it is not known to what extent increased attrition occurs during subsequent measurement waves among subjects who were hard-to-recruit at baseline and what characteristics the hard-to-recruit dropouts have compared to the hard-to-recruit retainers. Second, it is unknown whether characteristics of hard-to-recruit responders in a prospective population based cohort study are similar across age group and survey method. Methods First, we compared first wave (T1) easy-to-recruit with hard-to-recruit responders of the TRacking Adolescents’ Individual Lives Survey (TRAILS), a prospective population based cohort study of Dutch (pre)adolescents (at first wave: n = 2230, mean age = 11.09 (SD 0.56), 50.8% girls), with regard to response rates at subsequent measurement waves. Second, easy-to-recruit and hard-to-recruit participants at the fourth TRAILS measurement wave (n = 1881, mean age = 19.1 (SD 0.60), 52.3% girls) were compared with fourth wave non-responders and earlier stage drop-outs on family composition, socioeconomic position (SEP), intelligence (IQ), education, sociometric status, substance use, and psychopathology. Results First, over 60% of the hard-to-recruit responders at the first wave were retained in the sample eight years later at the fourth measurement wave. Hard-to-recruit dropouts did not differ from hard-to-recruit retainers. Second, extensive recruitment efforts for the web based survey convinced a population of nineteen year olds with similar characteristics as the hard-to-recruit eleven year olds that were persuaded to participate in a school-based survey. Some characteristics associated with being hard-to-recruit (as compared to being easy-to-recruit) were more pronounced among non-responders, resembling the baseline situation (De Winter et al.2005). Conclusions First, extensive recruitment effort at the first assessment wave of a prospective population based cohort study has long lasting positive effects. Second, characteristics of hard-to-recruit responders are largely consistent across age groups and survey methods.
Collapse
Affiliation(s)
- Esther Nederhof
- Interdisciplinary Center Psychopathology and Emotion regulation, University Center for Psychiatry, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands.
| | | | | | | | | | | | | |
Collapse
|
10
|
Beebe TJ, McAlpine DD, Ziegenfuss JY, Jenkins S, Haas L, Davern ME. Deployment of a mixed-mode data collection strategy does not reduce nonresponse bias in a general population health survey. Health Serv Res 2012; 47:1739-54. [PMID: 22250782 DOI: 10.1111/j.1475-6773.2011.01369.x] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022] Open
Abstract
OBJECTIVE To assess nonresponse bias in a mixed-mode general population health survey. DATA SOURCES Secondary analysis of linked survey sample frame and administrative data, including demographic and health-related information. STUDY DESIGN The survey was administered by mail with telephone follow-up to nonrespondents after two mailings. To determine whether an additional mail contact or mode switch reduced nonresponse bias, we compared all respondents (N = 3,437) to respondents from each mailing and telephone respondents to the sample frame (N = 6,716). PRINCIPAL FINDINGS Switching modes did not minimize the under-representation of younger people, nonwhites, those with congestive heart failure, high users of office-based services, and low-utilizers of the emergency room but did reduce the over-representation of older adults. CONCLUSIONS Multiple contact and mixed-mode surveys may increase response rates, but they do not necessarily reduce nonresponse bias.
Collapse
Affiliation(s)
- Timothy J Beebe
- Division of Health Care Policy and Research, Mayo Clinic, 200 First Street Southwest, Rochester, MN 55905, USA.
| | | | | | | | | | | |
Collapse
|
11
|
Davern M, McAlpine D, Beebe TJ, Ziegenfuss J, Rockwood T, Call KT. Are lower response rates hazardous to your health survey? An analysis of three state telephone health surveys. Health Serv Res 2010; 45:1324-44. [PMID: 20579127 DOI: 10.1111/j.1475-6773.2010.01128.x] [Citation(s) in RCA: 64] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022] Open
Abstract
OBJECTIVE To examine the impact of response rate variation on survey estimates and costs in three health telephone surveys. DATA SOURCE Three telephone surveys of noninstitutionalized adults in Minnesota and Oklahoma conducted from 2003 to 2005. STUDY DESIGN We examine differences in demographics and health measures by number of call attempts made before completion of the survey or whether the household initially refused to participate. We compare the point estimates we actually obtained with those we would have obtained with a less aggressive protocol and subsequent lower response rate. We also simulate what the effective sample sizes would have been if less aggressive protocols were followed. PRINCIPAL FINDINGS Unweighted bivariate analyses reveal many differences between early completers and those requiring more contacts and between those who initially refused to participate and those who did not. However, after making standard poststratification adjustments, no statistically significant differences were observed in the key health variables we examined between the early responders and the estimates derived from the full reporting sample. CONCLUSIONS Our findings demonstrate that for the surveys we examined, larger effective sample sizes (i.e., more statistical power) could have been achieved with the same amount of funding using less aggressive calling protocols. For some studies, money spent on aggressively pursuing high response rates could be better used to increase statistical power and/or to directly examine nonresponse bias.
Collapse
Affiliation(s)
- Michael Davern
- University of Chicago, 55 E. Monroe, Suite 3000, Chicago, IL 60603, USA.
| | | | | | | | | | | |
Collapse
|