1
|
Koriat A. Subjective Confidence as a Monitor of the Replicability of the Response. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2024:17456916231224387. [PMID: 38319741 DOI: 10.1177/17456916231224387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2024]
Abstract
Confidence is commonly assumed to monitor the accuracy of responses. However, intriguing results, examined in the light of philosophical discussions of epistemic justification, suggest that confidence actually monitors the reliability of choices rather than (directly) their accuracy. The focus on reliability is consistent with the view that the construction of truth has much in common with the construction of reality: extracting reliable properties that afford prediction. People are assumed to make a binary choice by sampling cues from a "collective wisdomware," and their confidence is based on the consistency of these cues, in line with the self-consistency model. Here, however, I propose that internal consistency is taken to index the reliability of choices themselves-the likelihood that they will be repeated. The results of 10 studies using binary decisions from different domains indicated that confidence in a choice predicts its replicability both within individuals and across individuals. This was so for domains for which choices have a truth value and for those for which they do not. For the former domains, differences in replicability mediated the prediction of accuracy whether confidence was diagnostic or counterdiagnostic of accuracy. Metatheoretical, methodological, and practical implications are discussed.
Collapse
Affiliation(s)
- Asher Koriat
- Institute of Information Processing and Decision Making, University of Haifa
- Department of Psychology, University of Haifa
| |
Collapse
|
2
|
Johns BT. Computing the Relativity of Word Meanings through the Construction of Individualized Models of Semantic Memory. COGN SYST RES 2023. [DOI: 10.1016/j.cogsys.2023.02.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/07/2023]
|
3
|
Drafting strategies in fantasy football: A study of competitive sequential human decision making. JUDGMENT AND DECISION MAKING 2022. [DOI: 10.1017/s1930297500008901] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Abstract
AbstractDrafting is a competitive task in which a set of decision makers choose from a set of resources sequentially, with each resource becoming unavailable once selected. How people make these choices raises basic questions about human decision making, including people’s sensitivity to the statistical regularities of the resource environment, their ability to reason about the behavior of their competitors, and their ability to execute and adapt sophisticated strategies in dynamic situations involving uncertainty. Sports provides one real-world example of drafting behavior, in which a set of teams draft players from an available pool in a well-regulated way. Fantasy sport competitions provide potentially large data sets of drafting behavior. We study fantasy football drafting behavior from the 2017 National Football League (NFL) season based on 1350 leagues hosted by the http://sleeper.app platform. We find people are sensitive to some important environmental regularities in the order in which they draft players, but also present evidence that they use a more narrow range of strategies than is likely optimal in terms of team composition. We find little to no evidence for the use of the complicated but well-documented strategy known as handcuffing, and no evidence of irrational influence from individual-level biases for different NFL teams. We do, however, identify a set of circumstances for which there is clear evidence that people’s choices are strongly influenced by the immediately preceding choice made by a competitor.
Collapse
|
4
|
Haeussler C, Vieth S. A question worth a million: The expert, the crowd, or myself? An investigation of problem solving. RESEARCH POLICY 2022. [DOI: 10.1016/j.respol.2021.104453] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
5
|
Thomas B, Coon J, Westfall HA, Lee MD. Model-Based Wisdom of the Crowd for Sequential Decision-Making Tasks. Cogn Sci 2021; 45:e13011. [PMID: 34213800 DOI: 10.1111/cogs.13011] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Revised: 05/27/2021] [Accepted: 05/28/2021] [Indexed: 11/29/2022]
Abstract
We study the wisdom of the crowd in three sequential decision-making tasks: the Balloon Analogue Risk Task (BART), optimal stopping problems, and bandit problems. We consider a behavior-based approach, using majority decisions to determine crowd behavior and show that this approach performs poorly in the BART and bandit tasks. The key problem is that the crowd becomes progressively more extreme as the decision sequence progresses, because the diversity of opinion that underlies the wisdom of the crowd is lost. We also consider model-based approaches to each task. This involves inferring cognitive models for each individual based on their observed behavior, and using these models to predict what each individual would do in any possible task situation. We show that this approach performs robustly well for all three tasks and has the additional advantage of being able to generalize to new problems for which there are no behavioral data. We discuss potential applications of the model-based approach to real-world sequential decision problems and discuss how our approach contributes to the understanding of collective intelligence.
Collapse
Affiliation(s)
- Bobby Thomas
- Department of Cognitive Sciences, University of California, Irvine
| | - Jeff Coon
- Department of Cognitive Sciences, University of California, Irvine
| | - Holly A Westfall
- Department of Cognitive Sciences, University of California, Irvine
| | - Michael D Lee
- Department of Cognitive Sciences, University of California, Irvine
| |
Collapse
|
6
|
Bennett M, Mullard R, Adam MTP, Steyvers M, Brown S, Eidels A. Going, going, gone: competitive decision-making in Dutch auctions. COGNITIVE RESEARCH-PRINCIPLES AND IMPLICATIONS 2020; 5:62. [PMID: 33252772 PMCID: PMC7704846 DOI: 10.1186/s41235-020-00259-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/13/2019] [Accepted: 10/24/2020] [Indexed: 11/12/2022]
Abstract
In a Dutch auction, an item is offered for sale at a set maximum price. The price is then gradually lowered over a fixed interval of time until a bid is made, securing the item for the bidder at the current price. Bidders must trade-off between certainty and price: bid early to secure the item and you pay a premium; bid later at a lower price but risk losing to another bidder. These properties of Dutch auctions provide new opportunities to study competitive decision-making in a group setting. We developed a novel computerised Dutch auction platform and conducted a set of experiments manipulating volatility (fixed vs varied number of items for sale) and price reduction interval rate (step-rate). Triplets of participants (\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$N=66$$\end{document}N=66) competed with hypothetical funds against each other. We report null effects of step-rate and volatility on bidding behaviour. We developed a novel adaptation of prospect theory to account for group bidding behaviour by balancing certainty and subjective expected utility. We show the model is sensitive to variation in auction starting price and can predict the associated changes in group bid prices that were observed in our data.
Collapse
Affiliation(s)
- Murray Bennett
- School of Psychology, The University of Newcastle, Callaghan, NSW, 2308, Australia.
| | - Rachel Mullard
- School of Psychology, The University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Marc T P Adam
- School of Electrical Engineering and Computing, University of Newcastle, Callaghan, Australia
| | - Mark Steyvers
- Department of Cognitive Sciences, University of California, Irvine, Irvine, USA
| | - Scott Brown
- School of Psychology, The University of Newcastle, Callaghan, NSW, 2308, Australia
| | - Ami Eidels
- School of Psychology, The University of Newcastle, Callaghan, NSW, 2308, Australia
| |
Collapse
|
7
|
Rutchick AM, Ross BJ, Calvillo DP, Mesick CC. Does the "surprisingly popular" method yield accurate crowdsourced predictions? COGNITIVE RESEARCH-PRINCIPLES AND IMPLICATIONS 2020; 5:57. [PMID: 33175285 PMCID: PMC7658271 DOI: 10.1186/s41235-020-00256-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/29/2019] [Accepted: 10/18/2020] [Indexed: 11/10/2022]
Abstract
The “surprisingly popular” method (SP) of aggregating individual judgments has shown promise in overcoming a weakness of other crowdsourcing methods—situations in which the majority is incorrect. This method relies on participants’ estimates of other participants’ judgments; when an option is chosen more often than the average metacognitive judgments of that option, it is “surprisingly popular” and is selected by the method. Although SP has been shown to improve group decision making about factual propositions (e.g., state capitals), its application to future outcomes has been limited. In three preregistered studies, we compared SP to other methods of aggregating individual predictions about future events. Study 1 examined predictions of football games, Study 2 examined predictions of the 2018 US midterm elections, and Study 3 examined predictions of basketball games. When applied to judgments made by objectively assessed experts, SP performed slightly better than other aggregation methods. Although there is still more to learn about the conditions under which SP is effective, it shows promise as a means of crowdsourcing predictions of future outcomes.
Collapse
Affiliation(s)
| | - Bryan J Ross
- California State University, Northridge, Northridge, USA
| | | | | |
Collapse
|
8
|
Afflerbach P, van Dun C, Gimpel H, Parak D, Seyfried J. A Simulation-Based Approach to Understanding the Wisdom of Crowds Phenomenon in Aggregating Expert Judgment. BUSINESS & INFORMATION SYSTEMS ENGINEERING 2020. [DOI: 10.1007/s12599-020-00664-x] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
AbstractResearch has shown that aggregation of independent expert judgments significantly improves the quality of forecasts as compared to individual expert forecasts. This “wisdom of crowds” (WOC) has sparked substantial interest. However, previous studies on strengths and weaknesses of aggregation algorithms have been restricted by limited empirical data and analytical complexity. Based on a comprehensive analysis of existing knowledge on WOC and aggregation algorithms, this paper describes the design and implementation of a static stochastic simulation model to emulate WOC scenarios with a wide range of parameters. The model has been thoroughly evaluated: the assumptions are validated against propositions derived from literature, and the model has a computational representation. The applicability of the model is demonstrated by investigating aggregation algorithm behavior on a detailed level, by assessing aggregation algorithm performance, and by exploring previously undiscovered suppositions on WOC. The simulation model helps expand the understanding of WOC, where previous research was restricted. Additionally, it gives directions for developing aggregation algorithms and contributes to a general understanding of the WOC phenomenon.
Collapse
|
9
|
Nasiri N, Abedi L, Hajebi A, Noroozi A, Khalili M, Chegeni M, Nili S, Taheri-Soodejani M, Noroozi M, Shahesmaeili A, Sharifi H. Population Size Estimation of Tramadol Misusers in Urban Population in Iran: Synthesis of Methods and Results. ADDICTION & HEALTH 2019; 11:173-182. [PMID: 31839915 PMCID: PMC6904981 DOI: 10.22122/ahj.v11i3.240] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Background Estimating the population who use drugs is essential for planning, monitoring, and evaluation of substance use prevention and treatment. This study aims to estimate the population who misuse tramadol in urban population in Iran. Methods We used the wisdom of the experts (WOTE) and network scale-up (NSU) methods to calculate the population of tramadol misusers in 10 provincial capitals of Iran, in 2016. The WOTE was conducted among pharmacists in drugstores and the personnel of traditional medicinal herbs stores. They guessed the best estimation of tramadol misusers population in their cities. The NSU method was conducted among the general population and participants were questioned about ever and daily, non-medical use of tramadol during last 12 months in their network. The median of the methods was used to calculate the proportion of the adult population (15-49 years old). Findings The population size of tramadol misusers in studied cities was 83300 [95% uncertainty limits (UL): 47960-256220]. This corresponded to 6.6 per 1000 (95% UL: 3.88-20.30) of the 15-49-year-old population. The projected number of tramadol misusers for all 31 provincial capitals was 118290 (95% UL: 68100-363130840) and 212440 (95% UL: 122310-653410) for all urban areas. NSU also estimated the number of people who misuse tramadol on daily basis. These numbers were 52000 (95% UL: 19940-176570) for studied cities, 73840 (95% UL: 28320-250740) for all 31 provincial capital cities, and 132610 (95% UL: 50860-450310) for all urban areas in Iran. Conclusion This study presents information on high prevalence of tramadol misuse in urban population. We need national control measures and demand reduction programs to control tramadol misuse.
Collapse
Affiliation(s)
- Naser Nasiri
- Department of Public Health, School of Public Health, Jiroft University of Medical Sciences, Jiroft, Iran
| | - Leili Abedi
- HIV/STI Surveillance Research Center, and WHO Collaborating Center for HIV Surveillance, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
| | - Ahmad Hajebi
- Research Center for Addiction and Risky Behaviors AND Department of Psychiatry, School of Medicine, Iran University of Medical Sciences, Tehran, Iran
| | - Alireza Noroozi
- Department of Neuroscience and Addiction Studies, School of Advanced Technologies in Medicine AND Iranian National Center for Addiction Studies, Tehran University of Medical Sciences, Tehran, Iran
| | - Malahat Khalili
- HIV/STI Surveillance Research Center, and WHO Collaborating Center for HIV Surveillance, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
| | - Maryam Chegeni
- HIV/STI Surveillance Research Center, and WHO Collaborating Center for HIV Surveillance, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
| | - Sairan Nili
- Modeling in Health Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
| | - Moslem Taheri-Soodejani
- Social Determinants of Health Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
| | - Mehdi Noroozi
- Social Determinants of Health Research Center, University of Social Welfare and Rehabilitation Sciences, Tehran, Iran
| | - Armita Shahesmaeili
- HIV/STI Surveillance Research Center, and WHO Collaborating Center for HIV Surveillance, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
| | - Hamid Sharifi
- HIV/STI Surveillance Research Center, and WHO Collaborating Center for HIV Surveillance, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
| |
Collapse
|
10
|
Kvam P. The Price is Right: Analyzing Bidding Behavior on Contestants’ Row. AM STAT 2019. [DOI: 10.1080/00031305.2019.1592782] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Affiliation(s)
- Paul Kvam
- Department of Mathematics and Computer Science, University of Richmond, Richmond, VA
| |
Collapse
|
11
|
Mistry P, Liljeholm M. The Expression and Transfer of Valence Associated with Social Conformity. Sci Rep 2019; 9:2154. [PMID: 30770853 PMCID: PMC6377616 DOI: 10.1038/s41598-019-38560-4] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2018] [Accepted: 01/02/2019] [Indexed: 01/10/2023] Open
Abstract
Consensus seeking – abandoning one’s own judgment to align with a group majority – is a fundamental feature of human social interaction. Notably, such striving for majority affiliation often occurs in the absence of any apparent economic or social gain, suggesting that achieving consensus might have intrinsic value. Here, using a simple gambling task, in which the decisions of ostensible previous gamblers were indicated below available options on each trial, we examined the affective properties of agreeing with a group majority by assessing the trade-off between social and non-social currencies, and the transfer of social valence to concomitant stimuli. In spite of demonstrating near-perfect knowledge of objective reward probabilities, participant’s choices and evaluative judgments reflected a reliable preference for conformity, consistent with the hypothesized value of social alignment.
Collapse
Affiliation(s)
- Prachi Mistry
- Department of Cognitive Sciences, University of California, Irvine, USA
| | - Mimi Liljeholm
- Department of Cognitive Sciences, University of California, Irvine, USA.
| |
Collapse
|
12
|
Affiliation(s)
- Paul H. Kvam
- Department of Mathematics and Computer Science, University of Richmond, Richmond, Virginia 23173
| |
Collapse
|
13
|
Rich AJ, Lachowsky NJ, Sereda P, Cui Z, Wong J, Wong S, Jollimore J, Raymond HF, Hottes TS, Roth EA, Hogg RS, Moore DM. Estimating the Size of the MSM Population in Metro Vancouver, Canada, Using Multiple Methods and Diverse Data Sources. J Urban Health 2018; 95:188-195. [PMID: 28631060 PMCID: PMC5906378 DOI: 10.1007/s11524-017-0176-8] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Men who have sex with men (MSM) are disproportionately affected by HIV globally, regionally in Canada, and locally in Vancouver. Lack of reliable population size estimates of MSM impedes effective implementation of health care services and limits our understanding of the HIV epidemic. We estimated the population size of MSM residing in Metro Vancouver drawing on four data sources: the Canadian Community Health Survey (CCHS), a cross-sectional bio-behavioural MSM survey, HIV testing services data from sexually transmitted infection (STI) clinics serving MSM, and online social networking site Facebook. Estimates were calculated using (1) direct estimates from the CCHS, (2) "Wisdom of the Crowds" (WOTC), and (3) the multiplier method using data from a bio-behavioural MSM survey, clinic-based HIV testing, and online social media network site Facebook. Data sources requiring greater public disclosure of sexual orientation resulted in our mid-range population estimates (Facebook 23,760, CCHS 30,605). The WOTC method produced the lowest estimate, 10,000. The multiplier method using STI clinic HIV testing data produced the largest estimate, 41,777. The median of all estimates was 27,183, representing 2.9% of the Metro Vancouver census male adult population, with an interquartile range of 1.1-4.5%. Using multiple data sources, our estimates of the MSM population in Metro Vancouver are similar to population prevalence estimates based on population data from other industrialized nations. These findings will support understanding of the HIV burden among MSM and corresponding public health and health services planning for this key population.
Collapse
Affiliation(s)
- Ashleigh J Rich
- Epidemiology and Population Health Program, British Columbia Centre for Excellence in HIV/AIDS, 505-1200 Burrard Street, Vancouver, BC, V6Z 2C7, Canada.
- Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada.
| | - Nathan J Lachowsky
- Epidemiology and Population Health Program, British Columbia Centre for Excellence in HIV/AIDS, 505-1200 Burrard Street, Vancouver, BC, V6Z 2C7, Canada
- School of Public Health and Social Policy, Faculty of Human and Social Development, University of Victoria, Victoria, BC, Canada
| | - Paul Sereda
- Epidemiology and Population Health Program, British Columbia Centre for Excellence in HIV/AIDS, 505-1200 Burrard Street, Vancouver, BC, V6Z 2C7, Canada
| | - Zishan Cui
- Epidemiology and Population Health Program, British Columbia Centre for Excellence in HIV/AIDS, 505-1200 Burrard Street, Vancouver, BC, V6Z 2C7, Canada
| | - Jason Wong
- British Columbia Centre for Disease Control, Vancouver, BC, Canada
| | - Stanley Wong
- British Columbia Centre for Disease Control, Vancouver, BC, Canada
| | | | | | - Travis Salway Hottes
- British Columbia Centre for Disease Control, Vancouver, BC, Canada
- University of Toronto, Toronto, ON, Canada
| | - Eric A Roth
- Department of Anthropology, Faculty of Social Sciences, University of Victoria, Victoria, BC, Canada
- Centre for Addictions Research BC, Victoria, BC, Canada
| | - Robert S Hogg
- Epidemiology and Population Health Program, British Columbia Centre for Excellence in HIV/AIDS, 505-1200 Burrard Street, Vancouver, BC, V6Z 2C7, Canada
- Faculty of Health Sciences, Simon Fraser University, Burnaby, BC, Canada
| | - David M Moore
- Epidemiology and Population Health Program, British Columbia Centre for Excellence in HIV/AIDS, 505-1200 Burrard Street, Vancouver, BC, V6Z 2C7, Canada
- Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada
- British Columbia Centre for Disease Control, Vancouver, BC, Canada
| |
Collapse
|
14
|
Danileiko I, Lee MD. A Model‐Based Approach to the Wisdom of the Crowd in Category Learning. Cogn Sci 2017; 42 Suppl 3:861-883. [DOI: 10.1111/cogs.12561] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2017] [Revised: 07/26/2017] [Accepted: 09/13/2017] [Indexed: 11/27/2022]
Affiliation(s)
- Irina Danileiko
- Department of Cognitive Sciences University of California, Irvine
| | - Michael D. Lee
- Department of Cognitive Sciences University of California, Irvine
| |
Collapse
|
15
|
Population size estimation of female sex workers in Iran: Synthesis of methods and results. PLoS One 2017; 12:e0182755. [PMID: 28796847 PMCID: PMC5552099 DOI: 10.1371/journal.pone.0182755] [Citation(s) in RCA: 43] [Impact Index Per Article: 6.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2017] [Accepted: 07/24/2017] [Indexed: 11/19/2022] Open
Abstract
INTRODUCTION Estimating the number of key populations at risk of HIV is essential for planning, monitoring, and evaluating prevention, care, and treatment programmes. We conducted this study to estimate the number of female sex workers (FSW) in major cities of Iran. METHODS We used three population size estimation methods (i.e., wisdom of the crowds, multiplier method, and network scale-up) to calculate the number of FSW in 13 cities in Iran. The wisdom of the crowds and multiplier methods were integrated into a nationwide bio-behavioural surveillance survey in 2015, and the network scale-up method was included in a national survey of the general population in 2014. The median of the three methods was used to calculate the proportion of the adult female population who practice sex work in the 13 cities. These figures were then extrapolated to provide a national population size estimation of FSW across urban areas. RESULTS The population size of FSW was 91,500 (95% Uncertainty Intervals [UIs] 61,400-117,700), corresponding to 1.43% (95% UIs 0.96-1.84) of the adult (i.e., 15-49 year-old) female population living in these 13 cities. The projected numbers of FSW for all 31 provincial capital cities were 130,800 (95% UIs 87,800-168,200) and 228,700 (95% UIs 153,500-294,300) for all urban settings in Iran. CONCLUSIONS Using methods of comparable rigor, our study provided a data-driven national estimate of the population size of FSW in urban areas of Iran. Our findings provide vital information for enhancing HIV programme planning and lay a foundation for assessing the impact of harm reduction efforts within this marginalized population.
Collapse
|
16
|
The relationship between crowd majority and accuracy for binary decisions. JUDGMENT AND DECISION MAKING 2017. [DOI: 10.1017/s1930297500006227] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Abstract
AbstractWe consider the wisdom of the crowd situation in which individuals make binary decisions, and the majority answer is used as the group decision. Using data sets from nine different domains, we examine the relationship between the size of the majority and the accuracy of the crowd decisions. We find empirically that these calibration curves take many different forms for different domains, and the distribution of majority sizes over decisions in a domain also varies widely. We develop a growth model for inferring and interpreting the calibration curve in a domain, and apply it to the same nine data sets using Bayesian methods. The modeling approach is able to infer important qualitative properties of a domain, such as whether it involves decisions that have ground truths or are inherently uncertain. It is also able to make inferences about important quantitative properties of a domain, such as how quickly the crowd accuracy increases as the size of the majority increases. We discuss potential applications of the measurement model, and the need to develop a psychological account of the variety of calibration curves that evidently exist.
Collapse
|
17
|
Herzog SM, von Helversen B. Strategy Selection Versus Strategy Blending: A Predictive Perspective on Single- and Multi-Strategy Accounts in Multiple-Cue Estimation. JOURNAL OF BEHAVIORAL DECISION MAKING 2016. [DOI: 10.1002/bdm.1958] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
18
|
Sulaberidze L, Mirzazadeh A, Chikovani I, Shengelia N, Tsereteli N, Gotsadze G. Population Size Estimation of Men Who Have Sex with Men in Tbilisi, Georgia; Multiple Methods and Triangulation of Findings. PLoS One 2016; 11:e0147413. [PMID: 26828366 PMCID: PMC4734621 DOI: 10.1371/journal.pone.0147413] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2015] [Accepted: 01/04/2016] [Indexed: 11/18/2022] Open
Abstract
INTRODUCTION An accurate estimation of the population size of men who have sex with men (MSM) is critical to the success of HIV program planning and to monitoring of the response to epidemic as a whole, but is quite often missing. In this study, our aim was to estimate the population size of MSM in Tbilisi, Georgia and compare it with other estimates in the region. METHODS In the absence of a gold standard for estimating the population size of MSM, this study reports a range of methods, including network scale-up, mobile/web apps multiplier, service and unique object multiplier, network-based capture-recapture, Handcock RDS-based and Wisdom of Crowds methods. To apply all these methods, two surveys were conducted: first, a household survey among 1,015 adults from the general population, and second, a respondent driven sample of 210 MSM. We also conducted a literature review of MSM size estimation in Eastern European and Central Asian countries. RESULTS The median population size of MSM generated from all previously mentioned methods was estimated to be 5,100 (95% Confidence Interval (CI): 3,243~9,088). This corresponds to 1.42% (95%CI: 0.9%~2.53%) of the adult male population in Tbilisi. CONCLUSION Our size estimates of the MSM population (1.42% (95%CI: 0.9%~2.53%) of the adult male population in Tbilisi) fall within ranges reported in other Eastern European and Central Asian countries. These estimates can provide valuable information for country level HIV prevention program planning and evaluation. Furthermore, we believe, that our results will narrow the gap in data availability on the estimates of the population size of MSM in the region.
Collapse
Affiliation(s)
| | - Ali Mirzazadeh
- Global Health Sciences, University of California San Francisco, San Francisco, California, United States of America
- Regional Knowledge Hub, and WHO Collaborating Centre for HIV Surveillance, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
| | | | | | - Nino Tsereteli
- Center for Information and Counseling on Reproductive Health–Tanadgoma, Tbilisi, Georgia
| | | |
Collapse
|
19
|
Quaye S, Fisher Raymond H, Atuahene K, Amenyah R, Aberle-Grasse J, McFarland W, El-Adas A. Critique and lessons learned from using multiple methods to estimate population size of men who have sex with men in Ghana. AIDS Behav 2015; 19 Suppl 1:S16-23. [PMID: 25704987 DOI: 10.1007/s10461-014-0943-4] [Citation(s) in RCA: 39] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
Population size estimation of key populations at risk of HIV is essential to every national response. We implemented population size estimation of men who have sex with men (MSM) in Ghana using a three-stage approach within the 2011 Ghana Men's Study: during the study's formative assessment, the larger integrated bio-behavioral surveillance (IBBS) survey; and during the stakeholder meeting. We used six methods in combination within the three-stage approach (literature review, mapping with census, unique object multiplier, service multiplier, wisdom of the crowd, and modified Delphi) to generate size estimates from 16 locations (4 IBBS survey sites and 12 other locations) and used the estimates from the 16 sites to extrapolate the total MSM population size of Ghana. We estimated the number of MSM in Ghana to be 30,579 with a plausible range of 21,645-34,470. The overall estimate suggests that the prevalence of MSM in Ghana is 0.48 % of the adult male population. Lessons learned are shared to inform and improve applications of the methods in future studies.
Collapse
Affiliation(s)
- Silas Quaye
- Division of Global HIV/AIDS, Center for Global Health, Centers for Disease Control and Prevention, 24 Fourth Circular Rd., Cantonments, Accra, Ghana,
| | | | | | | | | | | | | |
Collapse
|
20
|
Evolution of a Metaheuristic for Aggregating Wisdom from Artificial Crowds. PROGRESS IN ARTIFICIAL INTELLIGENCE 2015. [DOI: 10.1007/978-3-319-23485-4_24] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
21
|
|
22
|
Lee MD, Steyvers M, Miller B. A cognitive model for aggregating people's rankings. PLoS One 2014; 9:e96431. [PMID: 24816733 PMCID: PMC4015936 DOI: 10.1371/journal.pone.0096431] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2014] [Accepted: 04/07/2014] [Indexed: 11/19/2022] Open
Abstract
We develop a cognitive modeling approach, motivated by classic theories of knowledge representation and judgment from psychology, for combining people's rankings of items. The model makes simple assumptions about how individual differences in knowledge lead to observed ranking data in behavioral tasks. We implement the cognitive model as a Bayesian graphical model, and use computational sampling to infer an aggregate ranking and measures of the individual expertise. Applications of the model to 23 data sets, dealing with general knowledge and prediction tasks, show that the model performs well in producing an aggregate ranking that is often close to the ground truth and, as in the "wisdom of the crowd" effect, usually performs better than most of individuals. We also present some evidence that the model outperforms the traditional statistical Borda count method, and that the model is able to infer people's relative expertise surprisingly well without knowing the ground truth. We discuss the advantages of the cognitive modeling approach to combining ranking data, and in wisdom of the crowd research generally, as well as highlighting a number of potential directions for future model development.
Collapse
Affiliation(s)
- Michael D. Lee
- Department of Cognitive Sciences, University of California Irvine, Irvine, California, United States of America
- * E-mail:
| | - Mark Steyvers
- Department of Cognitive Sciences, University of California Irvine, Irvine, California, United States of America
| | - Brent Miller
- Department of Cognitive Sciences, University of California Irvine, Irvine, California, United States of America
| |
Collapse
|
23
|
Using cognitive models to combine probability estimates. JUDGMENT AND DECISION MAKING 2014. [DOI: 10.1017/s1930297500005799] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Abstract
AbstractWe demonstrate the usefulness of cognitive models for combining human estimates of probabilities in two experiments. The first experiment involves people’s estimates of probabilities for general knowledge questions such as “What percentage of the world’s population speaks English as a first language?” The second experiment involves people’s estimates of probabilities in football (soccer) games, such as “What is the probability a team leading 1–0 at half time will win the game?”, with ground truths based on analysis of large corpus of games played in the past decade. In both experiments, we collect people’s probability estimates, and develop a cognitive model of the estimation process, including assumptions about the calibration of probabilities and individual differences. We show that the cognitive model approach outperforms standard statistical aggregation methods like the mean and the median for both experiments and, unlike most previous related work, is able to make good predictions in a fully unsupervised setting. We also show that the parameters inferred as part of the cognitive modeling, involving calibration and expertise, provide useful measures of the cognitive characteristics of individuals. We argue that the cognitive approach has the advantage of aggregating over latent human knowledge rather than observed estimates, and emphasize that it can be applied in predictive settings where answers are not yet available.
Collapse
|
24
|
Okal J, Geibel S, Muraguri N, Musyoki H, Tun W, Broz D, Kuria D, Kim A, Oluoch T, Raymond HF. Estimates of the size of key populations at risk for HIV infection: men who have sex with men, female sex workers and injecting drug users in Nairobi, Kenya. Sex Transm Infect 2013; 89:366-71. [PMID: 23761166 DOI: 10.1136/sextrans-2013-051071] [Citation(s) in RCA: 57] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Abstract
OBJECTIVES Size estimates of populations at higher risk for HIV infection are needed to help policy makers understand the scope of the epidemic and allocate appropriate resources. Population size estimates of men who have sex with men (MSM), female sex workers(FSW) and intravenous drug users (IDU) are few or non-existent in Nairobi, Kenya. METHODS We integrated three population size estimation methods into a behavioural surveillance survey among MSM, FSW and IDU in Nairobi during 2010–2011. These methods included the multiplier method, ‘Wisdom of the Crowds’ and an approach that drew on published literature. The median of the three estimates was hypothesised to be the most plausible size estimate with the other results forming the upper and lower plausible bounds. Data were shared with community representatives and stakeholders to finalise ‘best’ point estimates and plausible bounds based on the data collected in Nairobi, a priori expectations from the global literature and stakeholder input. RESULTS We estimate there are approximately 11 042 MSM with a plausible range of 10 000–22 222, 29 494 FSW with a plausible range of 10 000–54 467 FSW and approximately 6107 IDU and plausibly 5031–10 937 IDU living in Nairobi. CONCLUSIONS We employed multiple methods and used a wide range of data sources to estimate the size of three hidden populations in Nairobi, Kenya. These estimates may be useful to advocate for and to plan, implement and evaluate HIV prevention and care programmes for MSM, FSW and IDU. Surveillance activities should consider integrating population size estimation in their protocols.
Collapse
|