THOUGHTS ON SUBJECTIVE MEASUREMENT, SENSORY METRICS AND USEFULNESS OF OUTCOMES

Comparing Direct and Indirect Methods of Audio Quality Evaluation in Virtual Reality Scenes of Varying Complexity

Many quality evaluation methods are used to assess uni-modal audio or video content without considering perceptual, cognitive, and interactive aspects present in virtual reality (VR) settings. Consequently, little is known regarding the repercussions of the employed evaluation method, content, and subject behavior on the quality ratings in VR. This mixed between- and within-subjects study uses four subjective audio quality evaluation methods (viz. multiple-stimulus with and without reference for direct scaling, and rank-order elimination and pairwise comparison for indirect scaling) to investigate the contributing factors present in multi-modal 6-DoF VR on quality ratings of real-time audio rendering. For each between-subjects employed method, two sets of conditions in five VR scenes were evaluated within-subjects. The conditions targeted relevant attributes for binaural audio reproduction using scenes with various amounts of user interactivity. Our results show all referenceless methods produce similar results using both condition sets. However, rank-order elimination proved to be the fastest method, required the least amount of repetitive motion, and yielded the highest discrimination between spatial conditions. Scene complexity was found to be a main effect within results, with behavioral and task load index results implying more complex scenes and interactive aspects of 6-DoF VR can impede quality judgments.

Consumer Preference Testing of Boiled Sweetpotato Using Crowdsourced Citizen Science in Ghana and Uganda

Crowdsourced citizen science is an emerging approach in plant sciences. The triadic comparison of technologies (tricot) approach has been successfully utilized by demand-led breeding programmes to identify varieties for dissemination suited to specific geographic and climatic regions. An important feature of this approach is the independent way in which farmers individually evaluate the varieties on their own farms as “citizen scientists.” In this study, we adapted this approach to evaluate consumer preferences to boiled sweetpotato [Ipomoea batatas (L.) Lam] roots of 21 advanced breeding materials and varieties in Ghana and 6 released varieties in Uganda. We were specifically interested in evaluating if a more independent style of evaluation (home tasting) would produce results comparable to an approach that involves control over preparation (centralized tasting). We compiled data from 1,433 participants who individually contributed to a home tasting (de-centralized) and a centralized tasting trial in Ghana and Uganda, evaluating overall acceptability, and indicating the reasons for their preferences. Geographic factors showed important contribution to define consumers' preference to boiled sweetpotato genotypes. Home and centralized tasting approaches gave similar rankings for overall acceptability, which was strongly correlated to taste. In both Ghana and Uganda, it was possible to robustly identify superior sweetpotato genotypes from consumers' perspectives. Our results indicate that the tricot approach can be successfully applied to consumer preference studies.