Back to articles
Articles
Volume: 28 | Article ID: art00038
Image
Using individual differences to better determine normative responses from crowdsourced transcription tasks: An application to the R. E. MacLaury Color Categorization Archive
  DOI :  10.2352/ISSN.2470-1173.2016.16.HVEI-115  Published OnlineFebruary 2016
Abstract

Individual differences inherent in human perceptual and behavioral data pose challenges for researchers who aim to develop standardized models of phenomena and procedures for normative assessment. A common approach used when modeling individual variation is to adopt criteria for identifying and excluding the individual data of outliers. We present investigations that use an alternative approach for analyzing response variation, which makes use of individual differences in data, to define a robust process model of both response variation and the information shared by individuals in a group. Crowdsourced perceptual identification tasks and formal analysis methods – Cultural Consensus Theory (CCT) – are employed to evaluate participants’ responses to transcription tasks, towards the aim of digitizing approximately 23,000 handwritten pages of an irreplaceable cross-cultural color categorization survey by Robert E. MacLaury. Preliminary results show (1) utility of several original crowdsourced tasks for database transcription, (2) the appropriateness of CCT as a formal model for aggregating transcription data, (3) novel ways of addressing “expertise” using CCT analyses, and (4) the accurate derivation of correct transcription “answer keys”, suggesting the potential for CCT methods to contribute to accurate transcription results even in the presence of large individual differences in participants responses. Research presented suggests that crowdsourcing in conjunction with CCT considerably reduces, without loss of accuracy, the number of participants needed for expeditious transcription of large, handwritten, corpora.

Subject Areas :
Views 28
Downloads 0
 articleview.views 28
 articleview.downloads 0
  Cite this article 

Kimberly A Jameson, Prutha S Deshpande, Sean Tauber, Stephanie M Chang, Sergio Gago, "Using individual differences to better determine normative responses from crowdsourced transcription tasks: An application to the R. E. MacLaury Color Categorization Archivein Proc. IS&T Int’l. Symp. on Electronic Imaging: Human Vision and Electronic Imaging,  2016,  https://doi.org/10.2352/ISSN.2470-1173.2016.16.HVEI-115

 Copy citation
  Copyright statement 
Copyright © Society for Imaging Science and Technology 2016
72010604
Electronic Imaging
2470-1173
Society for Imaging Science and Technology
7003 Kilworth Lane, Springfield, VA 22151 USA