Tyler Burleigh Research Scientist PhD scientist w/ full-stack chops
Stuff I wrote

A Test of Time-Based Sample Stratification On MTurk

After reading Casey et al.'s (2017) paper on time-based sample variation with online samples, it occurred to me that this issue might be solved with time-based sample stratification. The logic goes like this: If the issue is sample bias resulting from posting large batches of HITs at specific times of day, then posting many smaller micro-batches at equal intervals spaced out across time should lead to better representation. Here I present the results from a secondary analysis of a study (N = 197) in which I apply this technique.

Preliminary Validation and Usability of a 5-Minute Web Stroop (Mini Report)

I wasn't satisfied with the default Stroop task that shipped with expfactory. It was too long and (in my opinion) the user experience left something to be desired. No surprise there—experimental cognitive psychologists tend to place a strong emphasis on experimental control, and may even see usability as a threat to validity. So, with the help of Vanessa Soch of expfactory, together we created a version of the task with better usability that would only take 5 minutes to complete—the 5-Minute Stroop Task. We also conducted a study to ensure that revisions to improve usability and brevity had not reduced the scientific validity or statistical power of the test. To do this, we recruited 50 participants from Amazon's Mechanical Turk (MTurk) to complete the task, and we also asked them to tell me about their experience. In this blog post I share the results of this study.