In this tutorial, I explain how to setup Qualtrics to generate survey codes for Mechanical Turk.
A common method for validating the participation of Workers on externally hosted surveys goes like this:
- Send Workers to a survey hosted on a platform like Qualtrics
- Have the survey software generate a random code
- Present it to Workers on the last page of the survey
- Ask Workers to submit the code on Mechanical Turk
- Compare codes generated by the survey with those submitted to Mechanical Turk; a code match is a valid proof-of-work
The code approach is so widely used that Mechanical Turk has a template called “Survey Link” which is designed for this exact setup. It looks like this:
The approach that I describe here uses randomly generated codes, for a very good reason. In the past, I have seen researchers give all participants the same code. This is problematic because Workers are known to share information about tasks (on sites like Turker Nation, mturkgrind, Mturk Forum and others) and if a single Worker makes the code publicly known, then it becomes impossible to track who actually completed the survey.
So how do we go about generating, recording, and presenting a random code?
The first thing you need to do is create a custom End of Survey message. To do this, first navigate Library -> Message Library.
Click Create a New Message.
For Category, select End of Survey Messages.
For Description, put “Mechanical Turk Survey Code” or something similar.
Finally, in the message box put the following text:
We thank you for your time spent taking this survey.
Your response has been recorded.
Your MTurk completion code is:
That last line in the message is key — it’s Qualtrics code for Piped Text. Basically, we are “piping” in the value of a variable that will get set elsewhere in the survey logic. We will set this variable in step three below.
Now we will tell the survey to use the End of Survey Message that we just created.
Navigate to Edit Survey -> Survey Options.
Scroll down to the Survey Termination section, then click the “Custom end of survey message” option, and select the message you created earlier, like so:
Then click Save Changes.
Now we will setup the survey to generate the survey code.
Navigate to Edit Survey -> Survey Flow for the survey you want to setup.
Click Add a New Element Here then select Web Service. For the URL, enter
Then click Test URL.
A window will pop-up that looks like this:
Check the box next to random, then click Add Embedded Data.Now set the “Embedded Data” variable name to MTurkCode (now you see why we used this in the Piped Text above), but leave the “Field From Web Service” field as random.
Next, we need to tell it to generate a certain number of digits for the code. Click the “Add a parameter to send to web service” link. Enter min for the “Parameter to Web Service” and 0 for the value. Then click the (+) symbol to add another parameter. Set this one to max and 99999999. We use 8-digits to ensure that there is a close-to-zero possibility that it will generate the same code twice (there are 100 million possible combinations).
Your Web Service should now look like this:
Finally, you need to Move the Web Service to the very top of your Survey Flow (click and hold the Move link, and drag it above all of the other elements), then click Save Flow.
That’s it! You’re all set now.
When testing, make sure to run the survey “live” and not in “preview” mode, because the code won’t display in preview mode.
Comments or questions? Post them below or send me an email!
As always, remember to test things first!