Attention check for a survey with 3 scenarios.

Question

Hello,

I need to implement attention check in my survey.
I have 3 scenarios, each participant will be randomly assigned to one of them, using Random generator. The attention check should look like selection of one out of 3 possible options. It will be placed on the next page after the scenario page to make sure that participants have read the scenario carefully and understood it. Participants should choose the correct option according to their scenario. How can I implement it and check afterwards that each participant has chosen the correct option for the scenario presented to them? I will need to exclude participants who failed the attention check when analyzing the final results of the survey.

Thank you!

Answer 1

Are you using the same selection for all participants? Or do you vary the options?

I will need to exclude participants who failed the attention check when analyzing the final results of the survey.

One option is to do that in the analysis. Exclude that where the chosen option and the random scenario do not match.

Another option is to use a bit PHP code and exclude them immediately. Might look like this on the page following the attention check:

$random = value('RG01');  // Random generator
$chosen = value('AC01');  // Attention check
if (
  (($random == 1) && ($chosen != 2)) ||
  (($random == 2) && ($chosen != 3)) ||
  (($random == 3) && ($chosen != 1))
) {
  text('XX01');  // Text do display when exluded
  buttonHide(); // We're done!
  pageStop();
}

Comments

Yes it will be the same selection for all participants.  No I don't want to exclude them immediately because I will need to mention it in my thesis how many participants are excluded and why. So my idea is, a participant reads the scenario, goes to the next page and should select the correct option out of 3 options. Next, I need to check if they have chosen the correct option or not. Can I do this check automatically by writing some code? for example, creating a pair scenario-answer or something like that. Or can I only manually check if the scenario matches the answer after the survey is completed?

One option is to do that in the analysis. Exclude that where the chosen option and the random scenario do not match. - Do you mean just to create the "Selection" question type and then manually check if the scenario matches with the answer?

---

> Can I do this check automatically by writing some code?

The code example for that is in my answer above.

What do you want to do with the check: Say them goodbye (you still have there data so far in the dataset) or just flag the record?

>  Or can I only manually check if the scenario matches the answer after the survey is completed?

That is possible as well. At least, you have the scenario and their anwers to the attention check.

> Do you mean just to create the "Selection" question type and then manually check if the scenario matches with the answer?

Yes. This is valid for all my answers above.

---

I would like them to complete the survey but flag the record that it doesn't match the scenario that was presented to them so that I don't need to check it manually.

---

Okay, than create an internal variable to strore the flag, and in the PHP code from above, replace the

  text('XX01');  // Text do display when exluded
  buttonHide(); // We're done!
  pageStop();

simply by setting the interne variable, for example IV01_01

  put('IV01_01', 1);

---

Thanks. So how I understand this code: 1 will be stored in an internal variable every time a scenario does not match the selected option from the Attention check section. Right?
How can I retrieve this information later when I analyze the results?

---

That is correct.

> How can I retrieve this information later when I analyze the results?

The internal variable is part of the data set. However, I recommend you just give it a try. This will  probably make it much easies to grasp. And make sure to put your PHP code a page after the attention check question, not on the same page.

---

Thank a lot!