Can SC-iAT be made optional

Question

Greetings,

I am including three single category IAT tests (SC-IAT) in my two day study. My IRB insists that all questions be optional but I have not found a setting that allows an SC-iAT to be skipped. Also, on whatever page I put an SC-IAT question on, the "next" and "back" buttons disappear. Is there an easy way to have the buttons appear at least until the space bar is pressed, or do I have to put a button on a prior page that will allow them to skip over the IAT page.

Answer 1

Is there an easy way to have the buttons appear at least until the space bar is pressed

You can use the following JavaScript code:

<script type="text/javascript">
window.addEventListener("load", function() {
  window.setTimeout(SoSciTools.submitButtonsDisplay, 500);
});
</script>

Or you can use the PHP function buttonCode() to add another button "Skip this test" to the page. This, however, would encourage the respondents to skip the question.

Comments

So I am sorry to report that I do not think this bug is fixed, in fact it may be worse.  I had another report of this issue this morning. The person sent me the following links to show the issue:
https://www.soscisurvey.de/DecisionSimularionSummer2020/index.php?i=I2ZU6SZBNLV2&rnd=UTZU
https://www.soscisurvey.de/DecisionSimularionSummer2020/index.php?i=I2ZU6SZBNLV2&rnd=QJFL
https://www.soscisurvey.de/DecisionSimularionSummer2020/index.php?i=I2ZU6SZBNLV2&rnd=YXJN
https://www.soscisurvey.de/DecisionSimularionSummer2020/index.php?i=I2ZU6SZBNLV2&rnd=PDMJ

When I saw this, I went into the data and looked at every case that was started after 11:00 pm on September 10, or roughly 30 minutes after I received your email reporting the most recent fix.  173 of a possible 283 are stuck on page 10 or 11 where the SC-IAT tests are.  That is over 60%.  

Given this data, I have taken your earlier advice and reset the IAT tests to not probe and simply allow no answers even though that was how the bug was first discovered.  I am hoping that the changes will allow that case to work.  

This is causing real problems for me.  If lots of people on Amazon Turk agree to do my study but can't complete it and thus don't get paid they will likely rate me low on Amazon. This could hinder my ability to use the platform for studies in the future.  Also, this is slowing my data collection down considerably.  Lastly, Amazon forces me to give them money to hold in reserve for all of the tasks I create. Since people aren't completing them this money is sitting in limbo and I won't be refunded until I can show that the tasks won't be used which requires various date limits to expire.  Hopefully this will work now that I have made the change but if it doesn't I really need a good working fix.

---

I just opened the URL (they are all the same except for a random string to prevent browser caching). I saw the IAT, then clicked on "next" and then saw a page

> Thank you for filling out these surveys. You are finished for today.

That may be because you have changed to non-probing. I could also set the interview one page back to replicate the issue.

> 173 of a possible 283 are stuck on page 10 or 11

Very strange ... I did several tests and was not able to provoke such a problem. Could you possibly download the question as XML (feature in the top of the question) and send this XML to info@soscisurvey.de? Then I can test this with your specific question...

I would, if you all right with that, also retrieve the IAT raw data from the case that you have linked above, and use this to test with your question.

> This is causing real problems for me.

I feel really bad for this any can only repeat my apologies. What is the current state without the probing? And more important: Is the IAT data in the cases that got stuck and what is the page time TIME*** stored for these cases? Are these people who actually did the test or did they just try to skip it?

---

Thank you again for you quick reply  The news is good.  Since I made the change to remove the probe, no one has written me to complain about getting stuck.  I probably would have heard from 1 or 2 by now.

 I went back to my downloaded data from yesterday and found the average time people spent on page 11 among those who's max page was 11.  It was much lower than I thought at only ~255.  The standard deviation was high however at roughly 508.  There were about 10 cases who spent 600 seconds or more on the page.  I am guessing that many fewer were stuck than I thought and so I am much less concerned today than I was yesterday.  I have an interrupt page on page 12 and so my assumption was that it would be the max page for anyone waiting on the interrupt.  My guess is that I had that wrong and that max page is 11 except for those who already got to the next page (page 13) which happens 18-24 hours later, typically.  If that is right than the vast majority of those 173 people were not stuck. It probably wasn't more than a dozen.  

Nevertheless I have mailed the XML of the question as per your request.  This is, however, the version with the probe off.

---

Thanks for sending me the question's XML file. I went into the database to CASE 4019 that you have sent me the links for.

And yes, I was able to identify the problem. The SC-IAT used a heuristic that any case with less than 4 valid trials in the last block (time > 350 ms and correctly answered) seems lacking data and was probably not even started.

But, really ... now I am quite sure that no useful data got lost. And I am also sure that nobody got stuck in the loop who did not earn that. Of course, this is not how research ethics work, so this (hopefully last) issue is also fixed now. The answer is either accepted (20+ trials of data sent, no matter if valid or not and counting all blocks) or there is no automatic re-submitting of the SC-IAT and more.

So, if you like, you could now re-enable the probing feature for the question. I assume you do not like, so this is only meant to be an update about the issue :)

---

Thank you very much for tracking this down and for the detailed explanation.  My data gathering is nearly complete so I won't change things now but if I use the SC-IAT again I will likely go back to the probe.  Incidentally I received no more complaints since my last post.