Questionnaire Randomization for Very Small Groups

Question

Hello,

I have a survey with a large number of different questionnaires. Upon clicking the survey link each participant should randomly get only one of the questionnaires. The tricky part is that each of the questionnaires needs to be filled out by a very small number of participants. The total number of participants for the entire survey will be set on an external platform where the participants are recruited.

I understand from the manual that with very small group sizes one may get unequal distribution of the groups if there are drop outs (which there often are). The way the survey link will be distributed is that many people will see and will be able to click on it at the same time. How do I go about making sure that I get only the necessary number of participants (and not more) for each questionnaire? Is there a mechanism that recognizes which questionnaires are currently being filled out and redirects the next participant to a questionnaire with still open participation slots? Also, if a participant drops out before submitting the last page, this should count as an incomplete questionnaire with a participation slot still available, is this correct? And finally, is there a way to set the exact number of respondents for each questionnaire or is this not necessary because the algorithm tries for equal distribution?

Thank you

Answer 1

The way the survey link will be distributed is that many people will see and will be able to click on it at the same time.

The question is: will they actually click at the same time or not?

If you want to be absolutely sure, you will have to use a random generator that takes care of an equal distribution at the time when the respondents start the questionnaire and you have to adjust the random generator very early (and a few times) to respond to drop-outs.

It is helpful if the randomization does not occur at the very beginning, but after one or two pages. Because the very most respondents drop-out on the first two pages.

If it's likely that responses will spread over time, a random generator that counts only completed interviews will be fine.

And finally, is there a way to set the exact number of respondents for each questionnaire or is this not necessary because the algorithm tries for equal distribution?

The algorithm (it's actually just an urn that is empied one piece aften another, and then refilled) will try to achieve an equal distribution. Therefore, there's no need to set a specific number.

Comments

Thanks so much! This clarifies many things! Yes, the randomization will not happen right from the first page - they all will see the consent form and the general instructions first and will then be redirected. I'll give it a try. Thanks again!