Can SoSci Survey accomodate larger projects with advanced randomization and if yes, how?

Question

I have a question regarding the feasibility of implementing an upcoming survey project in SoSci Survey. Our current survey provider does not appear to support the required allocation of stimuli and participants. The survey would consist of three parts:

1. Pre-survey: This section would determine participants’ eligibility and classify them into two groups: participants with and without a speicific trait.

2. Comment annotation: Each participant would be randomly shown 50 comments from a pool of 1,500 text-only social media comments. Each comment must be evaluated by six participants: three with and three without the specific trait. For every comment, participants would provide continous rating and one binary classification. Thus, each comment should ultimately receive six ratings and six binary labels, evenly distributed across the two participant groups.

3. Post-survey: This section would include several open-ended questions.

The balanced allocation of comments across the two participant groups, combined with the size of the comment pool, appears to be difficult to implement with our current provider. Would it be possible to implement and host such a project in SoSci Survey?

Thank you very much in advance!

Answer 1

Pre-survey: This section would determine participants’ eligibility and classify them into two groups: participants with and without a speicific trait

Do you want the categorization to be automated (for example based on a scale index) or will you do it in a manual analysis? For automation, the function valueMean() is what you're looking for, and you will need a little PHP filter.

Is there a gap (days/weeks) between pre-survey and annotation task? Do you know the respondents a priori (and have their contact data) or will you collect email addresses in the pre-survey?

Each participant would be randomly shown 50 comments from a pool of 1,500 text-only social media comments.

You would use a random generator for this task in SoSci Survey.

three with and three without the specific trait

Two random generators in this case ;) See block randomization.

For every comment, participants would provide continous rating and one binary classification.

Most likely you will do your analysis a favor, using the multilevel feature for the measure per comment.