Can SoSci Survey accomodate larger projects with advanced randomization and if yes, how?

Question

I have a question regarding the feasibility of implementing an upcoming survey project in SoSci Survey. Our current survey provider does not appear to support the required allocation of stimuli and participants. The survey would consist of three parts:

1. Pre-survey: This section would determine participants’ eligibility and classify them into two groups: participants with and without a speicific trait.

2. Comment annotation: Each participant would be randomly shown 50 comments from a pool of 1,500 text-only social media comments. Each comment must be evaluated by six participants: three with and three without the specific trait. For every comment, participants would provide continous rating and one binary classification. Thus, each comment should ultimately receive six ratings and six binary labels, evenly distributed across the two participant groups.

3. Post-survey: This section would include several open-ended questions.

The balanced allocation of comments across the two participant groups, combined with the size of the comment pool, appears to be difficult to implement with our current provider. Would it be possible to implement and host such a project in SoSci Survey?

Thank you very much in advance!

Answer 1

Pre-survey: This section would determine participants’ eligibility and classify them into two groups: participants with and without a speicific trait

Do you want the categorization to be automated (for example based on a scale index) or will you do it in a manual analysis? For automation, the function valueMean() is what you're looking for, and you will need a little PHP filter.

Is there a gap (days/weeks) between pre-survey and annotation task? Do you know the respondents a priori (and have their contact data) or will you collect email addresses in the pre-survey?

Each participant would be randomly shown 50 comments from a pool of 1,500 text-only social media comments.

You would use a random generator for this task in SoSci Survey.

three with and three without the specific trait

Two random generators in this case ;) See block randomization.

For every comment, participants would provide continous rating and one binary classification.

Most likely you will do your analysis a favor, using the multilevel feature for the measure per comment.

Comments

Thank you for your previous answer. I have now implemented the annotation task using the multilevel feature, as you suggested, and it works as intended.

I have a related question regarding the limits for free use and the paid cloud packages. The study consists of four separate survey projects, one for each country, with 200 completed participants per project. Each participant completes one main questionnaire and then rates 50 comments in a subordinate questionnaire. Consequently, each participant generates one main record and 50 subordinate records, i.e. 51 rows in the datasets.

The information about free use states that a survey project may collect a maximum of 5,000 “interviews (records in the dataset).” I have multiple questions regarding this:

1. Does the limit of 5,000 apply separately to each survey project or jointly across all projects in one account?

2. In this multilevel structure, does one participant count as one interview/respondent or as 51 interviews/records?

3. The paid cloud packages specify a maximum number of “respondents.” Would one participant count as one respondent there, regardless of the 50 subordinate records?

4. Can one paid package cover all four survey projects simultaneously?

5. The projects have already been created. Can a paid package be applied to these existing projects, or would I need to transfer them?

Thank you very much!

---

1) technically, the maximum refers to the number of records in a dataset. However, before we shut down any questionnaire, we will consult the project's responsible, and using a multi-level-structure is a valid reason for having a larger number of records in the survey projects. You won't have to worry about that.

2) Again, technically they would count as 51 records. Our licencing model does not fit diary studies and multilevel structures well. We didn't optimize it, because there are only few survey projects hitting that issue. Should you use a paid licence and the multilevel feature, just contact us via email...

3) ... we would then adjust the licence to match the participant numbers.

4) Yes, you can use one licence for multiple survey projects. By default, the overall number of records in all projects must not exceed the licence's limit. Again ... we'll take custom care for multilevel projects. Just send an email with the relevant information to info@soscisurvey.de.

5) The licence is automatically linked to all projects whose project administrator is linked to the customer account who bought the licence. You'll see the active licence in the project settings -> publish survey.