0 votes
ago in SoSci Survey (English) by s291395 (110 points)

Hi,

I'm looking to run a multifactorial survey experiment with a lot of randomization in SoSciSurvey and find myself a bit lost.

I have a factorial with 8 attributes with 2-4 values each (2x2x2x2x2x3x3x4) resulting in a total of 1152 vignettes. To be able to look at interactions I need to run the full factorial (all vignettes).

A vignette may look like this:

Please rate how likely you would be to buy a car with the following
attributes:
- Expensive
- White
- Automatic
- Four wheel drive
- No seat warmer
- Two doors
- Little storage space
- Electric

The vignettes are divided into 128 blocks of 9 vignettes each, with respondents being randomized to blocks.

Additionally, the order of the 9 vignettes should be randomized between respondents.

Finally, the order of the 8 attributes should be randomized between respondents.

I have already designed the blocks and vignettes and have that as a CSV file like below:

Block Run x1 x2 x3 x4 x5 x6 x7 x8
1 1 1 1 2 3 1 1 1 1
1 2 1 2 1 2 2 2 1 2
1 3 1 2 1 4 1 2 2 1
1 4 2 1 2 1 3 1 1 2
1 5 2 1 2 3 2 1 2 1
1 6 2 2 1 4 2 1 1 1
1 7 3 1 1 4 3 2 2 2
1 8 3 2 2 1 3 2 2 2
1 9 3 2 2 2 1 2 1 1
2 1 1 1 1 1 3 1 1 1
...
128 9 3 2 2 3 1 1 1 2

My first though was to use a random generator to divide respondents to a block between 1 and 128. But if I were to follow the instructions for randomization per block I would then need to create 128 additional random generators filled with the specific vignette combinations as shown in the guide for multifactorial experiments.

I feel like there should be a cleaner solution with php to first randomize to decks, then randomize the order of the attributes (example to x6x3x4x7x1x8x2x5), and then randomize the order of the vignettes (second column "Run"). This would then be three randomization per respondent, after which I would need to actually show these vignettes to the respondents. Alternatively, the order of the vignettes could just be a random draw without replacement if I can do it within the block only, and keeping the order of the attributes.

Any help is very much appreciated!

ago by SoSci Survey (315k points)
Sounds like you're facing some fun :)

> The vignettes are divided into 128 blocks of 9 vignettes each, with respondents being randomized to blocks.

First question: Are you sure you want to have 128 "fixed" blocks or would it be evenly (or even more) useful to randomly draw 9 out of the 1152 vignettes?

> the order of the 9 vignettes should be randomized between respondents.

If you decided to randomly draw 9 vignettes per respondents, this would not be an issue at all.

Second question: How many respondents will you have?

> I would then need to create 128 additional random generators

Thanks for seeing that this is a bad idea before doing a night-shift on creating those random generators :)

And also thank you for not overwhelming your respondents with a crazy number of vignettes per person. 9 may still annoy one person or another, but ther eis hope that a substanial part of the respondents will still read this number.
ago by s291395 (110 points)
Thanks for the response!

> First question: Are you sure you want to have 128 "fixed" blocks or would it be evenly (or even more) useful to randomly draw 9 out of the 1152 vignettes?

Yes, it would be much easier to fully randomize but then I might get confounding between main effects and interaction effects. As part of the focus is on higher order interaction the blocking is a necessity.

> Second question: How many respondents will you have?
About 1500, so with each rating nine vignettes, each unique vignette is rated about 11 times.
ago by SoSci Survey (315k points)
> As part of the focus is on higher order interaction the blocking is a necessity.

Good, I just wanted to be sure that there's a valid plan :)

Next Question: How many questions do you want to ask for each vignette? And which question types? This is important to decide if you should create 8 pages, work with loopPage() or better work with a multi-level-structure: https://www.soscisurvey.de/help/doku.php/en:create:multilevel
ago by s291395 (110 points)
>Next Question: How many questions do you want to ask for each vignette? And which question types? This is important to decide if you should create 8 pages, work with loopPage() or better work with a multi-level-structure: https://www.soscisurvey.de/help/doku.php/en:create:multilevel

We have five questions for each vignette. The first is a scaling slider 1-100, so question type slider, the other four are likert scales of 1-7 which I think is question type "Scale (extremes labeled)".

1 Answer

0 votes
ago by SoSci Survey (315k points)

We have five questions for each vignette. The first is a scaling slider 1-100, so question type slider, the other four are likert scales of 1-7 which I think is question type "Scale (extremes labeled)".

Okay, in my personal opinion, a multi-level-structure is the better solution.

I would recommend the following procedure:

  1. Give each vignette an ID (v1 to v1152) and
  2. Write a table with this key (v1...) in the first colum and the vignette data in the subsequent colums:

    v1 1 1 1 1 2 3 1 1 1 1
    v2 1 2 1 2 1 2 2 2 1 2

  3. Import this table (you can use an Excel file) inthe the database for contents.

  4. Create a random generator with 128 rows, each having the IDs of the above vignette blocks, that is

    1=v1,v2,v3,...v8
    2=v9,v10,v11,...v12

Again, use a script or excel table to do that.

  1. Create another random generator (for the vignette order) with only the numbers 1 to 8. Tell this random generator to draw 8 of those 8 codes per interview.
  2. Now, you draw one block from the first random generator, use value(..., 'csv') to split that into an array, then
  3. Draw the 8 codes from the second random generator an use some PHP to sort the array from above according to that order (you could also use shuffle(), but had to take care about page-reloads then).
  4. And store these 8 values into 8 internal varibles.

Now you have 8 variables with vignette numbers from one block, randomly sorted in the 8 variables of a "internal variables" question. Now comes the multi-level part. Use a loop to go through the 8 values.

Where do you need the "database for contents" now? You give the vigentte code (v1) to the subordinate questionnaire, make it look up the proper line in the database, and show the vignette accordingly - along with your questions. I recommend writing the vignette values in another internal variable to make your analysis easier.

And should these trivial eight steps (<- irony) pose any troubles on you, just ask :)

Willkommen im Online-Support von SoSci Survey.

Hier bekommen Sie schnelle und fundierte Antworten von anderen Projektleitern und direkt von SoSci Survey.

→ Eine Frage stellen


Welcome to the SoSci Survey online support.

Simply ask a question to quickly get answers from other professionals, and directly from SoSci Survey.

→ Ask a Question

...