Design of vignettes

Question

Good afternoon,
I have a question concerning a questionnaire design.
I would like to implement a vignette study in which each vignette is composed of three different independent variables (a,b,c) , having two levels each.(a1/a2, b1/b2, c1/c2).
Accordingly 8 different combinations of those independent variables/vignettes are possible.
(a1,b1,c1);(a1,b2,c1);(a1,b1,c2);(a1,b2,c2)......
Unfortunately I am not sure how I can implement a questionnaire design, in which each respondent evaluates all 8 potential combinations individually but according to the same question (Ideally the order of those combinations would be randomized). I already found a design allowing for such combinations of independent variables, but unfortunately one could only ones draw from this urn of combinations. Is there any option to draw from this urn of 8 potential combinations on different pages ? Subsequently I would like to analyze the date with an ANOVA analysis.

Thank you very much in advance !

Answer 1

questionnaire design, in which each respondent evaluates all 8 potential combinations individually but according to the same question

There are multiple options to implement such a design. The easiest would be the following one:

1) Create 8 copies of that quastion, adjust the content to present one vignette per question

2) Place these questions on 8 different pages

3) Rotate the page order.

You could as well work with with placeholders or question rotation, but the above solution is the least abstract one in my opinion.

Comments

Thank you very much!

Would in this case a multilevel design be necessary or is it easier to aggregate the results of the 8x the same question to a variable afterwards.
Moreover I'm wondering whats the easiest way to present the different stimuli, I tried it with a with-drawl of combinations from one urn, according to the Fruitbasket example in the Randomization by urn chapter, but this results in the same combination of the stimuli on each of the 8 pages instead of a different combination on each page. Unfortunately I could not find any solution how to use combinations with a random generator. How can I solve this problem ?

urnDraw('kombi', 'IV01');
$a = array(
  1 => ' u',
  2 => ' v',
);
$b = array(
  1 => 'w,
  2 => 'x'
);
$c = array(
  1 => 'y',
  2 => 'z'
);
html('
  <h1>Bitte stellen Sie sich folgenden Obstkorb vor:</h1>
  <p>• '.$a[value('IV_01')].'</p>
  <p>• '.$b[value('IV_02')].'</p>
  <p>• '.$c[value('IV_03')].'</p>
');

---

> Would in this case a multilevel design be necessary or is it easier to aggregate the results of the 8x the same question to a variable afterwards.

A multi level design is elegant - but in my opinion 8 variable (sets) are easy to handle as well. I assume you don't have more than a few variables per vignette?

>  I tried it with a with-drawl of combinations from one urn

As I said, try the page rotation code (the manual is linked in my answer from above). This uses shuffle() for the rotation. You don't need a urn or a random generator for that.

---

Thank you very much for the fast answer!

As I understand the rotation of the page order helps me to randomize the order of the vignettes, so that the respondents evaluate the vignettes in a different order.  

Unfortunately the implementation of this did not solve yet my problem since the urn I implemented was not intended to randomize the vignettes themselves, but the content of the single vignette.

For my research I am using 3 variables
-price (a)
-service (b)
-speed (c)
 which each having two levels:
-high (1)
-low (2)
Each vignette will describe the price, service and speed of a product which the respondents have to evaluate based on the same question. According to the previously described variables and levels 8 potential combinations of price, service and speed and hence vignettes are possible.
In my research I would like that each respondent rates all 8 potential combinations/vignettes, with each page showing one combination/vignette + rating scale. I tried to retrieve the possible combinations from an urn in which all potential level combinations were written down (a1,b1,c1 ;a1,b2,c1; a1,b2,c2;...). Afterwards I copied this code on 8 consecutive pages. But instead of displaying a different combination on each side (e.g. p.1 (a1,b1,c1), p.2 (a1,b2,c2), p.3 (a1,b2,c1)...) each page shows the same combination (p.1. (a1,b2,c1), p.2 (a1,b2,c1), p.3. (a1,b2,c1).
So instead of displaying all different combinations from the urn there is always the same combination displayed on all 8 pages.
Is there any way to overcome this!

I am very sorry for the inconvenience!

---

> Unfortunately the implementation of this did not solve yet my problem since the urn I implemented was not intended to randomize the vignettes themselves, but the content of the single vignette.

If I got you right, you have 8 vignettes and they are always the same, for each respondent.

If that is correct, you do not need any urn. Just create 8 questions (e.g. selections) with the contents of the 8 vignettes. If you use some HTML code, and (for example) a horizontal selection, you can present them quite nice.

---

You are probably right, thank you very much!

Could you give me a hint how I can implement the different levels of the variables (price, service, and speed) in one question for example for a product which has:
-a1:high price
-b2: low serivce
-c1: high speed
 so that I retrieve data showing the effect of for example 'a1(Price high)' on my dependent variable 'product' evaluation.
I am not quite sure how to implement this since I always present several variables in one vignette and subsequently would like to analyze each variables main effect (e.g. price -> product evaluation, as well as mixed effects e.g. price x service -> product evaluation)

So far I chose the approach of building a textblock for each level (price high; price low...) as well as a Internal variable for each level (price high; price low...).
Subsequently I added a PHP:
prepare_input('a1')
prepare_input('b2')
prepare_input('c1')

and added underneath the matching textblocks. When I start the quesionnaire in debug mode a pop up shows up saying 'Sie können auf dieser Seite via HTML ein Formularelement einbinden: name="a1"  is that normal ?

And again , thank you very much you're being a great help!

---

Well the HTML code look like this:

<table>
  <tr>
    <td>Price:</td><td>high</td>
    <td>Service:</td><td>low</td>
    <td>Speed:</td><td>high</td>
  </tr>
</table>

Just add two options to a horizontal selection, then open the options for editing in the list of question (menu) and then insert this HTML code (or whatever code you would like to use, to make it appear more beautiful).

prepare_input() is deprecated. If you really would like to work with separately placed inputs, please use them as follows (one question per pair).


<table>
  <tr>
    <th colspan="2">Option A</th>  <th colspan="2">Option B</th>
    <td>Price:</td><td>high</td>  <td>Price:</td><td>high</td>
    <td>Service:</td><td>low</td>  <td>Service:</td><td>high</td>
    <td>Speed:</td><td>high</td>  <td>Speed:</td><td>low</td>
    <td colspan="2">%input:AB01_01% I prefer Option A</th>  <td colspan="2">%input:AB01_02% I prefer Option B</th>
  </tr>
</table>

This is then something to store as text element (HTML code). The %input...% will insert the input elements at the appropriate positions.

---

Since I imagined my research to rather look like this:
"Imagine the following product:
• the price of the product is high (=a1)
• the linked service is high          (=b1)
• the speed of the product is rather low " (=c2)
"Please evaluate the product on the following scale (1=very bad...7=very good).

Accordingly I would like to be able to link "product price is high" to the internal variable of it a1, the "service being high" to b1 and the " speed being rather low" to c2, in order to be able to measure their individual influence on the rating. I tried this according to the manual 'Use Custom Form Elements'
Therefore I build a html textblock for each of the variable levels( e.g. for a1 "the price of the product is high" and wrote in the individual IDs "name=a1" and so on. Subsequently I added a PHP code saying prepare_input('a1')
prepare_input('b2')
prepare_input('c1') in order to activate the internal variables, which how I understood it subsequently allows me to measure their influence on the rating.
Under the PHP code I simply dragged the textblocks which were named after the single IV they represented and simply contained a html version of the text e.g. "the price of the product is high". Unfortunately I want to debuck I get three times the error notification saying "Die Frage mit der Kennung __00 gibt es im Projekt nicht."

Since I am unfortunately not too experienced with Sosci I wanted to stick to the previously mentioned manual but apparently there is a mistake from my side for the implementation.

---

> Please evaluate the product on the following scale (1=very bad...7=very good).

Well, in this case I would recommend to simply put the product description (pimped with HTML) as text or instruction into a scale question (plus one Item for the rating, you can set to display this in a single line).

You definitely won't need prepare_input() or %input:...% to implement this design :)

>  in order to activate the internal variables, which how I understood it subsequently allows me to measure their influence on the rating.

You do not "measure" the influence, but infer it from patterns in the data. What you do when you're data is collected is to compute a regression model to estimate the influence of each variable on the rating.

What you need for that is either a multi-level design (a bit overcomplex for that) or a restructured data set with one column for the product (was: variable number), one for the rating, and one for the CASE number, e.g.

1  7  123
2  5  123
3  6  123
4  7  123
...
1  5  124
2  6  124
3  2  124
...

And then in SPSS you recode the 1 to 8 in the first colum into 3 variables describing your independent variables. That's all you need to compute the regression model.

If we find some time, we will implement a conjoint module. But as long as this is done, it's not that hard to realize.

---

That sounds like my (research) life just got way easier.

Accordingly I could just implement the study like this:
- I prepare a textblock for each potential variable level (e.g one for price high, one for service high, one for speed low...)
- I prepare 8 times the same rating questions (scale ranging from 1-7)

Subsequently I place on each page:
- three textblocks, each of one variable, to achieve the specific combination of price, service, speed
- one of the 8 question copies measuring the product evaluation on a scale of (1-7)

This I repeat until all 8 combinations of variables are in the questionnaire.

After the respondents answered I can restructure the data I received, in order to analyze the individual influence of for example price high on the product evaluation. So finally this allows me to achieve similar data like from a multilevel analysis?

And thank you for your fast answers, you are amazing !

---

> - I prepare a textblock for each potential variable level (e.g one for price high, one for service high, one for speed low...)

Not exactly. What you do is to prepare a question (with text) for each combinatin/vignette that contains all three variables in the text.

>  I place on each page:
> - three textblocks

Yes, you could also create it this way. But honestly: It will look better if you put it all into one HTML code.

> This I repeat until all 8 combinations of variables are in the questionnaire.

Correctly. Then you should have 8 pages for the 8 vignettes. And if you like (you should!) you use setPageOrder() one one page before the first vignette to rotate the pages.

> After the respondents answered I can restructure the data I received, in order to analyze the individual influence of for example price high on the product evaluation.

Exactly. But try this procedure after you have collected some data in the pretest. Just to be sure - and you don't want to do that when you're close to the deadline.

---

Not exactly. What you do is to prepare a question (with text) for each combinatin/vignette that contains all three variables in the text.

--> So I would just put the text describing my vignette in the field 'Frage-Text', unfortunately I didnt find there the option how to edit this part of the text as a html.

Or do I add the text separately via 'Text hinzufügen' and hence create 8 texts for the 8 vignettes  and just put the text of one vignette on top of one of the 8 scales measuring the product evaluation.

---

> unfortunately I didnt find there the option how to edit this part of the text as a html.

Just type HTML code :)

You can, of course, use separate texts and questions as well. However, this is unnecessarily complex in my opinion.