Posts Tagged ‘ Ramona

Managing Questions in DDI3.1 – “Other, please specify”

A still difficult problem in managing complex questions in DDI is those questions that ask a respondent to pick from a list of options, and if no suitable ones exist, that they write in their own. Below are examples of this kind of question from the US, UK and Australian Censuses (censii/census/censes?):

UK Census Extract
USA Census Extract

ABS Census Extract

In all three questions, respondents are asked about their origins, and are given the option to select from a list of common responses or provide a write in response. The easiest way to manage this is through the use of a DDI <MultipleQuestionItem>. A <MultipleQuestionItem> is a way to capture a complex question that asks two or more separate questions that are highly linked.

In the above examples we can split the questions into two, as illustrated in the generic answer below:

<MultipleQuestionItem>
    <SubQuestions>
        <QuestionItem>
            <QuestionText>
                <LiteralText>
                    What is your ancestral origin?
                </LiteralText>
            </QuestionText>
            <CodeDomain>
                <!-- This CodeDomain would include a reference to the list of countries or races -->
            </CodeDomain>
        </QuestionItem>
        <QuestionItem>
            <QuestionText>
                <LiteralText>
                    Please Specify:
                </LiteralText>
            </QuestionText>
            <TextDomain/>
        </QuestionItem>
    <SubQuestions>
</MultipleQuestionItem>

Here we have been able to split the question, while still managing it in a single item. This is needed as without each other, each subquestion is incomplete. This is not a new concept, and is quite an obvious solution to many people who have tried to solve this issue.

However, there is still the problem that this metadata doesn’t contain the restriction that a respondent should only be able to enter a free text option if the “other” option is selected. While there have been a number of published and attempted solutions, none have been satisfactory. Spliting the question outside of a MultipleQuestionItem and using IfThenElse clauses complicates the structure, and leaving this out makes designing self-interviewed computer systems difficult to manage directly from the metadata.

A possible solution, that resolves both of these issues is through the use of the <SubQuestionSequence>. This is illustrated in the DDI Fragment below:

<MultipleQuestionItem>
    <SubQuestions>
        <QuestionItem>
            <!-- Ancestral origin QuestionItem -->
        </QuestionItem>
        <QuestionItem>
            <!-- Please Specify QuestionItem -->
        </QuestionItem>
    <SubQuestions>
    <SubQuestionSequence>
        <ItemSequenceType>Other</ItemSequenceType>
        <AlternateSequenceType formalLanguage="Name Of Language Here" >
            <!-- Proprietary command to control logic -->
        </AlternateSequenceType>
    </SubQuestionSequence>
</MultipleQuestionItem>

In this we have used the SubQuestionSequence to hold the logic used to indicate when the “Other” field should be allowable. This field is used to control the specific sequence that the SubQuestions are shown, and in this sense we are controling this ordering, just to specify when a member is not shown – an excusable use of the field. This choice can be further rationalised, as an unfamiliar agent, for example when moving to a new piece of software, can still interpret the bulk of the metadata, however when presenting the above question would allow a respondent to fill in both sections. But this is no different to how a respondent of a paper-based survey may answer, so it is no great loss of granularity.

How any given agency may choose to populate the commands contained in the AlternateSequenceType will be an individual choice, and a standard way of expressing this may be needed, but this should help other groups more easy solve this problem by indicating where the solution can go and reducing the problem size.

In the next day or two I will be putting a more solid example up into the DDI Examples Repository for people to work with. As always critiques of these ideas and examples are welcome.

Questionnaire design with DDI – Part 1: Will this survey ever end?

This is the first in a 5 part series of working with questionnaires and surveys managed using the Data Documentation Initiative XML standard. With DDI being an emerging technology, it is important that users are provided with best practises to ensure that they use the standard in a way that is logical, coherent and most importantly usable and reusable. This series of tutorials and discussions is aimed toward users who have some knowledge of DDI and would like to know how to effectively design markup existing and future questionnaires in DDI.

Part 1: Will this survey ever end?

This is a classic question that survey takers often ask, and not usually that politely. When a survey will end is an important question for users and designers alike, as the time users have to spend filling out a form can impact their likelihood of finishing or even starting a survey. So determining if a survey will end is of vital importance for users of DDI.
Unfortunately, the answer, based on simple analysis is that you can never tell. DDI uses a unique methodology of using complex referencing between sequencing objects to build up the structure of a questionnaire in a way that is highly reusable. Take for example the following pseudo-survey:

Survey: All about You!
    Part 1: Feelings
        Q1: Are you feeling happy today?
        Q2: How do warm summer days make you feel?
    Part 2: Favourites
        Q3: What is your favourite icde-cream?
        Q4: What is your animal?

Now, restructuring this in a way more analgous with DDI gives:

Sequence id="Part 1" title="Feelings"
    question reference: Q1: Are you feeling happy today?
    question reference: Q2: How do warm summer days make you feel?
Sequence id="Part 2" title="Favourites"
    question reference: Q3: What is your favourite ice-cream?
    question reference: Q4: What is your animal?
Instrument id="survey" title="All about You!"
    sequence reference: Part 1
    sequence reference: Part 2

The important thing to notice is that both Part 1 and 2 and the main survey are all the same structure: a DDI Sequence. In DDI, to increase reusability sequences can be included by reference within each other. However, the issue comes when a designer (either a person or piece of software) fails to account for this sending the survey taker into an endless loop. For example, looking at a different survey strucutre like DDI gives:

Sequence id="Verse 1"
    question reference: Q2n-1: Is this the song that never ends?
    sequence reference: Verse 2
Sequence id="Verse 2"
    question reference: Q2n: Does it go on and on my friends?
    sequence reference: Verse 1
# Apologies to Lamb Chop

An even shorter (but much less likely) example would be:

Sequence id="infinity"
    question reference: Qn: Do you believe how vastly, hugely, mind- bogglingly big Inifinity is?
    sequence reference: infinity
# Apologies to Douglas Adams

What these two examples show is how it is possible to send a user through the same sequence twice, potentially leading to an endless loop of questioning. With DDI providing appropriate mechanisms for Looping over questions, it is quite likely that this kind of structure will always be the result of a mistake. However, what it does highlight is how the flexibility DDI provides to allow users to reuse metadata when used inappropriately can cause issues.
Fortunately, as part of the research in developing a DDI web application for data collection, it was neccessary to create a module that was able determine the implied structure and possible ending of DDI Questionnaire. A web-service for this module (aka. Shari) is available online at http://sandbox.kidstrythisathome.com/dditools/sheri.
One of the notable features of Shari, is that is will create a hierarchical non-DDI based XML serialisation of a given questionnaire (the reasons for this will be covered in Part 2 of this series “Where am I?”). Along with this it will check that no two sequences have references to the same sequence to confirm that the questionnaire will halt. However, the ‘halting’ of the survey is however based on the provision that any DDI Loops within the survey are also able to end, but this is a true ‘halting problem’.
By throwing an error when ever a sequence is referenced twice, some may believe that this will lead to a perceived issue in which two sequences in different branches could link to a third sequence without causing an endless loop being rejected by the system. However, it is possible to rewrite any instrument that relies on using multiple references to a sequence to ensure convergence of a survey in a way that eliminates duplicated references. For example:

Instrument id="Main"
    sequence reference: Seq 1
Sequence id="Seq 1"
    question reference: Q1: Do you like A or B?
    if Q1 = A then goto sequence 2a else goto 3b
Sequence id="Seq 2a"
    question reference: Q2a: Why do you hate B?
    sequence reference: Seq 3
Sequence id="Seq 2b"
    question reference: Q2b: Why do you hate A?
    sequence reference: Seq 3
Sequence id="Seq 3"
    question reference: Q3: Wouldn't it be better if every one got along?

In this minimal example, it is trivial to see that this survey will always end. What is important to note is that this can be rewritten as:

Instrument id="Main"
    sequence reference: Seq 1
    sequence reference: Seq 3
Sequence id="Seq 1"
    question reference: Q1: Do you like A or B?
    if Q1 = A then goto sequence 2a else goto 3b
Sequence id="Seq 2a"
    question reference: Q2a: Why do you hate B?
Sequence id="Seq 2b"
    question reference: Q2b: Why do you hate A?
Sequence id="Seq 3"
    question reference: Q3: Wouldn't it be better if every one got along?

As the users steps through the branch, after either of the sequences 2a or b end, the survey steps ‘back’ from the inner sequence to Seq 1, not finding a following sibling for the If branch it steps back again to the parent instrument and finds Seq 1 has a following sibling and then steps into Seq 3.What should be taken away from this example, is that is is important before finalising a DDI questionnaire to understand the implied structure of the instrument and refactor it to ensure that it is minimal and logically correct as well as having the structure required by the survey designer.

In conclusion, it should be a little clearer about how to tame the flexibility that DDI allows when creating questionnaires and how to create logically correct survey instruments.

Next up… Questionnaire Design with DDI – Part 2: Where am I?- Examining why DDI has issues with non-predictability of movement through an instrument, and how to work around this.