Global Reach/Iraq Survey 1 Documentation

Overview

This toolkit is a supplement to our New Readers Iraq Phone Survey 1 results. It provides an overview of the context and methodologies through which the phone survey data were gathered and organized. To enable those interested in further investigation, we also provide a list of recommendations on how to utilize the raw data to optimize meaningful analysis and exploration.

1st Iraq phone survey

There are a total of 17 questions in the survey, addressing the following categories:

Internet use
Mobile phone use (smartphones & basic voice/SMS phones)
Awareness and use of Wikipedia
General demographics

Phone surveys were conducted in February 2017 by Votomobile prior to the Wikipedia Zero Launch. This is the 1st of 2 surveys in Iraq as it acts as a baseline for Wikipedia awareness measurement. The survey collected 2500 total responses, representing populations in 5 geographical regions served by 3 mobile Iraqi operators. 3 language choices (Arabic, English, Kurdish) were provided.

Here are the main questions this survey was designed to answer. However, analyzing the full data set allows you to conduct more in-depth data explorations and gain meaningful insights beyond the points presented here.

What is the actual number of people who use the internet?

(Real-world behavior makes this difficult to measure from industry reports, since people might have access to the internet through school, friends, internet cafés, public Wifi, etc.)

For internet users: What do people mostly use the internet for?
For non-internet users: Why not use the internet?
How many people use smartphones?
Do people with smartphones use the internet from just Wifi? Or just cellular service?
How many people think that they don’t use the internet, but still use Facebook or WhatsApp?
How many people have heard of Wikipedia? What do they use it for? How often?
If they have heard of Wikipedia, but aren’t using it, why not?

Compared to previous phone surveys in other countries, the 2017 Iraq phone survey presented new questions.

What are people’s awareness of other major internet brands in comparison to Wikipedia?
Can people find online content in their preferred language?
How does data cost impact internet use?

Survey Distribution Among Mobile Carriers

Our 2017 Iraq survey is a composite of individual surveys based on the market share distribution statistics of 3 mobile carriers provided by World Cellular Information Services.
The breakdown of the first 1500 responses: Zain (41%), Asiacell (39%), Korek (20%)
1000 additional responses were all from Asiacell users. Since we planned to launch our Wikipedia Zero program and awareness campaigns through Asiacell in the coming months, we wanted to gather more details about Asiacell users for future impact evaluations and analysis.

Where to get the data

This page shows graphs of the responses received for each question in the survey.

Flow diagram of survey questions

The full data set can be found at: Dan Foy (2017). Iraq phone survey 2017. figshare doi:10.6084/m9.figshare.5435110. This is the canonical version which contains a CSV including every answer from each of the 2542 responses.
The full text of the questions can be found here.

Using the data effectively for analysis

Looking at carrier-proportional results

For an overview of Iraq, you should turn on “Country subset” filter to obtain a subset of 1500 responses, with each carrier survey size contribution determined by its market share.

Looking at Asiacell specific results

Focusing solely on Asiacell survey results might provide additional insight as we conduct impact evaluation after the awareness campaign, followed by a 2nd phone survey. To select Asiacell-only results, create an “Asiacell” filter under “Carrier” and obtain 1518 results.

Facebook / WhatsApp questions

The questions asking if the respondents use Facebook or WhatsApp are only asked if they previously said that they do not use the internet. This is by design - we wanted to use this question to gauge how many people did not understand that Facebook was part of the internet. The responses to these two questions were not intended to measure the full use of Facebook or WhatsApp.

Awareness of Google / Facebook / Whatsapp question

In order to minimize the length of time each survey took the participants, we chose to avoid asking all participants about their awareness of all 3 companies. Instead, each participant was randomly asked about only one of the 3 companies. This gave us a complete enough idea of the total awareness, but this questions use in a more complex analysis is more limited than other questions.

Non-linear progression & Margin of Error

It is important to note that this survey is non-linear. Depending on how a question is answered, the flow of the rest of the survey may change. For example, if a respondent says that he or she does not have a smartphone, we skip the smartphone-related questions. You can review the flow diagram to see how the survey progresses. For proper statistical validity, our survey size of is large enough where the questions asked of all respondents have a 95% degree of certainty of being accurate within a 2% margin of error.

Methodologies

Addressing Biases

One issue with phone surveys is the tendency for some respondents to favor the first response to a question. To address this problem, most of the survey questions presented the responses in a random order for each call. This distributes any bias evenly among the responses instead of accumulating it all on one response. Note that questions that have a 'none of these' or 'other' response always kept this option as the last one presented. A couple of survey questions, however, have a strong order dependency of their responses and are confusing if they are presented in a completely random order. For instance, when we ask how often they use Wikipedia, asking in a non-sequential order would not make sense (e.g. an order of “once a week”, “once a month”, “once a day”). For these questions, we would randomly present the question in one of two orders: either from lowest to highest, or highest to lowest.