Learning and Evaluation/Events/2014/Wikimedia Conference 2014 Open Thursday Sessions

Writing Good Questions: A workshop on survey design and question construction (1–3 p.m.)

Are your surveys giving you the information you need to make decisions? This session will consist of a 75-minute presentation overviewing survey design strategies, question and response types and formats, as well as exampling common question writing challenges and corresponding review tips and corrective strategies. The presentation will be followed by a 45-minute hands-on activity in which participants will work together to identify, flag, and make recommendations for a set of survey question examples in need of improvement.

Goals of this session include that participants will:

learn core principles of survey design and question construction
learn to identify and correct common errors in question design
practice these skills through applied learning
be aware of available tools and evaluation supports for program leaders engaging in survey design and collection.

View the presentation or expanded learning module

Participants

JAnstee (WMF) (talk)
Misosoof (talk)
Daria Cybulska (WMUK) (talk) - Over February - June 2014 I will be in process of running a survey around the Wikimedian in Residence programme which was constructed with the help from PE&D team. Happy to share my experiences.
--CDG (talk) 08:21, 1 April 2014 (UTC)[reply]
Awang (WMF) (talk)
Adeliine (talk) 18:41, 8 April 2014 (UTC)[reply]
--Raimund Liebert (WMAT) (talk) 11:40, 9 April 2014 (UTC)[reply]
--Kulac (talk) 13:31, 9 April 2014 (UTC)[reply]
--Liang (talk) 07:28, 10 April 2014 (UTC)[reply]
--Reke (talk) 07:32, 10 April 2014 (UTC) - (WMTW)[reply]
HempalShrestha (talk) 15:18, 10 April 2014 (UTC)[reply]

Measuring Program Impact on the Projects: A strategy review and brainstorming session (3:15–5:15 p.m.)

This session will consist of a 15-minute review of the pilot metrics reported in the Evaluation Reports (beta) followed by a set of 45-minute breakout sessions for participants to engage in critical review of the pilot metrics for measuring core outcomes of this initial set of programs and brainstorming for the development of additional and/or improved measures.

The group will divide into two brainstorming groups; one for leaders of programs centered on mass photo/media uploads, and the other for leaders of programs centered on mass text production. Session participants will be tasked with reviewing the pilot metrics in terms of a handful of reflection points, such as:

What do the metrics tell us about a program's impact?
How meaningful is the measure in terms of (a) deciding how much time and resources to invest in a program, (b) setting expectations for a program's outcomes, and (c) measuring the success of a program?
What do the metrics NOT tell us about a program's impact?
What would better enable program leaders to tell the story of their program(s)?
How might we measure those other important program outcomes?

Following the breakout sessions, participants will regroup for 45-minute debrief session. In the debrief session, brainstorming session participants will share summary points from their breakout groups and work to prioritize potential next steps for improving and advancing evaluation metrics.

Goals of this session include that participants will contribute to the ongoing development of evaluation metrics by providing their careful review and constructive feedback to:

identify the strengths and challenges with the pilot metrics
identify critical gaps in measurement capability
strategize possible solutions to challenges and gaps in measurement
be aware of current and future metrics, their design, and relative importance
input into future discussions and decisions around metrics standardization

View the notes

Participants

Daria Cybulska (WMUK) (talk) - following from the recent exercise we have gone through at WMUK, Simon and I could contribute on the issues around identifying and implementing measures for evaluation