Grants:IEG/Co-Location Impact Study

status: not selected

Individual Engagement Grants	review grant submissions	visit IdeaLab submissions
eligibility and selection criteria

project:

Co-Location Impact Study

project contact:

David Randall
dpr47uw.edu

participants:

grantees: David Randall

advisors: David McDonald

summary:

This project will investigate the effect of participation in offline Wikipedia meetups, and how such meetings affect productivity in Wikipedia.

engagement target:

English Wikipedia

strategic priority:

Increasing Participation, Limit Attrition, Increasing Reach

total amount requested:

10,200 USD

2014 round 1

Project idea

What is the problem you're trying to solve?

Many members of the Wikipedia community participate in offline meetups, however very little is known about what effect such meetups have on the health and productivity of communities like Wikipedia.^[1] Scant prior research exists around this topic, but that which is available has shown conflicting reports of both increased^[2] and decreased^[3] participation following community members' attendance in offline meetups in other, similar, communities, and the question of how such meetups can be used to best leverage increased participation in the community remains unanswered. This is especially relevant to WikiProjects and other WikiProject like groups that exist across the Wikimedia platforms. Understanding how offline interactions improve or hinder the productivity of these teams would enable the Wikimedia Foundation to better allocate resources into these teams and create more tailored events that enable the groups to more effectively realize their potential

What is your solution?

To address the problem of determining how offline meetups affect participation in Wikipedia, data will be gathered concerning Wikipedia members' activities and experiences at in-person meetups, as well as their general Wikipedia use leading up to and following these events. Gathering this data will expose the existing situation regarding how meetups impact the health and productivity of the Wikipedia community. Allowing us to address issues that could be negatively impacting participation and to tease out positive aspects that could be further leveraged to increase productivity among both individual members and Wikiproject groups.

The study will include: The overall goal of this project is to discover what effect in person meetups have on the Wikipedia community, in an effort to provide recommendations for how to improve participation and productivity among current and future participants of these events. This goal will be met through:

A series of semi-structured interviews with Wikipedia members who are attending in-person Wikipedia social meetup events, where the primary purpose is to interact with other Wikipedians in a social context to discuss various topics including Wikipedia and beyond
- Interviews will cover both lone editors and members of Wikiprojects in order to reveal any additional productivity impact group membership may have.
Data collection surrounding interviewees edit histories leading up the and following both the event they are being interviewed at, and any prior events they indicate they have participated in.
Observations of individuals and their interactions at these events
Creation of a quantifiable definition of productivity in Wikipedia with which to measure the impact of meetup events.
Production of a workshop where the outcomes of this research are presented, to enable the community to benefit from the results when organizing future in-person meetup events.

This research will be focusing on the in-person social interaction between editors in Wikimedia projects, rather than the task-based interactions that are typical of edit-a-thons and other such events. By looking at the social interactions this research can identify how the relationships and interactions between individuals effect productivity, to add to the already existing body of research on more task-based interactions.

In gathering this data on community members experiences with in-person meetups and creating a defined measure of productivity in Wikipedia, this research can enable organizers of meetup events, both social and task-based, to better tailor those events to their participants. The aim being to create a totally positive and productive experience for editors, benefiting both their personal future experiences with Wikipedia but also the health and productivity of the small groups they may be a part of and the larger community in general.

Project goals

Identify current effects of in-person meetup events on the health and productivity of the larger Wikipedia community.
Put forward recommendations for future meetup events that will leverage positive effects and mitigate negative effects of current meetups to increase participation by community members.
Generate a quantifiable definition of productivity within Wikipedia.
Creation of a workshop where the outcomes of this research are presented to the community.

Part 2: The Project Plan

Project plan

Methods

This research will use a mixed-methods approach in data collection. Primarily data will be gathered using interviews with Wiki editors who have attended in-person social meetup events. These interviews will be mostly be conducted in-person following the meetup events conclusion, although some may be conducted during the meetup events, or online if that is more convenient for the interviewee. Quantitative data will also be collected on interviewee's editing history before and after these events by collecting their user data during the interviews. The researcher will also be observing the interactions at the meetup events as a participant observer and collecting fields notes.

Following the data collection phase of the study, the qualitative and quantitative data will be analysed using a Grounded Theory approach to form a definition of productivity within Wikipedia. The social interactions will be analysed and contrasted to various existing social theories following this analysis approach in order to identify the benefits and negative consequences of those interactions.

Scope

Activities

Data Collection Phase | June - September 2014
The first step in the project will be to collect data by attending various Wikipedia social meetup events. These events will range in size, given the dynamic attendance at this type of event, however they will mostly be small local meetups of around one to two dozen individuals. In order to get a diverse range of participants locations have been chosen across a number of countries in cities of varying size and cultural diversity, including non-English Wikimedia projects. Wikimania has been included in an effort to engage individuals from non-English Wikimedia platforms who will be attending the event, in order to further increase the diversity of the interviewees. However a direct contrast between interviews conducted at Wikimania and the smaller social events will not be made, due to their differences in nature, although smaller social satellite events that occur around Wikimania may be included.

The Proposed locations are: Seattle, WA; New York, NY; Vancouver, Canada; Toronto, Canada; London, United Kingdom; Birmingham, United Kingdom; Dublin, Ireland; Berlin, Germany; Utrecht, Netherlands.

At each of these events the following data collection procedures will occur:

Conduct semi-structured interviews with individuals who attend those Wikipedia events in a one-on-one environments.
Conduct observations of individuals who attend meetup events

As a follow up:

Collect user edit history data from interviewees

Additionally:

Produce and publish mid-point report (September)

Analysis Phase | October - November 2014
Upon completion of the data collection phase, audio recordings of interviews and field note observations will be transcribed and qualitatively coded using a grounded theory approach in Atlas.Ti. Simultaneously the edit history data will be aggregated and patterns in the data analysed, both data sets will then be contrasted to tease out themes and complete the following tasks:

Create a definition of productivity in Wikipedia using context from the interviews and based in the collected edit history data
Identify trends in positive and negative effects of co-location meetups of WIkipedia members
Put forward suggestions for how to better structure future Wikipedia meetup events to promote productivity and mitigate possible negative effects of co-located meetings.

Results Phase | December 2014-January 2015
After the completion of the data analysis phase the result of the study will be collated, and presented for both the academic and wikimedia communities.

Constant updating of the community on the results of the study will take place throughout the project, once the data has been analysed the project updates will cease and a final document containing results and actionable suggestions for improving meetups will be created and sent out to the community through Village Pump, Wikiproject Council and other appropriate areas.
At the conclusion of the project there will be the creation of a workshop, or workshop series, where the actionable suggestions can be put forward to the community allowing others who plan to organize meet ups to both benefit from this research and to also continue discussion on this topic. [This will be a virtual workshop at first, with the potential for in-person workshops at future meetups]
Publish final report (January 2015)

Post-Project Completion | 2015

Following the workshop creation and the completion of the 6-month project, one or more papers to be published in scholarly journals or academic conferences will be written.

Budget

Total amount requested

$10,200 USD

Budget breakdown

Category	Description	Cost	Currency	Notes
North American Travel	Travel and accommodation to/from in-person Wikipedia meetup events in North America	$500-$1500 per trip	USD	No costs should be associated with Seattle, WA events and Vancouver, Canada events would be a significantly reduced cost due to it's proximity to Seattle.
		$4000 Total
European Travel	Travel and accommodation to/from in-person Wikipedia meetup events in Europe	$6000	USD	Includes registration at Wikimania 2014 & studies of non-English Wikipedia projects (German, Dutch & Gaelic). Contacts in these locations and primary researcher's EU citizenship will reduce overheads in accommodation, travel costs and avoid potential immigration issues.)
Transcription Equipment	Dictation equipment to aid in rapid transcription of interviews in the limited time available	$200	USD

Intended impact

Target audience

The Wikimedia Foundation
Current and future Wikipedia contributors
Regular attendees to Wikipedia meet up events
Wikiproject groups

Community engagement

This project is about increasing productivity for the Wikipedia community at large, to that end prior to the start of research I will engage various Wikiproject groups and individual Wikipedia members through the Wikipedia Village Pump and Wikiproject Council pages. This engagement will continue throughout the length of the project, and I will engage community members at Wikimania to gather input and feedback on the research.

Fit with strategy

Increasing Participation, Limit Attrition, Increasing Reach: the aim of this project is to mitigate the negative elements of in-person meet ups and leverage the positive effects to increase productivity and participation in Wikipedia and other Wikimedia platforms. These aims would also have benefits in reducing attrition of members who may have negative experiences at meet ups or following meet ups, and increasing reach by attracting more casual editors into an increased participatory role from interactions they have had at these events

Sustainability

The project is a first step in creating Wikipedia meetup events that better leverage the opportunities such events present to increase participation and quality on Wikipedia. Following on from is project longitudinal studies of individuals and groups who participate in Wikipedia meet ups over several years could be performed, to better tease out more subtle and nuanced behaviour, interactions and other participatory elements that could effect their productivity in Wikimedia environments.

In addition the research will be submitted to one or more academic journals for publication, which could lead to further research in this area.

Measures of success

Conducting 40-50 interviews with Wikipedia members at in-person events
- 5-10 interviews per event, 10+ for Wikimania
Construction of a definition of productivity in Wikipedia, based in the quantitative data collected.
Creation of a workshop where the outcome of this research is presented to the community, putting forward suggestions for how to create, adjust and present their workshops, meetups and hack-a-thons in ways that will result in increased participation, productivity and user retention.
Positive community feedback from future events created following this research, using suggestions and feedback from the workshop

Post-Project Completion

A published research paper highlighting the positive and negative effects of in person meetings in Wikipedia, with recommendation for future events to help mitigate the negative aspect as and leverage the positive ones.

Participant(s)

David Randall is a 3rd year PhD student in the Information School at the University of Washington, he is advised by David McDonald . David's research interests are on the impact of offline meetups on groups that have formed complete or predominately online, with particular focus on productivity and interpersonal relationships. David has also conducted research in the areas of video surveillance in public libraries, cyberbullying and mentoring relationships within online communities. He is also a member of the UW's Computer Supported Collaboration Lab.

David McDonald is an associate professor in the Information School at the University of Washington. Dr. McDonald has ongoing projects studying mass interaction in Wikipedia and technology and media use in the home. He has published research on ubiquitous sensing for behavior change, collaboration in distributed contributor systems, collaborative authoring, recommendation systems, and public use of large screen displays. His general research interests span Computer-Supported Cooperative Work (CSCW) and Human-Computer Interaction (HCI). His role in the project is as an advisor and mentor to David Randall.

Discussion

Community Notification

Endorsements

Do you think this project should be selected for an Individual Engagement Grant? Please add your name and rationale for endorsing this project in the list below. Other feedback, questions or concerns from community members are also highly valued, but please post them on the talk page of this proposal.

Community member: add your name and rationale here.

This sounds interesting. I've occasionally wondered what the impact of offline meetups is on collaboration within Wikipedia, and while I am not a regular at Wikimeetups, they are definitely golden opportunities to stimulate collaborative work, and Wikimedia has a vested interest in trying to make them as effective as possible. This project looks like it could make some significant contributions to that end. Chubbles (talk) 07:34, 7 April 2014 (UTC)
Sounds interesting to me, too, but I must admit that my confidence in interviews and online questionaires and their interpretation is limited. Nevertheless, this is a valuable research topic that is worth to be pursued. -- Purodha Blissenbach (talk) 09:46, 17 April 2014 (UTC)
Endorse. The plan seems sensible, and this is a research area that's actually really important for how we allocate resources. Meetups and other non-conference offline activity are basically not supported in any systematic way. When they happen, it's because of individual initiative and following the examples of other similar activity. But if such things have a significant long-term impact on on-wiki activity for people who participate, then that would be strong indication that we should be putting more resources into making meetups easier to organize, advertise and run, and to put more effort into thinking up ways to incentive Wikipemedians getting together in the real world.--Ragesoss (talk) 14:20, 21 April 2014 (UTC)

References

[1] ttp://nms.sagepub.com/content/early/2013/09/19/1461444813504275.abstract

[2] ttp://www.tandfonline.com/doi/abs/10.1080/13691180903468954

[3] ttp://dl.acm.org/citation.cfm?id=2038566

[1]

[2]

[3]