Grants:PEG/WM Wikisym/WikiSym 2011/Report

Report accepted

This report for a Project and Event grant approved in FY 2010-11 has been reviewed and accepted by the Wikimedia Foundation.

• To read the approved grant submission describing the plan for this project, please visit Grants:PEG/WM Wikisym/WikiSym 2011.

• You may still comment on this report on its discussion page, or visit the discussion page to read the discussion about this report.
• You are welcome to Email grants at wikimedia dot org at any time if you have questions or concerns about this report.

Did you comply with the requirements specified by WMF in the grant agreement?

Activities

WikiSym 2011 panel "Apple to Oranges?" featuring (left to right): Jude Yew (moderator), Ed H. Chi, Sharoda Paul, Judd Antin, Aaron Shaw, Andrea Wiggins and Cliff Lampe.

WikiSym is the International Symposium on Wikis and Open Collaboration, the premier conference on open collaboration and related technologies for researchers, industry, entrepreneurs and practitioners worldwide. On its 7th year, WikiSym 2011 took place on October 3-5, 2011 at the Microsoft Research Campus in Silicon Valley (Mountain View, California). This is one of the most important interdisciplinary centers of Microsoft outside Redmond, WA.

Overall, WikiSym 2011 registered 115 attendees from many different countries all over the world, representing academia, research institutes, private companies and non-profit organizations over the 3 conference days. This includes 19 staff members of WMF.

Over the 3 conference days, multiple activities, research sessions, debates, panels and focused open space meetings were conducted. You can consult the full schedule of WikiSym 2011 to learn all details about our program of activities.

Some of the most important activites undertaken within the program were:

• Keynote sessions: WikiSym hosted 3 featured keynote sessions in 2011. The first one was delivered by Cathy Casserly, CEO of Creative Commons. The second keynote was associated to the WikiViz data visualization challenge (see below), and it was addressed by Jeff Heer, Associate Profesor at Stanford University. Finally, the last keynote was delivered by Google's principle research scientist Ed Chi.

• WikiViz data visualization challenge: Since April 2011, WikiSym launched a novel activity to foster collaboration and research in the very important field of data visualization, focusing on open data sources. As the first edition of any challenge, it was vital to choose an attractive data set to draw the attention of practitioners and enthusiasts. Therefore, it was decided that this first edition of WikiViz should focused on Wikipedia, and more precisely, on solving a challenging endeavour: visualizing the impact of Wikipedia outside its own community. A small team of 3 persons (Dario Taraborelli and Erik Zachte from WMF, along with Felipe Ortega representing the WikiSym 2011 Symposium Committee) outlined the master building blocks of this new initiative, the topic of interest, requirements to participate and a prospective list of reviewing panelists to select the winner. We were very fortunate to have on board some well-known experts in this first jury, and featured partners to improve dissemination and outreach. You can check the details on the WikiViz page. Jenn Lowe, from Data Telling was the winner of the contest with its awesome visualization "A thousand fibers connect to us", exploring the relationship of reading activity in Wikipedia versus Internet accessibility for all languages represented in Wikipedia. WikiSym covered the travel and accommodation costs of the winner to present its work at the conference, right before the keynote addressed by Jeff Heer.

• Research sessions: Up to 7 research sessions were hosted in WikiSym 2011, in which 22 papers where presented. Remarkably, 2 whole sessions were run about Wikipedia-centered topics exclusively: "Understanding Wikipedia", on the analysis of topics and issues of general interest to leverage our understanding of Wikipedia and its associated communities of users/editors, and "Wikipedia as a Global Phenomenon", tackling the problem of quantifying and explaining the impact and coverage of global breaking news and current affairs in Wikipedia. Additionaly, 4 demos were presented on a dedicated slot, and 15 posters were presented in a different session. In total, 10 research papers, 7 posters and 1 demo (on collaborative video editing) were focused on Wikipedia as a case of study/application.

• Invited panels: Two invited panels were featured in WikiSym 2011 program. The first one "Learning in a WikiWorld", moderated by Andrea Forte, explored the impact of new media and open collaboration in our understanding of the world and what happens inside it. The second panel "Apples to Oranges?" pursued a multidisciplinary perspective to present different but intertwined studies on peer-production and open collaboration in different contexts and communities, with the aim of finding similarities between different approaches to build an integrated framework for research in open collaboration.

You can also browse our blog entries related to WikiSym 2011, as well as a complete overview of our outreach impact on social networks and different media on our customized eventburn dashboard.

Project goal and measures of success

Project goal

The main goal of WikiSym 2011 was to gather together relevant researchers, practitioners, entrepreneurs and enthusiasts of wikis and open collaboration to share research results, details on work in progress and work on new ideas, projects and initiatives to boost the new generation of open collaboration tools and communities. Under the light of the numerous activities, sessions and initiatives that were carried out, along with the very active participation of attendees, the high number of top-level research contributions received and the satisfactory impact on social networks and media, we can conclude that the project goal was achieved in all major aspects.

Measures of success

In the grant proposal, several metrics for success were proposed in order to evaluate the outcomes of WikiSym 2011. Below, we provide these metrics that let us draw several important conclusions regarding the conference achievements.

• Number of attendees to the main conference: 115 persons
• Number of attendees to the Wikipedia Research Session: There were several sessions, average attendance per session was 20-30 persons, 10 for the demo session.
• Number of different research works on WMF projects presented: 10 research papers, 7 posters and 1 demo. All these works are openly accessible on the WikiSym 2011 website.
• Number of submissions received to the Wikimedia Data Challenge: 5 in total, 3 valid submissions.
• Number of new proposals and intiatives launched in WikiSym 2011: not exhaustive, but at least 5 new projects were arranged/prepared during the conference.

These scores confirm that most of the project goals have been successfully achieved. The only one exception was the outreach level for the WikiViz data visualization initiative. Despite the high quality and appealing of the winner work, the low number of submissions has been lately linked to the difficulties for entering the highly competitive field of data visualization in the current scenario, with several prominent challenges and contests backed up by important media and technology sponsors and offering substantive prizes (sometimes, orders of magnitude higher than the whole budget initially allocated for WikiViz).

In addition to the original measures of success we observe that in the follow-on year, WikiSym 2012, the number of research papers on Wikipedia and Wikimedia projects increased further. Thus, WikiSym is not only a major, probably the main, place for academic research on Wikipedia, but it is also facilitating research growth. In line with 2011, all Wikipedia research of 2012 at WikiSym will be freely available (Open Access) from the WikiSym website at http://www.wikisym.org/ws2012

Lessons learned

What lessons were learned that may help others succeed in similar projects?

In general, WikiSym performed as expected on the research and conference side of the event. On the other side, the novel data visualization challenge (WikiViz) rendered outreach results well below the expected level, specially given the well-known, respected profile of some of our sponsors (like Visualizing.org or Information Aesthetics). Without a doubt, entering data visualization initiatives these days must be accompanied by a substantial budget, specially to backup prizes for winners and finalists, in order to attract the attention of this highly demanded and quite selective audience.

What impact did the project have on WMF mission goals of Increased Reach, Increased Quality, Increased Credibility, Increased and Diversified Participation?

The impact of WikiSym on WMF mission goals was manifold:

• Increased reach: Research results were presented about the evolution of participation of new Wikipedia editors, their trends over time as well as possible causes to continue or cease their participation in Wikipedia. A whole session "Wikipedia as a Global Phenomenon" was devoted to analyze the impact of relevant affairs, breaking news and major society events in Wikipedia content, and how this affects the participation and interests of Wikipedia editors.

• Increased quality and increased credibility: The topic of increasing content quality in Wikipedia was covered in two different research contributions, explaining a novel feature-rich model to detect possible vandal edits in Wikipedia, together with another work characterizing the content deletion process in Wikipedia, and identifying copyright violations as the most problematic of all deleting behaviors highlighted in this study.

• Increased and diversified participation: The central issue of the gender gap in Wikipedia editing was addressed from multiple perspectives in two papers, identifying possible differences in the activity patterns of male and female editors in Wikipedia.

Reporting and documentation of expenditures

Documentation of expenditures has been received by WMF.

Did you send WMF documentation of all expenses paid for with grant funds?

Documentation on all relevant expenditures is currently being compiled by our treasurer and will be provided to WMF as soon as possible.

Details of expenditures:

Item	Planned	Actual
WikiSym passports (free tickets for WMF)	$3000 for 3-day, full conference registration for 10 attendees from WMF	Ultimately, 22 tickets for WMF, of which 3 people didn't show; these add up to $6600 if calculated at the same rate as the planned tickets
Contribution to coffee breaks/lunch	$8000, will display WMF logo sponsoring the corresponding break/lunch on conference program	Total catering costs were $16.201,47 to which we applied the $8000 from this grant; the Wikimedia sponsorship was widely announced, not just with logo and information, but in the main hall and thanked in speeches by the conference chair
Contribution to conference materials	$1,500: Totebag, pen-drives... Include WMF logo (monochrome) and promotional material (as per request, provided by WMF)	Total conference materials costs were $4.330,92 to which we applied the $1500 from this grant; it included WMF information material as requested
Wikimedia Data Challenge	$5,500 prize and/or grants for winners	Total expenses for WikiViz winner was $586,23, leaving a surplus of $4913,73 from the grant
Contribution to grants for on-site volunteers	$2,000, individual grants for students and community volunteers helping with on-site organization	There were no on-site volunteers because Microsoft wanted to handle facilities; however, we are prized like a community event to break even, and overall student participation is subsidized

Summary: We provided additional tickets to WMF employes at a value of $3600. Our Catering costs and conference materials cost was more than twice as high as the WMF contribution to it. We underspent prize money (see competition problems above) and had to go with Microsoft restrictions as to onsite volunteers.

Will you be requesting re-allocation of remaining grant funding?

The total unused funds from the prize and volunteers is $4913,73 + $2000. If you accept that we subtract the additional WMF tickets from that of value $3600 we have unused funds remaining of $3313,73. We would like to request that this money be spent equally on the catering costs and the conference materials. Please note while catering costs may feel expensive they are just average for the location (Silicon Valley) and we considered them fair.

Will you be returning unused funds to the Wikimedia Foundation?

Additional requests and related proposals

Will you be requesting an extension or were you granted an extension?

Please link to related grant proposals here: