Learning and Evaluation/Evaluation reports/Wikimania 2018 Evaluation

Part of the Wikimedia Resource Center

Learning and Evaluation

• Start
• Plan
• Report
• Contribute
  • Community
  • Case studies
  • Evaluation reports
  • Learning patterns
• Past News and Events

Wikimania 2018 in Cape Town -- Evaluation Results

Wikimania 2018 group photo at Museo Soumaya

This is the report page for the 2018 Wikimania evaluation survey from the event in Cape Town. The evaluation was carried out by the WMF Learning and Evaluation team in consultation from the WMF Wikimania event planners and leaders of the Wikimedia South Africa chapter. The purpose of this evaluation study is to better understand the immediate outcomes of Wikimania and the impact on our movement. We hope that the lessons learned this year help inform the creation of future issues of Wikimania. Please share, comment and discuss!

Evaluation Questions

The 2018 Wikimania evaluation was designed to respond to the following questions:

• Who participates in Wikimania?
• What do attendees think and feel about Wikimania?
• What do people learn at Wikimania?
• Do attendees apply what they learned at Wikimania?
• What is the impact of Wikimania on the movement?
• What should be the long-term strategy for Wikimania?

Methodology

• Post-conference Feedback Form - sent 3 weeks after the conference via email link to all registrants who checked in at the venue (Preview of questions)
• Follow-up Feedback Form - sent 4 months after Wikimania to everyone who replied to the initial survey, minus 3 people who opted out (Preview of questions)
• Survey Respondents
  • The initial feedback survey was sent via email to 636 individuals and 356 completed it for a response rate of 56%.
  • The follow-up survey was sent to 353 individuals and 201 completed it for a response rate of 57% (and 32% of all registered participants).
• Analysis
  • Survey results were analyzed using Excel and SPSS. This report includes descriptive analyses, and comparisons by gender, geography, language, scholarship receipt, and length of time attending Wikimania. Qualitative results were analyzed using thematic coding in NVivo and Excel.

Who are Wikimania participants?   

(Data below represent only those who responded to the post-conference feedback form)

• Respondents to the survey represent 71 countries across the world. The United States had the most respondents with 61, followed by South Africa with 56, then Germany with 38, and India with 24. Many countries in Africa and Asia are not represented in these evaluation results.
  

This graph portrays the number of attendees from each country for individuals who responded to the 2018 Wikimania post-conference feedback form.

• English was the most common native language of respondents (26%), followed by German (12%), Afrikaans (4%), Spanish (4%), and multiple native languages (3%).
• Men made up the majority of respondents at 63%, followed by women at 34%, and gender non-binary at 2%.
• 41% (n=356) of respondents indicated it was their first Wikimania.
  

This graph portrays the number of Wikimania conferences attended by respondents to the 2018 Wikimania post-conference feedback form.

What do people think and feel about Wikimania?

Over 90% of respondents were satisfied with various elements of the conference venue (n=355). Overall, 93% of respondents were satisfied with the Wikimania venue in Cape Town, and only 3% were not satisfied. Additionally, 90% felt safe at the venue, 95% felt the venue was language accessible, and 97% felt the venue was physically accessible. (Means are out of 5)

Venue Ratings, Wikimania 2018

While 90% of respondents felt safe at the venue, about a quarter of respondents commented that they did not feel safe in the surrounding neighborhood. A handful of people (n=8) mentioned that they do not prefer conferences at hotels and would like to see Wikimania at a community-based venue. Yet another group of people (n=5) commented that they would like a venue with more windows and natural sunlight.

The vast majority of respondents were satisfied with the conference program and the sessions, however, about a quarter were ambivalent about the plenaries and the conference theme (n=321 to 329). (Means are out of 5)

Program Ratings, Wikimania 2018

Most people who were ambivalent about the conference theme felt the content of the sessions was unbalanced. They hoped for more sessions on technical topics or sessions with concrete takeaways. In contrast, about 15% of respondents stated that the conference theme helped to focus the conference sessions. These respondents felt the program was coherent and they appreciated the sessions on knowledge gaps. According to the comments, respondents who were not satisfied with the plenaries felt that some were not engaging enough.

What did people learn at Wikimania?

Respondents are more likely to agree that they gained knowledge versus skills, and 91% intend to apply what they learned (n=327). This mirrors the findings from the 2018 CE Insights survey, in which people named the biggest benefit of global events being the discovery of new information. (Means are out of 5)

Outcomes Rating, Wikimania 2018

The most common next steps named by respondents include 1) Wikidata projects, 2) filling or supporting knowledge gaps, 3) starting a new partnership or collaboration, 4) organizing an event, and 5) sharing the information they learned with others.

Are people applying what they learned at Wikimania?

Four months after Wikimania, 84% of respondents (n=111) acted on their stated intention from the original feedback survey. Of those 84%, the most common next step implemented was either a Wikidata project or a project to fill a knowledge or language gap on a Wikipedia.  The main reason people were not able to act on an intention was a lack of time and competing commitments. Among all respondents to the follow-up survey (N=202), 111 also described how they applied what they learned to their Wikimedia activities. Below are some of the comments that illustrate the most common themes of filling knowledge gaps, collaborating with others, organizing local events, and sharing information:

“I decided to start learn more languages in order to translate from different languages and to speak with Wikipedians from other countries on their mother language.”

“Wikimania helped me connect with people from around the World. This helped me work with people across borders.”

“Next week, we will be meeting to devise an agreement for our user group's first GLAM partnership, which was completely inspired by my attendance…at Wikimania and Wikimedia Conference. I will also be organizing a Wikidata workshop…based on what I have learned at the conference.”

“Attending Wikimania exposed me to activities led across the world and look at my own community with a lens of that diversity. My own limited perspective got expanded and it helped me identify and even educate others about issues.”

“I have led an edit a thon which involves women with the African Womens Development Fund on their contribution towards decolonizing the internet through feminism. I have also gained in depth knowledge on wikidata and will have a WikiData workshop in the coming year.”

“I have organised local meet ups and told about different tools and influenced communities regarding contributing to Wikidata. I am currently working towards bridging the gender gap and planning to have a workshop in women’s colleges. I with another developer planning to make a new home page for mobile view of our local wikipedia.”

“I have shared the knowledge with some students from my communities and they are learning to translate articles in 11 South African Languages.”

What is the impact of Wikimania?

1)   Learning

According to the post-conference feedback form, the most common initial or immediate outcome after Wikimania included increases in knowledge about the Wikimedia movement and it projects. Additionally, 85% (n=169) of respondents to the follow-up survey said that their current Wikimedia activities are influenced by what they learned at the conference.

2)  Relationships

84% (n=166) of respondents to the follow-up survey said that their current Wikimedia activities are influenced by the people they met at the conference. In the open-ended responses, many named that they planned to continue collaborating with individuals and groups they met at the conference.

3)  Community organizing

Wikimania appears to be a catalyst for Wikimedians and those interested in the Wikimedia movement to heighten their involvement. Not only are individuals learning and building relationships, but they are identifying ways to expand and strengthen the Wikimedia movement and 84% are acting on their planned next steps. This may be especially true of scholarship recipients who were more likely to gain knowledge and skills immediately following the conference and almost all indicated an intention to apply what they learned. Furthermore, 74% (n=144) of follow-up survey respondents agreed that their Wikimedia-related activities had grown since attending Wikimania.

What should be the long-term strategy for Wikimania?

Continue to apply a theme to the conference. Of the 74% of respondents stating the theme added value to the conference program, nearly all of them felt it brought a necessary level of focus to the presented content.

Design the conference to highlight emerging trends in the movement. The most valuable information that respondents gained from the conference in 2018 included information about Wikidata and knowledge gaps in the movement. The conference can continue to benefit both new and veteran Wikimedians alike if the focus of the sessions and plenaries is on emerging trends and hot topics.

Focus future goals of the conference on gaining new knowledge and building relationships. Given that a wide variety of individuals attend Wikimania, it will be important to provide learning and networking opportunities for everyone from the average editor to the leaders and veterans of the movement. However, as mentioned in the section above, much of the evidence suggests that Wikimania may be a catalyst for average editors to become more involved in the movement. The most common actions taken after Wikimania included leadership actions such as organizing events, forming new partnerships, and sharing information with others. Future events should capitalize on this potential opportunity.

Consider the potential role and unique experience of Wikimania scholarship recipients. Results show that scholarship recipients are more likely to gain knowledge and skills than WMF staff and non-scholarship recipients. Future conference committees should consider the role that scholarship recipients play in designing the content, program, and goals of future events.