Community Engagement Insights/꯲꯰꯱꯸ Report

(Please help translate to your language)

Community Engagement Insights 2018 Report: Support & Safety

• Read summary
• Learn the survey design
• Browse team results

Watch the presentation about this report! The video is in English.

Wikimedians at the Asia meet up at Wikimania 2018

Community Engagement Insights 2018: Fostering learning to improve support to Wikimedia communities

In fiscal year 2017–2018, one of the Wikimedia Foundation's three strategic areas was Communities. The goal of this strategic area was to "increase volunteer retention and engagement through improved programs, experiences and resources." Since the plan was written and approved last fiscal year, the Foundation has engaged in a variety of programs to obtain this goal.Community Health was a major initiative during the last fiscal year and continues to be important. The Foundation also engaged in goals of supporting and providing resources to Wikimedia communities, and, in particular, to emerging Wikimedia communities.

Learning about the outcomes of the Foundation's work can be challenging, especially in a social and technical movement like Wikimedia. Community Engagement Insights is an annual survey for which teams at the Foundation design survey questions based on their goals. This survey helps paint a picture about our communities and provides information about progress on Wikimedia Foundation goals.

On this page, we summarize data from over 4,000 participants who answered 170 survey questions that are focused around our annual goals from 2016–17. These goals related to several themes.

• What is the diversity of Wikimedia communities?
• What is the current health of Wikimedia communities? Related to community health, are Wikimedia communities inclusive?
• What are some outcomes or stories about our programs supporting Wikimedia communities?

To learn more about how we did this survey, read about the survey design.

What progress has been made in the diversity of Wikimedia communities?

Diversity of contributors on the Wikimedia projects seems to remain unchanged.

• Between 2017 and 2018, no statistically significant changes were found in gender for Wikimedia contributors. We detected a decrease in gender diversity in emerging communities, but we are not sure if this is due to a change in the sampling strategy.Last year, we sampled a specific group of contributors who in addition to editing in their home language, also edited in English Wikipedia.This year, we removed this group and this may have contributed to the representation in our sample. Some work is needed to learn what effect this sampling change had, if any.
• The median age of all contributors is between 35 to 44 years and has slightly increased since 2017. While median is the preferred measure to describe age, we noticed an increase in the mean.Age was grouped into seven categories. The mean age across these seven categories increased from 3.97 in 2017 to 4.0 in 2018, which is closest to the 45–54 age category.
• Contributors were overwhelmingly from the Global North (81%). Although contributors from emerging countries decreased, we also changed our sampling strategy. We suspect that changes in regional representation are due to this change, but we cannot be certain until the survey is run again in April 2019.
• The large majority of contributors have post-secondary education. 85% of contributors have post-secondary education. This is the first year we have measured education in CE Insights, so we do not know if this has changed over time.

• 
  Gender for Contributors in 2018
• 
  Education for contributors in 2018

Looking at diversity across community audiences

• Gender: As with 2017 data, program organizers and affiliate organizers reported higher percentage of women compared to Wikimedia contributors and volunteer developers. For program organizers, 36% reported being women, and, for affiliate organizers, 26% reported being women. Among volunteer developers, 12% reported being women. Program organizers in our sample this year seems to have increased 10% from last year, while Affiliates seems to be 2 percent lower.
• ꯆꯍꯤ ꯃꯁꯤꯡ: The median age across all the audiences remains 35–44. Editors at the Asian and Indic Wikipedias, editors at the Middle Eastern and African Wikipedias, and volunteer developers reported lower ages; the median response was 25–34 from these groups. For these three audiences, at least 70% of the respondents were under the age of 35. English Wikipedia and Western European Wikimedians reported higher age. However, 53% of editors at the English Wikipedia and 46% of editors at the Western European Wikipedias reported being under the age of 35.
• ꯃꯍꯩ ꯃꯁꯤꯟ: The median completed level of education for each community audience is the first university degree. Program organizers and Wikimedia affiliates reported completing higher education levels, and over 90% of them had completed the first university degree or higher. Volunteer developers and editors of Middle East/Africa Wikipedias and Asia/India Wikipedias reported having completed less education than others. Each reported less than 70% with a first university degree or higher. These three groups are also the three youngest groups, which likely correlates with their completed level of education.
• Geography: Our sampling strategy is stratified, which means that we specifically target certain projects and languages. Geographic representation, therefore, is heavily influenced by how we sample. As with the previous year, geographic representation among affiliates, program organizers and developers is higher than editors.

• 
  Gender in 2018
• 
  Age in 2018
• 
  Education in 2018
• 
  Geography in 2018

What is the current health of Wikimedia communities? Related to community health, are Wikimedia communities inclusive?

Among measures of collaboration and engagement, community members' self-awareness and awareness of others remains lowest.

• "Awareness of self and others" continues to have a lower average by far. "Awareness of self and others" includes statements about how contributors perceive other's emotional awareness (e.g. "Most Wikimedia contributors are aware of their biases and patterns of behavior."). Respondents either agreed or disagreed with these statements. The average for this question is about 3.05, which is considerably lower than the other measures of collaboration and engagement. Among program organizers, there was a 45% increase from 2.1 to 3.0 on a scale of 5 in awareness of self and others.
• Changes across the measures for collaboration and engagement seem to be minimal from 2017. Changes seem to be slight, between 3 and 5%, across the survey questions these measures of community health.
• For all audiences between 2017 and 2018, perceptions of Wikimedia Foundation leadership decreased the most. The wording for this survey question changed minimally.It had a higher percentage change than other constructs with wording changes. There was a 5.5% decrease in the average response between 2017 and 2018 which is statistically significant. For Wikimedia affiliates between 2017 and 2018, perceptions of the Wikimedia Foundation's leadership decreased 18%. Some Wikimedia affiliate organizers may have attended the Wikimedia Conference, which co-occurred with the survey, and this may have influenced results.
• Among developers, there was 20% average increase from 2017 for all measures of collaboration and engagement. This change needs to be investigated further.
• Dutch Wikipedia contributors surfaced several times in both the "Collaboration & Engagement" and "Diversity & Inclusion" constructs as being lower than several other Wikimedia projects. Dutch Wikipedia was lower in measurements of collaborative intent, awareness of self and others, feedback and recognition, individual commitment to diversity, and inclusive interactions. Each of these differences are statistically significant.

• 

While diversity and inclusion measures are mixed, program organizers and affiliate organizers are showing stronger attitudes towards diversity.

• Individual commitment to diversity and frequency of discrimination have favorable outcomes. Individual commitment to diversity is highest among the measures, while the frequency of discrimination is fairly low.
• There is room for improvement in communities valuing diversity. The average response to "attitudes towards the importance of diversity" is 1.5 out of a scale from 0 to 4 (this question is formatted differently from the others). Compared with editors, program and affiliate organizers perceive that their communities place more value in diversity, about 2.5 out on a 0 to 4 scale.
• Sentiments towards a sense of belonging and having an inclusive culture in the Wikimedia community were two of the lower scores among the diversity and inclusion measures. The scores for these constructs were 3.59 and 3.57, respectively. For measurements of an inclusive culture, the average response for women in the survey was 8% lower than men, which is statistically significant. Similar differences in gender were found among developers and low-activity editors.
• Editors on the Wikimedia projects reported statistically significantly lower frequency of discrimination than developers, affiliates organizers and program organizers. The cause of this difference is unknown, so further investigation would be needed.

• 

Experience of harassment has not declined since 2017 and appears to remain steady

• Across Wikimedia audiences, an average of 22% felt unsafe or uncomfortable in any online or offline space in the last 12 months. In 2017, we asked the same question, but did not set a time limit of 12 months. So while it cannot be stated that harassment has decreased, we can say with some confidence that it has not worsened.
• Among contributors to Wikimedia projects who reported feeling unsafe, 71% reported being bullied or harassed on Wikipedia in the last 12 months. Wikipedia was highest among the Wikimedia projects. The changes in bullying and harassment across the projects from 2017 were small. After Wikipedia, the incidence of harassment seems to decrease sharply. The second project with the highest reported frequency of bullying or harassment is Wikimedia Commons, where 21% reported being harassed followed by Meta-Wiki, where just 15% reported being harassed.
• Although 54% of Wikimedians on the projects agree they are "freely able to express my thoughts without being attacked on Wikipedia", women reported statistically significant lower scores than men. The average response from female participants was 2.66 (between "Disagree" and "Neither Agree nor Disagree") and was significantly lower than males, who reported an average of 3.43 (between "Neither Agree nor Disagree" and "Agree").

What are some outcomes or stories about our programs supporting Wikimedia communities?

Wikimedia Foundation teams included questions related to their programs and goals. Some teams are interested in learning whether communities are aware of the support they provide, while others might be looking to learn the outcomes of their work on the people they support. The following are a few highlights from different teams.

AE = Low-activity editors, VAE = High-activity editors, DEV = Volunteer developers, PL = Program organizers, AFF = Affiliate organizers

Communications Department: Many contributors are not using any recognized channels for learning about the Foundation's work and women seem to use certain channels more than men

Among Wikimedia editors, 44.8% reported not using any channels for learning about features and services from the Wikimedia Foundation. From those who did report using one of the channels listed, Wikimedia project pages were endorsed the most. Wikimedia Affiliates reported a wider variety of channels for learning about updates from the Foundation. Besides Wikimedia projects, they reported using the blog, local conferences, and social media more than the other categories. Women seem to use channels differently than men. 68% of males and 80% of females reported using one channel or more to learn about Foundation updates. A higher proportion of men used Wikimedia projects pages. Women editors reported a higher use of mailing lists, social media, the Wikimedia Foundation blog, and conferences.

Community Programs team: Wikimedia Commons users would like better support of multilingual descriptions of media files and be able to easily discover new or unexpected media files

The most often selected features for Wikimedia Commons, for both high- and low-activity editors, were wanting better support of multilingual descriptions of media files and wanting to easily discover new or unexpected media to illustrate other Wikimedia projects (PR30). Both were voted on about 25% of the time. Other options included better options to edit multimedia files, better options for collaborating with other projects like Wikimedia projects, and more refined ways to embed media from external websites.

Community Resources team: Local and regional events show more learning and building skills as major outcomes than larger global conferences, while Wikimania excels in discovery of the new

Participants shared which outcomes were most important from attending Wikimedia conferences,and certain events seemed to be better suited for certain outcomes:

Participant outcomes for Wikimedia conferences

• Discovery of new projects and ideas is best at Wikimania: While all conferences had a high proportion of participants that reported discovering new projects or ideas as the most important outcome, Wikimania had the highest proportion of them all.
• Getting work done at all types of events: Across all the types of events, it appears that a high proportion of participants reported that they were able to start or improve a project because they attended the conference.
• Resolving conflict at Wikimedia Conference and national or local conferences: The annual Wikimedia Conference and other national or local conferences seemed have more reports of this kind of activity.
• Learning at thematic and national or local conferences: Participants who attended thematic events had a higher proportion who selected learning or improving a new skill as the most important result, which was followed by national or local conferences and regional events.
• Feeling recognized is less frequent at conferences: Across the different outcomes, it appears that feeling recognized or appreciated seemed to be on the lower end.

Learning & Evaluation team: Continuing to see an upward trend in evaluation capacity among program organizers & affiliate organizers

LE07 & LE08: Annual measurement of evaluation capacity

LE08: Measurement of evaluation activities

In general, preparedness for evaluation among organizers has grown since 2017 (LE07). Between 2014 and 2016, the Evaluation Pulse survey only reached program organizers who had been in contact with the Foundation through various programs.With CE Insights, the audience expanded to a broader community of program organizers who may or may not have participated in Wikimedia Foundation's capacity-building programs. Self-reported program management and evaluation capacity increased between 2014 and 2016 (in green in the graph). With the shift in audience, we continue to see an increase between 2017 and 2018 (in blue).

Results show that the team's strategy of capacity building through online resources, tools, and leveraging peer networks is on track. The data show that there is increase of use Program & Events Dashboard, that organizers gain skills through in-person trainings, and that program organizers mostly go to friends or to Meta-Wiki for support.This data is in line with the team's work on the Wikimedia Resource Center, learning workshops at conferences, and curating online resources for learning.

Legal team: Communities seem to be mixed about content policies related to Biographies of Living Persons and Paid Editing

Attitudes towards Biographies of Living Persons

Opinions about Foundation support to policies

From a sample of community members, results show that opinions are mixed as to whether the Foundation "should be doing more to curb paid editing on Wikipedia." From this sample of 184 editors, 52% agreed with the statement, while 32% selected "neither agree nor disagree" and 16% disagreed. No significant differences were detected among editors with different activity levels.

Wikipedia editors who participated in this survey are divided about whether a person's information should be removed from Wikipedia when that information is considered private in the person's country of residence. Wikipedia articles sometimes have information about living persons that is legally allowed on Wikipedia, but that would be considered private information where the article subject lives. Wikipedia editors were asked whether private information should be removed when this situation happens. The results are also mixed. From 516 editors, about a third agree, a third disagree, and a third selected "I don't know."

Partnerships & Global Reach team: Focus on emerging communities shows through awareness of partnerships support.

Awareness of services provided by partnerships team

The majority of affiliate organizers (61%) were unaware of the Partnerships and Global Reach team. 74% affiliate organizers reported being unaware that the Global Reach and Partnerships team could help support the creation of partnerships.

Emerging countries reported being more aware (59%) of the Partnerships and Global Reach team than program organizers from Global North countries (41%). We asked all organizers whether they were aware that the team is available to help with partnerships. Only 26% of all organizers reported being aware of this support resource. More organizers from emerging communities reporting being aware of this support (41%) than organizers from emerging communities (19%).

Technical Collaboration team: Contributors prefer giving feedback about new software features on the wikis, and Tech News is by far the preferred way to get technical news

Tech News is most preferred for receiving updates and news about Foundation software development.

Editors reported highly preferring Tech News for receiving updates and news about WMF software development.For high-activity editors, Wikimedia community pages such as local village pumps and Wikimedia news sites like the Signpost or the Kurier were also highly selected. Wikimedia mailing lists and Wikimedia Foundation products seem to be the third group of preferred sites among high-activity editors. For low-activity editors, social networks, the Wikimedia Foundation blog and community pages were preferred after Tech/News. Differences between 2017 and 2018 CE Insights survey were checked, but no significant differences were found.

Conclusions and next steps

• There is room for improvement in diversity of the movement, as well as in improving specific aspects of both collaboration and engagement and diversity and inclusion, which are essential for achieving knowledge equity, one of the areas of focus in the 2030 strategic direction. The focus of the strategic direction is to be the essential infrastructure for free knowledge in the world. There are two branches to this direction, and one is knowledge equity. This direction demands that we examine how well we are working to "break down the social, political, and technical barriers preventing people from accessing and contributing to free knowledge." Being a diverse, inclusive community that is welcoming to all is essential to reach this goal. Awareness of self and others, a sense of belonging, and attitudes towards valuing diversity could be improved to reach this aim.

* Eleven teams at the Foundation are working to decide how they will use their data. Some teams have already published their next steps on their report pages. Currently, teams continue to explore the results and making decisions on how to use the results. In every report, teams are reporting their most useful findings as well as how they intend to use the results. * Continue to improve the survey experience, in order improve response and completion rates. The response rates this year improved for contributors, but more work needs to be done to improve the experience for users. We aim to shorten the survey as well as to conduct user testing to improve the experience for contributors. This survey will be used to measure aspects of the Community Engagement Department's strategy.