Wikimedia Meta-Wiki

Grants:Project/WM HU/Editor retention program/Final

< Grants:Project | WM HU/Editor retention program


Report accepted
This report for a Project Grant approved in FY 2018-19 has been reviewed and accepted by the Wikimedia Foundation.
  • To read the approved grant submission describing the plan for this project, please visit Grants:Project/WM HU/Editor retention program.
  • You may still review or add to the discussion about this report on its talk page.
  • You are welcome to email projectgrants(_AT_)wikimedia.org at any time if you have questions or concerns about this report.




Welcome to this project's final report! This report shares the outcomes, impact and learnings from the grantee's project.

Part 1: The Project

edit

Summary

edit

This grant funded the editor retention program in the Hungarian Wikipedia between April 2019 and December 2020. The project helped the Hungarian Wikipedia community in decreasing the negative experiences and strengthening the positive experiences of the contributors; helping, motivating and retaining both new and old editors; improving the community atmosphere and strengthening the community cohesion, the Wikipedian identity, the sense of mission and pride in Wikipedia. Key outcomes of the program:

  • Research and community surveys were made and statistical pages were created.
  • New tools were introduced guiding newcomers in their first days and weeks, and the former mentor program was reactivated.
  • Community service awards and other regularly updated statistics were created for motivating experienced contributors.
  • Regular meetups and a WikiCamp were organized, online web conferencing tools came into service.
  • Platforms for communication between editors inside and outside Wikipedia were introduced, improved or updated.

Project Goals

edit

Through solving the problems that have negative influences on the editor retention in the Hungarian community, the goals of the project are

  • decreasing the negative experiences and
  • strengthening the positive experiences

of editors in order to allow a larger part of the newly registered users to become active members of the volunteer community, and to keep the activity and enthusiasm of the experienced editors.

Additional goals of the project

  • improving the community atmosphere and friendliness and
  • strengthening the community cohesion, the Wikipedia identity, the sense of mission and pride in Wikipedia.

Project Impact

edit

Important: The Wikimedia Foundation is no longer collecting Global Metrics for Project Grants. We are currently updating our pages to remove legacy references, but please ignore any that you encounter until we finish.

Targets

edit
  1. In the first column of the table below, please copy and paste the measures you selected to help you evaluate your project's success (see the Project Impact section of your proposal). Please use one row for each measure. If you set a numeric target for the measure, please include the number.
  2. In the second column, describe your project's actual results. If you set a numeric target for the measure, please report numerically in this column. Otherwise, write a brief sentence summarizing your output or outcome for this measure.
  3. In the third column, you have the option to provide further explanation as needed. You may also add additional explanation below this table.
Planned measure of success
(include numeric target, if applicable) 		Actual result Explanation
Surveys, statistics
  • Regularly updated activity statistics will be available for the different user groups
  • Regularly updated list of the new editors
  • Survey about the experience of the new editors (with at least 10 editors)
  • Survey about the experience of the old editors (with at least 30 editors)
  • Survey about the experience of the inactive editors (with at least 10 editors)
 Surveys, statistics List of newcomers: This is a temporary solution only, planning to make it dynamic/live.
Guidelines
  • Updating and expanding the already existing guidelines with the changes that took place in the previous years
  • Checking and updating the Hungarian translation of the most important interfaces, policies and guidelines outside of Wikipedia (Wikimedia Commons, Wikidata, Meta-Wiki) where the editors are redirected from Wikipedia. Translating these pages where there is no translation yet.
  • Creating interactive tutorials for easier understanding the processes in Wikipedia (at least 1 tutorial as an experiment)
  • Developing a new, alternative learning method with small learning steps, a gradual learning process
 Guidelines
  • Updating and expanding the already existing guidelines: this activity hasn't been done yet.
  • Large number of interfaces and pages have been translated and updated in the course of the project (see in #Methods and activities)
  • Creating interactive tutorials for easier understanding of the processes in Wikipedia (0 tutorial until now)
  • Newcomer tasks (done by the Growth Team)
Welcoming new editors
  • Creating a tool and starting a community discussion about systematic, preferably personal welcoming, guiding and warning solutions for the new editors
  • Discussion with the patrollers and administrators, including case studies and analysis of reaction patterns
  • Developing a proposal for changing the community policies and guidelines and thee way they are applied in order to improve the communication between the contributors and make the community atmosphere more friendly.
  • Reactivate the former mentor program
 Welcoming new editors
Motivate the contributors
  • Developing and introducing a general motivation system based on editor contribution
  • Developing a system which helps in the recognition of editors doing background and maintenance activities
  • Developing a system which helps in the recognition of editors who reach major milestones
 Motivate the contributors Editors doing background and maintenance activities: Planned to be part of the statistical portal. This module is not finished yet, it is scheduled beyond the time frame of the grant.
Events for in-person meetings
  • Organizing regular meetups with presentations and trainings (at least 5 meetups with at least 80 participants in total)
  • Organizing a Wikicamp with educational programs (with at least 20 participants)
 Events for in-person meetings
  • 8 meetups with 155 participants in total, and 8 online meetups with over 160 participants in total (plus 6 participant at a workshop)
  • Wikicamp with 14 participants
 Meetups 1: Because of COVID-19 measures, after February 2020 we moved all our events online.
Meetups 2: The original 12 month time frame of the project was extended to 20 months at the end. Averaged for a 12 month time span, these metrics are 4.8 meetups with 93 participants in total, and 4.8 online meetups with over 96 participants. Average number of participants on the meetups was 20.
Communication, online community building outside of Wikipedia
  • Setting up a support service for beginner Wikipedians on social media
  • Setting up contact channels on platforms outside Wikipedia and building up an online community around them
  • Publishing blog posts to spread news and events about Wikipedia and Wikimedia (at least 10 blog posts)
 Communication, online community building outside of Wikipedia Note 1: Development of the Wikimedia Space prototype was stopped and the project is frozen.
Note 2: Implementation of web conferencing tools was originally not part of the project.
Technological environment, tools, gadgets
  • Overview and research about the technological environment, tools, gadgets and their problems on Hungarian Wikipedia (which tools do we have, which ones do we use, where do we need development, which are the critical bugs)
  • Enable UploadWizard locally
  • Checking and completing translations and developing features necessary for a complete functionality of the VisualEditor (Citoid, TemplateData, i18n etc.)
  • Promoting Wikidata integration and improving the existing Lua modules in Hungarian Wikipedia
  • Introduction of Structured Discussions
 Technological environment, tools, gadgets
  • Local technical wishlist, collecting and evaluating long-standing bug tickets
  • Enabling UploadWizard was not done yet.
  • Support for the functionality of VisualEditor (Citoid, TemplateData, i18n etc.) is not done yet.
  • Wikidata integration and Lua modules: not done yet.
  • Preparation for Structured Discussions was done, then building community support for the Talk page project
 Note 1: Structured Discussions has been frozen, and deployment of the extension is not possible anymore.
Note 2: We focused on somewhat different tasks than what was in the proposal. We focused on technical support for the community service awards, on the statistical portal, on surveys, on evaluating the effect of Flagged revisions configurations, on Wikimedia Space, on Matrix/Element, on web conferencing tools; and on general support of the efforts of the Growth project and the Talk pages project.
We consider the project successful, if at least two goals within each field of activity will be completely realized. Interpreting this measure charitably, 18 of 26 goals have been reached and at least two within each field of activity have been realized, therefore this target can be considered fulfilled. Note: There are several tasks which are not yet completely finished or were originally more ambitious. On the other hand, there are some activities which were realized in the time frame of the project, which were not included in the original proposal.
The project will measure the metrics related to the community health We couldn't measure these metrics with the tool yet. Some measurements are available as part of the Growth project, which show positive effect on the editor retention on the wikis using the Growth features (though, these metrics are not available for individual wikis yet). The project planned to use the community health toolkit. In 2018, the launch of the Metrics kit was expected before end of June 2019, but the project status is on hold since first half of 2019. Since the toolkit is not available yet, we would like to make some of these metrics manually later.
There will be an online survey at the end of the project, and the final report will summarize and evaluate its results. The goal for success is: at least 75% of the participants agree with the statement that the project helps retain editors in the Hungarian Wikipedia. Without the answers I don't know, 96% of the participants agreed that the project helped the retention of the editors on the Hungarian Wikipedia. Survey question: "Overall, do you think the project supported its goal and retained the editors of the Hungarian Wikipedia?" See more in the #Survey(s) section.


Story

edit

Looking back over your whole project, what did you achieve? Tell us the story of your achievements, your results, your outcomes. Focus on inspiring moments, tough challenges, interesting anecdotes or anything that highlights the outcomes of your project. Imagine that you are sharing with a friend about the achievements that matter most to you in your project.

  • This should not be a list of what you did. You will be asked to provide that later in the Methods and Activities section.
  • Consider your original goals as you write your project's story, but don't let them limit you. Your project may have important outcomes you weren't expecting. Please focus on the impact that you believe matters most.

One of our goals was providing inclusive options for off-wiki communication as well; to achieve that, we introduced the friendly space policies and applied them to all of our events. Based on experiences from earlier meetups and the feedback from participants we placed an event ban on one community member. It was a hard decision, because the positive effect was to be achieved through a negative action. Fortunately, it worked: the atmosphere of the meetups improved significantly, and community members who had not visited meetups for a long time showed up in the subsequent meetups again. We had positive experiences with the online meetups as well. We started organizing online meetups only because of the restrictions due to the pandemic, but it also resulted in the participation of a number of community members whom we had never met before or met only many, many years ago. It turned out that most of these community members live far away (in other countries or continents), and this is the only way they can attend community meetups. Beside meetups, we also tried other forms of communication, for example a Matrix channel, and it already has several active members who are not (active) contributors of Wikimedia projects but contribute to other open source or similar projects (e.g. OpenStreetMap) and are interested in Wikipedia. It seems to be a good platform for keeping in touch with other communities.

Survey(s)

edit

If you used surveys to evaluate the success of your project, please provide a link(s) in this section, then briefly summarize your survey results in your own words. Include three interesting outputs or outcomes that the survey revealed.


A survey was performed at the end of the project in order to collect community feedback and provide metrics about the success of the project. The survey was running in the first week of January 2021, and 45 editors filled it out. The survey itself (questions and answer options) is described here, while the detailed results of the survey available here (translation to English is in progress).

  • Majority (over two third) of the responses were received from experienced active editors.
  • More than half of them heard about this project already.
  • About 84% of them found research and studies about the community trends made in the frame of the project useful.
  • Over 80% responded that they use the statistical pages created as part of the project at least occasionally. 44% answered that it motivates the editors' activity and 36% answered that it maybe or partly motivates them.
  • 78% of them found the new Discussion Tools useful.
  • 84.4% of them found the in-person meetups useful for the community, and 11% answered that they are maybe useful. Nobody chose the option No, they are not useful.
  • 44% answered that the in-person meetups are better, 27% answered that both the online and in-person meetups are good (no preference), 4-5% chose the online meetups (mainly because of the geographical distance), and about 10% does not want to participate in meetups.
  • There were questions about the communication tools and platforms outside Wikipedia. Over 80% use the Hungarian Wikipedia Facebook group and over 50% read the Hungarian Wikipedia Magazine (blog). Interestingly, for the question "which platform would you prefer using in the future", only 60% answered Facebook, 43-43% selected the blog and web conferencing tools, 26% selected instant messaging tools (like Matrix), and 23% chose a forum (like Wikimedia Discuss Space).
  • Overall, 73% found this project useful for the Hungarian Wikipedia (97% without the "I don't know" answers), and 60% answered that this project helped the retention of the editors on the Hungarian Wikipedia (96% without the "I don't know" answers).
 
Overall, do you think the project's activities so far have been useful for Hungarian Wikipedia?
 
Overall, do you think the project supported its goal and retained the editors of the Hungarian Wikipedia?

Other

edit

Is there another way you would prefer to communicate the actual results of your project, as you understand them? You can do that here!

Methods and activities

edit

Please provide a list of the main methods and activities through which you completed your project.

Project start

edit

The first month of the project was spent on setting it up and starting it:

  • consultation with a lawyer, accountant;
  • finalizing and signing the agreement with the WMF;
  • composing and publishing a job posting;
  • evaluating the applicants, making a decision;
  • preparing and signing the contract with the project manager;
  • setting up the Project Team;
  • creating the project page on Meta;
  • creating the project page on the Hungarian Wikipedia;
  • making announcements about the project;
  • preparing a task list, defining first tasks and priorities.

Research, surveys and statistics

edit

The first part of the project focused on gaining a better understanding of what the actual editor retention trends are and what do the community members find good or bad, what would they change.

  • The Flagged Revisions extension was enabled on the Hungarian Wikipedia in 2008. The effect of that change had been a perennial discussion topic. After a lot of debates, many theories and hypotheses, in 2018, the community decided to change the FR setting for an experimental time period in such a way that all visitor can see the latest versions of the articles (which in a sense amounts to partially disabling Flagged Revisions). Half a year later, we started discussing whether the experiment was successful or not, what changed and how should we measure it. Part of the community proposed to change back to the pre-experiment setup, others wanted to keep the experimental settings. The consensus was that we wait a bit longer until we see some objective data which can help in the decision. A few editors tried to make some manual measurements (for example they took samples from both periods and evaluated the ratio of vandal edits), but it was not enough for a well-founded decision, and over time later more and more editors became angry about the situation. Therefore we decided to help with a statistical evaluation. The evaluation started in June based on the user activity data available on stats.wikimedia.org/v2, but it had to be restarted in July because of a methodological change in the data set. The evaluation was complimented with an analysis of the vandalism ratio based on ORES data (for which the model had to be retrained first, and a dataset created). The results were presented to the Hungarian community in August (and then in October), and a report in English is available with the main conclusions and learnings. Since the community still could not make a decision, a community discussion was started, a voting proposal and a voting were created. Finally, the community decided to switch back to the original configuration settings and hide unpatrolled contributions from readers despite the significant increase in (mainly anonymous) contributions in the experiment.
  • Activity statistics and toplists of users are very popular among editors; a set of such pages (such as monthly, yearly or all-time activity counters, or a list of editors ordered by their first edit) was generated using software developed by one of our community members, and these pages are updated regularly. Some of these pages are useful for monitoring as well, for example a list of new editors or a list of inactive editors. Community service award is an other activity statistics for which the software support was developed in the frame of the project (see below).
  • Statistical portal is a long-time wish, and the first module of the portal was realized in the frame of the project: this is the funnel or pyramid module, which can support further researches by providing data about the size of different user groups at given dates. Other (more contributor-oriented) modules of the portal are already planned, and they are scheduled for early 2021.
  • We started more consultations where we asked the community members about their experience, for example about the usage of talk pages or about friendly communication. More details about these activities can be found below.
  • We started to collect feedback from our events. The responses help us improve our future events and understand what the participants felt was useful or interesting and what wasn't, why they came and what they would like to see or experience next time. Beside collecting feedback after the meetups, we used the opportunity to make some personal interviews during the meetups with a few volunteering participants. We received responses from active editors, currently inactive contributors, and newcomers (who have never edited before) as well.
  • To understand community needs and problems better, community surveys were prepared separately for active, inactive and new contributors. A bot was developed to send out large number of messages (through the wiki's "e-mail user" functionality if it is available, otherwise via the talk page). After the list of users were generated, the messages with the survey were sent out to a randomly selected subset of the users. 212 responses were received for the survey, and the results were published in Hungarian and in English, and presented on the (online) Wikimedia CEE Meeting 2020.

Guidelines, community health

edit
  • One of the most important factors in retaining active contributors is a friendly community. Therefore, we would like to put a great emphasis on improving community health, and reaching a more welcoming, helpful and friendly community. As a first step, a community consultation took place, and it resulted in a set of behavioral recommendations and best practices in online communication.
  • In-person events have an important role in an online community: they help to bring the members closer, into a more personal relationship with real faces attached. They are also usually attended by a few people with no former wiki involvement. Therefore, it is very important to provide a calm, friendly, welcoming and harassment-free space for everybody. In order to help this goal, the Friendly space policies and the Event Ban policy were translated and presented to the Hungarian community, and then enforced during the in-person events. As a result of feedback from event participants, an event ban was applied on one of the community members.

Welcoming new editors

edit
  • One common piece of community feedback was that it's a bad experience for anonymous editors that they often receive unintended messages (like Do not delete!, Stop doing that!, You are blocked! etc.) after their first edit, because they see the messages sent to an earlier user of the same IP. Even when the messages are not offensive, they see conversations on 'their' talk page from the previous user, which is very confusing. Therefore, all anonymous pages with dynamic IP address were completely emptied using a bot. This is only a temporary solution (fix) until the short term solution or the long term solution will be developed.
  • A common wish was to have a list of newcomers, so we can see which editors need more attention. The list of newcomers has been generated and is regularly updated.
  • We realized that many of the tools and ideas we planned in order to welcome, orient and onboard newcomers better on the Hungarian Wikipedia, are covered by work done by the WMF Growth Team, so we requested to join the Growth prototype. After the team approved, we started a community discussion to reach consensus about this move, translated the project pages, software interface and documentation, and looked for volunteers for the mentor program. The Growth experiment started on the Hungarian Wikipedia at the end of January 2020, the community was informed and the upcoming questions were answered. The project tried to support the continuous development of the Growth tools.

Motivating editors

edit
  • Activity statistics, which provide metrics and overview to the contributors about their activity in comparison with other contributors, can help the motivation of active contributors and wikiaddicts. In order to make use of this, regularly updated activity statistics (such as a list of editors based on their monthly activity, a list of editors based on their yearly activity, a list of editors based on their all-time edit count, and a list of editors based on their first edit) were created and maintained.
  • Community service awards had been discussed for a long time (more than ten years) on the Hungarian Wikipedia, but despite the popularity of and support for community discussions, they had not been realized until 2020. As part of the editor retention project, we wanted to provide software support to make these lists easier to maintain. Therefore a contractor was hired to develop a tool for this task, and the service awards were presented to the community.
  • Participating in regional or international conferences or trainings can energize and inspire participants as they get to meet and talk to many similar enthusiasts, and allow them to learn from each other and exchange ideas and best practices. Therefore community travel scholarships were offered on behalf of Wikimedia Hungary for Wikimania 2019 (two scholarships) and for the Wikimedia CEE Meeting 2019 (two scholarships). Unfortunately, conferences and trainings were not possible in 2020 because of the COVID-19 pandemic.

Events

edit
  • Wikicamp 2019 had been organized in Kecskemét. The camp offered a three-day occasion for contributors to get to know each other. The event provided cultural, natural and educational programs, and time for the community members to spend time together. Beside community building, it was an ideal occasion for deep and thorough discussion of important community issues or project ideas, for developing new proposals, and also for effective educational programs.
  • Organizing regular programs and meetups offers the possibility for meeting in person, discussing questions about online activities, building relations, and for learning from each other and from invited speakers. These events offer a personal learning space for people who are interested in Wikipedia but not familiar with it, or just started to get involved in the community, and also provide meeting space for experienced contributors to strengthen their sense of community. In order to reach these goals, in-person meetups were organized in May, June, July (birthday party of the Hungarian Wikipedia), September, November, December (Christmas party) in 2019, and in January (Wikipedia birthday party) and February in 2020. These events were more organized than the usual spontaneous wikimeetup, with socializing programs, trainings and presentations.
  • After February 2020, in-person events were not possible anymore because of the COVID-19 pandemic, and the organized March and April meetups had to be cancelled. Instead of the in-person events, online meetups were organized in March, April, May, June, July, August, September and October. Digests were created for each online meetup. During this experiment, several web conferencing platforms were tried out (see below).

Communication

edit
  • The online social life of the community moved more and more outside of Wikipedia in the recent years, mainly to Facebook groups and closed mailing lists. The editor retention project tries to serve these needs, therefore we updated our Facebook group and pages and catalyzed some activities there. The Hungarian Wikipedia Facebook page gets daily updates, and we tried to boost its reach by a small paid campaign as well. We started a community discussion in order to see what would be the reception if links to the the Facebook pages and groups would be displayed more prominently on the Hungarian Wikipedia. In parallel, we prepared for introducing Wikimedia Space to the (Hungarian) Wikimedia community during summer and fall 2019 (translations, testing, development, category for the Hungarian community) and introduced to the community at the end of the year, but the experimental project was closed by the WMF at the beginning of 2020. After that, we also tried Matrix out using a private instance of Riot/Element, as instant message tool was the top request by newcomers based on our surveys. We are planning to introduce this as a basic tool for the Hungarian community as soon as the user experience is improved (for example the Wikimedia authentication can be used instead of a separate registration).
  • In connection to preparing and executing our project in the last couple of months, we met many enthusiasts, who work in the same field improving the capability for retaining volunteer contributors. In order to join our efforts, work together and exchange ideas, we started to build up an international group focused on editor retention. As a first step, we organized a meetup at Wikimania 2019. At the meetup we decided that we will set up and use a communication channel on Wikimedia Space for future collaboration where we can easily keep in touch and share our ideas and results, however, after closing Wikimedia Space in early 2020, the collaboration was not as active as we planned.
  • The blog of the Hungarian Wikipedia has not been very active for a while, but in 2019 we posted short stories about WikiCamp 2019, about Wikimania 2019 and about the malicious attack on Wikipedia in September, and then in 2020 about the reply tool of the Talk page project, about the effect of COVID-19 pandemic on the number of page views, about Abstract Wikipedia, about the new Wikipedia tools and surfaces, about the new activity statistics and the statistical portal, about the local and global community wishlists, and about the results of Wiki Loves Earth 2020.

Technical environment, software development

edit
  • The technical tasks related to editor retention on Hungarian Wikipedia and to the grant were collected and organized on Phabricator in order to have an overview and to make managing them easier.
  • One of the most important technical improvements we planned to offer for the community was introducing a more modern, user friendly interface for editing talk pages and community discussion pages. This is mainly important for newcomers, who often have problems with asking or answering questions, or commenting about mistakes. The existing user experience is confusing because of the different appearance and behavior of Visual Editor (which can only be used in articles, not talk pages) and the wikitext editor. And even if they can solve the technical challenges of editing using wiki code, they often face unfriendly reactions because of missing signatures, wrong formatting etc. That said, a modern interface would provide more convenient user experience for experienced contributors, too. Therefore (already before the start of this project), a community consultation was performed about enabling the Flow extension (since then renamed to Structured Discussions) on the Hungarian Wikipedia. Some motivated community members invested a lot of energy into translating all interface messages of the extension to Hungarian, and other blocking technical problems were also solved. Meanwhile, the global talk page consultation started, and we provided a summary of the feedback given by the Hungarian community in the local consultation. During the global talk page consultation, we were told we have to wait for enabling the extension until the end of the consultation. And after the consultation ended, the responsible team at WMF decided to change the conceptual direction of talk page improvements, and declined the request to enable Structured Discussions due to its discontinued status. That was a big disappointment for many involved participants and community members, but we were happy to be one of the first wikis participated in the new Talk pages project (development of the DiscussionTools extension), where we tried to provide useful community feedback on the prototype and beta design of the tools, and communicated the changes to the community. The first version of the reply tool was deployed as beta feature in March 2020, and version 2.0 is active as opt-out feature since September 2020. The community expressed its support of the planned new discussion tool in October, and it was informed about the first prototype at the end of December.
  • We planned to present Wikimedia Space for the Hungarian community as a new communication platform. To prepare for that, we spent a lot of energy in the second half of 2019 trying out the software and its services, and joined the testing, translating and developing efforts in order to fix the most critical problems before it can move from the prototype to the production phase. However, the early participants loved the tool (especially comparing with other used alternatives), and we had large expectations about it, it was closed before reaching the production phase. Unfortunately, this is another experiment in the project without success.
  • Since demand for a user friendly instant messaging platform is often expressed in the community, and the former very popular IRC channel has not been in use for many years, we tried to use Matrix through an open local Element instance installed by the president of Wikimedia Hungary. There is a new Hungarian Wikipedia channel, a (closed) channel for the Board of Wikimedia Hungary, and many more. We have a plan to integrate this platform into Wikipedia in the future.
  • The COVID-19 pandemic made it urgent to provide good web conferencing tools for the community. Therefore Wikimedia Hungary registered for the Google for Nonprofits program to have the option of offering web conferences to up to 250 people on Google Meet. In parallel, we wanted to offer free and open-source alternatives as well, and first we installed and tested Jitsi. Based on our experience, it worked well only for a low number of participants, therefore we also tried out BigBlueButton, which scaled better.
  • One of our community members wrote a tool (as a contractor in the project) to support the long-time planned community service awards. With the help of the tool, the page can provide a full statistical table, a summary table of the award holders and a list of editors with new awards. There is a template which can show the progress of an editor and how far are they from the next award (level), and there is a userbox. Both of these templates update automatically using a Lua module.
  • Our plans involve creating a wide range of activity statistics for the different user groups inside the Hungarian Wikipedia. For this aim, we made plans for a statistical portal and identified needed features. We proposed it as a Google Summer of Code (GSoC) 2019 project, evaluated four applicants for it, but in the end had to withdraw the project due to changes in our of mentoring capacity. At the Wikimania Hackathon two volunteer developers expressed their interest, and wanted to continue the work with them, but this trial didn't work either in the end. After that, we looked for a developer, who agreed to work on these tools in the form of a paid contract. The whole project was divided into separate modules and a development plan was prepared for the next several months. The first, research-oriented funnel or pyramid module was realized until the end of this project, but the following modules are prepared and planned to be developed in the following months, too.
  • ORES is a machine learning service for Wikimedia projects, and it helps (beside other features) with identifying bad faith and damaging edits, which helps the work of the recent changes patrollers. The Hungarian Wikipedia is using this service since its experimental, first phase, but the used model had some bias. Therefore a new set of labels was created, and with that the model was retrained and updated in order to make better predictions.
  • As already mentioned above, we expressed our interest to join the Growth prototype and its tools, which can help newcomers in their first, critical period through an improved software interface. After a longer preparation period, the experiment was enabled on the Hungarian in January 2020 with the following tools: Understanding first day, Welcome survey, Newcomer homepage and Help panel, which were followed by the Newcomer tasks module later in 2020. The Growth Team continuously develops the tools and it seems to be effective for retaining newcomers. It is a great synergy here that one of our project member is member of the Growth Team as well.
  • We made a local wishlist and survey in order to identify the most painful technical problems and needs of the Hungarian Wikipedia community. The results were evaluated by importance and difficulty (and the required skills, resources, etc.), and we will try to provide support to solve as many of these problems as possible in 2021.

Project resources

edit

Please provide links to all public, online documents and other artifacts that you created during the course of this project. Even if you have linked to them elsewhere in this report, this section serves as a centralized archive for everything you created during your project. Examples include: meeting notes, participant lists, photos or graphics uploaded to Wikimedia Commons, template messages sent to participants, wiki pages, social media (Facebook groups, Twitter accounts), datasets, surveys, questionnaires, code repositories... If possible, include a brief summary with each link.

 
Statistical analysis
 
Wikicamp group photo
 
Wikicamp discussion
 
Wikicamp praxinoscope
 
Wikicamp museum
 
Meetup with discussion
 
Meetup with training
 
Cake on the anniversary meetup
 
Wikimania presentation
 
Statistical portal


Project page on the Hungarian Wikipedia


Activity statistics

We created regularly updated activity statistics for the Hungarian editors:

Eventually we intend to incorporate this functionality in the planned statistical portal, but these pages are a useful stopgap solution until then.

Community service award
Statistical portal
Community dynamics and trends
Event surveys

We started making surveys and collect feedback after the organized events (meetups, trainings, Wikicamp) to improve the experience of the participants.

Community surveys

The talk page consultation was integrated into the project. Local page for reactions on the Hungarian Wikipedia.

We prepared community surveys for three important target groups:

We also made some personal interviews about the user experience of Wikipedia during meetups, and we used this experience in the activities of the project.

Further collaborations

We are aiming at building an international network of people and organizations which are active in the field of volunteer retention and research about volunteer communities, including

This is more of an idea than an existing collaboration at this point, but we

We shared our experience on international conferences, like Wikimania 2019 (+ panel discussion), the Wikimedia CEE Meeting 2019 (+ editor retention meetup, editor retention roundtable discussion), the VSN Meeting 2019, and the Wikimedia CEE Meeting 2020.

Community health, guidelines
Welcoming new editors
Motivating editors
Events
Communication
Technical environment, development

Learning

edit

The best thing about trying something new is that you learn from it. We want to follow in your footsteps and learn along with you, and we want to know that you took enough risks in your project to have learned something really interesting! Think about what recommendations you have for others who may follow in your footsteps, and use the below sections to describe what worked and what didn’t.

What worked well

edit

What did you try that was successful and you'd recommend others do? To help spread successful strategies so that they can be of use to others in the movement, rather than writing lots of text here, we'd like you to share your finding in the form of a link to a learning pattern.

What didn’t work

edit

What did you try that you learned didn't work? What would you think about doing differently in the future? Please list these as short bullet points.

  • Work on the project started more than a month later than scheduled because of administrative holdups, see
  • We had to abandon some plans and wishes due to unmet external dependencies, for example we could not implement the Flow extension (discontinued by the WMF) or the Wikimedia Discuss Space (the project is frozen and the development has stopped).
  • Several in-person events had to be cancelled because of the pandemic.
  • Teaching and training community members about the importance of friendly and civilized behavior, and achieving real change and results is a hard and very slow process.

Other recommendations

edit

If you have additional recommendations or reflections that don’t fit into the above sections, please list them here.

Next steps and opportunities

edit

Are there opportunities for future growth of this project, or new areas you have uncovered in the course of this grant that could be fruitful for more exploration (either by yourself, or others)? What ideas or suggestions do you have for future projects based on the work you’ve completed? Please list these as short bullet points.


The community health and long-term growth of the Hungarian Wikimedia community are the most important priorities for Wikimedia Hungary. The Board of the chapter decided that a large part of the chapter activities and resources would serve these goals, not only in the last two years, but in the following years as well. We believe that we are going in the right direction, and we would like to continue our journey on this way. In this pilot program, we completed several tasks and reached some of our goals, but many others are still incomplete. As next steps, we would like to realize further activities, among others for example:

  • Continued community support for the ongoing development of the Growth and Talk pages projects (in which the Hungarian Wikipedia is an experimental partner wiki).
  • Continued efforts towards a welcoming atmosphere and friendlier communication.
  • Better guidelines, tutorials, help pages, short videos for different platforms and tasks (desktop, mobile web, mobile app; Visual Editor, wikicode editor; wiki tools; complex task).
  • Continued efforts for regular meetups and trainings.
  • Based on the results of the surveys and consultations performed in this project, we plan different activities; for example based on the community technical wishlist we plan to provide technical (developer) help to the community with solutions for the most critical local technical problems.
  • Further development of the statistical portal with the planned modules.

As it was planned, the successful realization of the goals of this program prepares and helps the success of the second phase of the planned multi-year project, which in addition to editor retention, will target recruiting new editors as well: involve readers of Wikipedia and people who do not yet use Wikipedia in contributing to it, and transform them into useful and active members of the community.

Part 2: The Grant

edit

Finances

edit

Actual spending

edit

Please copy and paste the completed table from your project finances page. Check that you’ve listed the actual expenditures compared with what was originally planned. If there are differences between the planned and actual use of funds, please use the column provided to explain them.

Expense Approved amount Actual funds spent Difference
Community and project manager 2,804,298 HUF 1,402,149 HUF 1,402,149 HUF
Occasional contractors (developer, translator, tutorials) 618,182 HUF 223,645 HUF 394,537 HUF
State taxes 2,309,752 HUF 1,078,805 HUF 1,230,947 HUF
Event space 730,578 HUF 489,748 HUF 240,830 HUF
Wikicamp 337,190 HUF 211,409 HUF 125,781 HUF
Total 6,800,000 HUF 3,405,756 HUF 3,394,244 HUF


Remaining funds

edit

Do you have any unspent funds from the grant?

Remaining funds are retained by the grantee with WMF's permission until after the report review has been completed and report has been accepted, at which time the WMF program officer will also provide instructions regarding the grantee's reallocation or return of the unspent grant funds.

Please answer yes or no. If yes, list the amount you did not use and explain why.

  • Yes. The project used almost exactly half of the funds, therefore the other half of it remained unspent:
    • Less money was spent on Wikicamp 2019 compared to plan, because participation was lower than expected, and we could not realize Wikicamp 2020 because of the COVID-19 pandemic (37.3% remained unspent).
    • Less money was spent on (in-person) events (33.0% remained unspent), because from March 2020 onward we could not organize any in-person events due to the COVID-19 pandemic, and the substitute online meetups had no practical costs.
    • We spent less money on personal costs and taxes from the project budget (52.8% remained unspent), because of budget reallocation at Wikimedia Hungary. About 90% of the chapter budget comes from a kind of tax credit where Hungarian tax payers can donate one percent of their income taxes to a selected charity. These funds can only be spent on certain programmatic expenses and must be spent within two years. Because of the COVID-19 pandemic, the chapter could not realize several programs and activities that were planned for 2020, therefore an amount equivalent to about 10,000 USD (3 million HUF) was at risk of being reclaimed by the state. The Board of Wikimedia Hungary decided to cover part of the costs of this project from the state funds, as having to return funds to the WMF is a preferable outcome to having to return them to the state.

If you have unspent funds, they must be returned to WMF. Please see the instructions for returning unspent funds and indicate here if this is still in progress, or if this is already completed:

  • No, the unspent funds have not been returned to WMF yet:
    • As improving editor retention is a long-term task (it is the priority also in the next year activity plan of the chapter) and several goals of this project have not been completed yet (see #Next steps and opportunities), Wikimedia Hungary would request to reallocate the unspent funds from this project to a follow-up project with the same scope, to be realized in the following year. To this end, we would submit a project plan to WMF. As this project partly belongs to Community Organizing and partly to Research or Software Development, we could submit two separate proposals to the 2021/2 Round of Project Grants calls, or preferably one proposal as a continuous follow-up project of this proposal.

Documentation

edit

Did you send documentation of all expenses paid with grant funds to grantsadmin wikimedia.org, according to the guidelines here?

Please answer yes or no. If no, include an explanation.

  • Yes

Confirmation of project status

edit

Did you comply with the requirements specified by WMF in the grant agreement?

Please answer yes or no.

  • Yes

Is your project completed?

Please answer yes or no.

  • Yes

Grantee reflection

edit
Retrieved from "https://meta.wikimedia.org/w/index.php?title=Grants:Project/WM_HU/Editor_retention_program/Final&oldid=21937616"