Grants talk:Project/WM HU/Editor retention program

Project Grant proposal submissions due 30 November!

Thanks for drafting your Project Grant proposal. As a reminder, proposals are due on November 30th by the end of the day in your local time. In order for this submission to be reviewed for eligibility, it must be formally proposed. When you have completed filling out the infobox and have fully responded to the questions on your draft, please change status=draft to status=proposed to formally submit your grant proposal. This can be found in the Probox template found on your grant proposal page. Importantly, proposals that are submitted after the deadline will not be eligible for review during this round. If you're having any difficulty or encounter any unexpected issues when changing the proposal status, please feel free to e-mail me at cschillingwikimedia.org or contact me on my talk page. Thanks, I JethroBT (WMF) (talk) 23:20, 27 November 2018 (UTC)

Eligibility confirmed, round 2 2018

This Project Grants proposal is under review!

We've confirmed your proposal is eligible for round 2 2018 review. Please feel free to ask questions and make changes to this proposal as discussions continue during the community comments period, through January 2, 2019.

The Project Grant committee's formal review for round 2 2018 will occur January 3-January 28, 2019. Grantees will be announced March 1, 2018. See the schedule for more details.

Questions? Contact us.

--I JethroBT (WMF) (talk) 03:13, 8 December 2018 (UTC)

Questions from Superzerocool

Hi Samat, Bdamokos, Nyiffi and Hirannor, thanks for your efforts to make more easily to retain new users. Before I do my questions I have a few background from Hungarian Wikipedia, so, please be kind if I don't see all the image in my questions.

1. How do you will include to users outside of WMHU in your plan?.
2. In the statistics you said: "different user groups", Do you mean Wikimedia User Groups or user groups like newbies, veteran, active users, etc.?
3. About the surveys: How do you run it?, Do you have identified the key aspects to ask to users?, How do you will take the feedback from the survey?
4. About the motivation to users:
   1. Is there any difference between the automated greetings in the 1st, 10th, 100th edits in any Wikipedia?
   2. The planned tools, it will be an external tool (in ToolForge) or a Mediawiki extension?.
5. About Publishing blog posts to spread news and events about Wikipedia and Wikimedia (at least 10 blog posts), Which is the blog or publishing media selected to do it: WMHU page, Facebook note, WMF blog?
6. About the metrics, I feel some disconnected from the efforts to establish the statistics and make some improvements in huwiki, and the goals (improve the retention). I know that the retention doesn't depend on you, but it has a lot of planned activities and no-measurable metric from it. I see your note, but you can set up a metric about new users mentoring retained over 30 days (just thinking).

Kindly (and Merry Xmas!) Superzerocool (talk) 18:59, 24 December 2018 (UTC)

Dear @Superzerocool: thank you for your questions.

(Sorry for my late reply: this is the only week of the year when I've visited my relatives and I haven't been available here.)

1. We will organize and perform most of these activities online on the public surface of Wikipedia, and we will involve everybody who is interested or intend to participate. The project aims not members of WMHU, but the wider Wikimedia community.
2. "User groups" in this project refers to editors/contributors of a Wikimedia project in different roles (such as recent change or image patrollers, advanced rights holders, OTRS agents, mentors, bot operators, tools, templates and modules developers, contributors of community processes, like featured articles or deletion requests etc.), but does not refer to Wikimedia User Groups (Wikimedia affiliates). The second option in your question.
3. We plan to perform surveys using one of the existing platforms (Google Forms, Qualtrics or similar). For each survey we have a specific topic we would like to measure and clarify, and we will try to involve the community already in the survey design process. Results will be published similar to the earlier surveys, for example this one.
4. Motivating contributors:
   1. I personally don't feel automated greetings effective. Personal greetings or only a few words from an other editor is much more effective. I don't think that there is a big difference for example between greetings after the 1st, 10th or 100th edits, but there is a difference if there is a feedback or not. One of the main motivation of volunteers, that they feel their contribution valuable and recognized.
   2. The planned system is rather set of statistics and updated list of users, which helps editors to follow (their own and others') activities. We will mainly use and integrate existing tools. I can imagine smaller software development in the frame of the project, but it probably won't be a MediaWiki extension. The newly developed software parts will be open source and available for anybody else.
5. The blog of the Hungarian Wikipedia is the primary platform for longer posts now. (I can imagine that we will migrate the blog to our own platform later, as we had plans for it already.) If there is a new post on the blog, we share its link in the Hungarian Facebook group. If there is a post about the Hungarian community which can have wider, international interest, we can offer a post to WMF blog as well.
6. We are open for any ideas and recommendations what is the best way to measure the impact of the different project activities. If they do not need high resources and we can measure these metrics, we will measure them.

Let me know if I misunderstood any of your questions above or if you have further questions about this project. I wish you a Happy New Year! :) Samat (talk) 10:18, 31 December 2018 (UTC)

Thanks Samat for your answers, thanks to the team for the efforts and planning this project, it could be a great opportunity to retain new users on Hungarian Wikipedia, and your learnings could inspire others to try it. About the metrics, it could be an A/B test about how many users are retained before and after the new tools are developed or how many users will you expected to be retained in the first 30 days after registration (ie: a 10 % of newbies will have 10 editions in the first month) Superzerocool (talk) 20:05, 31 December 2018 (UTC)

Superzerocool (please note this is a non-authoritative answer) a proper A/B test might not be really realistic without having your own developer team, and also due to the small sample size (the current baseline is ~700 new editors a year who make it to 10+ edits; we don't have stats for better activity metrics, like editors who remain active over multiple months, but that's probably a magnitude smaller); before/after comparisons are hard due to seasonal variations. Also they limit you to testing one thing at a time which is not always practical. If something is super effective, we'll see it in the numbers (fingers crossed!), otherwise I think we might have to fall back to qualitative metrics (asking new editors which things they thought were helpful; surveying new users about what they saw as problems and seeing if the survey numbers change). --Tgr (talk) 05:34, 4 January 2019 (UTC)

Tgr, thanks for the feedback: I know the impractical things about A/B tests in many ways and how isn't applicable in many contexts. In this project could be interesting measure if there is a improvement in the metrics in newcomers. Anyway, surveys are a good way to ask to participant some variables non measurables (like as feelings, experiences, etc.). About the metrics, as I said in the questions message, I feel a bit disconnected from the possible outcomes of the project, but it's true that isn't a easy way to think about the retention without a clear scope, tools and aspect to be measured (ie, we want around 100 new actives users in next two years). Thanks Tgr for your comment, and as I see, the question was intended to explore a possible metrics related with the efforts. 12:11, 4 January 2019 (UTC)

Dear Superzerocool, a late comment from my side: we will measure and evaluate the impact of the project many ways, and we will try to measure related statistics where it is possible and reasonable. Thank you for your suggestions and comments! Samat (talk) 19:33, 17 January 2019 (UTC)

Question

Red this grant proposal I have only one question, why is better make a survey for 26000 $ with wholly inexperienced volunteers? When you wanted make such a survey in one of declining Wikipedia it should be more effective make an order for a survey by one of pollster firm or by one of university sociology department. They have the necessary know-how. (instruments, knowledge, software etc.) Incidentally I do not think, that after this project will grow the number of editors. To grove the number of new editors in this small declining Wikipedias as Hungarian should be done radical organizational changes. 195.228.73.42 10:16, 29 December 2018 (UTC)

Hi! Thank you for your question and comment. Surveys are only small part of this project to understand our users' motivation and expectation, and to measure the success of the project. For these small surveys (where we already know who we would like to ask and what are the main questions), I don't think it is reasonable to involve an external, professional company. This does not mean that we are not open for any recommendation or to learn from others' experience. The main part of the project will focus on organizational (and other) changes inside the Hungarian Wikipedia community to make the environment of editors as friendly, easy to learn and use as possible, similar how you recommended. Samat (talk) 10:36, 31 December 2018 (UTC)

A question about the trend

I have one over-arching question about your proposal, Samat, Bdamokos, Nyiffi, and Hirannor. You were clear about the trend in decreasing editors, "Compared to the peak values in 2008-2009, both the number of editors and number of edits halved until 2015, and the number of new registrations dropped to one fifth." In general, your proposed activities involve helping new editors feel more welcome and readily-equipped to engage in their work, though it is unclear to me that these are factors directly involved in the decline of editors. Do you have data to support this, or will you engage in this data-gathering to confirm these are contributing factors? If so, how will you reach former editors to confirm these claims? Thanks in advance for your reply. --- FULBERT (talk) 19:16, 5 January 2019 (UTC)

@FULBERT: thank you for your question. In the previous years we had a larger number of personal talks with active editors, with former (currently inactive) editors, with people who failed to join and with people who never tried to join. Based on these discussions, we already have an impression what could be the (main) factors and weak points we could improve to be more successful in retaining new and the old editors. If we retained more editors and increased their activity, it would directly help solving the phenomena mentioned above. In the frame of this project, we will definitely reach former (currently inactive) editors to ask about their opinion, experience and advice (using their email address and user talk page). (This is easier with formerly more active and known editors, but not with editors, who left the project after a few edits or with anonymous editors who never registered. For the latter groups we can only hope that they will answer.) And we will reach out the active editors as well, asking their view if the project was useful for their retention, and how do they feel which activities of the project helped the best the main goal. Do you have maybe better idea, how can we measure better the connection between the activities and the basic problem, or how can we confirm if these factors have an effect on the number of editors and their activity? Samat (talk) 22:35, 9 January 2019 (UTC)

Thanks for your reply Samat. All of what you say makes sense, I am mainly wondering if the reason people stop editing is for the reasons you identified, compared to other reasons (i.e., too text heavy, clunky old-fashioned wiki software, it takes too much time, etc.). Reaching people who stopped editing and trying to gather data about them seems to valuable, as it can help move your proposed steps forward. Given what you identified, trying to reach people as you shared may be a very practical manner for attaining that information. --- FULBERT (talk) 18:46, 11 January 2019 (UTC)

@FULBERT: I suppose there is some streetlight effect there, in that in that a small project by an affiliate with no staff has no hopes of significantly improving the software, not to mention participants' free time... but to the best of my knowledge existing scholarship supports the notion that the largest bottleneck in new editor retention is probably due to social factors such as aggressive reverts / deletes, complex help pages and policies, and not enough mentoring. --Tgr (talk) 07:08, 14 January 2019 (UTC)

Thanks for that Tgr. --- FULBERT (talk) 15:15, 14 January 2019 (UTC)

Dear FULBERT, we strongly believe that the activities planned in this project have effect on the basic problem and they could help the Hungarian community in the editor retention. Of course, there are other (technical, social, legal, economical, political etc.) factors we will not handle. Can happen, that some of these factors have larger effects on the decrease of the editor number than factors we handle. But we planned the project based on our past experience: what could we manage using our own resources in the given time frame. Other problems, which we probably will not be able to solve, left out from the project. Best regards, Samat (talk) 20:00, 17 January 2019 (UTC)

Aggregated feedback from the committee for WM HU/Editor retention program

Scoring rubric	Score
(A) Impact potential Does it have the potential to increase gender diversity in Wikimedia projects, either in terms of content, contributors, or both? Does it have the potential for online impact? Can it be sustained, scaled, or adapted elsewhere after the grant ends?	7.7
(B) Community engagement Does it have a specific target community and plan to engage it often? Does it have community support?	7.0
(C) Ability to execute Can the scope be accomplished in the proposed timeframe? Is the budget realistic/efficient ? Do the participants have the necessary skills/experience?	6.5
(D) Measures of success Are there both quantitative and qualitative measures of success? Are they realistic? Can they be measured?	7.8
Additional comments from the Committee: The project fits within Wikimedia's strategic priorities and has potential for online impact although its long term sustainability is more of question. High impact potential, although we are more speaking of a longer-term impact. As community health is one of the most important topics in the movement now, this is a good fit with strategic priorities. Slight concerns on sustainability of some points like live answers but overall looks quite good. This is a relatively small community, and as such is it important to be better able to retain editors than they have previously done. This project seems poised to do that. Geographically speaking, I am not sure if the project perfectly fits with the strategic direction of Wikimedia. However, regarding emerging Wikimedia communities, I believe the Hungarian community needs support to make it sustainable. Since the project plan itself is grow the number of active editors on Hungarian Wikipedia, the project does promise good online impact if executed well. If the project is successful, I don’t think there should be a need to scale this up again, because once the Hungarian Wikipedia gets active users and are developing content, further investment is not required. However, this project is very suitable for adaption elsewhere. Weak community and less number of active editors is a concern for many Wikipedias, and I am confident that this will be a good case study to see how we can deal with these problems. The approach is more iterative than innovative: the participants are just going to try applying the methods used before to huwiki. There are some potential as well as risks. The main risk is that nothing will really change despite all efforts. The success can be measured at least in part. I find it a very innovative and well-planned projects. This might even be the next Big Idea as many communities are facing similar editor retention problems, and this project can bring solutions they can also use. My only concern is about measures of success as targets are not easily measurable here. This proposal iterates on a solution that has been tried elsewhere, though here it seems a comprehensive solution in that it both collects data to inform what is needed as well as implement what is learned to help address the problem. It is a new approach to grow a community. We often see proposals related to edit-a-thons, campaigns or GLAM-related work, to grow the community. This approach of a year-long plan is very sustainable, it teaches us new things, and there is a lot to learn. Also, the proposer’s did good research of the grassroots level problems. There is a discussion on the talk page that the issues they’ve identified may not be the reasons. But I would like to highlight the point that these the factors that can be controlled, and tried to address. Other factors such as such of Wiki-interface, local issues, etc. are something which can’t be worked on, or at least not controllable now. The plan seems to address the factors which can be of our control. The measures of success or evaluation are clear. It can be seen how impactful this project has been in growing a sustainable community and also online impact. The timeline seems to be realistic and can be accomplished in 12 months. The budget appears to be weighted too heavily to funding the project manager. Can more money be spent of activities instead of management? Quite confident in ability to execute, the project is run by multiple editors in good standing with a very significant track record, with both on-wiki and affiliate experience. The only concern is regarding the budget: taxes look really high, should check if something can be done about it... There are several people involved in this proposal, and as such it seems sufficiently staffed to implement their solution. The participants have good contributions to the Hungarian Wikipedia, and I am confident of their ability to train newbies to contribute and organize training sessions for the same. Some of them also organized Wiki-activities in the past, and I believe that they are well positioned to execute this project. The project also nicely makes use of Wikimedia Hungary by involving its board members and also seeking their help wherever needed. The budget is very reasonable for a year-long program. However, some costs can be saved by partnering with institutions for in-kind donations of event space. These partnerships will be useful in the long term. The community engagement seems to have been significant. Excellent community engagement, a very thoughtful plan of engaging with the Hungarian community and good support from the community in return. There appears to be sufficient input from the community along this process. It has a good number of endorsements from the community, and participants are community members themselves. The plan also engages the community at various levels. The problem is important and needs a solution, but all those project management and staff give the feeling that a solution is not approached correctly. I am willing to support this project but its budget should be modified to shift at least some money from managing the project to some real activities. I think this idea really deserves full support. Editor retention is an important problem of our movement, and here we have a good example of a local community interested in doing some work on it. The only point to check is taxes that look extremely high: if possible checking with some expert in Hungarian fiscal system should help. No concerns about any parts of the project, everything looks good to me. The problem they want to solve is too big. For now, it's a global problem for Wikimedia. If we offer the fund for this proposal, there are too many risks we need to suffer. And, I can't believe we can get the significant improvement by the project. At this time, I think we need to waiting for the discussion about Wikimedia 2030. This seems an ambitious project which I can only hope will be shared with larger communities across the movement that are also facing similar challenges. I believe this can be a real test case to then consider sharing what is done to scale up across the movement. Everything seems reasonable and fair enough to me. However, the state taxes seem too high and are taking a significant chunk of the budget. This needs to be discussed with the Grants Admin or the finance department of WMF, and see if there any way to avoid these. It is close to 40% of the total budget. Often governments don’t tax non-profits, there can be a fiscal sponsor to route these funds (they charge 10-20%), and it saves a significant amount. By doing so, the money can be used more productively, for example, more edit-a-thons, more participants for the Wikicamp, etc.

This proposal has been recommended for due diligence review.

The Project Grants Committee has conducted a preliminary assessment of your proposal and recommended it for due diligence review. This means that a majority of the committee reviewers favorably assessed this proposal and have requested further investigation by Wikimedia Foundation staff.

Next steps:

1. Aggregated committee comments from the committee are posted above. Note that these comments may vary, or even contradict each other, since they reflect the conclusions of multiple individual committee members who independently reviewed this proposal. We recommend that you review all the feedback and post any responses, clarifications or questions on this talk page.
2. Following due diligence review, a final funding decision will be announced on March 1st, 2019.

Questions? Contact us.

I JethroBT (WMF) (talk) 17:17, 6 February 2019 (UTC)

Thanks for the thorough feedback! As I read it, the two main recommendations for improvements were around taxes and high ratio of project management costs vs. activity costs. Hungary has one of the highest tax rates within European countries, so not sure we can do much about the first (although paying a professional to look into alternative tax options might be worth a try). As for cost structure, in my opinion (note I am a volunteer who was involved with developing some of the background material for the proposal, this is my personal opinion and I am not speaking for the chapter or the grant participants) underfunding project management would be a major risk, in the past couple years we have tried to spend on program activities while hoping for the project management / community management part to happen on a volunteer basis and that has not worked out. (For some background, the chapter did not apply for grants in the last couple years but received around $10K/y donations, mainly from people donating 1% of their income taxes, which is a special donation method in Hungary, and comes with some legal restrictions which mean it is much easier to spend it on activities than on people.) So for example we budgeted money for event spaces but there wasn't enough volunteer time spent to organize the events, so we ended up with a smaller number of events than planned, they were just unstructured discussions and so less useful than they could have been, and no one collected feedback so we weren't sure what to do better next time. Ideally, all these things would happen on a volunteer basis but since low volunteer participation is what we are trying to solve here, it is something of a chicken and egg problem. My hope is that with the help of a paid project manager setting up the fundamental structure of an editor retention project, we can bootstrap a virtuous cycle (more editors -> more chapter volunteers -> more volunteer time for editor retention projects -> more editors) that becomes self-sustaining. --Tgr (talk) 20:36, 6 February 2019 (UTC)

Dear Committee Members, thank you for your detailed review and very useful feedback! I would like to react to your concerns about the tax amount and about the project management costs. The tax ratio of employment in Hungary is between 44-45% (which means, that there is 79% tax on the net payment). There are other ways for the payment, where the tax amount can be lower, but we have to choose very carefully to avoid tax violation (especially, that non-governmental organizations are in the crosshairs). We asked an expert's opinion on this question. Regarding the concerns about the cost of the project management: most of the activities planed in this project need no budget, only time, knowledge and effort of the people who will do, therefore the relative weight of the human costs is high. The role of the project manager in this project is not only management of others work, but (s)he has to drive the whole complex project, (s)he needs an overview of all running tasks, has to execute tasks where there is no volunteer, responsible for schedule and time management, and for the success of the project. The Hungarian community could not solve these issues on volunteer basis, and we would like to give a chance to solve them on a more organized way. Samat (talk) 21:48, 11 February 2019 (UTC)

Based on the questions of the Committee Members and the emails with the Grants Team, in accordance with the legal possibilities in the Hungarian tax system I reduced the tax amount to $8220 and the total amount of grant request to $24200 in the grant proposal. According to the offer of the Grants Team we would like to request 6.8 million HUF instead of USD. Thank you for your valuable feedback and ideas again. Samat (talk) 07:41, 16 February 2019 (UTC)

Round 2 2018 decision

Congratulations! Your proposal has been selected for a Project Grant.

The committee has recommended this proposal and WMF has approved funding for the full amount of your request, HUF 6,800,000/ $24,275 USD

Comments regarding this decision:
The committee is pleased to support your efforts toward editor retention on Hungarian Wikipedia. We appreciate your efforts to conduct preliminary research, to identify areas of need and concern amongst current and past editors, and to use several approaches to motivate, recognize, and support contributors in your community.

Next steps:

1. You will be contacted to sign a grant agreement and setup a monthly check-in schedule.
2. Review the information for grantees.
3. Use the new buttons on your original proposal to create your project pages.
4. Start work on your project!

Upcoming changes to Wikimedia Foundation Grants

Over the last year, the Wikimedia Foundation has been undergoing a community consultation process to launch a new grants strategy. Our proposed programs are posted on Meta here: Grants Strategy Relaunch 2020-2021. If you have suggestions about how we can improve our programs in the future, you can find information about how to give feedback here: Get involved. We are also currently seeking candidates to serve on regional grants committees and we'd appreciate it if you could help us spread the word to strong candidates--you can find out more here. We will launch our new programs in July 2021. If you are interested in submitting future proposals for funding, stay tuned to learn more about our future programs.

I JethroBT (WMF) (talk) 15:02, 1 March 2019 (UTC)

Grantees would like to thank the individual volunteers, the members of the Project Grants Committee and the staff members of the WMF for their time and valuable comments during the review process. We are glad that the proposal received positive decision and support, and we start to work on the realization of the project without any delay. Samat (talk) 09:27, 4 March 2019 (UTC)