Grants talk:PEG/Shipmaster - Community of Arabic Wikipedia/Producer Prize-2014

Evaluation by the GAC

GAC Members who approve this grant request

1. Hi, I have no concrete objections, and therefore approve. I still think it should address more specific problems as per mine and MADe's comments. Cheers! NLIGuy (talk) 06:33, 22 May 2014 (UTC)
2. Me too. Polimerek (talk) 21:36, 22 May 2014 (UTC)
3. I'm glad that the members of the evaluation committee will have firm commitment in the contest and that there are several rounds the entries will have to pass in order to be awarded. I have no more questions and would like to give my support to this. Good luck!--Kiril Simeonovski (talk) 07:39, 26 May 2014 (UTC)

Discussion

Comments MADe

Hey Shipmaster, nice to see projects in Arabic.

My main concern is that you stay too much in your comfort zone with this grant request. The Arabic Wikipedia probably has some specific problems (as all languages versions have). It would be nice if this grant request would address those issues - think of articles supporting neutral edits in female / gay / Israeli/Palestinian topics.

You've done this since 2010... I challenge you to reinvent the competition this year! MADe (talk) 18:42, 16 May 2014 (UTC)

Hi MADe, thank you for your comments. They raise very interesting points, but I would really like to keep our eyes on the ball for this contest. The Arabic Wikipedia is one of the severely under-represented languages in Wikipedia, the community of editors is small, editor retention is a challenge, and community outreach in general is relatively weak. So while the contest has been very successful in general, that's still a drop in the ocean of the work we need to do to bring the Arabic wikipedia to a level of representation of content and community that is comparable to the number of native speakers. We do tweak the contest goals every year in terms of volume and general areas of editing we would like to focus on (e.g stubs vs. creating featured articles for example), but that tweaking is being kept as general as possible with the goal of not to discourage participation from an already small community (notice that our goal is 20 submissions, last year we had 15 total submissions). I think that fighting systemic bias is a worthy cause on any wikipedia, but it's just not going to encourage the level of growth that we are hoping for. --Shipmaster (talk) 05:08, 17 May 2014 (UTC)

editor retention is an issue? so add a category for returning editors (with contribs over 2 yrs, for example). Need more featured articles? create a category for that! I think that's what MADe meant, and I totally agree. NLIGuy (talk) 05:54, 20 May 2014 (UTC)

NLI, thanks for filling me in. Shipmaster, it would to see the competition more 'targetted' instead of the very generic competition you proposed MADe (talk) 17:57, 20 May 2014 (UTC)

Comment from ProtoplasmaKid

Hi Shipmaster and community. I'm some worried about give money for editions. I don't see a clear logistic of the contest, and how it will be reflected on the winning of the contest (if it is only in Arab, please translate to English) and mainly, why money and no prizes (laptops, readers or a simple barnstar in the discussion). In my opinion, there is no money for defeat that "under representation" if there is no more additional community resources apart from funds: institutional support, sponsoring of the prizes, or donation of them. We need more info. Regards, --ProtoplasmaKid (WM-MX) (talk) 23:44, 19 May 2014 (UTC)

Hey PPK, I support your point. It would be nice if you could join up with a local organisation (eg. google for the Teylers challenge). MADe (talk)

Thank you for your replies, Shipmaster. For me, this answers are satisfactory. Go on with the contest :) Regards, --ProtoplasmaKid (WM-MX) (talk) 05:36, 28 May 2014 (UTC)

Just a comment, I don't read an after activities of this prize, for an example, have a written testimony. Do you plan some post for the Wikimedia blog or a campaign on social media channels? If not, this can be a nice idea. I'm glad for get more news from Arab Wikipedia. Regards, --ProtoplasmaKid (WM-MX) (talk) 05:41, 28 May 2014 (UTC)

Thanks for your support ProtoplasmaKid! And that's a very good idea! No, we have not been doing any campaigns or publications beyond the global noticeboard on ar.wp, but I realized that there are several Facebook groups that have cropped up over the last year that we can make use of, and we can get past winners to help us do the outreach. I will discuss with the prize steering committee having a structured approach to social media outreach and report back in this year's report if we hopefully get the grant... --Shipmaster (talk) 06:43, 28 May 2014 (UTC)

Comments from Kiril Simeonovski

Thanks for the submission. In general, the idea about the prize is fine and it's worth supporting, but my concerns are mostly related to the quality of content that the contestants will create on the Arabic-language projects. The risky thing when organising one such contest is that it also encourages inexperienced users to edit low-quality content and thereby harm the average quality of the project in question. So, what does the community plan to overtake as a preventive measure to stop increasing the amount of low-quality content? Do you plan to constitute a committee that will monitor the edits of the contestants? What is the current process of monitoring such users on the Arabic-language projects? What was your approach when facing the same problem last year? Additionally, do you plan any supporting activities (e.g. workshops, editathons or presentations) in order to increase the number of interested contestants who are keen on editing? The latter may be difficult provided that there is wide dispersion of the Arabic-language speakers, but organising online workshops and presentations might work very well to reach out to people from different geographical locations.--Kiril Simeonovski (talk) 09:50, 20 May 2014 (UTC)

Comments from Tony1

Procedural:

1. Would be nice to be able to click "Add topic" at the top for a new personal section.
2. A better title would be "Arabic Wikipedia Producer Prize", rather than starting with the applicant's name. Any chance of changing it?
3. Please change the project start date. In principle I'll vote oppose to a retrospective application.

Comments on the application:

1. I'm very attracted to the general idea. Where is the background description?
2. Twice yearly ... yeah, could work. Have you considered doubling the prize money for an annual set of prizes (by now, awarded July 2015, since we've missed the boat for the first half of 2014? Might allow a bit of fat to designate one or two with themes. For example, a women's prize, either for female contributors or on women's topics? And one on a hot topic that could generate editorial activity and is under-served currently: I know nothing, but "History of Arabic science"? See how ignorant I am ... but you know better. I just like the idea of picking a different thematic area for each round of prizes—sends out signals to the community, and is focused and more measurable. I'm not saying to ditch the idea of a general prize, though.
3. Measures of success: 20 submissions? That's not many. Rather than relying on submissions, why not have the judging panel select from new featured content? I'm actually confused as the process: could you think it through and put a draft sequence of what will happen. We can think it through then. "1000 new articles"—how to isolate these from those that would have been produced anyway? Same for the pics and improvements to articles. My leaning, by the way, is very much for the improvement of existing articles rather than necessarily creating new ones.

Tony (talk) 10:44, 20 May 2014 (UTC)

Replies from Shipmaster

Hi everyone, thanks for the valuable opinions, since some questions were repeated, I am putting consolidated answers here.

• More information on Contest structure and rules: I added a new section.
• More targeted measures: We can definitely add a new metrics to reflect editor retention and how many Producer prize alumni turn into Community leaders (Admins ... etc), I consider those examples to be part of the "general tweaking of measures" that is not only needed but required from year to year. However , please refer to the above bullet point for a detailed explanation of contest individual goals, I think you guys have understood what I added as metrics of success for the grant to be contest goals, we have a "set your own goal, get compared to other contestants" rule for the contest itself, and we put metrics for ourselves to measure whether this was successful or not.
• Why money: We give money for two reasons : first it serves best the editor needs (we might give away cameras, but the winner might not need one) second : we actually offer purchase and deliver items based on their requests and within the range of the prize, but in many cases the cost of shipping eats out large proportion of the prize.
• Ensuring quality of entries: Although there is no mechanism to constantly monitor contestants, contestants are required to set quality standards for their contributions before they start (e.g. articles should be more than 2 kb .... etc). In addition, the evaluation committee at the end of each round checks on the quality of the contribution and assigns their voting points based on it.
• Supporting activities: This is a really good idea, but unfortunately the organizing committee does not have time (or logistics, since they are in dispersed countries) to run any supporting activities on our own, but the contestants are free to run such activities and include them in their activity report for extra consideration from the judges.
• Name change: I am cool with changing the name if Alex is ok with it, the name as it is now comes from the template from grant request which is [Name]/[Project title], it also reflects last year's name for consistency.
• Date change: I changed it to the time where we will actually start using the money for the prizes.
• Measures of success: 20 submissions equals more or less the number of new featured articles per year, so we've used a bit of correlation here, and it is only one of several measures that span both participation and quality. This process puts less work load on both the steering committee and the judging committee because as I mentioned above, the measures of success are only indirectly correlated to the contest goals of each contestant, these are criteria we compile after the contest to make sure that we made more of a difference than just average editing frequency from people not involved in the contest and thus assess contest effectiveness.
• Expanding the contest and adding themes: I like the idea of doubling the amount over two years (puts less work on us) but with our small judging team we don't have the capacity for thematic contests since they require much more detailed scrutiny of entries (If I am comparing two history articles for example I have to somehow assess the usefulness of information beyond just the quality of the article). Also we prefer that all editors are free to bring the best in them rather than enforcing themes that not all editors might be good at (we are working with a small community, so if we pick a theme that is not popular, we may end up with one or two submissions).

I hope I have addressed all the points, please let me know if you have further concerns --Shipmaster (talk) 07:28, 22 May 2014 (UTC)

Rejoinder from Tony1

Hi, I can't reply to your bullets by number, which is a pity.

• (a) I asked about themes because goal 4 of the project is stated as: "The contest goals reflects areas that need improvement in the Arabic wikipedia".
• (b) You commented: "I like the idea of doubling the amount over two years". My suggestion was for one round, doubling the prize money, during this year. More visible.
• (c) How many people have typically been on the panel? I presume that their choice avoids any COI issues in being personally or editorially familiar with any of the contestants.
• (d) I'm not sure I understand or am comfortable with this "set your own goal". What is the advantage over simply stating that the judges will have reference to the FA criteria [link] in terms of judging quality of edits. FA criteria should be best practice on the site—at least a benchmark even if not every edit or body of editing comes up to the precise specifications. Should not the FA criteria be disseminated among the editing community, especially if there are only about 20 FA promotions per year (very few, I must say). I've had a look at the criteria, via gtrans. They look fairly similar to en.WP's, although modified. They look like a good direction to set for judgement, even in terms of article improvement: balance and citations should be taken into account.
• (e) I find it hard to make sense of the "Measures of success": photos?
• (f) What have the prizes been thus far?

Tony (talk) 11:58, 22 May 2014 (UTC)

Replies:

• (a) That is probably my fault, it needs to be reworded to be more clear. We try to keep what we mean by area very general, for example, we added improving 500 existing articles as a measure of success this year because we felt we needed expanding of existing content. We make those overall measures of success based on the discussion happening when approving individual contestant goals.
• (b) We more than doubled the prize between 2011 and 2012, it went from 150$ first prize to 500$ first prize. Of course more money is good, however it just occurred to me that as we increase the prize value more and more we hit a logistics issue of transferring the money, its rather hard to transfer money to some countries in the middle east and the more money it is the harder it gets.
• (c) On average we have 12-14 people on the judging panel, COI issues that we know of factor in selection, also judges are warned to make clear any COI issues that we do not know of.
• (d) The target is not to just create FAs. It can vary depending on candidate goals, but we do have a minimum criteria for each goal. If your goal is to create FAs, then the measure the judges will look at is whether or not your articles became FA, if your goal is to create new articles, then our minimum requirement is that they are 4K each and not bot-created, if your target is to improve stubs, our minimum requirement is to expand it to 10K, have at least 2 sources and 2 categories. We review those goals on a contestant basis, because we accept a wide variety of goals, last year we even had one contestant with his goals targeted towards improving the Arabic Wikiversity. I realize that it is less structured than what some people would like, but the judges job is to measure "Effort" and "Quality" over a wide variety of goals, this keeps people motivated to work on what they like and keeps the contest relevant.
• (e) We accept contest submission from people who are passionate about photography, more CC photos documenting areas of the middle east is good, no? Why shouldn't that be a measure of success?
• (f) 500$ for first prize, 250$ for second and 100$ for third, some people ask us to buy something with the money, e.g a laptop or a camera.

--Shipmaster (talk) 05:23, 24 May 2014 (UTC)

Comments from Léna

Hi ! I really don't see the point in giving money for the producer prize, for two reasons :

1. Firstly, we have a similar event in French Wikipedia which does not need monetary prices to work. Giving external motivation (money, books, etc) can not only be useless, but it can also be hurtful, by turning off the internal motivation (I edit Wikipedia to share knowledge).
2. Secondly, because I think it shouldn't be needed to participate to the Producer Price in order to have financial support to contribute ; I think any trusted editor who needs books, references... to edit Wikipedia should be helped.

I really do not see any problem with giving USD 1,800 (or more !) to help trusted editors of the Arab Wikipedia, but I think it should be given to anyone asking.

Thus, I suggest :

• To give USD 900 or less (the first prize might be too much) for the prizes of the next context ;
• To give USD 900 or more (maybe thus asking for a larger grant) to prepare the next context : any trusted editor needing ressources (books, etc) for the next context will be helped before the context takes place, in a kind of "micro-grant" system for the Arab Wikipedia. Léna (talk) 16:18, 25 May 2014 (UTC)

Great idea Léna! The Wikimedia Foundation is currently launching a pilot micro grants program on the Arabic Wikipedia. The micro grants will provide books to individual contributors. Please read more about the pilot here.

Thank you for your feedback, Léna. I don't think that we are asking people to edit for money. Rather, I think of it simply as a community event that has physical prizes to increase competition. Some of our editors are much more active than the contestants and they choose not to participate, some of our contestants participate very competitively and win then they donate the prize back to the contest pool, so it's not always about the money. Some of our contestants do compete very hard for the prize because they need it. Please understand that the region is very diverse, so some areas are as affluent as a first world country, and in some other areas a first or second prize may mean a much needed replacement for their 4-year-old computer, hard-won through competition. --Shipmaster (talk) 06:34, 28 May 2014 (UTC)

My point was : why not giving money to remplace 4-year-old computers when people ask for it instead of only when people win the competition ? Léna (talk) 21:58, 1 June 2014 (UTC)

A monetary prize can be put towards replacing a four-year-old computer; but we certainly don't want to demotivate editors who have a newer and fully working computer. Tony (talk) 04:09, 4 June 2014 (UTC)

Approved

Thanks to the GAC and Shipmaster for your engagement on this request. We are happy to fund this project again as it has been proven effective. We appreciate your providing the cultural context on why holding a general competition is preferable to doing a themed one. Using social media to market the competition is a great suggestion as is reaching out to past participants. Please also see our program resources for writing competitions for more information on tracking participant contributions over time and metrics for measuring success. Alex Wang (WMF) (talk) 20:59, 4 June 2014 (UTC)

Tracking submissions

It would great if this year's report could include links to the photos submitted for the contest (this way we can see how many photos submitted are included in wiki projects using the glamorous tool) and links to articles submitted. Have categories for each of these types of submissions would be very helpful. If you have categories for last year's submissions, please add them to the 2013 report. Thanks, Alex Wang (WMF) (talk) 23:31, 2 July 2014 (UTC)