Talk:Learning and Evaluation/Evaluation reports/2015/Writing Contests

This section is meant to encourage a discussion around next steps, both as potential areas for further investigation and areas in which to improve tracking of inputs, outputs, and outcomes to improve future reports. Please reach out on the talk page if you would like to contribute to the conversation.

Potential areas for further investigation

Median cost aso.

Das ist mehr so ein Spaßvergleich oder? Ich könnte mir vorstellen (aber natürlich nur völlig theoretisch ^^), dass, ein Projekt in der deutschen Wikipedia leicht höhere Kosten hat als ein Projekt in der ukrainischen. C. p. natürlich. - Aber nur so ein theoretischer Gedanke. ...Sicherlich ^Post 19:08, 13 August 2015 (UTC)

Zusatz: In euren limiations fehlt die Frage nach Opportunitätskosten. Wenn in einem Wettbewerb 1000 Artikel geschrieben wurden; wurden die wirklich zusätzlich geschrieben oder wurden dafür einfach an anderer stelle 1000 weniger geschrieben. gerade im hinblick darauf, dass ihr schreibt, dass es "83% existing users" sind. wenn die 83% ihren aufwand nur anders fokusieren, dann ist der effekt ein ganz anderer (muss nicht schlecht sein; aber es ist dann eben kein "mehr" sondern nur bsw. ein "anderes thema"). ob die 18% new users (hehe, now we are at 101 % :D ) dann auch wirklich neu sind? Oder doch alte hasen halt als IP? -

die opportunitätskosten sollten nicht so schwer sein zu analysieren: in welchem zeitraum war der wettbewerb; gab es ein mehr an artikeln/bearbeitungen/qualitätsauszeichnungen im vergleich zu ähnlichen zeiträumen (bischen statistikspielen mit vorjahreszeitraum ggf. unter beachtung aktueller trends usw.) ...Sicherlich ^Post 19:21, 13 August 2015 (UTC)

Hello Sicherlich. Thank you for your comments. I am using google translate, thus I am unsure as to accuracy of the translation. With regard to your first comment/question, it is my understanding that you are asking whether costs are comparable between events in Germany vs. the Ukraine. Most certainly they are not and a quick google search for :global economic comparisons" will yield a good number of indexes for making such comparisons on a variety of indicators. We advise that when interpreting or using these data to help set reasonable targets that everyone be cautious and attend to context as not only are there vast economic differences between different community contexts, there may also be different goals.

With regard to your second query, which I understand to be a question as to whether productivity is actually increased or rather just directed differently - no one has reported about this and we did not pull metrics from time periods other than what has been specified. We did explore a similar question somewhat last year in terms of editathon participant cohorts to some extent. Looking at whether participation in an edit-a-thon reduced or increased user productivity for the month of the editathon compared to the preceding and following month - the data did not support any conclusions, as the sample lacked representativeness and statistical power, but the trend appeared to be increased editing behavior. The design is even more complicated when it comes to writing contests and we have events that last 6 and 12 months and comparison windows become rather tricky to define and compare given the variety of editing behavior patterns over time. Of course, the productivity here is also only measured in terms of the writing contest content, perhaps it would also be useful to look at the proportion of editing which is directed at the writing contest vs. elsewhere as well. Depending on various program leader goals and objectives, this evaluation question may be of interest to others as well, thanks for calling it to attention as we consider next steps for evaluating the impact of writing contests. Please let me know if anything was missed in the translation JAnstee (WMF) (talk) 17:01, 19 August 2015 (UTC)

Thanks for the answer JAnstee and you got it right :)

1st part can't see this consideration somewhere in the output. there is median and min and max but considering the different values it actually is meaningless :) ...

2nd: thx. for pointing out the difficulties. I see the problem of the small sample (but thats probably with all findings) and did not consider these long term events what for sure will always be difficult to evaluate.

one more thing I know I'm a pain in the ass :D : If you want a broad public recognition you should in the future consider translations, at least for the summary. Maybe some major languages (numbers of speakers and/or size of Wikipedia and/or target regions) would probably be usefull.

So thanks again for the answers! ...Sicherlich ^Post 21:05, 19 August 2015 (UTC) maybe its just "everything used to be better in the old days - but translations by the community do not work very well anymore. Maybe its because there is too much stuff by WMF. Maybe its the money: why should I do work others get paid for? - money is IMO slow affection poison to Wikipedia :/ - or maybe it never worked :D

Other contests

It would have been nicer if the report had included the publicly available data from other known contests/incentives. From a quick glance at the appendix, several seem missing. --Nemo 21:50, 30 August 2015 (UTC)

Thanks Nemo for the comment. What would you want to know from these other contests or incentives? Did you have any particular analysis or questions you wanted answered? Thanks! --EGalvez (WMF) (talk) 17:27, 31 August 2015 (UTC)