Wikimedia Meta-Wiki

Learning and Evaluation/Evaluation reports/2015/Wiki Loves Monuments/Outputs

< Learning and Evaluation | Evaluation reports | 2015 | Wiki Loves Monuments
(Redirected from Grants:Evaluation/Evaluation reports/2015/Wiki Loves Monuments/Outputs)
Wiki Loves Monuments


What does a typical Wiki Loves Monuments produce in terms of on-wiki content?

We examined how many participants contributed to each program implementation, how many of those participants were new users, existing users, and active users, as well as used this output along with budget to learn examine the dollar cost for each participant.

Read this page to learn about the amount of on-wiki content you might expect and costs that have been associated


Participation

edit
The average Wiki Loves Monuments implementation had 153 participants. 65% of those participants were new users.

We were able to obtain participant counts from all of the 72 Wiki Loves Monuments contests. The number of participants in the national contests ranged from 5 to 1,485, which averaged 153 participants [1] (2013[2] , 2014[3]). Of the total participant base (18,682 users),11,429 were newly registered. [4]

Cost per Participant
edit

On average, Wiki Loves Monuments implementations cost around $25 USD per participant.

With budgets reported for 43% of the implementations, we were able to calculate the cost per participant for Wiki Loves Monuments implementations. This ranged from $0.64 to $326.95 USD per participant, with an average cost of $25.39 USD per participant [5] (2013[6] ,2014[7])

Graph: Cost per Participant.

This boxplot presents the distribution of the cost per participants for the 2013 and 2014 contests, and for both combined. While 2014 has a higher minimum cost ($1.86 USD as opposed to $0.64 USD in 2013), the distribution is more narrow in 2014 (Quartile 1 = $4.59 USD and Quartile 3 = $35.38 USD versus Quartile 1 = $5.66 USD and Quartile 3 = $47.68 in 2013.


Content Production and Quality Improvement[8]

edit
The average number of media uploaded to Commons by Wiki Loves Monuments implementations from 2013 and 2014 was just over 3,700. The number of media placed in Wikimedia articles per event was, on average, just over 350.

We collected media upload counts for all 72 WLM implementations. Uploads totalled 539,875[9] and counts ranged from 22 to 45,255 (Graphs: Cost per participant - above - and Budget, participation, and media uploaded - below.). The average event uploaded 3,709[10] (2013[11], 2014[12]).

Regarding media use, the number of unique media used on Wikimedia project pages per contest ranged from 6 to 10,762. The average number of media used was 363 [13] (2013[14] ,2014[15]) (see Graph ?).

Budget & Content Created
edit

The average Wiki Loves Monuments 2013 and 2014 photo upload costs just under one US dollar. However, more money spent on a national Wiki Loves Monuments event does not necessarily relate to more participants or more image uploads.

The graph below plots event participation, budget, and number of media uploaded. With larger bubbles representing larger media upload counts for an event, the graph seems to show that there may be a link between the number of participants and the number of media uploaded as there seems to be more smaller bubbles collected to the left of the horizontal axis, a link which is slightly supported by correlation analysis [16]. Still, there is no clear collection of bubbles or certain bubble sizes along the vertical axis, which suggests that budget size does not predict either media uploads or participant counts, which is supported by correlation analysis [17].

Graph: Budget, Participation and Media Uploaded.

This bubble graph displays that more media uploaded tended to be coupled with increased participation. You will notice that for the bubble graphs some of the data points are missing, this is because only the 31 contests which had report of all three metrics could be included in the graph

Cost per Media Uploaded
edit

For the 31 photo events that reported their budget, the average cost per media uploaded was $0.90 USD.

We calculated the cost per media uploaded for 31 events for which we had budget and media upload data. The average implementation costs $0.90 USD[18] (2013[19], 2014[20]) per upload. Based on reported budgets for Wiki Loves Monuments 2013 and 2014 implementations and on the number of media uploaded to Commons, the amount of dollars spent for each upload ranged from $0.04 to $4.54 USD, with 50% of costs falling around the median ranging from $0.32 to $1.59 USD per upload.

Graph: Cost per media uploaded.

This boxplot displays the distribution of the cost per media uploaded for the 2013 and 2014 contests, and for both combined. Interestingly, the boxplot shows that there was a much wider distribution of cost per photo added for the 2014 contests compared to those in 2013. The Quartile 1 amounts are similar ($0.33 USD for both 2013 and 2014) but the 2014 median and the Quartile 4 extends nearly a dollar higher per image ($1.36 USD for 2013 and $2.14 USD for 2014) suggesting a larger number of contests with higher costs per photo in 2014.
Across all events captured in this reporting nearly 70,000 media (13%) had already been added to the main article namespace on over 95,000 Wikimedia pages.

The number of unique media used on WikipediaWikimedia article pages ranged from 6 to 10,762. Most often the number of unique media used per event was 68, while the median average was 363[21] representing article use rates ranging from 1% to 41% with an average of 11%.[22]

We also looked at how much it costs in terms of Wiki Loves Monuments media which were used on Wikimedia article pages. The costs ranged from $0.23 to $115.48, and averaged $7.60[23] with 50% of costs falling between $2.99 and $29.19 USD per upload use overall. (2013[24], 2014[25]).

As the graph below illustrates, media use, represented by bubble size and which can be used as a marker of image quality, is quite evenly spread along the vertical axis of program budget. This suggests, along with correlation analysis, that budget and media use are not linked.[26]. However, with smaller bubbles concentrated to the left of the horizontal axis, there does seem to be a relationship between media uploaded and media use, which has support from correlation analysis[27].

Graph: Budget, Media Uploaded, and Media Used.

This bubble graph displays that more media uploaded tended to be coupled with increased media use, but not necessarily. Budget did not affect use rates in any consistent way. You will notice that for the bubble graphs some of the data points are missing, this is because only the 31 contests which had report of all three metrics could be included in the graph

With regard to media uploads and media use, an average of 11% of the media uploaded for a contest were used on Wikimedia projects [28], with a lowest proportion of 1% and a highest of 41% and with 50% of proportions ranging from 4% (Quartile 1) to 19% (‘’Quartile 4’’) (2013[29], 2014[30]).

Hours, Participation, and Media Used
edit

Events with more organizer hours per week per participant tended to have more media uploaded but not more unique media used.

We have the number of organizer hours per week (staff + volunteer),[31] number of participants, media uploaded, and unique media used for 15 contests. Organizer hours per week per participant ranged from 0 to 1.41 with an average of 0.24 hours.[32](See graph below, Organizer hours, media uploaded and unique media used.) Events with more organizer hours per week per participant tended to have more media uploaded but not more unique media used per participant.[33] A wider sample of data on organizer hours may reveal more or more nuanced trends between these inputs and outcomes, as well.

Graph: Organizer Hours, Media Uploaded and Unique Media Used.

This graph displays organizer hours per week per participant vs media uploaded per participant. The bubble size reflects the number of unique media used per participant.

Participation and content
edit

Graph: Participation, Media Uploaded, and Media Used.

This bubble graph displays that increased participation led to increased media uploaded and was often coupled with increased media use, but not necessarily. The relationship was inconsistent.

Although it appears that events with more participants tended to have more media uploaded and more unique media used on Wikimedia project pages based on the appearance of the increased bubble size in the bubble graph (See graph above, Participation, media uploaded and media used), the relationship between the two is inconsistent, and weak according to basic correlation analysis [34].

In terms of the type of participants, correlation analysis suggests that the number of existing users involved in a contest can have a positive effect on media uploads and media use [35]. With regard to the proportion of new users, a less strong link is evident. [36]


  1. ↑ ’’Mean’’= 259; ‘’SD’’= 297
  2. ↑ ‘’Median’’=151; ‘’Mean’’=250; ‘’SD’’=293.
  3. ↑ ‘’Median’’=154; ‘’Mean’’=269; ‘’SD’’=305.
  4. ↑ For 2013 Events: 5,430 (48%);For 2014 events: 5,999 (52%).
  5. ↑ ’’Mean’’= $41.15, ‘’SD’’= $62.34
  6. ↑ ‘’Median’’=$29.76; ‘’Mean’’=$44.79; ‘’SD’’=$73.02
  7. ↑ ‘’Median’’=$24.18; ‘’Mean’’=$35.39; ‘’SD’’=$42.60.
  8. ↑ Note: Although content production is a direct product of the program event itself and technically a program output most of the program leaders who participated in the logic modeling session set that level as their end outcome, hence it can be considered differently.
  9. ↑ 268,743 from 2013 events and 271,132 from 2014 events
  10. ↑ ’’Mean’’= 7,498; ‘’SD’’=9,736
  11. ↑ ‘’Median’’=4,026; ‘’Mean’’=7,263; ‘’SD’’=8,999
  12. ↑ Median 2,714; Mean=7,747; SD=10,587.
  13. ↑ Mean=967; SD=1,670
  14. ↑ Median=476; Mean=1,102; SD=1,866
  15. ↑ Median=343; Mean=825; SD=1,447
  16. ↑ Correlation Coefficient=0.58.
  17. ↑ Correlation Coefficient for budget vs media uploads=0.03; Correlation Coefficient for budget vs participants=-0.26
  18. ↑ Mean= $1.09; SD= $1.01
  19. ↑ Median= $0.77; Mean= $1.07; SD= $1.14
  20. ↑ Median= $0.91; Mean= $1.11; SD= $0.85
  21. ↑ Mean=967; SD=1670
  22. ↑ Mean=14%; SD=11%
  23. ↑ Mean= $18.82; SD= $24.06
  24. ↑ Median= $9.84; Mean= $16.05; SD= $15.04; 50% of costs ranged from Quartile 1 $3.19 to Quartile 4 $26.80 USD;
  25. ↑ Median= $6.27; Mean= $23.20; SD= $34.27; 50% of costs ranged from Quartile 1 $2.81 to Quartile 4 $28.29 USD
  26. ↑ Correlation Coefficient=0.09
  27. ↑ Correlation Coefficient=0.65.
  28. ↑ Mean=14%; SD=11%
  29. ↑ Median=13%; Mean=16%; SD=12%; 50% of costs ranged from Quartile 1 6% to Quartile 4 23%;
  30. ↑ Median=8%; Mean=11%; SD=10%; 50% of costs ranged from 4% (Quartile 1) to 17% (Quartile 4)
  31. ↑ Number of organizer hours per participant divided by the length of the event in weeks.
  32. ↑ Mean=.31; SD=0.35.
  33. ↑ Organizer hours per week per participant to media uploaded: ‘’Correlation coefficient’’=0.65; Organizer hours per week per participant to unique media used: Correlation coefficient=0.06
  34. ↑ ‘’Correlation Coefficient’’=0.39)
  35. ↑ ‘’Correlation Coefficient’’ for existing users vs media uploads=0.67 and for existing users vs media use=0.61
  36. ↑ ‘’Correlation Coefficient’’ for new users vs media uploads=0.51 and for new users vs media use=0.28


Retrieved from "https://meta.wikimedia.org/w/index.php?title=Learning_and_Evaluation/Evaluation_reports/2015/Wiki_Loves_Monuments/Outputs&oldid=24893342"