Grants talk:IEG/Modernize the Wiki Loves Monuments PA Jury Tool

comment from another jury tool user and developer

• Hi, I'm the developer of another WLX_Jury_Tool (putting it clear:)). The tool that I developed does not have some shortcomings this jury tool has (WLM PA jury tool, PA from Panama, for the reference in the future text). You mention the following goals:
  • Porting the code to Symfony 2.7 from Symphony 2.3: My WLX Jury Tool is based on another framework - Play but I follow the latest versions of it. I started the development in 2013 with Play 2.2; Play 2.3 was released on May 30 2014, and I updated to it on Jan 2015; Play 2.4 released on May 28, 2015, and my development version already runs on it.
  • Improve existing features and add requested features.
    • For instance, this tool failed to support the use case of large contests like Wiki Loves Monuments UK, which had more than 10,000 images to rate. - It's very strange limitation, as this number is very low. My tool worked successfully and fast for about 50,000 images from WLM Ukraine in 2014. As a test I also filled it with all the images from some WLM year (about 300,000) and also did not have any problems. I tried to use WLMPA for Ukraine on WLM and every time I tried I failed to import any significant number of images into it, and did not get any feedback from you as developer and supporter of WLM PA Jury tool. I've only been told that there may be some errors and I should try again, but now I see in this request that the number of images Ukraine had on WLM was not supported at all.
    • incorporate the many features that have been requested throughout these years, and iron out bugs that have been reported. - I've been doing this in 2013 - 2015 for WLX_Jury_Tool and still continue to do. Also I need to mention that many countries either could not use the WLM PA jury tool, or did not like it because of bugs and missing features. For example Ukraine, Germany, Poland (three countries with the most number of images on WLM), Brazil and Catalonia developed their own jury tools. United Kingdom decided not to use jury tools at all. Brazil and Poland later went with my WLX_Jury_Tool for WLE 2015. Czech Republic first used your WLM_PA for WLM 2013 but faced many bugs and on WLM 2014 switched first to Poland jury tool and then to my WLX_Jury_Tool. I think you should be more clear on exactly what features and bugs you are going to fix, how and in what timeframe. Without this it's not clear what you are implementing, fixing, and if you have actually planned (or have) enough time and(or) other resources for this.
    • Redesign the UI/UX. - For rating the images my WLX Jury Tool already has features WLM_PA does not. I has gallery mode, (real) large view mode, thumbs bar, tabs to view images by the rate, comments. I also have some features in admin mode like viewing the images by the rate, juror and round, both as a gallery and a file list. While the admin interface is not finished and polished either, and this is a very serious shortcoming for WLX Jury Tool, I have a plan to finish it before WLM 2015.

Also I volunteered to WLM 2015 team to provide and support WLX Jury Tool for WLM 2015. While I'm not against competition and variety of tools, it may be the case many of the asked features will already be there for WLM 2015 done by me even without this grant or maybe even before it is granted. I think maybe we may come up with competitive request of exactly what features and how we are going to implement. But granting one tool when there is another one(s) without any discussion and good grounding is not a good idea --Ilya (talk) 06:58, 14 June 2015 (UTC)

Nice! DMaggot, what about building on Ilya's tool instead? Ilya, do you have a list of bugs/feature requests that can be worked on? --Nemo 09:28, 14 June 2015 (UTC)

Hi, the main problem with my jury tool is the same as with DMaggot's "time constrains ... caused many last-minute features to be poorly integrated in the code", maybe even much worse because of greater number of features. I'm going to cover the tool with automated tests and clean up first. --Ilya (talk) 15:45, 14 June 2015 (UTC)

I second Ilya's comments. FYI, I developed a jury tool a long time ago, which was used each year but at least a few countries, which doesn't have the above problems. Ilya's tool seems even better, and might be better suited for non-WLM competitions. —Ynhockey (talk) 15:11, 14 June 2015 (UTC)

I moved the page and rewrote some parts to make it clear that this is just one of the tools. I initially left out the feature list that I would be working on because it is probably meaningless for people that have not used the tool, but I included three features that I think are easy to grasp. DMaggot (talk) 19:40, 14 June 2015 (UTC)

WLX Jury Tool already has all these 3 features --Ilya (talk) 23:22, 14 June 2015 (UTC)

Many jury tools

I would welcome a grant for the development of a WLM jury tool, but not in the current form. What indeed would be very beneficial for the movement is the following:

1. Spend time and document the existing jury tools with their features, advantages and disadvantages. We need to have this information in one place.
2. Choose the base tool for improvement (obviously the tool with most features and/or least disadvantages should be chosen).
3. Make this tool THE Wiki Loves X Jury Tool, so that it has all features implemented in other tools (and meets expectations of all users of existing tools).
4. Work on improvements (e.g. UX) and updates (e.g. new features requested by local teams)

I don't think that improving one of the tools without studying available solution is a good approach. I would definitely endorse, however, the project that would bring THE jury tool for the movement, and I think that everybody would agree that making such a tool is a good investment — NickK (talk) 00:57, 15 June 2015 (UTC)

I have organized (or helped the organization) the WLM in my country and I have experimented at least three jury tool. In my opinion there is no problem of diversity, there is no official jury tool. This is a tool developed personally (and it fits correctly the aim of the IEG) and we have so different needs for each countries to have space for many jury tools. So the best question is to know how many people are using it, how many people used it and if there are good feedbacks. --Ilario (talk) 12:09, 15 June 2015 (UTC)

I would say that a jury tool supported by the Foundation becomes from a certain point of view the official jury tool (as all other tools were designed by volunteers in their spare time, and this one was officially supported by the Foundation). My expectation from investment in this grant would be at least a jury tool with best features of existing alternative tools — NickK (talk) 12:14, 15 June 2015 (UTC)

To be financed by the IEG is different than to be supported by the Foundation. The same Wiki Loves Monuments or Wiki Loves Arts or Wiki Loves Earth can be considered "clones" if we would reduce them to a photo contest, but they are more than a photo contest, so they are different. Imagine if there is the proposal of the Foundation to merge the three photo contests only in one? This may be the solution to have a single jury tool. But we have several photo contests with different needs, it makes sense to have different jury tools. In addition a single jury tool can become a "legacy" system and time by time it can be a risk, instead of having many small jury tools less expensive than a big one and in addition can be a valid option to offer several alternatives. It's natural that this number should be smaller, but there is no sense to have a single monolithic one. --Ilario (talk) 12:22, 15 June 2015 (UTC)

Please be more specific, how do the jury tools need to differ from contest to contest? When the tool and its support satisfies the needs, organizing committees do not care about developing or using different one, they are only very satisfied with the tool they use. When the tool is buggy, not supported, missing features, many people start writing their own to overcome bugs and limitations. That's from my experience of running jury tool for 17 countries for WLM and WLE. --Ilya (talk) 12:36, 15 June 2015 (UTC)

Upcoming evaluation of jury tool by end users

There is a large amount of countries that where using (or trying) WLM PA Jury tool in 2013 and using WLX Jury tool in 2015. We initially planned to ask for feedback just on WLX Jury tool, but now we can come with more detailed and broad questionnaire together with Ilario and DMaggot to better understand what jurors need. --Ilya (talk) 13:27, 15 June 2015 (UTC)

Existing feedback and comparison with existing tools

DMaggot, can you provide the feedback or suggestions

• from users that used WLM PA jury tool?
• from users of other jury tools?
• from users that used several jury tools? --Ilya (talk) 13:36, 15 June 2015 (UTC)

I only have feedback from uses of the WLM PA tool DMaggot (talk) 08:18, 22 June 2015 (UTC)

then do you want to do joint evaluation in July as in previous question? I have feedback from many users of different jury tools, but they are in private mails and I don't want explicitly invite them to share it here, as it's may seem as explicit seeking of people with specific POV by specific person, it's better to have some open and unbiased jointly developed questionnaire for all the participating countries. --Ilya (talk) 08:28, 22 June 2015 (UTC)

Usage numbers

37 countries asked to try the jury tool in 2013, only 12 in 2014 (both years include countries that did not use the tool, as, for example, Ukraine evaluated the tool twice, but never used). How many countries used the jury tool, and why there was such drop of interest? --Ilya (talk) 16:56, 21 June 2015 (UTC)

What was the total number of countries participating in both years? DMaggot (talk) 08:21, 22 June 2015 (UTC)

It's available Commons:Wiki_Loves_Monuments_2013/Participating_countries, Commons:Wiki_Loves_Monuments_2014/Participating_countries --Ilya (talk) 08:24, 22 June 2015 (UTC)

Test coverage

Do you plan to cover the tool with automated tests? What percentage of code do you expect to cover? --Ilya (talk) 15:34, 15 June 2015 (UTC)

Yes, I plan to write functional tests. I expect to cover 100% of the code. DMaggot (talk) 08:18, 22 June 2015 (UTC)

9/29/15 Proposal Deadline: Reminder to change status to 'proposed'

Hi DMaggot,

This draft is looking like it's well on its way. I'm writing to remind you to make sure to change the status of your proposal from 'draft' to 'proposed' by the September 29, 2015 deadline in order to submit it for review by the committee in the current round of IEG. If you have any questions or would like to discuss your proposal, let me know. We're hosting a few IEG proposal help sessions this month in Google Hangouts. I'm also happy to set up an individual session. Warm regards, ----Marti (WMF) (talk) 20:33, 20 September 2015 (UTC)

April 12 Proposal Deadline: Reminder to change status to 'proposed'

The deadline for Individual Engagement Grant (IEG) submissions this round is April 12th, 2016. To submit your proposal, you must (1) complete the proposal entirely, filling in all empty fields, and (2) change the status from "draft" to "proposed." As soon as you’re ready, you should begin to invite any communities affected by your project to provide feedback on your proposal talkpage. If you have any questions about finishing up or would like to brainstorm with us about your proposal, we're hosting a few IEG proposal help sessions this month in Google Hangouts:

• April 5th, 16:00 - 17:00 UTC
• April 8th 1:00 - 2:00 UTC
• April 12th 16:00 - 17:00 UTC

I'm also happy to set up an individual session.

Warm regards,
--Marti (WMF) (talk) 06:09, 2 April 2016 (UTC)