Grants talk:IEG/Visual editor- gadgets compatibility

Congratulations! Your proposal has been selected for an Individual Engagement Grant.

The committee has recommended this proposal and WMF has approved funding for the full amount of your request, $4500

Comments regarding this decision:
We look forward to learning more about the gadgets ecosystem along with you, and to seeing useful results as you adapt the first few gadgets for Visual Editor and chart a course for the future.

Next steps:

1. You will be contacted to sign a grant agreement and setup a monthly check-in schedule.
2. Review the information for grantees.
3. Use the new buttons on your original proposal to create your project pages.
4. Start work on your project!

Questions? Contact us.

Related topics

See also:

• Requests for comment/Global bits and pieces
• mw:Gadgets 2.0
• Wikimedia Scripts
• Global-Wiki
• User talk:Krinkle/Le Tour de Wikí/2011 Resource Walker#Automated script updates (see b:pt:User:Helder.wiki/Tools/jsUpdater.js and https://github.com/Krinkle/mw-gadgets-jsUpdater, which might be out of sync)
• User talk:Krinkle/Le Tour de Wikí/2011 Resource Walker#Global page history

Helder 13:42, 30 September 2013 (UTC)

Thanks for the links, and especially for "Le Tour de Wikí" links, jsUpdater and global page history which I didn't know before and seems to be very useful for this project. The project idea is actually very similar to "Le Tour de Wikí" (the first stages), though there it was manually reviewed. It seems according to log that not so many projects gadgets were reviewed, and here we want to focus more on static code analysis on the gadgets so we can cover all the projects [we plan to do static code analysis to classify possible gadgets that are relevant for VisualEditor]. If you have any idea for rules to look for common pitfalls in gadgets JS [in more sense of mw specific than jshint] we may try to add them :) eranroz (talk) 17:40, 30 September 2013 (UTC)

What comes to my mind are things like

• Use of protocol relative URLs instead of http, wherever it is possible
• Removal/migration of deprecated code (some of it is on jsUpdater.js, but I haven't touched that list for a while, so there is probably more to add...)
• Check if all dependencies of the scripts are declared (either at MediaWiki:Gadgets-definition or by means of mw.loader.using( ... ) calls)
• Make the gadgets more translateable by separating code from interface and using mediaWiki.message (this would help in a future transition to Gadgets 2.0)

Maybe Krinkle has some suggestions too? Helder 20:42, 1 October 2013 (UTC)

I see that Helder added a mention of gadget 2.0[1] but you didn't know about it. From your notification to wikitech-ambassador it sounded like you were trying to make a central repository like that, so it would be nice to clarify that this project has (apparently) no overlap with that one.

The first point has already been done a year ago by Hoo, though some wikis reintroduce problems (bugzilla:34670). Hoo also fixed other problems as he encountered them, a work not listed in the tour de wiki log. --Nemo 05:56, 21 October 2013 (UTC)

Feedback

I think auditing gadgets is a great idea. I'd really like to see us develop a system in which we can centralize gadgets (and Scribunto/Lua modules and ...), though I realize that this grant request is largely focused on the work that would precede such a centralization. The wonderful Helder covered most of what I would've pointed to; I'll only add w:en:Wikipedia:Database reports/User preferences/Configuration (the configuration page for the English Wikipedia report you've already discovered).

Soliciting feedback is wonderful, but I think it might be more helpful if you could ask more specific questions. For example, are there any parts of this grant request that you think will be particularly difficult or tedious to complete? Are there any concerns you have (re: data collection, cross-wiki analysis, etc.)? Is there any part of the current technical infrastructure/ecosystem that you're confused about or have questions about?

One of my concerns is that unless you have a Toolserver account or you can be deemed a researcher by the Wikimedia Foundation, I think you'll have difficulty getting gadget usage statistics. Gadget preferences are considered private data and are not available in the public dumps or at Wikimedia Labs. --MZMcBride (talk) 19:59, 3 October 2013 (UTC)

• I have access to the toolserver, and now we have Gadgets based on the data available there, which gives cross project statistics based on user preferences. I thought that the data available in the toolserver is available in Labs as well, but it seems that user_properties_anonym isn't there (although user_properties_anonym doesn't damage privacy) so I have concerns that such statistics can't be done in the long run. Another approach is to create such statistics based on projects and not user preferences - which may not give us real "popularity" measure but still some kind of metric for highly used gadgets.
• We have experience with gadgets and bots, but haven't yet dived into the VE source so much. So I'm not expecting problems or difficulties from the first stages of doing stats and publishing gadgets across projects, but maybe in the VE stuff and conversion of gadgets to support VE. I think we can overcome such difficulties, either by ourselves with enough diving into code or some help on #mediawiki channel in the IRC.

Thanks for the feedback, eranroz (talk) 16:44, 5 October 2013 (UTC)

Hi eranroz.

It's great to hear that you already have Toolserver access. The Toolserver, over time, created custom MySQL views that allowed for partial data release of private information. The user_properties_anonym table is really just a MySQL view operating on the user_properties table. Labs will likely eventually have similar access, however it may take a while (probably even longer now given the recent incident). To measure gadgets on a per-project basis, it seems like it'd be fairly trivial to scan the contents of MediaWiki:Gadgets-definition on every wiki, though that would only find exact title matches, not functionality matches, of course.

Re: VisualEditor, it shouldn't be too painful to plug in to it, I don't think. In addition to the #mediawiki IRC channel, there's also #mediawiki-visualeditor, which usually has helpful folks, particularly during San Francisco business hours (Monday through Friday, 9 a.m. to 6 p.m.). #mediawiki-scripts has also existed for years, but has very little activity unfortunately. (Maybe you can change that!) For what it's worth, just the other day I saw someone come on IRC from the Spanish Wikipedia and complain that most gadgets there weren't compatible with VisualEditor. I'm hopeful that the work you (all) have planned improves this situation. :-) --MZMcBride (talk) 01:51, 6 October 2013 (UTC)

Eligibility confirmed, round 2 2013

This Individual Engagement Grant proposal is under review!

We've confirmed your proposal is eligible for round 2 review. Please feel free to ask questions here on the talk page and make changes to your proposal as discussions continue during this community comments period.

The committee's formal review for round 2 begins on 23 October 2013, and grants will be announced in December. See the schedule for more details.

Questions? Contact us.

Siko (WMF) (talk) 05:51, 4 October 2013 (UTC)

Missing budget breakdown

Would this be the cost of development time? For how many people? (I understand that it seems yes, for two, but assuming is hard.) Gryllida 10:02, 13 October 2013 (UTC)

Yes, it is the cost of development time for 2 developers to work on a bot for static code analysis and migration of some of the most popular gadgets to VE. eranroz (talk) 18:28, 15 October 2013 (UTC)

Thanks for confirming, eranroz. Can you please update the budget breakdown section of your proposal to include this information (and explain whether the 2 developers are the 2 grantees listed on your proposal, and the amount would be split equally or?)?. This information will help the reviewers greatly. Siko (WMF) (talk) 23:43, 18 October 2013 (UTC)

Missing community notification

I bet there are lots of people who would be interested in this project. We'd like to see that you've notified some relevant community members about your idea, before we start scoring proposals next week. This can be as simple as posting on a wikitech mailing list, village pump, etc, with a link to your proposal and a request for input, and then pasting a link to that notification in the "community notifications" section of your proposal here. I'd really encourage you to do this before October 23. Let us know if you need further help. Thanks, Siko (WMF) (talk) 23:49, 18 October 2013 (UTC)

Consider also posting to https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors. Helder 21:51, 19 October 2013 (UTC)

It seems the proposal is similar to work routinely performed by some global interface editors, so it would be interesting to know from them too. --Nemo 05:57, 21 October 2013 (UTC)

Measurement of popularity and scope

Following are a couple random questions to encourage discussion and refinement. I'm not supporting or objecting anything in this proposal.

How do you intend to measure which gadgets are popular, and what amount of popularity do you plan to use as cut-off to decide whether upgrade/migrate/patch gadgets to use VisualEditor? Consider using database reports from Toolserver or Tool Labs to determine usage of a gadgets through user preferences.

From looking at "Project idea" table, I think this proposal sets a too wide scope. Either focus on gadgets landscape in general (to find broken gadgets and upgrade them all to ResouceLoader) or focus on rewriting gadgets to become VisualEditor plugins.

Also note that the range of authors, maintainers, code quality, language and structure varies significantly. Most don't have tests, so verifying a change can be difficult. A generic focus on gadgets seems very hard to do, if it is possible at all. It would require deep knowledge of javascript, MediaWiki and the Wikimedia community's habits. I also suspect it will take a significant amount of time to understand each one (I speak from experience). Gadgets copied to other wikis can be modified over time ("forked"). Moreover, it is not always feasible to contribute a patch as these are managed by local admins who may have different reservations for a particular gadget. One shouldn't just start changing things as gadgets, much more than articles, are often owned by a set of people who maintain it – for good reasons – which means it is best to ensure the change will be welcome first before setting out to refactor it. Besides, by reaching out first, they might want to do it themselves, thus better spreading the load and providing wider participation and awareness (win-win!).

As for the focus on VisualEditor in particular, contrary to the change from the classic editor to the WikiEditor in 2010, the upcoming change to VisualEditor is a much bigger change that goes much further than just the program interface to the wikitext document. The document is now not wikitext-based anymore, which means the conversion cannot (or should not) be automated. One has to interpret the intent of the wikitext operation and rewrite it to use the new document model instead. User interface components are also quite different.

Please don't be discouraged by this. For one, an abstraction library would probably go a long way. Also, I'd recommend staying in touch with the VisualEditor team and verify that an abstraction library is actually needed. Certain utilities may be more suited to become part of VisualEditor itself. –Krinkle^talk 23:51, 28 October 2013 (UTC)

As for queries, he said he has already done them: Gadgets. --Nemo 06:58, 29 October 2013 (UTC)

A small note

Please make the "bot to leave message on various village pumps regarding new versions of gadgets" opt-in. Gryllida 10:16, 30 October 2013 (UTC)

Aggregated feedback from the committee for Visual editor gadgets compatibility

Scoring criteria (see the rubric for background)	Score 1=weakest 5=strongest
Potential for impact
(A) The project fits with the Wikimedia movement's strategic priorities	4
(B) The project has the potential to lead to significant online impact.	4
(C) The impact of the project can be sustained after the grant ends.	3.5
(D) The project has potential to be scaled or adapted for other languages or projects.	3.5
Ability to execute
(E) The project has demonstrated interest from a community it aims to serve.	3
(F) The project can be completed as scoped within 6 months with the requested funds.	3.5
(G) The budget is reasonable and an efficient use of funds.	4
(H) The individual(s) proposing the project have the required skills and experience needed to complete it.	4
Fostering innovation and learning
(I) The project has innovative potential to add new strategies and knowledge for solving important issues in the movement.	3.5
(J) The risk involved in the project's size and approach is appropriately balanced with its potential gain in terms of impact.	4
(K) The proposed measures of success are useful for evaluating whether or not the project was successful.	4
(L) The project supports or grows the diversity of the Wikimedia movement.	3.5
Comments from the committee: Very useful idea to inventory existing tools and compare similar tools that Wikipedians rely on. Strong argument to build capacity for Visual editor integration. May help develop support for Visual Editor by making it more integrated with existing tools. Concerns about the difficulty of collecting information on individually maintained and differently implemented gadgets - this task may prove challenging for this project. Reasonable and moderate budget appropriate for the proposed tasks, but some concern about the scope of the plan being too broad for a 6-month project. Little community endorsement, perhaps because Visual Editor is still being gradually rolled out, but meaningful talk page discussion showed interest from developers.

Thank you for submitting this proposal. The committee is now deliberating based on these scoring results.

Funding decisions will be announced by December 16. — ΛΧΣ²¹ 00:21, 14 November 2013 (UTC)

Interest from wikisource

I - and I'm sure I'm not alone - am very interested about the VE-gadgets (and other local scripts) compatibility, as you know wikisource is a peculiar project among wikis, since other wikis can state some formatting standard & orthography while wikisource is forced to render different styles and different orthographies, and is faced with mistakes from OCR. This means that we need many tools, sometimes work-specific, surely language-specific. Proofreading time can be extremely shorted by appropriate, customized tools. They must be simple, easy to create and to put at work. They are needed both in first steps of proofreadin - to fix scannos - and in final retouch - to add "smart links" and other advanced formatting features. Personally I'm a tireless "gadgets creator", any of them coming from suggestions by personal proofreading.

How can I help you? --Alex brollo (talk) 13:36, 19 November 2013 (UTC)

According to our list in Gadgets/wikisource, it seems that the most popular wikisource specific gadget is TemplatePreloader. This may be a bit misleading since scripts loaded through Common.js aren't listed (aren't considered as gadgets), such as s:MediaWiki:OCR.js, but may give good estimation on the popularity of gadgets and which gadgets should get more attention in wikisource. However according to s:Special:Version the VisualEditor isn't currently installed on Wikisource, and until it is installed, it isn't very helpful to create "VE compatible" gadgets for popular gadgets on Wikisource. So in Wikisoruce context:

• VE should first deployed there. You can ask the VE team on #mediawiki-visualeditor (on IRC) or Jdforrester (WMF) what are the current plans for Wikisource. (The VE had many improvements since its first deployment and maybe it is ready for more projects)
• Meanwhile the currently used gadgets on Wikisouce can be re-factored. One important issue is to replace old deprecated methods such as addOnloadHook (should be replaced with $ e.g addOnloadHook(function(){....}) => $(function(){}). You can find what are deprecated method using ?debug=1 (for example: [2]), and open the console panel (F12 in most browsers). eranroz (talk) 23:10, 18 December 2013 (UTC)

@ערן and Alex brollo: VisualEditor is not yet deployed to Wikisources because it is not yet compatible with ProofreadPage, which is a necessary step before it can be remotely useful. I know that @Rtdwivedi: and @Tpt: are looking at what it would require to get this working, which is excellent. Jdforrester (WMF) (talk) 19:42, 19 December 2013 (UTC)

Note about navpopups

I made the start of a list at w:Wikipedia talk:Tools/Navigation popups#List of places Popups is used, which might be useful. I believe that's one of the most widely used Gadgets, across all languages/wikis. Quiddity (talk) 19:32, 17 December 2013 (UTC)

Thank you for the link. According to toolserver, it is the 3rd most popular gadget (Gadgets/wikipedia). BTW, do you know of any categories (in English Wikipedia or other Wikipedias), or other wikidata items that organize documentation pages for gadgets? (for example en:Category:Wikipedia scripts contains both documentation pages and code pages and isn't very useful for users. eranroz (talk) 23:10, 18 December 2013 (UTC)

I don't think there has ever been any standardization in gadget/script setup - some are documented in code-comments, some on the talkpage (either at the top, or scattered throughout), and some on a dedicated documentation page.

(The same problem exists throughout the Template: namespace. There's even a proposal right now on Enwiki to change that, but it's not doing well.). HTH. Quiddity (talk) 19:43, 3 January 2014 (UTC)

See also bugzilla:51651 and w:pt:Wikipédia Discussão:Ptwikis#JSDuck para Gadgets. Helder 10:45, 13 January 2014 (UTC)

Congratulations on funding

Congratulations on securing funding from the grants committee. This looks to be a great piece of work, and of course I (and the rest of the VisualEditor team) will be happy to help if you have any issues. The easiest way to get us is probably IRC (we're in #mediawiki-visualeditor amongst other places), but on-wiki works too if you prefer it. Jdforrester (WMF) (talk) 22:53, 17 December 2013 (UTC)

Thank you. eranroz (talk) 23:10, 18 December 2013 (UTC)

Advisor

Hi. I am willing to serve as your advisor through the next 6 months and provide help and support on anything you might need. Is that okay with you? — ΛΧΣ²¹ 22:51, 10 January 2014 (UTC)

Yes. Thanks, eranroz (talk) 19:54, 26 January 2014 (UTC)

Internationalization, localization to Spanish and improvements on the ProveIt gadget

Hi! Given the nature of this project, I figured that you may be interested in knowing that I've recently finished internationalizing, localizing to Spanish and generally improving the ProveIt gadget. My clone of the official repo is here, and I've already shared my work with the Spanish Wikipedia here. The next steps are gathering feedback, fixing bugs, doing a few enhancements, contacting the ProveIt team to merge the code, and localizing the gadget to other languages. Regards! --Felipe (talk) 16:07, 26 January 2014 (UTC)

Thanks for adding note here. ProveIt is very useful gadget and it is cool to hear it is now available also in eswiki. Consider to add it as gadget to es:MediaWiki:Gadgets-definition so users will be able to easily "install it" (if you use the same convention as enwiki, e.g es:MediaWiki:Gadget-ProveIt, it will be easier to understand it is a localization of the same script). eranroz (talk) 19:54, 26 January 2014 (UTC)

Once I deal with the most obvious bugs and do a few basic fixes, I'll add it to the list of default gadgets, though I don't have edit rights on it, I'll have to request it. I'll also follow the enwiki convention, of course! --Felipe (talk) 14:01, 27 January 2014 (UTC)

Default gadgets

Note that the statistics at Gadgets don't account for gadgets that are turned on by default for all users. For example, the Vietnamese wikis all have an input method turned on by default and the Vietnamese Wiktionary has various default gadgets for a new entry wizard, a dynamic table of contents while editing, and auto-linking the selected text (more info). Some of these gadgets are loaded by other gadgets instead of being turned on via MediaWiki:Gadgets-definition. Because no one has ever needed to turn these gadgets on, many of them don't appear at all in the statistics. – Minh Nguyễn (talk, contribs) 09:27, 23 April 2014 (UTC)

Minh Nguyễn, this is good point. We didn't add default gadgets to the list, as the "number of users" measure is unknown (we can add all the site users but than there is bias for default gadgets). We will create a seperate list of default gadgets. Thanks, eranroz (talk) 05:42, 24 April 2014 (UTC)

@Minh Nguyễn, we updated Gadgets page to include default gadgets. BTW, note that all the gadgets there link to "gadget page", which give more details about the gadget, including screenshort and installation instructions (Gadgets/Rechtschreibpruefung for example). Would be nice to see there more details about the Vietnamese gadgets (in Gadgets/linkify, Gadgets/start) - it seems they may be useful for other projects. thanks, eranroz (talk) 21:08, 28 April 2014 (UTC)

You have a new topic here ('Examples' section).

--Gryllida 23:59, 29 November 2016 (UTC)