Grants:Project/DannyS712/Create a global watchlist extension/Timeline

This project is funded by a Project Grant

Timeline for DannyS712

Timeline	Date
Deploy extension to beta cluster (phab:T268268)	November 18, 2020
Pass security review (phab:T260466)	January 5, 2021
Pass performance review (phab:T260860)	January 12, 2021
Deploy extension to testwiki (phab:T260862)	February 09, 2021
Deploy extension to metawiki (phab:T260862)	February 17, 2021

Monthly updates

Please prepare a brief project update each month, in a format of your choice, to share progress and learnings with the community along the way. Submit the link below as you complete each update.

June

I found someone to help with code reviews, as required in the grant conditions (see Grants talk:Project/DannyS712/Create a global watchlist extension#Round 1 2020 decision) but the WMF has yet to assign the technical advisor called for in those conditions.

Started the extension - set up the repo, registered with CI, added to translatewiki, created mediawiki.org documentation page, etc. Implemented a frame extension that registered the special pages and the client side javascript modules, though the modules only wrote to the console that they were loaded successfully.

Wrote the starting php backend for saving user settings to a user's global.js (for now) and the base javascript for the Special:GlobalWatchlistSettings page, including saving settings via a custom api module. Later added validation with errors to that api module, to warn on invalid settings being chosen. The current front end for the settings page should be fully functional.

Wrote the initial version of the javascript for Special:GlobalWatchlist (converted from the global userscript), ended the month awaiting code review for the patch.

Progress towards meeting grant requirements noted at Grants talk:Project/DannyS712/Create a global watchlist extension#Round 1 2020 decision:

"Find a second person to assist with code review as a best practice." Done
User:Jack Phoenix has agreed to assist

July

Both of the special pages are now fully functional, and both remain implemented in javascript. At some point after deployment, Special:GlobalWatchlistSettings will likely be converted to an HTMLForm implemented in PHP. This is being delayed until after deployment to simplify the translation for users of the user script to the extension by enabling use of javascript hooks to allow scripts to define a user's settings if they have none saved. See phab:T258770 for details.

I have begun migrating the javascript for Special:GlobalWatchlist to use HTML templating where possible. This makes the code clearer by including the underlying HTML structure rather than creating everything with jQuery. This began with adding a template to display when a site has no changes to show on the watchlist:

<li class="globalWatchlist-emptyWatchlist">
	<a href="{{{special-watchlist-url}}}" target="_blank">{{{site-name}}}</a>
	{{! Ensure there is no whitespace between the parentheses and the link }}
	(<a href="{{{special-edit-watchlist-url}}}" target="_blank">{{{edit-watchlist-msg}}}</a>)
</li>

For now the templating is done with mustache, but it will eventually migrate to Vue in line with phab:T241180. The eventual migration will be at phab:T259216, and I've started some preliminary work on it.

I've also done some work simplifying and splitting up the javascript frontend for Special:GlobalWatchlist. As a user script, it wasn't minified, so I sought to ensure that the code was as compact as possible. That is no longer a concern, since ResourceLoader includes automatic minification. Thus, code can be clarified and documented without significantly increasing the bandwidth cost. The code was migrated to use mw:ResourceLoader/Package modules, which also made it easier to add qunit testing for individual parts independently.

Progress towards meeting grant requirements noted at Grants talk:Project/DannyS712/Create a global watchlist extension#Round 1 2020 decision:

"Include tracking/metric count measuring how many people are using the script at some set time interval." Done
Extension now uses an injected IBufferingStatsdDataFactory to track adoption. Currently tracked are:
1. globalwatchlist.settings.new - user saves settings, previously had none
2. globalwatchlist.settings.change - user saves settings, previously had settings set
3. globalwatchlist.load_special_page - user loads Special:GlobalWatchlist
"Find a second person to assist with code review as a best practice." Already done
"Wikis used should be all opt-in (eg. by default you would only see your Meta watchlist). The current userscript has 4 default wikis and that's too many." Done
Current default in getSettings.js is to only include the local watchlist if the user has no options set (or if they have invalid options set). Extracted via mw.config.get( 'wgServer' ).replace( /.*?\/\//, )

August

Following-up on the preliminary work to migrate to Vue, there is now a fully-fledged Vue display that can be enabled instead of the one built with jQuery. It doesn't look as nice for now, and its use is controlled via $wgGlobalWatchlistUseVue, which by default is false. Part of the reason it doesn't look as polished is that the buttons used in the jQuery/non-Vue version are standard OOUI widgets, while the buttons in the Vue version are minimally styled <button> elements. A library for shared components is currently being created for use by MediaWiki developers, and once that is available it will be used.

At this point, the extension is, from a code perspective, essentially ready for deployment. I have filed the security review (T260466) and performance review (T260860) tasks. The overall task for deploying the extension to metawiki is at T260862.

Progress towards meeting grant requirements noted at Grants talk:Project/DannyS712/Create a global watchlist extension#Round 1 2020 decision:

"Extension must go through proper security and performance review" In progress…
Tasked filed for both reviews, see above
"Include tracking/metric count measuring how many people are using the script at some set time interval." Already done
"Find a second person to assist with code review as a best practice." Already done
"Wikis used should be all opt-in (eg. by default you would only see your Meta watchlist). The current userscript has 4 default wikis and that's too many." Already done

September

During September I made less progress since I was waiting on the results of the security review to move forward with deployment. I

Added proper handling for API failures when fetching the watchlist
Added performance metrics for timing the initial display on Special:GlobalWatchlist, in preparation for the performance review

I also, since I had some time

Added a GuidedTour for Special:GlobalWatchlistSettings. The tour is only shown if the user has no settings currently saved. There is a minor bug with the tour being displayed again after the form is submitted, which will be addressed soon (see phab:T264341)
Switched the JavaScript documentation to us w:JSDoc
Added tracking of manual changes to global watchlist settings via the options API

I have continued testing the GlobalWatchlist, and testing it is now easier thanks to the work at phab:T263934, which enabled using the GlobalWatchlist extension with the Patch Demo tool. I have filed a few tasks for adding some functionality that is currently lacking, though the features are not critical and do not block moving forward.

This month I was also assigned my technical advisors from the WMF: User:NKohli (WMF) is the primary advisor, and User:MusikAnimal (WMF) is serving as the secondary technical advisor. They, along with User:Mooeypoo, clarified the grant requirements regarding limiting the number of wikis, and there is a patch pending at phab:T258935 to add a new configuration option, $wgGlobalWatchlistSiteLimit, to limit the number of sites that a user can include in their global watchlist. By default, the limit will be 5, though this can be configured differently for the WMF deployment (and the default can also be changed).

Progress towards meeting grant requirements noted at Grants talk:Project/DannyS712/Create a global watchlist extension#Round 1 2020 decision:

"A technical advisor should be assigned to help throughout the life of the project (deployment, consideration of scalability, performance, etc.)" Done
User:NKohli (WMF) is the primary advisor, and User:MusikAnimal (WMF) is serving as a secondary technical advisor
"The extension should only be deployed on a limited number of wikis (not all wikis)." In progress…
This was clarified to mean that the extension, which will only be deployed on meta, should have a limit on the number of wikis that a user can include in their global watchlist, and a patch is pending to add this limit.
"Extension must go through proper security and performance review" In progress…
The security review at phab:T260466 is still in progress

The performance review at phab:T260860 is stalled pending deployment of the extension to the beta cluster (which cannot occur until after the security review is concluded)
"Include tracking/metric count measuring how many people are using the script at some set time interval." Already done
"Find a second person to assist with code review as a best practice." Already done
"Wikis used should be all opt-in (eg. by default you would only see your Meta watchlist). The current userscript has 4 default wikis and that's too many." Already done

October

Very little work done since the security review hasn't been completed yet. I did address phab:T264833 by hiding the guided tour behind a feature flag, which is off by default, so that it isn't enabled for WMF production.

Progress towards meeting grant requirements noted at Grants talk:Project/DannyS712/Create a global watchlist extension#Round 1 2020 decision:

"A technical advisor should be assigned to help throughout the life of the project (deployment, consideration of scalability, performance, etc.)" Already done
"The extension should only be deployed on a limited number of wikis (not all wikis)." In progress…
Patch pending to add this limit, awaiting review.
"Extension must go through proper security and performance review" In progress…
The security review at phab:T260466 is still in progress

The performance review at phab:T260860 is stalled pending deployment of the extension to the beta cluster (which cannot occur until after the security review is concluded)
"Include tracking/metric count measuring how many people are using the script at some set time interval." Already done
"Find a second person to assist with code review as a best practice." Already done
"Wikis used should be all opt-in (eg. by default you would only see your Meta watchlist). The current userscript has 4 default wikis and that's too many." Already done

November

The security review hasn't been completed yet. But, the extension was deployed to the beta cluster after AEzell (WMF) agreed to be responsible for the risk of doing so. Once it was deployed to beta, the performance review could begin, and GDubuc (WMF), who is doing the performance review, has already given some preliminary feedback that I began addressing.

Progress towards meeting grant requirements noted at Grants talk:Project/DannyS712/Create a global watchlist extension#Round 1 2020 decision:

"A technical advisor should be assigned to help throughout the life of the project (deployment, consideration of scalability, performance, etc.)" Already done
"The extension should only be deployed on a limited number of wikis (not all wikis)." Done
Patch to add this limit was merged, see phab:T258935. The default limit is 5.
"Extension must go through proper security and performance review" In progress…
The security review at phab:T260466 is still in progress. The latest update from JCross (WMF) is that the review should be done "by the end of the quarter" (phab:T260466#6649374), so I may need to request an extension on the grant timeline in order to be able to reach the deployment and post-deployment support.

The performance review at phab:T260860 has started, and I've begun addressing the issues raised.
"Include tracking/metric count measuring how many people are using the script at some set time interval." Already done
"Allow time for post-deployment support" Waiting
As noted above, I may need to get an extension given the delay in the initial deployment relative to the originally-scheduled end for the grant.
"Find a second person to assist with code review as a best practice." Already done
"Wikis used should be all opt-in (eg. by default you would only see your Meta watchlist). The current userscript has 4 default wikis and that's too many." Already done

December

Both the security and performance review are still in progress. The extension was available to test on the beta cluster, where it was deployed in November. During December I spent some time addressing the issues raised in the performance review so far. I requested and recieved an extension on the overall grant timeline, given that the security review is taking longer than expected.

Progress towards meeting grant requirements noted at Grants talk:Project/DannyS712/Create a global watchlist extension#Round 1 2020 decision:

"A technical advisor should be assigned to help throughout the life of the project (deployment, consideration of scalability, performance, etc.)" Already done
"The extension should only be deployed on a limited number of wikis (not all wikis)." Already done
"Extension must go through proper security and performance review" In progress…
The security review at phab:T260466 is still in progress. The latest update from JCross (WMF) is that the review should be done "by the end of the quarter" (phab:T260466#6649374).

The performance review at phab:T260860 is still in progress.
"Include tracking/metric count measuring how many people are using the script at some set time interval." Already done
"Allow time for post-deployment support" Waiting
As noted above, I requested an extension on the overall grant timeline.
"Find a second person to assist with code review as a best practice." Already done
"Wikis used should be all opt-in (eg. by default you would only see your Meta watchlist). The current userscript has 4 default wikis and that's too many." Already done

January

This month, the security and performance reviews were both finished, unblocking the extension for deployment. After getting a few last changes in before it goes live, I plan to have the extension deployed early in February.

Progress towards meeting grant requirements noted at Grants talk:Project/DannyS712/Create a global watchlist extension#Round 1 2020 decision:

"A technical advisor should be assigned to help throughout the life of the project (deployment, consideration of scalability, performance, etc.)" Already done
"The extension should only be deployed on a limited number of wikis (not all wikis)." Already done
"Extension must go through proper security and performance review" Done
The security review at phab:T260466 was finished. Done

The performance review at phab:T260860 was finished. Done
"Include tracking/metric count measuring how many people are using the script at some set time interval." Already done
"Allow time for post-deployment support" Waiting
As noted above, I requested an extension on the overall grant timeline. The extension should be deployed in early February.
"Find a second person to assist with code review as a best practice." Already done
"The extension should only be available on Meta." In progress…
Extension is ready for deployment
"Wikis used should be all opt-in (eg. by default you would only see your Meta watchlist). The current userscript has 4 default wikis and that's too many." Already done

February

The extension was deployed to Meta on February 17. I continued to work on the extension until I finished the 250 hours of work that the grant funded, after that it became less of a priority.

Progress towards meeting grant requirements noted at Grants talk:Project/DannyS712/Create a global watchlist extension#Round 1 2020 decision:

"A technical advisor should be assigned to help throughout the life of the project (deployment, consideration of scalability, performance, etc.)" Already done
"The extension should only be deployed on a limited number of wikis (not all wikis)." Already done
"Extension must go through proper security and performance review" Already done
"Include tracking/metric count measuring how many people are using the script at some set time interval." Already done
Worked on creating grafana dashboard to show the counts, see https://grafana.wikimedia.org/d/ZIvCK9EMz/globalwatchlist?orgId=1
"Allow time for post-deployment support" Done
Until the 250 hours of work funded by the grant are done, I continued to work on the extension. After that, it was less of a priority.
"Find a second person to assist with code review as a best practice." Already done
"The extension should only be available on Meta." Done
Per feedback from system administrators, it was also enabled on testwiki, which appears to be standard practice, but the use on testwiki is not being advertised and should not conflict with the use on meta.
"Wikis used should be all opt-in (eg. by default you would only see your Meta watchlist). The current userscript has 4 default wikis and that's too many." Already done

Is your final report due but you need more time?

Extension request

New end date

February 2020.

Rationale

The security review for the extension is taking longer than expected, and is only estimated to be done some point this quarter. Until the security review and performance review are completed, the extension cannot be deployed the WMF prod. See phab:T260466 for more.

Thanks, --DannyS712 (talk) 23:15, 7 December 2020 (UTC)

Extension Approved

This request is approved, with a new end date of February 28, 2021, and Final Report due March 30, 2021. --Marti (WMF) (talk) 21:29, 10 December 2020 (UTC)