Grants:Project/WikiProject X/CollaborationKit MVP

This project is funded by a Project Grant

statusselected

WikiProject X Continuation

summaryToolkit for WikiProjects on the English Wikipedia: Complete work for deployment of CollaborationKit in order to better to address live issues and assess results

targetEnglish Wikipedia

amount10,000 USD

grantee• Isarra

advisor• Bawolff

contact• Isarra

this project needs...

volunteer

give feedback

join

endorse

created on23:59, 31 January 2018 (UTC)

Friendly space expectations

This is a follow-up proposal to the IEG WikiProject X, intending to pick up the work on CollaborationKit, the prototype WikiProject extension, where the second round of WikiProject X left off, in order to complete deployment to and testing on the English Wikipedia.

Project idea

What is the problem you're trying to solve?

Initial definition: WikiProject limitations

From the original WikiProject X proposal: WikiProjects on the English Wikipedia are intended to foster collaboration on articles related to subject areas by providing resources and centralized discussion. There are a few very successful WikiProjects which have an active role on Wikipedia by building subject-area communities, establishing best practices, and providing active encouragement and feedback for those improving the encyclopedia. However, most WikiProjects fail to achieve the critical mass necessary to be of use to most editors, and so they fall apart. WikiProjects are a huge missed opportunity to develop these more formalized editing communities within the huge English Wikipedia community.

Follow up: Results of round one research

Findings and methodology are documented in the round one midpoint report.
Several issues with current WikiProjects came up in particular:
1. Unnecessary overhead of maintaining projects by hand (a major blocker for smaller projects with fewer members to split this between)
2. Difficult nature of starting new projects
3. Lack of discoverability of projects for potential new members
4. Lack of effectiveness of retaining members over time

Issues with round one prototype

For the second half of round one, we used a combination of gadgets, templates, and bots to create a framework for a more managed WikiProject experience.
While reception of the concept and design were generally positive, the effectiveness of the template prototype, Module:Load WikiProject Modules, suffered from similar limitations to those of existing wikitext WikiProjects:
1. Template calls proved even more difficult for regular users to edit than normal wikitext, and very easy to break by messing up the syntax
2. Too many pieces made it difficult to edit/maintain over time
3. Full project creation using prototype required manual creation of every object, as well as admin privileges to set up the member signup, and was never fully documented, and so required pretty much a single person (James Hare) to create any new ones by hand as well
4. Unreliable; some modules would simply break the entire hub for no apparent reason if loaded, such that no others would load after them either
The bot-generated worklists and FormWizard-based member signup proved more effective, but likewise had issues with the difficulty of the setup process and porting to new projects.
In the end, WikiProject Women in Red, our flagship pilot project, opted to quit using the onwiki template prototype entirely and we recreated their hub in simpler wikitext, though they retained and continue to use the member signup and bot-based worklist generation tools.

Incomplete status of round two prototype (CollaborationKit)

In round two, the renewal of the original WikiProject X grant, we decided to stop work on the bulk of the original prototype (the bots being the notable exception), and instead focus on creating a second prototype with a more centralised implementation that would not suffer from the issues and limitations of the on-wiki templates and gadgets: the MediaWiki extension CollaborationKit. This extension uses new JSON content models to store and display data for special Collaboration Hubs and Lists that are edited using guided forms, but is in an incomplete state, with some buggy workflows, various rendering issues, and a lack of some features such as metrics tracking and easy removal, making it a poor candidate for rolling out and testing until these are resolved.
CollaborationKit has been deployed to testwiki (WMF production), but due to the above blockers is not yet available on the target project, the English Wikipedia.
A full overview of what was accomplished and where things left off is documented in the renewal midpoint and final reports.

What is your solution to this problem?

We pick up where the previous grant (the renewal/round two) left off and complete a minimum viable product of the round two prototype, CollaborationKit, which seeks to address these issues by providing a better user experience through a more maintainable, secure framework than onwiki gadgets and wikitext templates, and providing forms with clearly labelled inputs for all the things users need to input.

From there, we will deploy it for testing on the English Wikipedia, iterate on the development, and assess the overall effectiveness of the product at addressing the discovered issues, creating learning patterns and results that should also help to guide similar projects down the road.

Project goals

The initial goal of WikiProject X was as follows: to address the causes of WikiProject failure on the English Wikipedia. By addressing these causes, we hope to increase the activity on WikiProjects and make WikiProjects more central to the experience of editing Wikipedia. This is based on the hypothesis that WikiProjects help facilitate Wikipedia-editing in a given subject area by organizing contributors around a cause and by providing resources and social support.

This goal remains unchanged.

Project impact

How will you know if you have met your goals?

For this project's outputs, we will measure success in terms of the following, which are further discussed in the Activities section:

CollaborationKit deployment
- Deployment and testing blocker tasks are resolved
- Consensus for deployment is established on the English Wikipedia and carried out
CollaborationKit testing success: that CollaborationKit is objectively better than the status quo
- Are users happy with it? Do they seem to want it around, want to try it out, and not want it removed from the project?
- Do WikiProjects that try it keep using it, or wind up converting their projects back to plain wikitext?
- Are WikiProjects that use CollaborationKit active/successful over time?
- Measure of success: that the number of active (as defined by the WikiProject Council) WikiProjects using CollaborationKit goes up over time.

Do you have any goals around participation or content?

Of the 3 shared metrics, it is uncertain how much will be measurable in practice, but this proposes to meet the of the following:

Total participants
- We can track this by number of participants creating WikiProjects and other workspaces as Collaboration Hubs (1) as well as number of members signing up on these hubs and/or actively participating using the tools provided (2).
- Targets: 5-10 active Hubs by the end of project; 100-500 involved members.
Number of newly registered users
- Directly impacting this metric was deemed out of scope for this project.
- Indirect cause of account creation would be an outcome as a result of user initiative, such as the use of CollaborationKit for facilitating editathons and other events.
Number of content pages created or improved, across all Wikimedia projects
- WikiProjects, and thus Collaboration Hubs, exist for the purpose of facilitating user collaboration creating and improving articles.
- The previous round of WikiProject X achieved 88 completed requests through Wikipedia Requests. If CollaborationKit, as another tool, can facilitate the creation and improvement of similar numbers of articles within a similar timeframe, this would indicate a measure of consistency across outputs of the overall project.

Project plan

Activities

To begin:

Set up and announce some way for people who want to make a test hub on testwiki but don't have admin permissions to just provide a list of needed things to start the process (change initial content model) such that they can then edit and test it properly
- (This will later most likely be converted into process for requesting admin help to create a new project onwiki)
Newsletter:
- Announce the above
- Start actually looking for pilot projects for after eventual deployment (recommend setting up clone on testwiki now so they know what they're getting into, per above?)
- Proper announcement of actual plans, predicted timeline, etc (outline of below?)

Deployment plan:

Complete full design and worlflow review
Run compatibility and functionality reassessment to account for the one-year gap in both development and maintenance
Triage and update existing tasks (phabricator:project/board/1778/)
Resolve existing blockers for deployment/testing (note that the main blockers for deployment have been resolved already, as it is currently in production - this is specifically for the English Wikipedia):
- Overhaul visual design to be more in line with MediaWiki core styles (mediawiki.skinning.interface) (phabricator:T192327) (t)
- Fix page not correctly updating on save issues (caching problem; Timeless has a similar issue as well?) (phabricator:T192328?) (t)
- Create export script (phabricator:T192326) and special page (phabricator:T192329) for reverting CollaboratonKit content model pages to regular wikitext/json (d/t)
- Set up automated metrics reports/logging for collaboration hubs (phabricator:T129251, phabricator:T192426) (t)
- Look into situation regarding more general rights for changing content models (blocker for non-admins creating/uncreating collaborationhubs/lists, but can be worked around) (t)
- Resolve any other major issues raised by above review and triage that would cause issues with initial deployment as well as general testing effectiveness (such as UX stumbling blocks) (d/t)
- Collaborate with Release Engineering to determine and address any other blockers (d)
Establish on-wiki consensus for deployment
- Success: Deploy to English Wikipedia (phabricator:T123028)
  or:
- Failure: Address issues raised by lack of consensus; iterate
- Total failure (no consensus is likely to be possible, technical blockers or Wikimedia policy prevent it, etc): End project there, thoroughly document results up to this point, create report(s) with findings and recommendations based on everything thus far, including previous WPX grant rounds, and publish for use with any future attempts at similar/related things.
(This is unlikely. CollaborationKit is already deployed to production, has passed review, and only requires a config change for further deployments to specific projects. If content can be easily converted back to wikitext if people try it and don't like it (phabricator:T192329) and it can also be completely undeployed without losing anything (phabricator:T192326) if it comes to it, this should address the most likely concerns that would block consensus for trying it out.)

Followup (after deployment):

Help set up and maintain pilot projects (requires admin/editinterface rights to change content models; clearly outline what's needed so other admins can also do it)
Fix technical bugs that arise, or file upstream, depending on the nature of the bugs
Resolve major workflow and UX issues that arise, or at very least properly document them
Track and evaluate general effectiveness of CollaborationKit in practice

End project:

Thoroughly document results, create report(s) with findings and recommendations based on findings throughout the WPX grants progression, and publish for use with similar/related projects. Example: a scalable extension implementing more structured interfaces for facilitating inter-user communication.

Budget

This grant request is for the allocation of up to 20hrs/week over the course of 3-6 months up to 250 hours total for me to complete the MVP for, deploy, test, and iterate upon the extension CollaborationKit in order to effectively assess it in a production environment. Using a standard rate of 30-50 USD/hour and assuming accuracy of the 181 hours calculated below, that comes out to 7,240 USD total. Given the inherent unpredictability of software development as new issues arise and the potential for other expenses (hardware, travel for outreach purposes, etc), we will round up to 10,000 USD (200-250 hours total).

Estimated time breakdown:


Item	count	time per	total hours
Design and development
Feature development	4	10 hours	40
Interface and workflow design	3	4 hours	12
Task triage	50	2 minutes	1.67
Bugfixes	20	4 hours	80
Outreach and coordination
Pilot project coordination			20
Hub creation and facilitation			5
Announcements/newsletter			6
Reports and documentation			16
Total
Base hours			181
Cost		7,240 USD
Rounded hours			250
Total cost after rounding		10,000 USD

Notes:

This is a stipend, per previous rounds, but now calculated based on existing metrics and tasks on the project workboards, with a specific target as a technical project.
The reason for the part-time approach is to account for delays in inter-team collaboration, necessary reviews, and complications that inevitably arise in the deployment process, as well as the time required for changes and announcements to reach users once deployed, and user responses to trickle back after actual use. A six-month scope also allows for more meaningful data to be acquired regarding buy-in and practicality of the extension over time, as initial responses and impressions may be misleading (per what we discovered with Women In Red).
Resolving a task (implementing a fix or feature) can be expected to take about 3-5 hours on average. Some will be one-liners. Some are simply horrible.
Likely design tasks include the general overhaul and new interfaces for the metrics reports, among other things. These can take anything from a few minutes (where most of the design work is done while implementing it in the code itself) to several hours wireframing, prototyping, and tracking workflows, depending on the nature of the problem.
There is some overlap in terms of bugs, backend concerns, and general progression of the project with the Timeless proposal being submitted concurrently, as both concern frontend content-rendering, have caching problems, and involve deployment and testing across Wikimedia projects. There is also overlap in terms of timeline with the Timeless proposal. As both are part-time projects, they should not interfere with each other, and if anything doing both concurrently should be beneficial to both, as there are likely to be fixes, processes, and practices that apply to both, and thus I will be able to go through these much more quickly on the second after having already done it once on the first.

Community engagement

I will maintain existing WPX channels: newsletter, hub, and discussion pages. Progression requires an on-wiki consensus to deploy CollaborationKit to the project itself, so I will be working closely with potential pilot projects to establish needs and expectations on the user end, and as testing begins, implementing their feedback, as this will be imperative to effectively develop and assess the extension from there.

I will also be setting up spaces for requests and better directions for how to request/set up test projects on testwiki prior to English Wikipedia deployment (my initial instructions in the first newsletter were somewhat unclear, and I was also unaware at the time that I had global rights to do change content models myself), and subsequently for creating CollaborationHubs for WikiProjects onwiki once it is, and will ensure that all requests are properly carried out.

Get involved

Participants

Isarra is the designer and one of the developers for WikiProject X. As a user experience designer, she is experienced in designing MediaWiki interfaces with a focus on extension frontends and skins, and has also been a MediaWiki developer for ~6 years.
Bawolff is advising in a volunteer way. I am committing to doing code review of any code that results from this project and also providing advice about MediaWiki as needed. I am a long term (~8 years) MediaWiki developer, and have +2 rights on MediaWiki and all extensions, and was one of the original developers of CollaborationKit as part of round 2.
James Hare, who began WikiProject X and managed the previous two rounds, is unfortunately no longer able to commit to the project as he now works for the WMF as part of the Cloud Services team, but will continue to advise as he is able.

Community notification

Notified wikitech-l and Wikimedia-l
A WikiProject X newsletter was sent out with updates on the project in general, and news of this proposal
Added a note on the previous grant talkpage

Endorsements

Do you think this project should be selected for a Project Grant? Please add your name and rationale for endorsing this project below! (Other constructive feedback is welcome on the discussion page).

Makes sense to complete all the work that's already been put in. Bawolff (talk) 18:33, 14 February 2018 (UTC)
Support: WikiProject Women in Red has benefited from Project X and is keen to support its further development.--Ipigott (talk) 11:53, 15 February 2018 (UTC)
Has the potential to engage new editors and work together on a common goal (often improving the quality of articles with high-traffic). Daylen (talk) 19:53, 15 February 2018 (UTC)
There is a lot of potential in improving WikiProjects, and it would be great to complete the work which was already started here. the wub "?!" 00:14, 20 February 2018 (UTC)
Support - Women in Red has been with WikiProject X since Day1. It has developed our mainpage and several other modules, keeping our look fresh. Together, we've gone through several learning-curves. I strongly endorse continued efforts. --Rosiestep (talk) 15:45, 22 February 2018 (UTC)
Meh While I think the idea behind Project X might be a good one, to automate some of the content and reduce the administrative burden of outreach, I really dislike the look of the implementation of this project. It puts form over content, and makes things really hard to read. I used to be able to go to the Women in Red project space and find information, and feel like the information was clearly and concisely laid out. No longer. Now there are barriers to information everywhere. Maybe I have just gotten used to plain Wiki presentation and this automated look is akin to the WMF "snazzy" reportage where it's about slick design and lack of simple displays of information. So unless that is addressed, I don't support this project. I understand that there has been a lot of work done here, but the outcome / end-use implementation isn't an improvement to communicating information. Until that improves I don't endorse its further implementation. -- Erika aka BrillLyle (talk) 03:11, 27 February 2018 (UTC)
Support - Very interested in using for Wikipedia: WikiProject Women's sport as well as task forces such as Wikipedia: WikiProject Football/Women's football task force. Hmlarson (talk) 03:25, 27 February 2018 (UTC)
yes Slowking4 (talk) 23:58, 11 March 2018 (UTC)
Great project and initiative in improving collaboration on Wikipedia. Full support. – Craig Davison (talk) 11:38, 5 April 2018 (UTC)
Support Gerat. Web SourceContent Management System 08:30, 28 July 2018 (UTC)