Community Wishlist Survey/Staff instructions

This page documents how to setup and administer a Community Wishlist Survey. Technical and Movement Communications (MoveComms) instructions are aimed at Community Tech staff. Some tasks may require administrator or translation admin access, while some of the technical tasks require access to Toolforge.

In this documentation, we refer to the page title of the new survey as Community Wishlist Survey NEW. The NEW should be replaced with appropriate year.

Phases

Testing phase

1. First setup all the templates (see #Templates for descriptions on what these templates are for)
   1. Create the Schedule template by copying Community Wishlist Survey 2022/Schedule to e.g. Community Wishlist Survey NEW/Schedule and change only the dates (months and days).
   2. Create the Phase template at Community Wishlist Survey NEW/Phase and set the content to testing
   3. Create the edit notice by copying {{Editnotices/Group/Community Wishlist Survey 2022}} to e.g. Template:Editnotices/Group/Community Wishlist Survey NEW and editing as required. The proposal category names used here are explained below.
2. Create the main survey page by copying [[Community Wishlist Survey 2022]] to e.g. [[Community Wishlist Survey NEW]] and edit as desired. Most of this page can be customized for the individual survey. Keep as much as possible the same, to reduce the burden on translators. Do not mark for translation yet.
3. Create the MediaWiki category pages (not to be confused with proposal categories) by copying the content from Category:Community Wishlist Survey 2022 and adjust accordingly.
4. Create the Untranslated page at Community Wishlist Survey NEW/Untranslated and add the content These proposals need translation. Once translated, the Wishlist team will move them to the appropriate category. (or whatever you want it to say...).
5. Update the year, dates and/or language in the messages for each AbuseFilter
   1. MediaWiki:Abusefilter-warning-wishlist-invalid-move
   2. MediaWiki:Abusefilter-warning-wishlist-invalid-location
6. Enable all AbuseFilters, updating the year in the filter description and code.
7. Change the bot configuration accordingly.
8. Mark the main survey page for translation. Note that because of how Extension:Translate works, at least one translation unit (a new message) must be present. So far this has never been a problem as you would normally want to add some year-specific content.
9. Run the import_translations script (WishlistSurvey bot) so refactored translations show up in the <languages> bar.

MoveComms

1. Create a communication plan for the CWS as a project.
2. Review the FAQ and Prioritization pages. Check if everything is up to date and reflects the current team agreements. Update documentation if needed, mark for translation, inform the translators.
3. Inform all communities about the incoming CWS.

Proposal phase

Tech

1. Run the add_category_pages script (WishlistSurvey bot) to create all the blank category pages.
2. Turn on the SurveyCop bot on Toolforge:
   1. become community-tech-tools
   2. kubectl create -f /data/project/community-tech-tools/SurveyCop/deployment.yaml
   3. If the bot was still running from last year, kill it with kubectl delete deployment community-tech-tools.bot and recreate the job.
   4. It’s possible the bot will stop working somewhere during the survey. If it does, repeat the above steps to kill the existing job and create a new one.
3. Enable the cronjob for the WishlistSurvey bot:
   1. become community-tech-tools
   2. become musikbot
   3. Edit the jobs.yaml file and uncomment the section for wishlist-survey
   4. Reload the jobs configuration with toolforge-jobs load jobs.yaml
4. Test out the automation. Create a test proposal and verify the bot transcluded the subpage properly, the number of proposals was updated, and that the Tracking page has the newly created proposal.
5. Once you’re ready to actually open up the survey, change the Phase template and the "phase" value in the bot config to proposal.
6. Update the year (and categories, if applicable), and enable Special:AbuseFilter/145, 146, 147 and 186.

MoveComms

• Set up the banners about the proposal phase.
  • (Optional) Decide about the (1) traffic limit and the (2) number of impressions. This is optional because we have pretty good settings from the previous years
    1. What likelihood for the banners showing up you want [recommended: 100%; 50% as a minimum] and
    2. What number of times the banners would show up per user you want [recommended: 5; no fewer than 2, no more than 5]
  • [requires CentralNotice admin permissions] Set up the banners
    • Reuse the "Community Wishlist Survey 2023" and "Community Wishlist Survey 2023 Vote" campaigns and the banners
    • Update the landing page link (otherwise, it'd point at the latest CWS edition)
  • Add a CentralNotice calendar entry
  • Document the decision about the banners on the Asana Movement Communications board
• [requires translation admin permissions] Make sure that all the fully added proper translations are marked as "published" (keep in mind that there may be vandalism there)
  • During the proposal phase, particularly in the first hours and days, check these banners every now and again to see if there are any updates to the existing translations or new translations. Mark these as published.

Everyone

HELP:

Monitor proposals as they come in, cleaning up any formatting errors. Look for proposals on the Untranslated page and handle them accordingly. Get an early start on the review phase and ask proposers for clarification as needed. You can monitor the log of Special:AbuseFilter/146 to find proposals that were likely malformed. Frequently monitor the talk page and respond to any inquiries.

Review phase

This phase is known internally as The Grand Undertaking™.

Tech

1. Change the Phase template to review.
2. Enable Special:AbuseFilter/148.

MoveComms

Consider the following criteria and score 0/1:

• Not only for Wikipedia - this is a DEI-related criterion met if the proposal is Wikimedia project-agnostic, like the ones in the "Anti-harassment" category. The reasoning is: most active users are Wikipedians, and most CWS participants are Wikipedians. They might outvote anyone else with Wikipedia-specific issues. Autosave edited or new unpublished article is an example of a proposal scoring 1.
• Sister projects and smaller wikis - this is a DEI-related criterion met if the proposal is not about the needs of the "typical" ~top 20 Wikipedia communities, but about sister projects, incl. Commons and Wikidata, or smaller Wikipedias. The reasoning why we consider this criterion as separate from the above is: it's not enough not to make it easier for Wikipedians to outvote others. That would still be equality. We strive for equity, so we actively support the weak/under-supported ones. Tool that reviews new uploads for potential copyright violations is an example of a proposal scoring 1.
• Supports critical productive groups - this is a DEI-related criterion met if the proposal is about stewards, CheckUsers, admins, and similar groups serving and technically supporting the broader community. The reasoning is: if to you, this looks contrary to the above, you're wrong :) Do you remember that graphic with the entire system built upon a small piece? This is why need to additionally support group doing thankless jobs. Show recent block history for IPs and ranges is an example of a proposal scoring 1.
• Lowers barrier for entry - this is a strategy-related criterion met if the proposal is about communication or helps making the first contributions. The reasoning is: the threshold is too high, and making Wikimedia projects more welcoming, intuitive, easy to use, easy to collaborate on, etc., is a strategic thing. Show editnotices on mobile is an example of a proposal scoring 1.
• Readers (actually: content consumers, viewers, listeners, readers, etc.) impacted - this is a strategy-related criterion met if delivering the wish would change the consumers' experience. The reasoning is: most proposals may be focused on contributors. But just as in the case of smaller wikis, content consumers are under-represented in the considerations about software development. Select preview image is an example of a proposal scoring 1.
• Non-textual content & structured data - this is a strategy-related criterion met if the proposal is related to multimedia, graphs, etc. The reasoning is: most Wikimedia content is textual. Strategically, we need to support other kinds of content and those who add/curate these. Mass uploader is an example of a proposal scoring 1.
• UX urgency - this is like a special category. Community Tech is a go-to team with perennial bugs; there are also recurring proposals, and sometimes, people ask for relatively easy changes which would make contributing particularly smoother. Sometimes, the designer or PM argues that delivering a particular wish would be a game-changer. Fix search and replace in the Page namespace editor is an example of a proposal scoring 1.

This particular set of criteria was created in the 2021 edition and improved in 2022. The strategic criteria should be pretty future-proof because strategic challenges change slowly. However, if you believe there is a strategic change, do talk about that with the team, and consider changing the criteria accordingly.

Everyone (together!)

HELP:

Review every single proposal, evaluate feasibility, and decide whether it should go into voting or not. Proposals not fit for voting should be moved to the Archive. This involves simply moving the page and changing the proposal category part of the title to Archive. Special:AbuseFilter/149 prohibits non-staff from moving proposals in or out of the archive, so be sure to use your WMF account. Once a proposal is deemed "final" and suited for voting, set the proposal up for translation.

Voting phase

Tech

1. Run the add-voting-sections script of the WishlistSurvey bot:
   1. become community-tech-tools
   2. become musikbot
   3. toolforge-jobs run cws --command ./cws-voting-sections.sh --image ruby2.7
   4. You can monitor Special:Contributions/Community Tech bot for progress.
2. Disable Special:AbuseFilter/147.
3. Change the Phase template to voting.
4. Set the "phase" to voting in the bot config.

MoveComms

• Set up the banners about the voting phase.

Closed phase

1. Change the phase template to closed.
2. Change the "phase" value in the bot config to closed.
3. Turn off the bots:
   1. become community-tech-tools
   2. kubectl delete deployment community-tech-tools.bot
   3. become musikbot
   4. Edit the jobs.yaml file and comment out the lines for wishlist-survey
   5. Delete the WishlistSurvey job with toolforge-jobs delete wishlist-survey
4. Enable Special:AbuseFilter/147.
5. Disable all other filters (except 147).
6. Run MusikBot's --sock-check script to generate a report about new users (in terms of registration date and edit count), evaluate if they may be legitimate votes or sockpuppetry and remove the votes accordingly.
7. If you'd like, you can run the bot's main task once more to see the updated tracking page (using the musikbot account on Toolforge): toolforge-jobs run cws --command ./cws-main.sh --image ruby2.7
8. If you wish, you can get a head start on the Results page, which often includes year-specific content. It's safe to proceed to step #1 of the Complete phase below, but don't change the Phase template until you're ready to announce the results.

Complete phase

1. Run the results task of the bot, which will create all the Results pages and redirect the Tracking pages to them: toolforge-jobs run cws --command ./cws-results.sh --image ruby2.7 You can monitor Special:Contributions/Community Tech bot for progress. Note that it may take a minute or so to collect all the data. Also note that this task is safe to run during the closed phase, if desired. Any changes to the /Results page should be made after this script has been ran.
2. Add any additional content as desired to the /Results page, then mark it for translation.
3. Once you're ready to announce the results, change the Phase template to complete.
4. Celebrate!
5. Some weeks later, make sure to disable Special:AbuseFilter/147.

Analytics

The bot offers a few analytical tools that you might find interesting. Unless otherwise specified, the output will be written to cws.out in MusikBot's home directory, so you may want to run truncate --size 0 cws.out before running any tasks. Additionally, as with all tasks, these go by the current survey_root value specified in the config, so if you want data on a previous year, run the task before setting up a new survey.

Using the same methods to run the bot as described in the phases above, you have the following tasks at your disposal:

• Get a list of all participants: toolforge-jobs run cws --command ./cws-get-participants.sh --image ruby2.7
• Get a tab-separated list of all participants, along with their home wiki, registration date, global edit count, as well as edit counts to Commons and Wikidata (since users there often have higher edit counts due to use of automated tools): toolforge-jobs run cws --command ./cws-analyze-participants.sh --image ruby2.7 The output in cws.out can be copied and pasted into spreadsheet software, or into a wiki table using VisualEditor.
• Get the total number of words and characters throughout all category and proposal pages, including the Archive: toolforge-jobs run cws --command ./cws-prose-stats.sh --image ruby2.7

CWS Manager gadget

To assist you in common tasks, a gadget is installed on Meta that is available for use by all staff. To enable, go to your gadget preferences and enable "CWS Manager: Used by staff to manage proposals in the Community Wishlist Survey." under the "Editing gadgets" section.

After enabling, you should see a "Manage" button in the heading of each unapproved proposal. This buttons open a dialog that allows you to do several tasks on the proposal.

Renaming or moving proposals

In the Community Wishlist Survey, all proposals live as a subpage of the category page, and the category page is a subpage of the home page for that year's survey. The subpage title is the same as the proposal title, so to rename it, you move the page. A bot will handle (un-)transcluding the proposal from the corresponding category pages. If you attempt to move to an invalid category, Special:AbuseFilter/145 should throw a warning message.

This is made even easier with the CWS Manager gadget. Simply use the "Manage" button to open up the CWS Manager dialog, then select "Rename proposal or move to a different category."

Archiving proposals

To archive a proposal is to say it is not fit for voting in the survey. This can happen for any number of reasons, and ultimately comes down to the judgement of whoever is reviewing the proposal. The FAQ goes into detail on our inclusion criteria for proposals, as well as reasons a proposal might be archived.

Archived proposals live on the /Archive subpage of the main survey page. Only WMF staff should move proposals in or out of the archive. Archiving involves leaving clear rationale as to why the proposal was archived, then changing the category portion of the title to Archive. This can be done easily using the CWS Manager gadget, which also gives you a dropdown of common criteria used. The list of decline reasons can be changed by editing the bot configuration.

Unarchiving proposals

Sometimes you may archive a proposal and later the proposer addresses the concerns. In this case you can use CWS Manager to unarchive it. This will automatically remove the archive rationale from the proposal, then move it to the specified category.

Note that if there may be a redirect in the old location, which in some cases may have to be deleted before the proposal can be moved back. If this happens, you should get an error when trying to move the page using the gadget. You will have to ask an admin to delete the old page first.

Approving and preparing proposals for translation

Some instructions here require translation admin rights

To "approve" a proposal is to say that it is ready for voting and does need any further modifications to the proposal content (the discussion section may continue be edited freely). Approving proposals usually happens during Grand Undertaking (a.k.a. "Review" phase), but if a proposal is obviously suitable for the survey, any staff may approve it before the Review phase actually starts.

Approving proposals involves moving the proposal content to a /Proposal subpage using the {{Community Wishlist Survey/Proposal}} template, and setting the page up for translation. You can do this easily using the CWS Manager gadget.

After you approve a proposal using the gadget, a new tab is opened with the /Proposal subpage. From here you should add any <tvar> syntax, as deemed necessary, and then actually mark the page for translation (requires translation-admin rights). Note that for technical reasons, the <translate> tags are visible to readers until the page is marked for translation, so it's important to act quickly. If you are not a translation admin, please ping one on Slack or elsewhere to get it quickly marked for translation.

Instructions for translation admins

• For very long proposals, please try to take some time to make them into multiple <translate> tags so that it can more easily be translated.
• Use <tvar> where possible. For instance [[page title|foobar]] should be [[<tavar name=1>page title</tvar>|foobar]] so that translators don't accidentally translate the page title.
• CWS Manager should automatically uncheck the "Allow translation of page title" option seen towards the top of the Special:PageTranslation interface. This is intentional. The box should not be checked, as there's a dedicated message with the proposal title.
• After submitting, you should see a notification of whether or not the proposal was added to our message group. If it wasn't, you can manually add it at Special:AggregateGroups. The group should be called "Community Wishlist Survey [year] Proposals".

Managing proposals after they've been approved

There is currently no way to unapprove a proposal. This requires deleting translation subpages, so if needed please seek assistance from a Meta administrator who also has translation admin rights.

If you want to rename a proposal after it has already been approved, you will need assistance from a translation admin. Additionally, due to caveats with the Translate extension, there may be a delay up to a few minutes before changes take effect. For this reason, it is advised to not rename already-approved proposals unless absolutely necessary, particularly during the voting phase when there is higher traffic.

To rename an already-approved proposal, follow these steps:

1. First browse to the /Proposal subpage and attempt to move it. If there are too many translations, the Translate extension will disallow this action, in which case you are out of luck and the only solution is to delete the proposal entirely along with all translations, then recreate them under the new title.
2. Assuming the move was successfully queued (the Translate extension won't move them instantly), use Special:MovePage to move the parent proposal page to the desired location, ensuring it exactly matches the name chosen above in step #1.
3. The bot should have automatically update the proposal header template, but in case it didn't, edit the proposal page and manually change the 1= parameter in the Proposal header template on the first line.
4. Things should look correct now. If they don't, refer to Special:Log/pagetranslation for errors. If something went wrong, you may need to start over. Seek assistance from other translation admins if necessary.

Note that CWS Manager unfortunately cannot automate this process.

Untranslated proposals

When viewing the main page in non-English, you are shown a form to create a non-English proposal. Using this, the preload template is shown in their language. After saving, the proposal is transcluded by the bot at Community Wishlist Survey NEW/Untranslated.

From there, we must translate it, including the labels. E.g. the “Phabricator tickets” label should be in English, so that the bot can parse it. Similarly, the syntax should match the expected format exactly. Each field should look like * '''[field name]''': [value].

After translating, move the page to the appropriate category, giving it an English title. The bot should automatically update the proposal header template so that it matches the new English title.

Proposal categorization

The intention behind proposal categories (not to be confused with MediaWiki categories) is to make it easier for participants to navigate the survey and find wishes that are relevant to them. A proposal can only belong to a single category. This is mainly for technical reasons, but also to prevent unfair visibility of proposals during the voting phase.

Overpopulated categories

When a category page contains too many proposals, or too much content in general, it can become slow to load. Editing tools like the voting gadget and DiscussionTools may also fail to function. For this reason, it's important to move proposals from overpopulated categories to less populated categories where the proposal still logically fits. For example, the Miscellaneous category often becomes too large. Often you may see things broadly related to multimedia, so it may be more appropriate for the Multimedia and Commons category.

Changing categories

Sometimes you may want to introduce a new category. First, note that category names cannot contain slashes.

When changing categories, it is critical to keep these pages up-to-date:

• Community Wishlist Survey NEW/Proposals
• Template:Editnotices/Group/Community Wishlist Survey NEW – in the {{#switch}} statement.
• Special:AbuseFilter/145 – Filter that detects if a user is trying to move a proposal to an invalid category. Note that this filter should also include “Untranslated” as if it were a category (even though it technically is not).
• Special:AbuseFilter/148 – Filter that detects if a user is trying to create a proposal in an invalid category. Should also include “Untranslated” as if it were a category.
• User:Community Tech bot/WishlistSurvey/config – bot configuration. This should NOT include “Untranslated”.
• At the top of each category page, please also make sure the arguments to the category header template are correct. The first argument is the name of the previous category, the second is the name of the next category. Re-running the add_category_pages script should also work, but if the survey has already started, you need to manually update the pages.

Bots and automation

Instructions here require Toolforge access

There are bots, and one gadget:

• WishlistSurvey (tools.musikbot, written in Ruby) – runs every 10 minutes and rotates proposals, along with counting the number of contributors and edits to each proposal. It will also update the tracking page.
• SurveyCop (tools.community-tech-tools, written in Node) – uses EventStreams to look for edits to the survey in real time. It will automatically transclude newly created proposals as soon as they are created. It also keeps the Proposal header template up-to-date with the name of the proposal (which is the page title). Lastly, this bot keeps the proposal count templates up-to-date.
• AddMe – the voting gadget, also used for other purposes across Meta. No changes should need to be made to this during the survey.
• CWS Manager – the gadget used by staff to manage proposals (docs)

Templates

Here is a list of the templates used throughout the survey pages, for reference. All are subpages of Template:Community Wishlist Survey, unless otherwise noted.

• Community Wishlist Survey NEW/Phase – master switch for what phase we’re in. This affects the language and appearance of many of the other templates. Must be one of: testing (before proposal phase), proposal, review, voting, closed (after voting ends) or complete (after results are published).
• {{Community Wishlist Survey/Category button}} – button shown on the main page that links to the given category. It will also show the number of proposals in that category.
• {{Community Wishlist Survey/Category header}} – template that is transcluded at the top of every category page. This shows:
  • The name of the category
  • Number of proposals, contributors and voters (the latter only during the voting phase).
  • A form to create a new proposal.
  • Navigation between categories. For this, the first argument passed to the {{Community Wishlist Survey/Category header}} template is the name of the previous category, the second is the name of the next category.
• {{Community Wishlist Survey/Create proposal}} – form that is shown at the bottom of the Category header template, used to create a new proposal.
• {{Community Wishlist Survey/Create proposal-non-english}} – non-English variant of {{Community Wishlist Survey/Create proposal}}. Submissions are placed on the /Untranslated page.
• {{Community Wishlist Survey/Preload}} – preload template that is supplied when creating a new proposal.
• {{Community Wishlist Survey/Proposal header}} – proposal header that should be transcluded at the top of every proposal page. This shows a “Back” button when viewing the proposal directly, or shows a level 2 section heading when viewing the category page. The template is automatically added by the Preload template (see above), but sometimes users remove it. Please monitor Special:AbuseFilter/146 to identify these mistakes, and add the necessary code. Refer to other correctly submitted proposals to see what the code should look like.
• {{Community Wishlist Survey/Random proposal button}} – Button that when clicked shows a random proposal.
• Template:Editnotices/Group/Community Wishlist Survey NEW – Edit notice shown when editing any page of the survey. There is a {{#switch}} statement in here so that we can selectively show a message when the user edits a category page, asking them not to change it since it is completely managed by the bot. This “switch” statement relies on each category page being listed, please update if the categories change. A new edit notice must be created each year. Simply copy the one from the previous year, and update the parameters for the edit notice template accordingly.
• {{Community Wishlist Survey/Edit notice}} – used in each year’s edit notice.
• {{Community Wishlist Survey/Top}} – deprecated; still shown in older surveys

Automated counts

The bot keeps track of various counts for categories and proposals. Each lives on its own page which gets transcluded.

• Community Wishlist Survey NEW/Editor_counts/Category_name – the number of edits to all proposals for the given category.
• Community Wishlist Survey NEW/Proposal_counts/Category_name – the number of proposals for the given category.
• Community Wishlist Survey NEW/Vote_counts/Category_name – the number of votes across all proposals for the given category.
• Community Wishlist Survey NEW/Tracking – the sortable table of all proposals, including individual vote counts.
  • Community Wishlist Survey NEW/Tracking/Heading – To customize what is shown at the top. The bot will override any manual changes to the tracking page itself.
• And for the totals across all the categories:
  • Community Wishlist Survey NEW/Total_editors
  • Community Wishlist Survey NEW/Total_proposals
  • Community Wishlist Survey NEW/Total_votes

AbuseFilters

Instructions here require administrator rights and knowledge of AbuseFilter

Note that the messages shown by these filters might change conditionally based on the phase of the survey. If you're trying to make copy edits, try editing the message page to see where the changes need to be made as it may look like the wrong place when simply viewing it.

• Special:AbuseFilter/145 – Disallows moving or creating proposals in an invalid category.
  • Shows MediaWiki:Abusefilter-warning-wishlist-invalid-location
• Special:AbuseFilter/146 – Tracks users who created malformed proposals, such as missing the Proposal header template.
• Special:AbuseFilter/147 – Disallows addition of {{support}} or other voting templates until the voting phase has begun.
  • Shows MediaWiki:Abusefilter-warning-wishlist-invalid-voting which automatically shows the appropriate text based on the current phase.
• Special:AbuseFilter/148 – Disallows creating proposals outside the proposal phase.
  • Shows MediaWiki:Abusefilter-warning-wishlist-no-proposals which automatically shows the appropriate text based on the current phase.
• Special:AbuseFilter/149 – Logs non-staff moving proposals to or from the Archive page.
  • Shows MediaWiki:Abusefilter-warning-wishlist-unarchive
• Special:AbuseFilter/186 – Detects manual renames of proposals. Renames should only happen with Special:MovePage. The bot will update everything else.
  • Shows MediaWiki:Abusefilter-warning-wishlist-manual-rename

Caveats

This section explains the pitfalls that exist with the Wishlist Survey system. These issues are merely documented here for completeness. You as a Survey maintainer should not normally have to be concerned with them.

• Proposals names cannot end with /Proposal or /[lang] where [lang] is a valid language code. This is because the bot recognizes /Proposal and /en, /fr, etc. as translation pages, which should not get transcluded on category pages. Fortunately the likelihood of this happening by pure circumstance is quite low.
• Proposals can only belong to a single category. The system is hard-wired with this assumption. Any attempts to bypass it will be overridden by the bot. There is also the advantage that limiting proposals to one category will prevent them from having too much visibility during the voting phase.
• Proposals can only have one proposer. This is solely because we've never added support for it, which may change in the future.
• Proposals that have been approved (or more specifically, marked for translation) cannot be renamed easily. See #Managing proposals after they've been approved.

Glossary

Proposal subpage (or translation subpage)

The /Proposal subpage of the proposal that is used for translation.

Proposal page

The same page where the Discussion section is (not the /Proposal page, if it exists).

Proposal category

Topics a proposal can be placed into.

MediaWiki category

See mw:Help:Categories.

Untranslated proposal

A proposal submitted in a language other than English, which appear on the /Untranslated page.