Research:FlaggedRevisions investigation

Created

15:28, 28 February 2025 (UTC)

Contact

Claudia Lo

Wikimedia Foundation

Collaborators

Tara Rooks

Wikimedia Foundation

Duration:  2025-February – 2025-March

---

Research:Projects

This page documents a completed research project.

This work is tracked in Phabricator: task T381044

Findings and results are available in the Report subpage.

The Flagged Revisions extension (FlaggedRevs) is used in nearly 50 Wikimedia projects. It has two primary featuresets, patrolling tools and, more controversially, limiting visibility of new edits from readers until an experienced editor has reviewed the edit. This tool therefore drastically changes the “normal” workflow of edit patrolling, from a post-publication process to a pre-publication process.

Flagged Revisions was developed 15 years ago as a new ‘milestone’ in how Wikimedia projects could review and moderate incoming edits. Framed as a way to balance the open nature of editing Wikimedia projects with the need to ensure that readers see good content, the extension was made available to any Wikimedia project to enable and configure.

However, deployment of Flagged Revs has unofficially stalled since 2014, and officially halted since 2017. It has not been deployed anywhere since, but neither has it been undeployed, except on a few wikis where communities decided to stop using it.

The following 10 years have left FlaggedRevs in limbo. On many of the projects on which it is deployed it is the definitive way that editors patrol new edits, and many community members believe that hiding edits from readers before review is a positive feature. However the software is unmaintained by any WMF team, and there is no clear plan or consensus on whether it should continue to be used or deployed, or more broadly whether pre-publication edit review has positive or negative consequences for Wikimedia projects.

Methods

We conducted interviews and workflow observation of FlaggedRevisions users (those with the editor user right, generally part of patroller or reviewer user groups) on five target wikis:

• German Wikipedia, which use the overwrite mode of FlaggedRevs (where the last stable revision is the default shown for all page)
• English Wikipedia, which uses the protect mode of FlaggedRevs (as Pending Changes, where the last stable revision is the default shown for specified pages only)
• Finnish and Ukrainian Wikipedia, which have FlaggedRevs installed but have set neither override or protect mode (where the most current revision is the default shown for all pages)

We also contacted users from Polish Wikipedia for this study, but were ultimately unable to schedule interviews with Polish Wikipedia edit reviewers.

Timeline

We conducted user interviews during the month of March 2025, with final reports available at the end of March.

Policy, Ethics and Human Subjects Research

All interviews are accompanied by a privacy statement. This study follows the Wikimedia Foundation's data retention guidelines and data publication guidelines.

Results

Please view our Results page for the full report. The key findings have been reproduced below.

This report details the risks associated with FlaggedRevs’s lack of maintenance, grouped by the three core features most useful to patrollers, as well as potential areas where product development may benefit from applying lessons from how FlaggedRevs is currently used.

Page Stabilization

Many editors worry the overall credibility of Wikipedia - specifically their home wiki - would decline without page stabilization.

Recommendations:

• Expand watchlist capabilities to include “types” of articles, e.g. colors, dates, years to speed up fast edit reviews to overlooked article types
• Provide further information and guidance on when and where to set pages in Semi-Protect mode

Unexpiring queue

Relying on RecentChanges filters alone would dramatically shorten the patrolling window.

Recommendations:

• Make it easier to recreate the FlaggedRevs queue in RecentChanges with a premade filter
• Expand watchlist capabilities to include “topics”, allowing known experts or Wikiprojects groups to add the relevant “topics” to their watchlists

Codebase for automation bots

Vandalism would be more difficult to control without access to FlaggedRevs features, used by bots.

Recommendations:

• Build new automated anti-vandalism tooling that both
  • Rejects high probability “bad” edits to decrease the need for human review
  • Approves high probability “good” edits to prioritize human review on more ambiguous edits