Wikimedia Foundation Annual Plan/2023-2024/Reports/Core Metrics Q2

Q2 Core Metrics Report

Quarterly metrics are intended to measure progress toward the Foundation’s Annual Plan goals. Reviewing and publishing these core metrics throughout the year will allow us to evaluate which work streams and interventions have been most impactful and make data-driven decisions about what to prioritize through the remainder of the year. These reports will also allow the movement to track the progress we make with the many initiatives we’re running. The current report summarizes Core Metrics for Q2. The report includes comparisons to Q2 last year, as well as progress towards Annual Plan year-end targets.

What is Q2? Quarter 2 (Q2) refers to October, November, and December

What is FY 23-24? Fiscal Year (FY) 23-24 refers to July 2023 through June 2024.

What are Core Metrics? Core Metrics are small, well-defined set of measurements used to guide strategic decisions.^[1] As part of this year's Annual Plan, four Core Metric areas were developed: "Contributors", "Content", "Relevance", and "Effectiveness".

During Q2, we made progress toward the Contributors and Effectiveness metrics, detailed below. We have not yet begun to do the work that will feed the Content and Relevance metrics–those metrics remain at baseline. We expect to see progress on these through interventions in the coming quarters.

Contributors

While we develop a more comprehensive metric to measure the health of our contributors, we aim to improve and maintain the health of the Movement's communities of contributors by delivering at least one significant intervention per quarter that supports editors with extended rights.

Status: On track

In Q2, the Editing, Structured Data, and Apps teams improved three workflows for production users: Edit Check, UploadWizard, iOS Watchlist.

Edit Check improvements
- What? This set of improvements for the visual editor helps new volunteers understand and follow some of the policies and guidelines necessary to make constructive changes to Wikipedia projects.
- Why? This intervention will help decrease the amount of time editors with extended rights spend reverting low-quality edits and posting messages about policies and guidelines, by surfacing relevant guidance when inexperienced volunteers are in the midst of making a change to Wikipedia, equipping inexperienced editors to make high-quality changes.
UploadWizard improvements
- What? These design improvements to the Wikimedia Commons UploadWizard make it clearer to new users how to provide the right information when uploading new media, reducing the possibility of uploading material that would be flagged for deletion.
- Why? This intervention will help decrease the burden for volunteers with extended rights, by minimizing the possibility of uploading media that might trigger a deletion request
iOS Watchlist improvements:
- What ? This improvement brings the Watchlist feature to the Wikipedia app for iOS, enabling users to maintain a list of articles they're interested in, making it easy to stay updated on their changes and edits.
- Why? This intervention will help volunteers with extended rights mitigate vandalism on Wikipedia more quickly by having access to the Watchlist feature on their mobile device.

Content

We aim to increase quality and reliability of encyclopedic content by increasing the percentage of Wikipedia articles that are in high-impact topic areas (beginning with gender and geography) and meet a shared quality standard.

Status: Work impacting this metric has not yet begun

In Q2 the Wikimedia Foundation's Campaigns, Community Growth, and Growth teams worked on 6 projects that supported community-led initiatives related to geography and/or gender:

Event discovery: the event discovery tool aims to help editors discover community-led events on the wikis, such as those related to high-impact topics such as geography and gender.
Event registration: the event registration tool aims to help editors register for community-led events on the wikis, such as those related to high-impact topics such as geography and gender.
Community spotlight: this Homepage module aims to offer editors content translation tasks and help bring editors’ attention to content gaps on high-impact topics such as geography and gender.
Homepage content translation: this Homepage module aims to help communities highlight important community events and initiatives, including those related to high-impact topics such as geography and gender.
Women’s Health Project support: the Community Growth team is supporting the Women's Health Project, a community-led project that aims to map knowledge gaps related to women’s health on Wikipedia and curate resources that are useful to support the movement’s efforts in bridging these content gaps.
Celebrate Women campaign support and Wikipedia Needs More Women: the Community Growth team began providing campaigns support to a community project focused on adding content to close the gender gap on Wikipedia and other projects. Comms is also profiling the gender gap and funnelling newcomers into the community campaign through media targeting in Anglophone African countries and India.

Because the projects above are still in the stages of development and planning, the Wikimedia Foundation has not yet delivered programs or products to impact the Content metric; so the changes showcased below are related initiatives not led by the Foundation. Understanding fluctuations in the content area of geography will be critical when we report on our progress in upcoming quarters, as we will need to take into account how ongoing trends, external factors, and community-led initiatives are playing a role.

We also note that changes to articles about places in overrepresented regions can cause spikes and drops in these metrics. For example, in Q2, bot activity altering citations in a set of articles about France caused the percentage of new quality articles about overrepresented regions to be lower, resulting in a higher percentage of new quality articles about underrepresented regions.

Do you have ideas about community activities or external factors that might be impacting the Content metric? Let us know on the talk page!

Geography

In Q2, 37% of new quality articles about geographic content were about underrepresented regions.

37% is a 5 percentage point increase from Q2 last year.
Not all regional content had an increase. View breakdowns by region.
Not all Wikipedia editions had an increase. View breakdowns by Wikipedia language edition.
Product interventions focused on improving contributor experiences are planned to launch throughout Q3 and Q4. These interventions are targeting a +1% impact on this geographic content metric by the end of the 2023-2024 fiscal year.

Which regions are "underrepresented"? The following regions are classified as underrepresented within the Content metric area: East, Southeast Asia, and Pacific; Latin America and the Caribbean; Middle East and North Africa; South Asia; and Sub-Saharan Africa. Content about these five regions is underrepresented on Wikipedia when compared to global population distributions.

Gender

As part of our annual plan goals around gender equity, we aim to create "more inclusive, gender-equitable and safe spaces to contribute intersectional content on women's biographies and material that women want to read." This fiscal year, we will support, and report on the outcomes of, two community-led initiatives aimed at closing gender gaps in article content.

In Q2, the Community Growth team began work supporting supporting two community-led programmatic areas: one in Women's Health (Women’s Health Project) and the other in Biographies (Celebrate Women campaign and Wikipedia Needs More Women).

Relevance

We aim to ensure our relevance and sustainability to a broad audience world-wide by increasing the number of unique devices accessing Wikipedia. Unique devices refer to the number of distinct electronic devices (e.g. mobile phones, desktop computers) that visited, in this case, Wikipedia sites. Learn more about unique devices here.

Status: Work impacting this metric has not yet begun

In Q2, the Data Center Operations team continued work on a project to build out a new data center in São Paolo, Brazil. Deployment of this new data center is scheduled for the beginning of Q4 and is on-track to complete by the end of this FY. The intended impact of this new data center is a decrease in latency for users in the region, and an increase in readership in South America. From previous research, we have an estimation of impact to relevance metric: +5.1M unique devices in the LATAM+C region (0.3% increase to global unique devices).

Because the new data center is still in the development stages, the Wikimedia Foundation has not yet delivered products or programs that would impact the Relevance metric; so the increase in unique devices showcased below are related to initiatives not led by the Foundation. We will be doing more work to understand how we can grow unique devices moving forward, including an intervention next quarter. Do you have ideas about community activities or external factors that might be impacting the Relevance metric? Let us know on the talk page!

In Q2, Wikipedia was accessed by 1.66 billion unique devices per month.

1.66 billion is an increase of approximately 40 million monthly unique devices compared to Q2 last year.
Monthly unique devices increased in some Wikipedia editions, and decreased in others. View breakdowns by Wikipedia language edition here or visit Wiki Stats to explore language-edition trends further!
Monthly unique devices increased in some regions, and decreased in others. View breakdowns by region here.
1.66 billion is approximately 60 million above our Annual Plan year-end target of 1.60 billion unique devices per month. We expect fluctuation of this number to occur between now and the end of the fiscal year, and will continue to monitor our progress towards our year-end target in Q3 and Q4.

Effectiveness

We aim to ensure our long-term sustainability by improving how the Foundation operates and scales. One way we aim to do this is by increasing our programmatic expense ratio, which is the proportion of budget that funds work that directly supports our mission.

Status: On track

Last FY, 76% of our budget went to programmatic expenses. We set a target of increasing that number by 1 percentage point this year, which represents a substantial financial increase towards programmatic work to support our mission.

Based on Q2 data, we are currently on track to reach our 77% target by the end of Q4.

References

↑ Research and Decision Science/Data glossary

[1] Research and Decision Science/Data glossary

[1]