Wikimedia Foundation Report, January-March 2015

See also the original PDF version of this report
and (for online presentation) the Google Slides version




Wikimedia Foundation

Quarterly Report

January — March 2015
Q3 of the fiscal year

Foreword

edit

We are pleased to bring you the WMF Quarterly Report for Q3 of 2014/15. This is the second report since switching from a monthly cycle, to align with our quarterly goal setting process, and contains several changes that focus more on results and achievements.

One major change is that goals are counted as either a success (green) or a miss (red). This report is now a comprehensive overview of all the Q3 objectives rather than a subset of the quarter’s goals, as we delivered in the Q2 report. In a mature 90 day goal setting process the “sweet spot” is for about 75% of goals to be a success. Organizations that are meeting 100% of their goals are not typically setting aggressive goals.

We have also been able to reduce the amount of time and effort that goes into producing the report by flowing information straight through from the Quarterly Review Meetings.

In the report you will also find a couple of new pieces of data. The overview slide shows the status of all 130 goals from Q3 on a single page, broken down by organization. Additionally we have been able to include site speed metrics as part of our overall report card.

Terry Gilbey, Chief Operating Officer

Quarterly metrics scorecard (beta)

edit
         Participation
Sign-ups 1,507k -1.4% from Q2
+53.3% y-o-y
New editors 16.5k/mo +6.9% from Q2
+1.3% y-o-y
Active editors
(5+ edits/month)
80.9k/mo +6.8% from Q2
+5.5% y-o-y
          Site reliability
Read uptime
(Eng. WP main page)
99.99% 100% in Q2
100% in Q3 2013/14
Read latency
Median first paint time
1.4 seconds N/A
Write latency
Median page save time
2.0 seconds N/A
         Readership
Page Views
Crawlers excluded
18.5B/mo +2.4% from Q2
+4.0% y-o-y
Visitors Per-project unique visitors
to come later in 2015
comScore desktop UVs
(deprecated):

449M/mo in Q3
          Content
New articles 8.0k/day +6.1% from Q2
+7.2% y-o-y
Edits (in WP articles) 9.49M/mo -1.7% from Q2
-4.0% y-o-y
         Fundraising
Amount raised $9.9M
(exceeded $3M target)
Q2: $44M
Q3 2013/14: $6.1M
On mobile 21% Q2: 16%

Q3 2014/15: Successes/misses by team

edit
 

Q3 2014/15 objectives

  • 130 objectives
  • 32 teams
 
Successes: 52%, Misses: 48%

Misses include anything not delivered as planned, including “yellow” goals and those that were intentionally revised or abandoned due to changing priorities.
Objective Impact on Goal Status
Top notch, quick legal advice and support on wide host of issues (e.g. 7/day turnaround of 85% contracts; fundraising legal review; litigation mgmt; trademark permissions; successful Board meeting; privacy protocols; product counseling) Support of WMF mission by eliminating legal barriers and slow downs, by offering solutions to support users, and by stewarding values of WMF and community, including more knowledge. Done (approx. 75% of capacity)
  • 33 core workflows managed well (see Appendix A for metrics and details)
  • 90% of contracts reviewed/negotiated/approved w/i 7 days
  • legal@: 100% turn-around w/i 3 days
  • No significant adverse litigation judgments
  • Strong pushback against censorship (though transparency report 3 weeks late)
  • Strong focus on trademark registration and defense
  • Significant HR support
  • Solid Board support
Success
  1. 33 Core Workflows. We delivered on 33 core legal workflows - including contracts, trademarks, litigation & threats, privacy & transparency, governance, technology, fundraising, internal content review, public policy, grants & FDC, HR, and C-level support - despite resource challenges. We filled an open legal counsel position with an exceptional candidate.
  1. NSA lawsuit.  We filed a complex lawsuit against NSA to fight for our community's values and to support our mission, resulting in positive global support.  Details may be found here.
  1. Mobile trademark enforcement.  We put a strong focus on mobile trademark enforcement by sending notices for apps that infringe Wikimedia marks.  We are taking a supportive approach in an effort to encourage proper use of our trademarks.  The positive result: large proportion of developers started to address infringements.  We intend to continue these enforcement efforts as part of our ongoing trademark protection strategy.
Miss

Transparency report (Core).  We released our transparency report late by three weeks, though the report showed strong pushback on censorship requests.  This delay was due to our failure to anticipate competing pressures on dependent internal partners with resource challenges.

Learning -  We need to streamline and simplify long-term, recurring commitments - like the 6-month transparency report - to ensure on-time delivery.

Action - Sensitive to others' capacity limitations, we must better manage expectations, obtain clearer buy-ins, and minimize high dependency when possible.

Appendix
Legal Scorecard Q3 QoQ (Q3/Q2) YoY
Legal@ Requests 149 23% ↓  (149/195) TBD
Contract Requests 93 3% ↑ (93/90) 31% ↑ (93/71)
Trademark Permission Requests 43 13% ↑ (43/38) 14% ↓ (43/50)

For more detail, see the slide deck and minutes from the quarterly review meeting of the Legal team

Objective Impact on Goal Status
Purchase Order Implementation. Standardized process for purchasing at WMF to incorporate the purchase of servers along with all other contract processes. In progress.  Delay in project, will be added to Q4 goals as it will not be completed in Q3.
5th Floor Project Update and prepare 5th floor for occupancy Completed.
Office Wiki Makeover Provide easier access to information for staff.  Staff input on what is needed in Office Wiki. Survey at the end of the project.  Test and verify one click access to most requested data. Will be completed by April 30.
Metrics Report Implementation Report for annual and quarterly metrics for decision making and progress mapping of WMF. Project assigned to Terry with support from Garfield.
Success

5th Floor project was completed by the end of March.  The project was completed on time due to good front end planning by Lynette, good vendor management and keeping all tasks on time for completion.

Miss

Purchase Order Implementation was delayed to Q4.  Project failed to move forward as planned due to a limitation in the software which was not identified in the planning process.  Software was unable to give us segmented approvals by department and dollar value.  New software solution (Concur) was identified for this project and we are resuming implementation in Q4.  Lesson learned is to verify twice, all key requirements to ensure software is able to implement current process or we are comfortable with the needed changes to our process.

Office Wiki Project was delayed to Q4.  Is scheduled to be completed by April 30, 2015. Delay in project due to extra time taken for review by other departments.  Need to build into project plan additional time for review by other departments. Also delayed due to policies on the Office Wiki which protect user privacy.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Finance & Admin team

Objective Impact on Goal Status
Recruit key hires Hire VP of Strategic Partnerships

Hire Director of Engineering

Hire Legal Counsel

Kourosh hired 3/30

Wes hired 3/30

Victoria accept 3/27 (4/20 start)

Zhou accept 4/9 (4/27 start)

Rebuild Recruiting Team Hire Director of Recruiting

Hire Technical Sourcer

Hire Recruiting Coordinator

Interviewing for Director

Carla hired as Sourcer 2/9

Maria hired as Coord 1/26

Compensation Plan Update Approve Y path comp grid for more options between SME and Mgr paths

Approve Radford comp matching method

 
International Payroll & Benefits Pilot enrolled for April international payroll

Complete rollout meetings to Managers

Complete rollout meetings to Employees

Contract review by Legal

5 person pilot group set for April

Manager meeting 3/6

Employee meeting 3/10-11

Contracts reviewed by counsel

Objective Impact on Goal Status
ADP Upgrade Upgrade ADP to current version, to allow for performance review and analytic modules for Q4

Create interim org chart

 
All Hands Review & publish All Hands survey data:

staff satisfied with event and feel connected

Published 2/18 (60% participated)
Learning & Development C2A Community related strategy support

Call to Action

Support organizational changes & transitions (Community Engagement, Wikipedia Zero)

 


Success

Recruiting is making progress, with 12 req# hires in FQ3. This includes attending career fairs at Stanford and CCA for employer branding & sourcing technical candidates.

International contractor solution is launched with 5 staff in the pilot group for April, with 15 agreed for May & 2 for June.  73% of eligible staff confirmed so far (22 of 30). Work continues in Q4.  

All Hands was the biggest yet with 210+ attendees.

Benefits has been working closely with the Admin team on a Safety Program. They have already created more clarity on the Worker’s Comp process and staff options. In Q4, they are planning to roll out more.  Benefits has also researched an ACA compliance vendor in Q3 and is getting in place for Q4.

Miss

The FQ3 date for the compensation approvals was missed by a day, due to changes from the unexpected announcement of Gayle’s departure.

The ADP upgrade was pushed back due to ADP’s ongoing issues with customer service. Once the upgrade is done, if the software is still lacking, then we’d recommend getting a new HRIS & Payroll system. Garfield & Joady are doing a preliminary meeting with Workday on 4/30.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Talent & Culture team

Objective Impact on Goal Status`
Staffing: Fill Jr Comms Mgr position; Develop WP0 marketing, internship program Staff blog and core writing functions; develop WP0 regional marketing strategy. - Movement Comms Manager - Fabrice Florin

- JCM intern (contract) - Sam Lien

- WP0 marketing volunteer - Amy Bebbington

- 6 volunteers/interns; Recruiting 6-8 summer

Reporting: State of the Wiki report, parts I & II of 3. Deliver part 1 and 2 of State of the Wiki report; include report launch plan for community. - State of the WMF published 4/2 - 1,400 PVs in first 90 days

- Rolled out via Signpost; wiki, PDF versions.

- Parts II & III repurposed for WP15 campaign.

Product: Product & Comms workshop; product launch workflow Co-develop product launch process with product team for more effective public product marketing and PR. - Messaging and PR support for 5 features/product: WikiGrok, Gather, error pages, Share-a-Fact and lead images; VisualEditor  

- Workshop completed 4/9

Objective Impact on Goal Status
Brand & Content: Baseline brand and identity data; preliminary brand book; Content strategy for blog & social media Deliver preliminary brand strategy exploration. Deliver content strategy.

- Interim brand book. Planning in progress for brand project plan; WP15 anniversary

- Collected content metrics, ran blog survey.

- Prepared content strategy for the blog, started social media and video plan.

External collaboration: Workshop w/ PBS, NPR, chapter affiliates on future of knowledge and public media Introduce WMF in content and media space, initiate discussion on new knowledge, explore possible brand partners. - Workshop with PBS/NPR completed with Wikimedia NYC & Wikimedia DC

- Several immediate and longer term projects; incl. plans for edit-a-thons

Media/PR: Ongoing support for media events (60 Min, ACLU, emerging threats, etc.) 100% max 24 hour response to inquiries. Reinforce positive brand perception. Advance brand values.

- Wikimedia v. NSA launch (Mar 10)

- Art + Feminism editathons (Mar 7)

- Introducing SLA tracking in Q4

success

  • Launched first-ever digital Annual Report, in easier to access format
    • 13,688 views from March 7 to April 7
  • State of the WMF community release
  • Initiated support, collaboration, and process-building for Product teams
    • Managed 2 integrated product launches; brand and messaging support for 3 product features
    • Offsite workshop with representation from C-level, Product, UX, Engineering, CE, Communications
  • Wikimedia v. NSA op-ed placement in The New York Times
    • 300+ additional global media coverage
    • Exceptional message delivery, almost all coverage pulled quotes and information from WMF material
  • Drove campaigns for gender initiatives throughout Women’s History Month
    • Visual design for Inspire Campaign and distribution support; campaign exceeded goals
    • 30+ global media coverage on Art + Feminism editathons
    • Original WHM content social and blog campaign with 10k+ impressions on blog alone
  • Initiated development of integrated content strategy
    • Introduced single-focus themed campaigns with integrated content across multiple channels (blog, video, social, external media); started content research for data-informed decision-making
  • Developed baseline brand book for Wikimedia Foundation identity
  • Supported 5 high-level executive and board announcements and transitions
  • Held first team retreat to clarify mission, priorities, planning, and staffing

misses

  • Executive Director global visits
    • Developed calendar but had to cancel trip to South Africa due to lack of planning
    • No further process in planning for remaining visits
  • Social media strategy
    • Did not develop strategy due to lack of resources (headcount, time)
    • Did not successfully acquire social media management tool
  • State of the Wikis and State of the Internet
    • Reevaluated importance of these projects/formats and postponed development
      • Aim to incorporate in major report for Wikipedia 15 campaign
  • Wikipedia 15 planning delayed
    • Coincided with Wikimedia v. NSA , annual planning, and leadership announcement support
  • Annual Report challenges
    • Made request for data too late for full capture, leading to incomplete statistics
    • Production of print version of Annual Report took longer than planned
  • Unsuccessful media outreach
    • Open Access Policy - no significant coverage secured, possibly due to short timeline
    • Funding free knowledge the wiki way - only negative coverage despite significant time investment; community response centered on report being commissioned by WMF
  • Product/Comms offsite pushed back
    • Underestimated amount of time required to align internally with stakeholders  

For more detail, see the slide deck and minutes from the quarterly review meeting of the Communications team

Objective:

Overall goal: ensure resources go to effective programs to increase participation and quality content

Impact on Goal Status
Objective 1.

Outreach for program effectiveness data

Follow-up with grantees & non-grantees to fill gaps in data

Increase L+E’s access to program data through direct data reporting from at least 35 program leaders on at least 58 implementations.

[“implementation” in this context is a specific time when a program has been carried out. For example, Wiki Loves Monuments is a program; Wiki Loves Monuments in Argentina in 2013 is an implementation.]

Surpassed target by 150% for program leaders and nearly 400% for the implementations target.

Directly obtained reporting data from 49 different program leaders on 222 program implementations surpassing improvement targets.

Objective 2.

Capture better data

(a) Targeted outreach for strong programs and key metrics.

(b) Define and obtain metrics for data gaps

- Increase program leader adoption of metrics and reportability (Increase the number reporting directly by 33%; prepare and update an overview of evaluation and reporting)

- Improve evaluation reports and identification of impactful programs (Identify 10 positive exemplars for writing contest toolkits investigation)

- Assessment of quality and retention (Metrics pulls complete for 3 programs targeted for Q3 reporting)

90% Complete

Behind schedule on pulling on-wiki user activity and content metrics due in part to some Wikimetrics issues in January as well as some time loss to department transition needs. We have completed for the majority (70%), but not all, programs data to be pulled.

We have adjusted our timeline accordingly.

Objective:


Overall goal: ensure resources go to effective programs to increase participation and quality content

Impact on Goal Status
Objective 3.

Promote effective program design

Complete 3 Programs Evaluation Reports:  

WIki Loves Monuments

other Photo Events, and

Editing Contests  

Increase knowledge about Wikimedia programs, their design, and impact through

- Publication on Meta of Reporting Overview (A review of state of reporting and the programs evaluation report series)

- Publication on Meta of 3 Programs Evaluations (Wiki Loves Monuments, Other Photo Events, and On-wiki Writing Contests) to be followed by official announcements and RFC

- Program toolkits development for Writing Contests and Photo Events*

85% Complete

Behind on the third program evaluation report (On-wiki writing contests) due to shifting some of team members time to the director search and transition team needs.  

We have adjusted our timeline accordingly.

*Initiating interviews outreach and process for toolkits mapping.

 
 

Success

✯ Increased the number of implementations captured six-fold over last year’s data collection

✯ Doubled the number of program leaders who provided direct supplemental data reporting

✯ Quadrupled the number of implementations with primary data collection support to 222
~ 30% of the identified program implementations

Increased data access has been helpful to fill in gaps so that we can better map program impacts. For instance, we have username lists for nearly 50% of all implementations identified, compared to less than 20% last year, allowing us to better understand retention this round.

Misses

Behind schedule on:

  • Obtaining user activity and content metrics: We have completed data queries for the majority (70%), but not all, programs with key query parameters available. The lag is also partly due in part to some Wikimetrics issues in January, as well as some time loss to department transition needs.
  • 3rd program evaluation report: One of the three program evaluation reports due (On-wiki writing contests) is behind two weeks due to shifting some team members time to the director search and transition team needs.  

We have adjusted our timeline accordingly.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department

Objective Impact on Goal Status
Inspire campaign:

Create pipeline to source and support 20 new gender-focused projects w/ impact potential

March campaign targets:

*500 participants (10x increase in IdeaLab‘s monthly)

*100 ideas (20x increase in IdeaLab’s monthly)

*At least 30 grant proposals

*629 participants (13x IdeaLab monthly)

*266 ideas (50x IdeaLab monthly)

*57 grant proposals total

*43 proposals eligible for funding review

(# of new grant-supported projects TBD April 31)

Emerging communities:

Improve capacity development for select Global South communities

*Capacity Development Framework (CDF) published on-wiki
*10 major GS communities mapped in terms of CDF
*5 GS groups self-assess and identify support requests

*Changed plans: more GS community research was needed before CDF could be created.
*15 in-depth interviews with 12 communities have been completed
*Analysis and actionable CDF pushed to Q4.

Organizational benchmarking:

Build a more impact-focused culture by socializing benchmarking research

2 research projects are socialized with key stakeholders (WMF staff, committees, and affiliates in the movement):
*Organizational Effectiveness (OE)
*Participatory Grantmaking (PG)

*Organizational Effectiveness study finished and presented within WMF and to Affcom. Discussions w/ affiliate orgs scheduled for May at WMCon.

*Participatory Grantmaking study finished and shared via blog post and Signpost. Discussions w/ grant committees scheduled for May at WMCon.

Success

Inspire: Campaign participation and ideas exceeded expectations (2x targets). Successful first step in broadening the pipeline for proactively distributing resources focused on a targeted strategic issue.

  • First ever on-wiki “friendly space” expectations pilot in IdeaLab demonstrates potential to set higher tone standards. Community began to enforce these expectations, laying ground for more supportive engagement even on very controversial topics like gender. Survey outcomes on participant experiences coming in Q4.
  • Alex Wang & Jonathan Morgan (CR), and Patrick Earley (CA) were particularly instrumental in preparing for and running this successful campaign

Emerging Communities: Interviews already surfacing insights with impact potential for less-known community practices and challenges in emerging communities

  • Building trust as a foundation for impactful community health pilots with active emerging community members who are instrumental for community development. Several interviewees expressed happy surprise by the level of interest/attention from WMF.
  • Sati Houston & Asaf Bartov for adjusting course to gather more qualitative data in over 30 hours of community interviews. No small effort to gather qualitative data on community health.
Misses

Emerging communities: Timeline for original objectives was ambitious.

  • Learning: more research needed into emerging communities to validate community development hypotheses. Research on community health requires significant staffing investment. Action: prioritizing staffing in CR, L&E.

Organizational benchmarking: Time for socializing benchmarking research in the community was reduced in departmental transition, and some research has not yet been used as hoped.

  • Learning: socializing research takes staff time and needs clear prioritization to have impact. Action: In Q4, prioritizing the Organizational Effectiveness work more highly again.
  • Learning: Participatory grantmaking research sparked more community debate on how it was commissioned/reported than interest in the report’s substance. Staff transitions reduced our ability to address this narrative of concern w/ clear info. Action: future commissioned research must be more clearly planned/documented

Inspire: Many proposals developed in the campaign still require significant mentorship in order to become project plans with high impact potential

  • Learning: significant staff time is needed to support development of impactful plans, in each round of grantmaking. In department transition, less staff resources were available to support this work for Inspire. 1 PO’s time reprioritized to lead new CR team, other PO also operating PEG’s biggest quarter ever (see slide 8). Action: backfilling Individual Grants PO

For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department

Objective Impact on Goal Status
Actively mentor program leaders worldwide Start tracking interactions with volunteers in order to establish a baseline during Q4 Initial tracking numbers:
  • Contact with 100 volunteers in 53 countries
  • 4 Collab members involved in mentoring: helped 11 volunteers in Q3
  • 44% of contact initiated by WMF, 56% by community members
Add content to Arabic Wikipedia through continuing and expanding education program (No specific goals for Q3 because annual goal was achieved in Q2)
  • Continue adding bytes to Arabic WP
  • WikiArabia attendance, speaking about education
  • Preparing for trip to Oman to investigate education program pilot
Add content to Spanish Wikipedia (and sister projects) Targeted mentoring in Mexico and Argentina to help them run successful education programs
  • Recognition & motivation trip to Mexico
  • Co-organizing education pre-conference at Wikimania
  • Use of Wikimetrics is less than expected, so results are unclear
Objective Impact on Goal Status
Provide materials for program leaders and encourage community involvement in material development Launch Education Toolkit by March 2015. Invite community engagement with toolkit Launched Education Toolkit:
  • Consists of 22 wiki pages and 46 Learning Patterns
  • 777 pageviews of landing page
  • 58 viewers of “launch” hangout
Share lessons learned with broad education and Wikimedia audience and facilitate discussion and networking Publish monthly newsletter jointly with community

Publish one blog post about education per month

Start Facebook group, share one update per week.

  • 17 community members contributed to newsletter
  • Newsletter pageviews went up from ~50 in March 2014 to ~460 in March 2015 (yearly growth of >900%)
  • Launched Facebook group on Feb 26:
    • 155 members
    • 27 posts in 6 weeks (50% community, 50% WMF)
    • Average likes per post: 9
    • Average views per post: 91
Success
  • Toolkit launch: Comprehensive overview of materials and best practices, organized logically, ready for international use. Joint L&E and Education project with huge community input, managed and written by Kacie Harold.
  • Facebook group launched: Fills a networking need flagged by community members. Successful launch and daily management thanks to having a dedicated education communications intern, Samir Elsharbaty.
  • Tracking team’s interactions with community: First attempt at tracking shows huge number of interactions on a range of topics. Tracking made possible through L&E guidance.
  • Arabic: Achieved annual goal for bytes added in Q2, allowing us the opportunity to further diversify the Arab World program (continue support in Jordan, and explore an education program pilot in Oman).
Misses
  • Education Collaborative” challenges: Education Collaborative was intended to improve team effectiveness by helping community members help each other, but engagement was lower than expected. Targeting Q1 for improvements to the Collab model, based on learnings from other CE teams.
  • Weak data for Spanish Wikipedia mentoring efforts: Spanish-language education program has been a focus, and anecdotal signs are encouraging, but promoting the use of WikiMetrics has not resulted in much data. WMF staff currently lacks time to move forward on this focus region; instead focusing on working through Iberocoop education coordinator to increase impact.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department

Objective Impact on Goal Status
Integrate community engagement process with product development process Create product development process that incorporated community feedback in structured, productive fashion Process still in draft form; new methods of facilitation enacted within Editing Team
Support strategy consultation with Community Advocacy Facilitate feedback from C-Levels to participating community members Completed
Absorb the m:Tech/News newsletter Take over publication of the TechNews newsletter from Erik’s team. Initiative has been stalled until new team member onboarded at end of Q4
Implement Single User Login (SUL) Project manage necessary engineering work

Communicate change to community and resolve conflicting renames

Rename 2.8 million accounts

On track to rename affected accounts beginning 15 April
Success
  • New mechanisms for collaboration: Sherry and Erica supported weekly bug triage meetings and a Qualtrics survey (“papercuts”) targeting five groups of editors that either use VisualEditor significantly or offered other feedback on VisualEditor in the past. This has broadened the base of community members giving product feedback in structured, measurable ways.The papercuts initiative has not surfaced any new Phabricator tasks.
  • Strategy consultation: Nick and Moushira supported CA in the community strategy consultation by gathering feedback, identifying active community contributors and drawing attention to topics of interest. The consultation ended on March 16th with 2,216 unique participants.
  • SUL: Keegan has project-managed Single User Login (SUL) since mid-Q2. Under his guidance, the necessary tooling was successfully written, and his collaboration with “Global Renamers” volunteers supported the renaming of 6,000 accounts with much lower-than-expected community friction. Kunal (Engineering) was key in supporting technical objectives, and User:DerHexer (Steward) played a vital role in handling difficult renaming cases and supporting broad communication.
Misses
  • Product community engagement process stalled: The goal of an integrated product development/community engagement process reached only draft form, because of changed priorities (greater emphasis on Visual Editor and strategy consultation) and turnover in product leadership (Howie).
  • Tech/News: We did not begin the planned transition of the Tech/News newsletter due to changes in priorities (e.g., strategy consultation) and delayed hiring. Staff being hired for end of Q4 may be able to take on some of this task.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department

Objective Impact on Goal Status
Roll out community strategy consultation

Create strategy consultation framework

Publicize appropriately to community

Provide basic translations where possible

Clean up vandalism

Summarize feedback on spreadsheet

Engage community directly

Achieved; community consultation wrapped on March 16th with 2,216 unique participants.
Protect communities through a global ban policy. Create and implement a global ban policy

Staff and implement refined community protection processes

In process. Global bans policy implemented and initial bout of requested bans deployed in Jan.

New CA staff, with part time focus on community protection  role and other CA duties as assigned, anticipated on 4/20.

Objective Impact on Goal Status
Coordinate and facilitate visits to the Wikimedia Foundation office from the Affiliations Committee and Wikimedia Russia

Coordinate with community members, travel, and administrative teams

Coordinate with other WMF teams to plan and facilitate successful meetings

Both meetings were successfully held in January, with two delegates of WMRU and nearly the entire body of AffCom  (95%) + 2 Board liaisons in attendance. Across both groups, meetings lasted 5 days.
Support new initiatives in the Call to Action. We committed to supporting the Call to Action, internally and externally, with community stakeholders. The CtA  was rolled out in stages, with the first round of presentations in January. CA has supported the process of developing the new CE team and has joined in discussing implementation of several initiatives, but staged rollout of the CtA has limited practical implementation.
Success

Strategy consultation: This engaged the entire team through Q3, with technical monitoring and community engagement from all, and Kim and the strategy team seem extremely satisfied with the outcome.

AffCom visit: January Affcom visit was highly successful, with 95% turnout, high attendee satisfaction, and reported improvement in committee alignment/motivation. Maggie, Janet Renteria and the travel team were particularly helpful in dealing with unexpected challenges.

Inspire: CA put considerable time into supporting the Inspire Campaign, particularly Patrick. We expect learnings from this process will be broadly applicable as we seek to build safer spaces.

Misses

Staffing: We made an offer to a new trust & safety hire towards the end of the quarter, but she will not start until mid-April, so little formal progress has been made on new initiatives. Long hiring lead times were further lengthened by the unexpected reformation of a new department. For future hires, we need to find ways to streamline this process.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department

Objective Impact on Goal (Q4 targets: no Q3 targets set) Status
Add major partners (including non-English) and extend reach * Add 5 major partners

* Add 5 non-English partners
* 1500 users, 500 new users

* Added 12 partners, 3 non-English:
Project MUSE, DynaMed (EBSCO Health), Women Writers Project, Royal Pharmaceutical Society, Royal Society of Chemistry, Loeb Classical Library (Harvard Press), RIPM, Newspaper Archives.com, L’Harmattan, SAGE Stats, MIT Press
* 150 new unique users, 398 accounts distributed
* Seeing some English signup saturation, expect impact to increase w/ Global Branches

Start global satellite rollout * Add 10 new global branches

* Finished setup guide, reviewed with CE Global South and L&E
* Started setup process with 3: Spanish, Norwegian, French
* Rekindling with 3: Chinese, German, Arabic
* Doing 3-6-10 staged rollout over next 6-9 months
* Implementation going well but slower due to learning curve

Objective Impact on Goal (Q4 targets: no Q3 targets set) Status
Train volunteers to run own branches

* Onboard 15 global coordinators
* Outreach, Comm, Technical coord

* Reached out to 3 WP language groups for new branches
* New Metrics coordinator,new Outreach coordinator
* 4 new Account coordinators
* No Technical coordinator yet due to low technical focus

Develop reference tech

* Echo notifications built
* Library Card Platform reviewed
* Linkypedia (or GWT) 2.0 team funded

* Initial discussions with Engineering on Echo Notifications
* Backburnered other technical development in lieu of priorities
* Facilitating DPLA involvement in Glam Wiki Toolkit

Extend our network of influence

* Present at 5 major conferences
* GLAM, University, and OA coords
* 5 new visiting scholars

*Presented: ALA (Chicago), GLAM-US Consortium
*Accepted: GLAM-Wiki, DPLAFest, Project Muse KEYNOTE
*Upcoming: ALA (San Francisco) Booth, Wikimania x3
*Two New Library Interns Coordinators
*On track for recruiting 10+ Library Interns,  5+ Visiting Scholars
*Started WMF Partnerships Brunch

Success

Already exceeded *Q4* goals for new partners, easily: Outreach to new partners is now very reliable, requiring little logistics and with recent partnerships coming in on their own with very low effort.

Volunteer recruitment and coordination growing scalably:Breaking large programmatic requirements into manageable volunteer responsibilities — Partner, Metrics, Outreach, Account coordinators — is helping us scale.

Leveraging successful networking: To help raise visibility in the library community, team has networked and spoken at conferences. Example: Invited to give Project MUSE Keynote speech through connection, introduced to us by OCLC at an ALA conference.

Building Interconnections w/ Community and across Foundation: Invited to serve on US GLAM-Wiki Consortium. Organized internal “Partnerships Brunch” — already seeing results.  Sharing learnings with Kourosh, Edu, and APG. Teamed up with CA. Using L&E and GS guidance for program growth.

Misses

English-language Wikipedia signup saturation

New partnerships not filling as rapidly. We suspect a lack of direct invitation, unbundled individual resources with delayed turnaround, resource unfamiliarity, and high requirements (1000+ edits and 1 year). Experimenting with lower baseline (500 edits and 6 months). Q4 onward should see raised awareness and amplify signups as new global branches start.

Tech Development Backburnered

Team lacks strength in product development — we rely on outside consultation and coding. Signup management tool would scale and speed up distribution. Initial talks about Echo stalled without easy test-case. We likely need a technical coordinator (even if volunteer) and vastly increased tech capacity for superuser projects in either CE or Tech departments.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department

Advancement: Fundraising

edit
Objective Impact on Goal Status
Raise $3 million in Q3 primarily through:

1) Banner and Email campaigns in Israel, Hungary, Sweden, Denmark, Norway, and Spain.

2) Foundations, Major Gifts: Send out annual report; Send proposals to four new institutional donors and secure a multi-year commitment from one new funder

Raise $58.5 million in FY 2014-15 Done. All country campaigns were completed. Raised $7 million, exceeding the goal. The final number is still settling.
Research: Complete a study of our December fundraiser using a professional public opinion research firm Develop a strategy for the long term financial health of the organization Done; Released on Signpost
Objective Impact on Goal Status
Implement payment processor for China;

Prepare to implement payment processor for Latin America in early Q4

Raise $58.5 million in FY 2014-15 Complete
Partnerships: 1. Hire a VP of Partnerships 2. Write a partnerships plan Develop a strategy for the long term financial health of the organization VP of Partnerships has been hired! We have an initial six month partnership plan to be refined further next quarter.
Success

The Advancement team has exceeded our annual fundraising goal by 14%. We have raised $66.4 million by the end of Q3 2014-15. Our annual goal is $58.5 million.

The Advancement team has exceeded our quarterly fundraising goal by 133%. We raised $7 million in Q3 2014-15. Our quarterly goal was $3 million.

All teams within Advancement have contributed to this success. Both Online Fundraising and Foundations and Major Gifts have exceeded their individual team goals.

Misses

We need to further develop our partnership plan.

Example:

We did not complete the partnership plan by the end of the quarter. We did hire our VP of Partnerships just before the close of the quarter. We will be working to develop a comprehensive partnerships plan next quarter.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Advancement department

Objective Impact on Goal Status
PCI Compliance Assessment Security Done. Exceeded.
New payment method: China partner Enable fundraising in China Done. Disabled for legal reasons.
Start new 3rd party integration: LATAm partner Enable fundraising in Latin America Started coding.
Upgrade for nightly reconciliation file parsing with current vendor Security In progress.
Ongoing refactoring and code hygiene improvements in Donation Interface whenever possible. More flexible (borrowed) and happier developers Phase 1 complete.
Reporting Improvements
  • Improvements to CentralNotice tracking
  • A/B components for the internal dash
  • Custom civicrm reports (to be upstreamed)


Increase visibility In progress.
Success
  • PCI: The assessment (using an external vendor) is complete and we’ve begun implementing the recommendations.
  • China Partner: We implemented our China Partner in January, but our account was disabled by the processor as soon as it became operational due to legal changes on their side.
  • DonationInterface refactor: We accomplished the first phase of our goals, and are beginning the exciting second phase. See [1] for the roadmap.
Misses
  • LATAm Partner: This goal was to “start”, according to the master goals doc [2], but this is squishy (and was misrepresented in the QR), hence a “miss”. We are now in initial phases of implementation and plan to finish this quarter. This is our top-priority Q4 goal.
  • Nightly Reconciliation: Deprioritized due to sliding upstream deadlines.
  • Reporting improvements: The subgoals are each in progress, but this goal as stated was too ambitious.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Advancement department

Objective Impact on Goal Status
Release iOS app with lead images, image gallery, read more, and Tweet a Fact features The features will be out in production The release was pushed back to due regressions that were found by us during testing.

These regressions were fixed and the subsequent update has been approved by Apple and is awaiting release.

The update is planned for Wednesday 15th April.

Release update to the Android app containing Tweet a Fact, search / featured article widgets The features will be out in production The update was released on 30th March 2015
Trial single, more visually appealing “read more” result in Wikipedia Beta and select the best version for production The new feature must drive more engagement than the old one in order to be released to production

The new display was trialled in Wikipedia Beta on Android and found to not drive as much engagement.

The new version was not deployed to production

Success

The Android app was listed as a featured app by Google in the Play Store. Downloads of the app increased by 2-3x as a result. Impact on monthly uniques not yet determined.

We are prototyping a new service to deliver HTML to the apps. Initial tests show it is performing well, reducing HTML payload by around 50% and significantly reducing the number of network round-trips required to load articles.

The Android team continues to reduce technical debt and improve performance of the platform. Memory overhead of the app has been significantly reduced. We have seen a significant reduction in the number of bug and crash reports.

Due to increases in staffing, the iOS team has seen a significant increase in velocity this quarter.

Miss

Despite increases in staffing helping to mitigate this, development of the iOS app continues to be slow due to technical debt, and the release had to be pushed to April. Wikiwand released its iOS version before we were able to offer a superior Wikipedia reading experience to iOS users.

Search is central to content discovery in the app, and users continue to report that our search experience is subpar. Data continues to show users type multiple queries to find the results they want. The mobile apps team has reached the limit of what they can do with the frontend to increase the relevance of results, and does not have the expertise required to improve the search backend.

Poor monitoring caused an API account creation outage which was only detected due to users giving feedback on the app store. The time that the API broke could only be figured out using manual funnel analysis on the client-side logging in the Android app. The API was broken for over 24 hours before the problem was detected.

Deployment of an experimental version of the mobile apps content service to production is currently blocked for a variety of operational reasons with unclear timelines. In the mean time, users suffer with inflated data usage and reduced performance.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Mobile Apps team

Mobile: WikiGrok

edit
Objective Impact on Goal Status
Baselines for WikiGrok campaigns on English Wikipedia Measure response quality, engagement with feature, and short-term retention of WikiGrok campaigns on English Wikipedia. Done. Data available here: https://meta.wikimedia.org/wiki/Research:WikiGrok
Unsampled WikiGrok campaigns Run WikiGrok campaigns for 100% of readers, unsampled Work on Wikidata backend was prioritized over this objective.
Send data to Wikidata At least some of the data generated by WikiGrok campaigns should begin to migrate from the internal database we’ve been using to Wikidata The objective was not met and is on hold pending community discussion.
Success

Despite many technical and social challenges, the team was able to generate new WikiGrok campaigns, create a data aggregation heuristic, and build a backend framework for an entirely new mental model for contributing to Wikimedia projects (aggregated reader data vs. one-to-one user-to-revision).

Even in its very early stages, this model shows the most promising engagement numbers of any existing contribution feature, or any new feature the Wikimedia Foundation has ever built.

Leila, Ryan, and Max worked overtime to make all this happen and are total rockstars.


Miss

We were unable to launch existing WikiGrok campaigns to 100% of readers. Sending data to Wikidata was prioritized over this objective, and we were not able to do both in one quarter.

The technical and social requirements of sending data to Wikidata also proved too challenging/in flux to tackle in one quarter. The latter (changing community requirements) are summed up by this user:

“(...) there were a lot of freedom in [Wikidata] at the beginning but now we have to know if we want to continue our lonely game of data import without reference which will lead to the reject by [Wikipedia].”

For more detail, see the slide deck and minutes from the quarterly review meeting of the WikiGrok team

Objective Impact on Goal Status
Launch pilot of a new reader curation + sharing feature. Identify issues to inform product direction We launched pilot in beta.  We did not launch with built-in share features
Begin collecting baseline usage statistics (creating, adding, consuming) and characterizing use cases to inform further development Identify issues to inform product direction The feature-set is well instrumented.  

We have already identified and prioritized a few key issues that we will be  addressing next.

However, due to low traffic on mobile, we do not have enough metrics to drive other important decisions  prior to roll-out on stable (moderation being one of them).

(Cf. revised objectives from Q2 review meeting)

Success

In just over a quarter: 2 weeks of research and 8 weeks of development, we were able to go from inception to launching an mvp in beta for a feature that takes important steps in key areas: article presentation, identity, social, and casual contributions.

We realized we would not exceed our pre-quarter expectations within the first 2 weeks of the quarter and communicated as much in Q2 Quarterly review.  

When we made the emergency call for backend support, we were able to borrow Yuri from the Wikipedia Zero and had him on the team within 48 hours.

The team had outside support from design research, Aaron, Dario, and the rest of the mobile web team.


Miss
  • Team set aggressive goals
  • Did not start with necessary backend engineer
  • Culture and practices are not conducive to rapid innovation:
    • don’t have a great process in place for creating new extensions or making database changes
    • fears of community reaction played a big role in determining scope, delaying key features (like sharing), and promoting others (like watchlist functionality)
  • Mobile web beta population is not large enough for testing at scale

For more detail, see the slide deck and minutes from the quarterly review meeting of the Mobile Web/Gather team

Objective Impact on goal Status
Develop deep dive approach. Define plan (criteria & market, priorities, goals) Analyzed countries, scoped
Preserve free access after https default. Complete IP migration 59 countries total

Remaining: 24%

7 in progress, 7 to do

Grow to 90M free page views/month. Launch inbound requests

Manage existing partners

74 M pvs as of 3/31

Launched 12 Digicels, Uninor (4/1).

Lost Aircel and STC (March)

Provide for ongoing support for Zero with fewer tech resources. Complete portal phase 1

Messaging on projects

Messaging for termination

In progress, will complete by April 20
Objective Impact on goal Status
Ship Android preload Preload app with min. 1 OEM Preliminary yes 2 OEMs. Got de-prioritized by OEM. Needed marketing support.
Lay groundwork for industry coalition to promote open knowledge. Develop materials

Identify participants

Do initial outreach

Identified key players, and had initial discussions with Mozilla and CIS India (4/9).
Success

Big push on IP migration (from URL-based zero-rating) to support HTTPS and editing. Big win was saving Beeline Russia after Ops defaulted to https (Carolynne, Dan). Adele and Dan actively engaged with the partners in Africa, where partnerships were at risk due to contractual challenges. So while not a “green,” we do see the progress and the engagement as a success.

Wikipedia Zero successes: 1) Carolynne and Manprit renegotiated the [operator]  contract in line with operating principles, in signature process. 2) Smriti facilitated a big event in Bangladesh with WMBD and Grameenphone, where Jimmy spoke; resulted in substantial positive media coverage. 3) Smriti and Jorge engaged with partners at MWC, repopulating the active pipeline. 4) 8 partners have updated their IPs and contributed to keep the page views from declining.

We released the portal beta version to partners, who are now using it to set their test period directly. This reduces the partner testing time and allows quicker deployments.

We structured the framework for open knowledge coalition, identifying 45 key stakeholders to engage with. We started positive engagements with Mozilla and CIS India (April). We started socializing the initiative to community members (April).


Miss

Deep dive into one country to increase impact was put on hold while we worked through restructuring. The team completed the country analysis and selected two finalist countries. Adele organized a cross- functional team meeting for input and collaborative strategy. We’re ready to revisit the deep dive in Q4.

IP migration was not completed, because some partners were unresponsive, others in contract renewal. We will complete in-progress migrations in Q4, terminate unresponsive partners, and relaunch the ones engaged in contract renewal.

No growth in Wikipedia Zero page views. We lost Aircel India and STC Saudi Arabia, but still had a net gain of 1M in Q3. We intentionally did not pursue new outbound contacts. The team responded to inbound requests, but only Digicel launched. Managing the existing partners takes time away from new launches (especially the IP migration). The learning is valuable to understand the job demands and team capacity as we move forward.

Android preloads were a big disappointment. Carolynne had secured verbal agreement from [OEM], then they deprioritized. Same happened in Brazil. The Indian OEMs were unresponsive. We’re pitching mission, seeded content and search integration, but OEMs look for marketing or revenue. We will re-evaluate as a priority for Q4.

We did not complete the engineering work to set Zero up for low maintenance because Yuri was assigned to work on Gather. We have Eng time in April to complete (Ops dependency).

For more detail, see the slide deck and minutes from the quarterly review meeting of the Wikipedia Zero team

Objective Impact on Goal Status
Fix functionality and correctness bugs that affect VisualEditor Roundtrip testing accuracy stable at 99.8% (if false failures are fixed, would be closer to 99.9%) All VE Q3 blocker + some Q4 bugs fixed
Reduce generated HTML size 43% reduction in [[Barack Obama]] output • Unused data stripped from template content

• Eliminated duplicated HTML from <ref>s

Work towards Parsoid-HTML read views (stretch goal) Improved visual diffs

Language variants rendered

Cite customized via CSS

Marginal progress beyond whatever achieved as part of first objective.
Objective Impact on Goal Status
Initiate work to scope template output (stretch goal) Parsing performance improvements Wiki pages drafted; Wikimania talks submitted
Make the Parsoid cluster more stable Stable memory usage graphs in Ganglia. Load spikes correlated with actual traffic. • Handling more pathological pages

• Reliable timeout handling and process restarts

• Eliminated memory leaks

Gather performance metrics Performance regressions easier to detect. wt2html and html2wt endpoints instrumented in detail + dashboards established
Success

Sustained focus on visual editor bug reports and blockers helped us fix reported issues as they cropped up.

Our fairly robust test infrastructure has been key to the smooth regular deployments and catching problems early. That said, we nevertheless identified a testing hole (now fixed) after VE’s migration to RESTBase. But, given that the bug could have caused serious page corruptions, it also highlighted the remarkable accuracy of our selective serialization strategy.

OPW intern (Christy Okpo) helped us tackle previously neglected task of production performance instrumentation.

Miss

Our goals and targets are more ambitious than what we can realistically achieve with our team size and composition. Learning to scale down ambitions. But, having Tim Starling join the rebranded parsing team helps here.

The long tail of small things to fix is long / endless (second system problem). It is equally important to (a) enable read views with Parsoid HTML (b) make forward progress enabling tools to fix broken wikitext on pages (c) fix wikitext / template processing model -- this more fundamental work on wikitext improvements keeps getting pushed back.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Parsoid team

Objective Impact on Goal Status
RESTBase deployment Production deploy of RESTBase with Parsoid HTML and metadata storage https://rest.wikimedia.org/en.wikipedia.org/v1/?doc

- 15ms mean HTML revision load time

- No SPOF, horizontal scaling, multi-DC

VE speed-up - Store all current article revisions for low latency access

- Reduce HTML size by at least 20%

- Provide low-latency public REST API

VisualEditor HTML mean load time reduced by 36%.

Mean HTML size reduced by ~35%.

Further significant speed-up coming with VE loading HTML directly from RESTBase (currently live on mw.org).

Section editing and retrieval API Research and prototype section-level edit and view API in collaboration with Parsoid, Mobile and VE Research and design work, but no implementation yet.


Success

RESTBase performs really well: ~12ms mean HTML revision load times in production; 10k req/s peak throughput on six boxes. This enables high volume use and consistently low latencies.

API versioning guidelines and Swagger API docs have been well received, and are being adopted by other teams. We now have a flexible and performant API proxy which lets us expose internal services in a uniform and well-documented external API.

We have made some progress on streamlining the service development & deployment pipeline. We developed shared service infrastructure (libraries like service-runner, a service template, standards for testing, metrics & logging), and support other teams in Citoid, Apps, Graphoid service development. Marko Obrovac is driving much of this work.

Miss

The section edit API work was slightly delayed by deployment-related work taking longer than expected & tying up limited team resources. We have researched issues and developed a plan for the first iteration with the Parsoid and VisualEditor teams, which is our top priority for Q4.

Service deployments still use more time and resources than we'd like. We have made some progress on shared infrastructure, but there are still long lead times and heavy dependencies on operations for configuration and setup. We hope to improve this situation in collaboration with Operations and Release Engineering in the coming quarter.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Services team

Objective Impact on Goal Status
Modern authentication framework for MediaWiki Re-design MediaWiki authentication process to allow more complex authentication flows Implementation underway; ETA unknown as this will not be top priority in Q4
Service-oriented architecture authentication Document use cases for authentication and authorization by various non-MediaWiki consumers Not worked on; a lighter solution was found for the immediate needs of RESTBase
Wikidata query service Produce a working prototype of a graph search tool in the style of WDQ with a clear plan for production scaling Implementation underway; progress delayed by unexpected need to select new graph database in Q3
Multi datacenter MediaWiki Document impediments to active write-active read multi-DC operation; Begin removing those impediments Implementation underway
SUL finalization Support the implementation of SUL finalization On track
Success

Inter-team communications were greatly improved by having a full time interim Product Manager who was included in the PM team meetings. One PM for all of Platform will probably not be enough to successfully manage this communication for all teams in the Platform group but it's a great start.

The new Phabricator based process for signaling that an RfC is ready for discussion made scheduling discussions for both AuthManager and Multi-DC projects much easier than past workflows.

Changing the weekly meeting format from a report-up workflow to a topical discussion highlighted fragmentation of focus of the team members. This led to detailed discussions of possible solutions and the decision to reorganize into separate teams with more narrow focus starting in Q4.

Miss

Development across all projects was slowed because we took on too many projects, often assigning only a single full time developer. Reorganization into separate teams should improve focus but we still have some staffing challenges to overcome.

WDQ was set back significantly when the company developing the graph database was acquired. There are risks when working with bleeding edge technology.

The AuthManager project exposed some disconnects between the priorities of the team (developers and Product Manager) and those of management outside the team. Reaching agreement on goals prior to starting work on a project may reduce such tension going forward.

SUL Finalization should have been called out as a project for Q3 rather than being treated as something that would "just take running some scripts". Honestly it should probably be a Top N project for the whole WMF organization.

For more detail, see the slide deck and minutes from the quarterly review meeting of the MW Core team

Objective Impact on Goal Status
Scale up HTTPS infrastructure Scale up HTTPS infrastructure to be able to serve HTTPS-by-default. Capacity expanded according to measurements.

Additionally, HTTPS performance improvements & SPDY were rolled out.

IPsec on cross-data centre links for private data All PII (cache and logging traffic) between Wikimedia data centers is encrypted using IPsec ESP Still under testing, stability problems found. Expected to complete near end of April.
Virtualization for miscellaneous servers at codfw Virtualized cluster at codfw set up. At least 2 virtual services migrated to the virtualization cluster Virtualized clusters at both codfw & eqiad are set up. Services on track to be migrated.
Objective Impact on Goal Status
Ability to serve core site services (wikis) at codfw MediaWiki application servers deployed.

MediaWiki page loads functional.

Capacity meets or exceeds eqiad’s application cluster.

MediaWiki application servers and dependencies other than ElasticSearch and RESTbase deployed.
New external monitoring for performance and uptime metrics Redefined availability metrics monitored from the start of Q4, and available and reported in the Q4 quarterly review. Multiple providers evaluated. Contract with “winning” provider executed, monitoring has been setup.
Success

Our project of scaling-up our HTTPS infrastructure exceeded our stated goal. Not only was the infrastructure expanded to meet the required capacity, but also many additional performance improvements (including SPDY) were implemented. Brandon Black was instrumental in this work.

We were also able to complete the buildout of the MediaWiki application server cluster in the new data center (codfw), despite the large scale infrastructure and many components. Giuseppe Lavagetto led this project and coordinated with MediaWiki Core to resolve configuration and deployment challenges successfully.

Miss

We were unable to meet the deadline for full IPsec deployment. Insufficient subject matter expertise was available until later on in the quarter, so several technical blockers didn’t surface in time.

Our goal of deploying a virtualization cluster in codfw was also at risk due to delays at various stages of the procurement process, which resulted in the delivery of hardware very near the end of the quarter. We’ve started hiring for a Procurement Manager which will streamline this.

Appendix
 
Availability: Watchmouse, enwiki
These graphs will be replaced with data from our new external monitoring provider next quarter.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Tech Ops team

Objective Impact on Goal Status
Beta Cluster stability
Successfully re-integrate MediaWiki releases
  • Chris released 1.24.2 and 1.23.9 tarballs
Isolated CI instances
  • Packaged Zuul
  • Procuring Hardware
  • Arch review by Chase
QA Support for Editing and Apps
  • Increased the stableness for VE browser test this quarter
  • VE and Editing have quality releases
Team process improvements
  • began conversation with TPG, working on team offsite (pre-Hackathon)
  • On track with plan, culmination at offsite in May.
Success

The number of people involved with Continuous Integration maintenance has grown: special note to Kunal from MediaWiki Core Team and Timo Tijhof (as always).

WMDE is independently maintaining their jobs based on WMF templates.

The architecture review of the Isolated CI Instances work by Ops (especially Chase and Andrew) was very thorough and provided a lot of good suggestions.

Miss

We were unable to create the Staging cluster in just one quarter. This is mostly due to the unexpected issues that tend to come up. The support from Ops (especially Yuvi) was great, but sometimes things take longer than planned to do them right. And the goal is worth it (demonstrable cleanup which is better for everyone, and “Beta Cluster as a Service” - better name coming...).

Our goals from last quarter were predominantly all > 1 quarter long (hence all the red). Big projects are not inherently bad, but breaking them down into smaller sections can help gain traction/buy-in/support from others.

For more detail, see the slide deck and minutes from the quarterly review meeting of the RelEng team

Objective Impact on Goal Status
Upload Wizard Refactoring and unit testing

Tracking funnel metrics over time

The objective is complete, a tremendous amount of cleaning up happened
JS error logging Get visibility on which main errors/bugs are responsible for the Upload Wizard funnel drops Completed the week after the end of the quarter and the disbanding of the team
Platform Investigate a better solution for thumbnail storage Not worked on
Success

Mark Holmquist’s relentless focus on UploadWizard helped us greatly reduce the technical debt this extension is affected by. Mark kept a sharp focus on that objective despite the distractions that came our way during the quarter.

Gergő broke new ground on JavaScript error logging. The expertise he’s developed in that area will prove invaluable for the future of Wikimedia engineering. He kept working to complete this task despite the team being disbanded.

Miss

Originally we expected to deliver JS error logging for everything, but that objective was revised mid-quarter to focus on UploadWizard. Even that was slightly too much to fit in the quarter. Our main mistake was to attempt to implement the ideal complete system perfectly instead of focusing on a minimum viable product.

This quarter saw us putting out a lot more unplanned fires than usual. One of us having to serve as substitute PM also reduced our bandwidth more than we anticipated. We overcommitted when we estimated our quarterly goals, which is why we left an objective completely untouched.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Multimedia team

Objective Impact on Goal Status
Horizon dashboard proof-of-concept OpenStack Horizon available read-only to Labs users. Horizon dashboard has been made available with basic functionality.
Designate for Labs DNS Replace our internal DNS (PowerDNS) code with OpenStack Designate. Installed and being tested.

Due to an additional FQDN change, deployment was changed to opt-in by users, prolonging the migration.

Storage capacity & redundancy expansion Expand storage by 18 TB of usable space. All data replicated to codfw. A documented and tested procedure for (manual) storage failure switchover in place. Storage capacity was expanded, a cold spare deployed. Documentation and testing not yet complete. This will rollover into the next quarter goal (ToolLabs stability).
Success

The team’s goals were nearly met, despite a particularly difficult quarter for the infrastructure and team due to unexpected incidents and maintenance work (see next slide).

Yuvi Pandian’s collaboration with the Release Engineering team on its goal of Beta cluster stability was fruitful and resulted in significant improvements in the puppetization of Beta, addressing long-standing problems and technical debt.

Miss

While there was good progress on the team’s goals (both internal, as well as external commitments), limited extra effort was dedicated to longer-term improvements, as the team had to spend a significant amount of time fire-fighting.

The goal of deploying Designate for DNS, although nearly met, was modified slightly to take advantage of the opportunity to make an additional needed change. However this changed the migration to opt-in by users, delaying the completion.

Labs’ and Tools’ overall availability was considerably poor this past quarter with multiple outages for subsets of instances, and performance problems on underlying storage infrastructure (see the Appendix of the team’s quarterly review deck). This affected both foundation teams (e.g. Beta) and the engineering community (Tools).

This is being addressed by dedicating the next quarter to availability improvements and availability measurements, as well as engaging into discussions for increased staffing for the next fiscal year.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Parsoid team

Objective Impact on Goal Status
MediaWiki Developer Summit Successful organization of the first MediaWiki Developer Summit A success, according to the participants' survey and our lessons learned.
Data & developer hub A plan agreed and documented We focused on prototyping first. We have a broad plan that needs to be written down and checked with the stakeholders.
Phabricator migration Consolidate Phabricator as project management tool Good progress overall. Some teams still using Trello. Scarcity of maintenance resources.
Engage with established technical communities Identification of top 5 technical partners and first conversations established Not started.

Full ECT quarterly report: https://phabricator.wikimedia.org/T94165

Success

After organizing the MediaWiki Developer Summit successfully, Rachel Farrand compiled our lessons learned with recommendations for the next edition that are being integrated in ECT's future event plans and the 2015-16 budget ask (i.e. 3 days event instead of 2).

S Page joined our team in January and broke the vicious circle of theoretical discussions around the Data & Developer Hub by putting together a prototype with real content. This is forcing stakeholders and open tasks to be more specific and less opinionated.

S Page volunteered a Trello - Phabricator migration script, and he has driven the migrations of the Collaboration and former Growth teams, extending an offer to the remaining teams using Trello.

Miss

Rachel Farrand and Quim Gil struggled getting clear objectives and main themes with Engineering and Product management, and we were late when announcing the basic plan to our developers. Correction: ECT to own budget and organization of the Summit and hackathons, starting public planning sooner.

S Page has been contacting the right people in Core, Design, and other areas in order to obtain quick tactical progress with the Data and Developer Hub. However, the lack of a written plan and formally agreed dependencies keeps the resourcing of this important project out of the official plans of these teams. Correction: agree the plan with these teams, including committed resourcing.

Quim Gil had proposed 20% Phabricator maintenance time for Chase Pettet (Ops) and Mukunda Modell (RelEng), but didn't follow up for confirmation. While Chase agreed this time with his team, Mukunda has been totally invested in RelEng work. Correction: transfer Phabricator ownership to Greg Grossmeier (RelEng).

For more detail, see the slide deck and minutes from the quarterly review meeting of the ECT

Objective Impact on Goal Status

Prepare for bringing VisualEditor to all new users on wikis where it is currently opt-in.

We will get VisualEditor by the end of Q3 to be ready to be on by default ("opt-out") on the English Wikipedia and other wikis in desktop mode, subject to community happiness, for "new users" – anonymous users and logged in users who have never edited; doing nothing to existing accounts with edits.

Any identified major stability issues fixed – no regressions or data corruptions.

Improved performance, especially on load

Existing features provided, plus in-flight UX and product changes:

Fit-for-purpose special character inserter; blocking ~50 languages which need it, like Navajo and Welsh

GREEN
Objective Impact on Goal Status
Stretch goal

Test bringing VisualEditor to all new users on wikis where it is currently opt-in.

We aim to run a trial for new and IP users getting VisualEditor on by default for a week or two to test the value before the end of Q3. RED

Delayed for quality reasons until after the end of the quarter. To do in Q4 instead.

Success
 
 
 

We met our delivery criteria to improve the product greatly, notably including the auto-filled citations feature which has been very positively received.

We worked openly to burn down our work in partnership with the editing community to nominate blockers (top chart).

Load times hugely decreased: network load time down 41%  (middle chart) and code load down 37% (lower chart).

For many users on most pages, VisualEditor load is now comparable to, or faster, than loading the wikitext editor.

Huge thanks to the entire Editing team & our partner teams for their great work, and particularly Ori, Marielle, Moriel, and Alex who made it happen.

Miss

We did not achieve the stretch goal of running an A/B test of the product for new users on the English Wikipedia. It was delayed for product quality reasons until after the end of the quarter, as we did not feel the auto-filled citations tool was sufficiently reliable to put in front of users, and held that it was so core a part of the user experience that we could not usefully test the effect without it.

We will undertake the test in the coming quarter, instead.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Editing team

Objective Impact on Goal Status
French WP:  Deployment on Help & some Project pages Flow on all Help forum pages, and several Project pages in French WP Held up on pre-filled text and move board, will do full Forum des Nouveaux & Drafts in April.
Portuguese WP: Deployment on significant number of Project pages Flow on all Help pages, significant number of Project pages On Contact WP page. Held up on various feature reqs. Talking about next steps.
Catalan WP: Deployment on all new pages on one namespace Flow on Help, Village pump, and all new pages on either article talk or user talk 3 of 4 Village pump pages. Held up on edit patrolling, will do 4th Village pump & Help in April.
An active non-WP wiki: Convert LQT pages to Flow Flow used as the main discussion tool on Translatewiki, Mediawiki or Wikidata Started converting LQT on Mediawiki; will be done by  April.
Co-op project, English WP:  Flow is a part of the pilot project at launch Mentors and learners in the Co-op project use Flow to talk to each other, with a bot creating Flow boards for each learner BAG wanted better board-deletion features. Co-op excited to include Flow in April.
Success
  • VisualEditor integration in Flow, with a redesigned toolbar, including a custom “mentions” feature to support a core talk page use case. The toolbar can switch between VE and wikitext, and uses VE as wikitext’s “preview” mode.
  • Some much-requested features and tweaks: Editing other people’s posts, Undo functionality, Indentation model.
  • Built Special:EnableFlow, which makes deployments much easier.
  • Hebrew is testing Flow on some user talk pages; this can be a wiki where we can try opt-in Flow for user talk.
  • Made good progress with languages we didn’t expect: Chinese, Telugu, Punjabi.

 

Miss
  • Our first quarter focusing on increased deployment on language Wikipedias. We were too optimistic about the timing.
  • Great interest and engagement with Translatewiki in January, but lost touch suddenly & haven’t been able to engage since. Lost some time building features specifically for them.
  • Shifting focus to LQT transition on Mediawiki increased feature requests.
  • We realized too late that board deletion would be a crucial feature for the English WP Bot Approvals Group, which kept us from the Co-op pilot launch.
  • Dependency on code review from the Search team.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Collaboration team

Objective Impact on Goal Status
[CX] Respond to issues of initial deployment
  1. Triage bug reports and first response within 24 hours
  2. No high-priority bug remains open/unattended beyond 1 week
Talk page queries and Phabricator tickets promptly responded to.

Feature fixes and bigger bugs planned in sprints

[CX] Increase Language Coverage and support
  1. Increase in the number of  users
  2. Integrating new MT service and enable new languages with existing MT services.
  3. Expose user modifications on MT generated content.
Increased coverage from 8 to 20 Wikipedias. Blocked on negotiations with Yandex for legal clearance to use the MT service for more languages
[CX] Improve link editing and creation
  1. Create baseline number of articles translated for languages without MT
Delayed due to urgent development needed for CX Analytics
Objective Impact on Goal Status
[CX] Facilitate finding articles to translate i.e. entry points
  1. Increase of articles being translated per user
  2. Support suggestions from dashboard
Suggestions from interlanguage links, contributions menu & campaign messages during new article creation have been implemented. > 1200 articles translated
[Non-CX] Scoping for focus areas
  1. Collect and analyse relevant data
  2. Prioritize focus areas
  3. Prepare Q4 plan - scope & resource planning
Preliminary statistics collected. Translate and ULS need priority scoping.
Success
 
 
  • Increased precision in the new language activation process and follow-up on bugs and enhancements. 23 Wikipedias currently include CX. Collaborated with Tech Ops & Release Engineering to achieve this.
  • Community requests helped prioritize language deployments and continuous interactions identified failures & special cases.
  • Negligible deletion ratio of articles created with CX; only 2 articles deleted.
  • Entry points improved discoverability. Sharp increase seen in the number of users activating the tool through new entry points. Design team will continue exploration.
Miss
 
  • Collection of CX analytics data required additional developer overhead and out-of-turn planning. Increasing the scope of analytics support for the team would be helpful.
  • As a fallout, restructuring the links framework had to be delayed to Q4
  • Unable to complete the inclusion of new machine translation system - Yandex, due to pending legal clearance. This reduced the scope of supporting machine translation for more languages
  • Need to better anticipate translation patterns for individual languages and help users translate regularly. Given the diverse factors we have not yet found a model to better explain the trends.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Language Engineering team

Objective Impact on Goal Status
Fully instrument editing experiences across devices/platforms to support VisualEditor development Collect and continually visualize key metrics including but not limited to:

- initialization time and save time (perceived and actual)

- edit funnel from start of an edit action to successful save

- save rates, abort rates, error rates

across experiences: mobile, desktop, wikitext, VE.

Prioritization of metrics will be driven by VisualEditor success criteria.

First rev dashboard delivered early April.

Investigating data integrity issues with wikitext instrumentation.

Develop a prototype report and visualization on Unique Clients per project per day and month Delivery of report to provide insights on readership trends Still blocked on Ops support to integrate with Varnish
Success

Editing Dashboard and other VE analytics support

Close partnership with VisualEditor team this quarter has been key to developing consistent instrumentation and data visualization appropriate to the use case.

Unique Clients

The team designed a novel method to count unique visitors using a cookie and a last-access date; no pervasive tracking needed.

Miss

Editing Dashboard

Completed a week after Q3 due to lateness in getting wikitext editor data for comparison (VisualEditor-related work was prioritized more highly). Wikitext work landed late and there are still data integrity issues with the instrumentation.

Unique Clients

The project is blocked on prioritization with Ops. Need to establish clearer alignment around shared objectives early-on, possibly as part of goal-setting process changes.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Analytics Engineering team

Objective Impact on Goal Status
Data services

• Prototype revscoring service

Have a working prototype up on Labs of a public, query-able APIs of machine classified scores of Wikipedia revisions. Released libraries the service depends on, tools for handcoding  and project documentation.  API up on Labs
Traffic reporting

• Finalize Unique Clients requirements

• Support AnEng with the productization of Page Views reports

• Continued responsibility for ad hoc traffic analysis and reports

Clear all research dependencies for productizing traffic reports by Analytics Eng Completed QA of data based on new pageview definition and supported AnEng with the design of UDFs. PV reports as of Q4 will be automatically generated from Hadoop.

Provided methodology and requirements for counting Unique Clients, implementation is pending.

Mobile research

• Continue research support to WikiGrok and microcontributions

• Research support for Project Gather (aka Collections)

Deliver analysis of an AB test for WikiGrok on stable, perform all supportive analysis needed by the team to decide on next iteration Completed reports on WikiGrok test.

Provided ad-hoc support to Gather team prior to launch.

Objective Impact on Goal Status
Fundraising research

• Develop proposal to run FR as a minimization problem

[ Better AB testing methods,

Characterizing low value impressions ]

Deliver and publish AB test methodological proposals Completed analysis of Big English. Released tool with traffic projections.Published methodological reports: 1 - 2; #1 is  now used as part of ongoing FR analysis
Formal collaborations

Wrap up Improving linking structure

Kick off Increasing article coverage

Complete report on improving link structure Improving linking structure report in progress. Algorithm implementation being tested

Work on article creation recommenders started.

Success

Achievements

The team showed strong  leadership in solving complex problems:

  • Aaron took the lead of a team of grantees to do R&D work on first AI-as-a-service prototype.
  • Leila performed research on microcontributions, led research with Stanford’s InfoLab team, and started a new collaboration with Stanford’s DeepDive team.
  • Ellery redefined our testing methodology for FR and started work on big data services.
  • Oliver completed work towards a new, robust page view metric

Other Q3 achievements beyond stated goals

(received great support from Team Practices, Legal, Analytics Eng, Ops, Mobile for all these projects)

Miss

The completion of link recommendation is delayed due to the need (identified through research in Q3) for an improved algorithm. There are edge cases in which the algorithm doesn't perform well. We are 1) assessing the quality of the algorithm, 2) doing experiments with improved versions. We commit to having a working version of the algorithm by 5/8, along with its quality assessment, a research page report and a paper submission and a release of the underlying data (aggregate browsing traces) for the community.

The team provided continuous responsibility for ad hoc reporting and querying (both at team- and org- level), lacking self-service reporting infrastructure for key metrics: while we substantially met our stated goals, ad hoc reporting slowed down our progress.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Research & Data team

Objective Impact on Goal Status
Provide dedicated Team Practices Group support requested by product/engineering teams. Hire an additional TPG resource to work with the MediaWiki Core team Kevin Smith hired, started 3/9/2015
Identify patterns and trends in team health across WMF product/engineering teams. Publish Q2 health-check survey results Data published and presented to EMGT 3/9/2015
Measure and support engineering/product team health of WMF product/engineering teams. Iterate on and deliver updated health-check survey to same teams as Q2 plus Editing and Community Liaisons Q3 health check surveys completed week of 3/30/2015
Support team development across WMF engineering Conduct one structured team practices workshop Deferred due to org prioritization of VE release
Support realization of top priority organizational goals. Provide dedicated TPG support to the Editing team. Joel Aufrecht hired, started 4/6/2015
Success
  • Mid-quarter pivot to support Call to Action and top organizational priority
    • Built trust and laid groundwork for enabling dedicated TPG resourcing of Editing team
      • Grace Gellerman
    • Discovering capabilities and limitations of Phabricator as a PM tool
      • Executive ask for burndown charts
    • Streamlining organizational quarterly goals process
  • Launched cross-team initiative to decrease tech debt and increase automated testing
    • Discovered trend of not enough automated testing/too much tech debt from team health checks
    • Collaborating with Release Engineering to facilitate improvements
    • Kristen Lans, Grace Gellerman, Kevin Smith, Dan Duvall (RelEng)
Miss
  • Ability to support team health and facilitate changes in service to the Call to Action
    • Cross-org and inter-team communication issues
      • Exacerbating entropy, rumors, friction
      • Increasing stress, causing drag on team health and sustainability, slowing our ability to facilitate change
    • Phabricator is a better bug tracker than project management tool
      • Burndown chart functionality limited, misleading
      • Project management features lacking and require workarounds
        1. Collaboration with Release Engineering planned to facilitate improvements
  • Ability to support product/engineering and the broader organization
    • Could exceed expectations if we could move faster through the hiring process
      • Friction experienced with: securing req numbers for out-of-band hires, lack of sourcing, recruiter turnover, candidate interviews with execs, offer letter approvals

For more detail, see the slide deck and minutes from the quarterly review meeting of the TPG

Objective Impact on Goal Status
UI Standardization: Full coverage of iconography in core and WMF-deployed extensions with mw.ui icons Standard iconography available to all projects Completed (iconography)

Living Style Guide

Hire Sr. Designer with technical focus for partner projects and other feature areas Accelerate UX standardization efforts Not CompletedOne candidate reached final stage but did not pass; slow pipeline
Release Hovercards to selected wikis based on community requests   Completed**zh.wiki on hold pending review of API load analysis
Extend MediaWiki theme and Living Style Guide to include all components used by WMF projects   Not Completed

High VisualEditor priority has impacted OOUI/OOJS devs

OOJS controls for all in-use (core) controls completed   Not Completed

High VisualEditor priority has impacted OOUI/OOJS devs

Success

Achievements

Foundational work for UI Standardization

  • Icon library completed and integrated into core/extension
  • Technology for UI Standardization resolved to us OOJS, OOUI framework
  • Final designs for core component libraries completed

User Research

  • Extensive evaluative testing completed for top tier projects (VisualEditor, Flow, Apps)
  • Infrastructure and test plan for REFLEX qualitative readiness testing completed
  • Provisional persona content created and iteration workshop being defined.
Miss

Completion of UI Standardization library did not occur due, primarily to lack of engineering resources, which were redirected to VisualEditor.

Pilot tests for testing tools (Loop11 and UserZoom) to support implementation of REFLEX.

For more detail, see the slide deck and minutes from the quarterly review meeting of the User Experience team

Objective Impact on Goal Status
Implement standardized product development methodology which includes community input across teams working on user-facing functionality
  • Publicize integrated draft process which addresses all stages of a product's lifecycle and incorporates data-based decision-making and evaluation, qualitative research, and community input
  • Revise process based on feedback from Wikimedia communities (at least one well-publicized online meeting; outreach on lists and wikis).
Largely deferred:

Community Engagement group was moved out of Product in February (restructuring), and overall focus has shifted to prep for VisualEditor release.

Product process overhaul is being subsumed into engineering/product re-org, which was jointly led by engineering & product management.

New goalsetting/prioritization model for the organization is being developed under leadership of the COO.

Success
  • Leadership team in engineering/product have collaborated on development of new org structure designed to increase velocity of engineering/product work and to build next level of highly autonomous product leadership.
  • Clear focus on VisualEditor as top priority has galvanized teams to deliver a high quality product.
Miss
  • Pending or completed reorgs in engineering/product/community made for a turbulent quarter with significant handover and re-negotiation of commitments mid-quarter; process improvement goals were backburnered.
  • Product Management is under-staffed due to staff departures and delayed hiring on new PMs until the new organizational structure is in place. Teams are spending more time on technical debt / backend work due to lack of PM support.

For more detail, see the slide deck and minutes from the quarterly review meeting of the Product Management team