Wikimedia Foundation Report, October-December 2015


See also the original PDF version of this report





Wikimedia Foundation

Quarterly Report

October — December 2015
Q2 of the fiscal year

Foreword

edit

We are pleased to bring you the Wikimedia Foundation’s Quarterly Report for Q2 of the 2015/16 fiscal year, a comprehensive summary of how we did on the objectives defined earlier in our quarterly goal setting process. We have continued to optimize the report’s format and the organization’s quarterly review process based on the feedback that we have received.

Teams have highlighted key performance indicator (KPI) - with ongoing efforts to identify the best possible metrics - and to estimate how much time fell into each of the three categories from the 2015 Call to Action (strengthen, focus and experiment). We have organized the content of team reports to present all the information that is related to a particular objective in one place (description of the goal, measures of success, how we did on achieving the objective, and what we learned from working on it).

As before, we are including an overview slide summarizing successes and misses across all teams. In a mature 90-day goal setting process, the “sweet spot” is for about 75% of goals to be a success. Organizations that are meeting 100% of their goals are not typically setting aggressive goals.

Wikimedia Foundation Leadership Team


Quarterly metrics scorecard (beta)

edit
         Participation
Active editors
(5+ edits/month)
75.5k/mo +0.6% from Q1 (July-Sept)
+0% y-o-y
New editor activation
(not directly comparable to active editors)
22,531/mo -3.7% from Q1 (July-Sept)
-10.2% y-o-y
Edits via mobile
(on WP, percentage of total)
3.9% Q1: 3.0%
Q2 2014-15: 2.2%
         Site reliability
Read uptime 99.979% +0.037% from Q1 (July-Sept)
y-o-y: N/A
Write uptime 99.974% +0.039% from Q1 (July-Sept)
y-o-y: N/A
Read latency
Median first paint time
865ms Q1: 642ms
Write latency
Median page save time
1056ms Q1: 1042ms
         Readership
Page Views
Crawlers excluded
15.6B/mo +1.3% from Q1 (July-Sept)
approx. -8.8%  y-o-y
          Content
New articles 16.7k/day +119% from Q1 (July-Sept)
+126% y-o-y
Edits (in Wikipedia articles) 10.52M/mo -4.2% from Q1 (July-Sept)
+9.1% y-o-y
         Fundraising
Amount raised $46.9M
(exceeded $34M target)
+$2.9M y-o-y
On mobile

(Dec. 2015 only)

20% Q1: 12%Dec. 2014: 20.8%

Q2 2015/16: Successes/misses by team

edit
 

Q2 2015/16 objectives overall

edit

142 objectives
34 teams

 

   92 successes (65%)
   50 misses (35%)

Misses include anything not delivered as planned, including “yellow” goals and those that were intentionally revised or abandoned due to changing priorities.


Quarterly review

Q2 - 2015/16

6
For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department
Q2 - 2015/16
Approximate team size during this quarter: 7
Time spent: strengthen 20%, focus 50%, experiment 30%

Key performance indicators

 

SLA for Trust & Safety correspondence: Resolving 95% of emergency@ within three hours 100% +/- 0 change from Q1 (100%) +/- 0 change YTD (100%)
SLA for public correspondence: Resolving 95% of answers@ and business@ within two business days 100% +3.5% change from Q1 (97%) +3.5% change YTD (97%)
7
All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Community Advocacy  

Objective: Harassment strategy

 

Objective Measure of success Status
Innovate improved  community health through creation of a community-informed harassment strategy.
Team members involved: 6
  • Launch user experience survey
  • Launch first round community consultations (30 days), engage with community throughout
Survey released on schedule, with 3,845 responses from 14 projects. Consultation launched on schedule and generated many suggestions for next steps on addressing harassment issues.

Notes: Survey went smoothly given scale with some wrinkles. Many volunteers helped make it and the consultation happen, with pre-release feedback and translations. The consultation was kept open beyond 30 days to permit more time for feedback.

Learnings: Launching surveys in the modern movement is complex, and understanding the requirements and reaching out to stakeholders was more time-consuming than expected. Future surveys will build in more time for procedural complexity. We also learned that we need better advanced coordination with other teams, like Community Resources, on how to pipeline community-promoted initiatives generated by such consultations.

8

Q2 - Community Advocacy  

Objective: Refining T&S processes

 

Objective Measure of success Status
Improve execution of trust & safety by refining and clarifying processes.
Team members involved: 2
  • New child safety (images) policy created and submitted for manager approval
  • Implementation of new child safety (images) policy
The updated child safety (images) policy is underway and much research was put into it, including with the assistance of legal, but the policy update is not complete.

Notes: Other focuses at the Foundation particularly around strategy limited staff capacity, and the refinement of this process - given that we do have a functional policy - was deprioritized to allow us to complete other projects.

Learnings: The complexities of laws regarding processing inappropriate images of minors internationally makes establishing an international team for this workflow unlikely.

9

Q2 - Community Advocacy  

Objective: Functionary liaison

 

Objective Measure of success Status
Focus on community health by supporting and facilitating improved collaboration of key functionary groups.
Team members involved: 4
  • Successfully host conference of international stewards in San Francisco
  • Collect, analyze, and reasonably implement feedback on how we can improve services to this group
The conference took place in October, with 15 attendees out of 37 invited. All stewards - not just those in attendance - were polled about how we can better assist, and a report completed on 25 November. CA is coordinating with other departments and teams to meet steward needs.

Notes: The Stewards in attendance voiced strong opinions that the meetings were useful for them to connect on better ways to conduct their work and better ways to coordinate with the Wikimedia Foundation on supporting the movement through their role. As a cost-saving mechanism, it would be worth exploring whether we could in future add special functionary days to broader movement meetings, like Wikimania, with funding provided for the extra day of attendance or as part of the scholarship work.

10

Q2 - Community Advocacy  

Objective: Community communication.

 

Objective Measure of success Status
Devise and implement an experimental plan for a central community hub that will allow community predictable and reliable access to WMF staff.
Team members involved: 6
  • Present proposed community hub plan for DCE, C-level and ED review
  • Refined per feedback, launch experimental community hub
The design of the experimental hub was completed and approved. However, implementation was put on hold in order to maintain capacity for CA to more fully engage with the WMF strategy process.

Notes: The design was completed on schedule and cleared the review period, but put on hold to keep capacity for strategy. If capacity permits, it will be launched in Q3, but is not on Q3 Quarterly Goals due to need to retain availability for strategy consultation. It may be combined with hub pages serving communications and programs/affiliates into one hub.

Learnings: Remain flexible on release dates, but try not to release heavy engagement projects around major holidays.

11

Q2 - Community Advocacy  

Objective: Maintain the core.

 

Objective Measure of success Status
Maintain the core.
Team members involved: 7
  • Execute and maintain core workflows with 95% of inquiries responded to within 2 business days.
CA met KPIs related to defined inquiry paths and maintained core workflows in spite of staffing transitions.

Learnings: While CA grew in new ways this quarter with the addition to our team of senior strategist Haitham Shammaa to help guide us in new directions, core work continues to consume the bulk of our time and attention. Through consultation with Terry, Luis and Lila, CA learned that we need to protect the focus on our core even though that will mean committing to fewer additional projects. Our focus now is on using our added capacities to ensure that additional projects are properly focused to nurture global community health.

12
Q2 - 2015/16
Approximate team size during this quarter: 7 FTE (6 FT, 2 PT)
Time spent: Strengthen 30%, focus 50%, experiment 20%
Key performance indicators:

 

Individual contributors with 2+ edits in product pages
Q2/O - 915
Q2/N - 920
Q2/D - 929
+3.17% from Q1 N/A YoY
Page Views
Q2/O - 24,812
Q2/N - 22,891
Q2/D - 20,775
-7.31% from Q1 N/A ToY
13
All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Community Liaisons

Objective: Improve understanding and input

 

Objective Measure of success Status
Goal 1 - STRENGTHEN
Improve community understanding of and input to product development
Team members involved: 2
  • Creation of 5-10 Principles for Design and Development, submitted and ready for presentation in January
  • Finalize definitions of Community Liaison toolkit within product development
  • Community Liaison toolkit incomplete
The schedule for Design and Development Principles was built with delays for survey response and peer review, but end of year scheduling conflicts, holidays and vacation time created additional delays on obtaining signoff.
Discussions around the toolkit continue into Q3
14

Q2 - Community Liaisons

Objective: VisualEditor’s progress at enwiki

 

Objective Measure of success Status
Goal 2 - FOCUS
Focus on VisualEditor’s progress at enwiki
Team members involved: 4
  • VisualEditor on more accounts and logged-out users on the English Wikipedia; opt-in visible to all.
  • Goal changed after quarter start
  • Single edit tab development work still on-going, expected early Q3.
Goals changed after quarter start for quality reasons: brought forward work from Q4 as new blockers.

Erica was instrumental in supporting conversations around VisualEditor at es.wp, resulting in VE being turned on per request, as well as reactivating discussion about launching VisualEditor at nl.wp

15

Q2 - Community Liaisons

Objective: Future Planning

 

Objective Measure of success Status
Goal 3 - STRENGTHEN - Future planning
Team members involved: 3
  • Annual Planning process and finalize team narrative for upcoming year
  • Make an offer to a community liaison for the Discovery team
  • Annual planning process beginning Q3
  • Offer to CL candidate made after goal deadline of 11/15
Our goal was to extend an offer to a Community Liaison before Thanksgiving. After adjusting the job description in mid-November, we extended an offer which was accepted by the end of the quarter.
16

Q2 - Community Liaisons

Objective: Maintain current workflows

 

Objective Measure of success Status
Goal 4 - CORE
Maintain current workflows
Team members involved: 7
  • Providing designers, PMs, and engineers with community feedback for products
  • Participating in Reading strategy
  • Maintaining Tech/News
  • Teams assigned into: Editing, Collaboration, Community Tech, Reading, Analytics
 


17
Q2 - 2015/16
Approximate team size during this quarter: 3 FTE
Time spent: strengthen 60%, focus 20%, experiment 20%

Key performance indicator (NOTE: we are going to change KPIs)

 

Users of Wikimedia Web APIs N/A (T102079) N/A from Q4 N/A YoY
18
All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q1 - Developer Relations  

 

Objective Measure of success Status
Goal 1: STRENGTHEN

Successful organization of the Wikimedia Developer Summit 2016

Team members involved: 2
Logistics of a 3 day event in 2 locations are all solved.
The schedule reflects WMF Engineering priorities.
Scheduled sessions are preceded by online discussions involving their stakeholders.
25% of registration are volunteers or third party developers.
Successful logistics 2+1 days.
Successful program organized in 5 areas, with pre-scheduled sessions and unconference.
Successful prior discussions and note-taking.
BUT only 15% volunteers / 3rd parties (25 total, last year was 15 - 10%).
Rob Lanphier (program) and Valerie Aurora (unconference) were key contributors to the event.
Attracting more volunteers and 3rd party developers in the Bay Area requires a different type of event and promotion.
19

Q1 - Developer Relations  

Objective: Code Review performance

 

Objective Measure of success Status
Goal 2: STRENGTHEN
Define potential actions to reduce code review queues and waiting times.
Team members involved: 1
Proposal to address the code review queue of changesets submitted by volunteers, presented to WMF Product, Technology and Team Practices, and to our developer community.
Extensive research was made, and a draft identifying problems exists.
However, we still haven't presented a proposal yet.

This was a prominent topic at the Developer Summit, and we are integrating the outcome.

This goal is a stepping stone toward the definition of a Service Level Agreement by the WMF.
Partly thanks to DevRel's work in this area, today WMF developer teams agree that there is a problem with code review that needs solving.
20

Q1 - Developer Relations  

Objective: Third Party Developers

 

Objective Measure of success Status
Goal 3: FOCUS
Map current use of Wikimedia APIs
Team members involved: 1
List of known users of Wikimedia web APIs specifying main uses.
Metrics of quantitative use of these APIs
We ended up without resources for this goal.
Strategic Partnerships worked on a survey, but that is not enough to have the list of known users.
Quantitative metrics are still missing. We depend on Reading Infrastructure.
The root of the problem still is the lack of a clear WMF strategy around third party developers.
DevRel has no resources to research and reach out to 3rd party developers; our few hands are full with open source contributors to our projects.
21

Q1 - Developer Relations  

Objective: Code of Conduct

 

Objective Measure of success Status
Goal 4: FOCUS
Binding code of conduct for all Wikimedia technical spaces
Team members involved: 1
Code of Conduct for technical spaces approved
Committee created
The Code of Conduct is still being drafted. The main part has been reviewed. We are defining processes with support from specialists.
The approval process is not defined yet.
The creation of a committee has not started.
Matthew Flaschen, Moriel Schottlender, and Frances Hocutt are driving the drafting in a volunteer capacity.
WMF Community Advocacy, Community Resources, and Legal are contributing as well.
We agreed to contract services from Valerie Aurora and Ashe Dryden to provide specialist advice.
22

Q1 - Developer Relations  

 

Objective Measure of success Status
Goal 5: EXPERIMENT
Organize the start of Outreachy round 11
Team members involved: 2
Organization driven by volunteers.
5 featured project ideas.
Selection process completed and documented.
Tony Thomas is the main org admin. He is a volunteer, former mentor and intern.
We had 9 featured project ideas.
The selection was completed, refreshing the related documentation in the process.
The Winter round of Outreachy is a good occasion to onboard volunteer org admins because the activity is lower than in the Summer round with Google Summer of Code.
There were more candidates selected by our mentors, but a new rule by Outreachy discarded all candidates with full-time studies during the internship.
23

Program Capacity & Learning

edit
Q2 - 2015/16
Approximate team size during this quarter: 11.3 FTE
Time spent: strengthen 82%, focus 11%, experiment 7%

Key performance indicators:

 

TWL Unique recipients 2598 +  8.98% from Q1 + 23% YoY
Education program leaders served 229 (85 countries) + 22% (+ 39%) from Q1 YoY n/a
Community leaders engaged - learning and evaluation 96 - 49% from Q1 YoY n/a
24
All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Program Capacity & Learning  

Objective: Learning Infrastructure
(3 staff involved)

 

Objective Measure of success (direct project hours) Status
Focus
  • CR Quarterly Metrics roll-up & tracking support ✓
Complete.
Strengthen
  • Complete remaining program evaluation reports for 2015 ✓
  • Proposal Analysis for Round 1 APG ✓
  • Program design toolkits UX design research design ✓
  • Review of TWL branch outcomes: Review and revise setup guide ✓
Experiment
  • Global Metrics API tool successfully piloted in a partnership with analytics simplifying the pulling of global metrics by staff and volunteers by over 99% (estimated saving 800 hours of labor time per year for staff and volunteers reporting on grant funded programs)  ✓

The Global Metrics API (appreciatively referred to as the Magic Button) takes the pain out of data collection for staff and volunteers identified in the most recent grants consultation.   A collaborative partnership effort Amanda Bittaker (PC & L)  Dan Andreescu Analytics)  and Madhu Viswanathan (coding).

25

Q2 - Program Capacity & Learning  

Objective: Program Infrastructure
(6.3 staff involved)

 

Objective Measure of success Status
Focus
  • Sustain and expand TWL branch activity & sharing of best practices ✓
  • Monthly TWL branch check-ins, document new community models ✓
Complete
Strengthen
  • Support pilot of a university course in Oman & two in Palestine ✓

  • Enhance editor access to research with new major partners and by extending the reach of those accounts ✓

Complete
Experiment
  • Build for a more efficient, robust, inter-connected TWL platform for the future  X
Incomplete

  Mature Wikimedia programs demonstrating scalability and impact require infrastructure and support for

  continued  growth and impact.  This includes documenting best practices, community leadership development
  global program advocacy and tools to support community volunteers. The road map work is
  helping this integrated team prioritize supports across programs and communities.
  
26

Q2 - Program Capacity & Learning  

Objective: Community Leadership Development
(5 staff involved)

 

Objective Measure of success Status
Strengthen
  • Community Knowledge Sharing: Consultation with WMDE on program design for Wikimedia Conference 2016 and WMIT for Wikimania 2016 ✓
  • Transition AffCom to staff supported model under PC&L ✓
  • 10 Community leaders recruited & onboarded as conference faculty ✓

  • At least 2 established Central & Eastern Europe education program leaders mentor 2 less experienced leaders.

  • Identify and scope 2-3 projects with affiliates for Q3 and/or Q4 (e.g. Survey Bank) (10) X
Incomplete
Successes and Miss
  • We were able to launch the staff supported model for AffCom including 92 affiliates  reviewed for Wikimedia Conference eligibility status, 28 affiliates renewed through reporting updates.  Special appreciation to Jaime Anstee who has taken on a leadership role collaborating with Greg Varnum and Community Resources in this transition.  
  • Full Affiliate project  scoping delayed due to roadmap work and transition to AffCom staff model.
27
Q2 - 2015/16
Approximate team size during this quarter: 8.75 FTE (+2 part-time interns)
Time spent: strengthen+core 80%, focus 10%, experiment 10%
Key performance indicators

 

People supported
Global metrics from reports by resourced initiatives this Q
59,428
total individuals involved
2,324 new editors
(4% of total)
945 active editors
(2% of total)

 

Grants to
Global South
approved this Q
27 grants $527,291
48%
of total #
12%
of total $
-72%
# from Q1
+61%
$ from Q1
+93%
# YoY
+17%
$ YoY
28
All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Community Resources  

Objective 1: Pilot Simple APG

 

Objective Measure of success Status
Team members involved: 1
  • Pilot launched with 5 participating organizations; applications submitted and approved for first round
  • V1 best practices/guidelines for organizations
  • Advisory committee onboard
  • 1 application withdrawn (due to local regulations, in resolution)

  • 8-person committee with expertise from PEG, IEG, FDC, and AffCom delivered quality recommendations

Learning: Applicants need hands on support from staff and committee members to improve their applications throughout the review period; but with that support, results are great!

People: Winifred Olliff, 8 volunteers on the Simple APG Committee, the people behind our 5 amazing applicant organizations, and our volunteer and staff advisors, are making this pilot happen :)

29

Q2 - Community Resources

Objective 2: Community Capacity

 

Objective Measure of success Status
Pilot Community Capacity Development support with 2 emerging communities
Team members involved: 1
Q2: 2 pilot projects launched, each focused on developing 1 capacity with 1 emerging community.
Behind schedule: 3 communities on board, projects are still being designed until end of Jan, haven't launched yet.
Q3/4: Pilot communities meet planned measures of success, demonstrate a move from NO/LOW capacity to SOME capacity in the target area On track

Success: three emerging communities (Ukrainian [Ukraine], Portuguese [Brazil], Tamil [India]) engaged, toward building three different capacities, will allow us to maximize learning from first round of pilots

Miss: Planning phase took longer than expected; WMF turmoil and natural disaster (floods in Chennai) contributed to delay. Compressing community conversations to catch-up isn’t an option for building successful pilots, however.

30

Q2 - Community Resources

Objective 3: Wikimania planning

 

Objective Measure of success Status
Build a sustainable strategy for supporting Wikimania 2017-2021
Team members involved: 4
  • Decision finalized and announced  for 2017 team
Montreal confirmed as host for Wikimania 2017

 

  • Input gathered for 2018-2021 strategy from Wikimania steering committee, WMF staff and at least 50 community participants
  • 2018-2021 strategy draft begun on meta-wiki
  • Consultation launch was delayed due to other WMF priorities in Q2. Expected to close in January of Q3.

Miss: Q1 communications regarding selection process for Montreal were not clear. Revamped Wikimania planning process to come from consultation is aimed at avoiding repeat of this scenario for future years.

Learning: A dedicated Program Officer is needed to own events strategy overall, including integration between global and regional events. Much director (and other staff) time is spent filling in gaps.

31

Q2 - Community Resources  

Objective 4: Maintain Grants Core

 

Objective Measure of success Status
  • Maintain full coverage for
Annual Plan (APG),
Project and Event (PEG),
Individual Engagement (IEG) grant program workflows
Team members involved: 7 (CR)
  • All PEG, IEG and APG workflows are fully staffed for H1 2014/15
  • Meeting commitments for 10 of 10 grantmaking workflows on-time
  • APG core includes WMDE’s proposal for a restricted grant for Wikidata
  • Kacie Harold maintaining PEG, with 17 funded requests this quarter.
    • 2 new applicants from Inspire!

    • ½ are tools-focused (most ever!)

  • FDC recommended ~$3.77 million in Annual Plan Grants, including restricted funds for WMDE’s work on Wikidata.

Success:  ½ of new IEGs are focused on tool-building, with partnership with CE-Software team and easing of IEG eligibility restrictions. Thanks to Marti Johnson, Quim Gil, and the IEG committee!

Learning: More engineering support is needed for reviewing APG and IEG grant proposals, given increasing technical focus of some grant requests.


32

Finance

edit
Quarterly reviewQ2 - 2015/16
Approximate team size during this quarter: … 5.5
Time spent: strengthen 35%, focus 60%, experiment 5%
33
All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Finance  

Objective: Implement Purchase Order

 

Objective Measure of success Status
Implement purchase order system to be utilized by departments and teams to make purchases.
Team members involved: 3
100% of all purchases are processed via the purchase order system.  
Project was not started due to the departure of both the CFA and COO. Also, this is put on hold until we hire a Purchasing Specialist and finalize a purchase order workflow/process.
Project will be discussed and reviewed with the new CFO to determine when it’s feasible to commence and complete. Commencement date is TBD.
34

Q2 - Finance  

Objective: Annual Planning Process

 

Objective Measure of success Status
Create a master calendar for the Annual Plan process.
Team members involved: 2
A standard and sustainable calendar that can be used for the current annual plan as well as future years.  
For the FY 16-17 Annual Plan, a calendar was created to accommodate the FDC and community review and was approved by the C-Level team. Since this is our first year following the FDC timeline, we don’t know whether this can used as a master calendar going forward. However, we will incorporate all lessons learned from this year’s process and iterate as necessary.
35

Q2 - Finance  

Objective: Optimizing Concur For Credit Card

 

Objective Measure of success Status
To optimize Concur for our monthly credit card reconciliations
Team members involved: 4
All credit card reconciliations are being completed by the 8th of each month.
A majority of the reconciliations are actually submitted by the 4th business day of the month which is earlier that the 8th. At times, there are late submissions due to unforeseen events (e.g. lost supporting documents or difficulty in obtaining supporting documents from vendors).
36

Objective: Baseline Integration Process with HRIS and Payroll

Q2 - Finance  

 

Objective Measure of success Status
To create a baseline integration process between HRIS/Payroll and Intacct.
Team members involved: 2
Completion of integration process.
We currently have a manual integration process between ADP Payroll and intacct. Since we have not implemented a new HRIS/Payroll platform, we did not commence the work on creating a baseline integration process.
37

Talent & Culture

edit
Q2 - 2015/16

Quarterly review

Approximate team size during this quarter: 7
Time spent: strengthen 50%, focus 30%, experiment 20%
All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Talent & Culture - Boryana

Objective: T&C

 

Objective Measure of success Status
Boryana onboarding - Culture discovery process
Team members involved: 3
Staff involved: numerous
Document, share & seek calibration & final confirmation of cultural pillars Completed and posted summary on personal page for feedback and comments
T&C team strategy
Team members involved: 2
Staff involved: numerous
Defined & documented strategy for T&C team, gain leader consensus Strategy completed and shared with stakeholders. Strategy will get refined and project priority order may change depending on org needs.
Board Recruiting
Team members involved: 2
Other involved: staff, community, board, BGC, advisory board, etc.
2 trustees appointed by board
2 trustees appointed by board
39

Q2 - Talent & Culture - Boryana

Objective: T&C

 

Objective Measure of success Status
Boryana onboarding - Culture discovery process
Team members involved: 3
Staff involved: numerous
Document, share & seek calibration & final confirmation of cultural pillars Completed and posted summary on personal page for feedback and comments
T&C team strategy
Team members involved: 2
Staff involved: numerous
Defined & documented strategy for T&C team, gain leader consensus Strategy completed and shared with stakeholders. Strategy will get refined and project priority order may change depending on org needs.
Board Recruiting
Team members involved: 2
Other involved: staff, community, board, BGC, advisory board, etc.
2 trustees appointed by board
2 trustees appointed by board
40

Q2 - OD & Recruiting - Amy & Boryana

Objective: Recruiting

 

Objective Measure of success Status
OD Team Hiring
Team members involved: 5
Staff involved: numerous
Hire Learning & Development Manager PENDING - Finalists selected. Working through final logistics
Engineering Hiring Event - phase 1
Team members involved: 3
Staff involved: 3
Number of attendees
PENDING - Wikimedia Tech Talk scheduled for February 24 2016: 2 speakers (Moriel, Aaron H)
Agency vendor streamline
Team members involved: 3
Bring all existing agency relationships into Recruiting
Create process for Recruiting to be the only point of entry for agencies
Create a preferred and approved agency list that will have standard terms and conditions
POSTPONED - This goal became less of a priority, thus postponed,  as all recruiting needs were able to be handled in-house.
41

Q2 - Recruiting - Amy & Joady

Objective: Job Descriptions

 

Objective Measure of success Status
JD project
Team members involved: 6
JDs reviewed, streamlined & leveled (for Engineering - other depts to follow)
Recruiting created template level JDs with HR for Product for each band.
Draft Product level chart created in FQ1 by HR, to coordinate with Radford data.
Additional changes in Product/Engineering teams & roles made additional work on this goal challenging.  Draft work has been finished but additional work with managers is needed before it can be rolled out to staff.

Learning: Blanket JDs for all Product will not work considering the variations required for each team, but general levels can be detailed.  Closer coordination with Radford formats will provide the best matching to compensation and experience.

42

Q2 - HR Ops - Joady  

Objective: Open Enrollment

 

Objective Measure of success Status
Successful Open Enrollment
Team members involved: 3
Benefit budget increase kept under 13.6% (the budgeted amount for FY15-16)
Open enrollment completed on time by December 12/11/16
Increase reduced by 5.7%, to 7.9% (95k annual savings)
Open enrollment completed on schedule, with only 2 staff members requesting extensions due to their schedules.
It has been HR’s mission to create a quick and simple open enrollment, that minimizes work for staff, and maintains strong, stable & cost effective benefits.  

Learning: Staff continue to value high contact with HR during open enrollment.  Our brokers at Johnson & Dugan continue to represent WMF well to reduce carrier rates.  PlanSource had some back end glitches due to a recent upgrade on their part, but staff did not experience it.

Elena successfully managed open enrollment, including communications to staff, several open office hours for staff, updates of all changes into staff plan materials, and coordination of the entire process with the brokers & carriers.  Dan assisted with data coordination.
43

Q2 - HR Ops - Joady

Objective: Engagement Survey

 

Objective Measure of success Status
Run an updated engagement survey for req number staff, with high participation.
Team members involved: 3
Run 2015 engagement survey for req# staff with 75%+ participation
Exceeded participation by 18% with 93% (2013 was 58%)
Results shared on time
Expanded goal: Added contractor survey completed with 84% participation
This was a last minute goal that was added for HR, and was able to be run on short notice. The project was expected to be a challenge both in managing expectations.  

Learning: Next time conduct a simultaneous contractor survey, with questions limited to those that pertain to them. Include input from a broader group (than HR, Comms & Legal) in the planning process for the survey timing and sharing of results.

Anna successfully led the survey logistics, including: vendor selection process, question coordination with Culture Amp/HR/Legal/Comms, encouraging staff participation, running input sprints, and the added survey for contractors.  Dan assisted with demographics data & reporting.
44

Q2 - HR Ops - Joady  

Objective: Onboarding/Offboarding

 

Objective Measure of success Status
Req# onboarding & offboarding streamline - phase 1
Team members involved: 4 (plus 8+ non T&C staff)
HR, OIT & Administration has a fully documented process that integrates into a master process list.
Process maps for on & off boarding for HR, OIT, Admin, Finance & Tech Ops created on time and approved by respective teams.
With the process map created, this will help with the next phase of creating new systems to automate and streamline much of the processes, with the new HR Programs & Systems Manager.

Learning: Cross team collaboration can be challenging, with each team having different agendas and a different level of preferred detail.

Anna Lantz successfully led the project coordination, following up with all non T&C team members, making sure they stayed on track and informed.
45

Q2 - OD -  Joady

Objective: Online Training

 

Objective Measure of success Status
Implement Training Resources and System
Team members involved: 2 (and Legal)
Roll-out Mindflash for training access and tracking
Mindflash rolled out on time, with 5 T&C modules and 1 Legal module
HR and Legal needed a learning system that allowed for interactive training, that could be tracked and completion confirmed for risk management.  Legal selected Mindflash in FQ1, with the system rolled out in FQ2.

Learning: More coordination will be needed for linking what staff want to have on Office Wiki and what is in Mindflash.

Legal led the project, with Joady & Anna advising.  Anna led the addition and coordination of the T&C modules.
46
Quarterly reviewQ2 - 2015/16
Approximate team size during this quarter: 7 FTE
Time spent: strengthen 50%, focus 30%, experiment 20%
Key performance indicator

 

Positively impacting value delivered by supported teams 4.13 out of 5 (Likert scale) baseline metric
Positively impacting sustainability of supported teams 4.47 out of 5 (Likert scale) baseline metric
47
All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Team Practices Group  

Objective: Project Forecasting

 

Objective Measure of success Status
Improved metrics for product project forecasting
Team members involved: 3
  • Burn-up charts actively used by three teams
  • Five teams sample their maintenance fraction and two teams track it throughout the quarter.
  • Three teams were using burn-ups by end of quarter; six teams are currently using or configuring burn-ups.
  • Eight teams planned to sample and five teams sampled. Three teams are continuing to track beyond Q2.
48

Q2 - Team Practices Group  

Objective: Team Strategy

 

Objective Measure of success Status
Team Practices Group achieves consensus on the team’s strategy.
Team members involved: 7

Defined and documented strategy for the Team Practices Group

Miss this quarter due to change in strategic focus in light of a number of organizational changes and divided focus across TPG slowing the strategy process.
49

Q2 - Team Practices Group  

Objective: Tool Assessment

 

Objective Measure of success Status
Determine whether or not we can drive changes to Phabricator to enhance its usability as a project management tool in an environment with diverse workflows and data-driven needs. (multi-quarter)
Team members involved: 3
Five high-priority project management-related Phabricator feature requests upstreamed
3 feature requests have been successfully upstreamed; 2 became obsolete. Intensive coordination is required to upstream, and we should continue to assess whether the impact is worth the effort.
50

Administration

edit
Quarterly reviewQ2 - 2015/16
Team size during this quarter: 11
Time spent: strengthen 50%, focus 30%, experiment 20%
Key metrics and highlights of our core services can be found on final two slides.
51
All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Q2 - Administration   

Objective: Refine and revise onboarding and offboarding with HR.

 

Objective Measure of success Status
Refine and revise onboarding and offboarding with HR.
Team members directly involved: 3
Develop and produce collaborative workflow and documentation of all service teams supporting the initial onboarding of req# numbers. Detailed work flow has been developed by OIT, HR, TechOps, Front office and Facilities to show all tasks and dependencies of onboarding and offboarding a req#
Facilities, Front office, and department admin representatives worked with HR, OIT, Tech Ops, to develop a comprehensive understanding and workflow of the steps involved in onboarding, and the various types of offboarding of req#. HR owns the workflow document.
Learning included understanding of risk dependencies and timelines, need for process on all types of onboarding, conversions, and planned and unplanned offboardings.
52

Q2 - Administration   

Objective: Monthly Travel back-up submitted before 4th of month

 

Objective Measure of success Status
Create unique method of reporting transactional volume of centralized travel expenses (range of 200 - 500 transactions)
Travel expenses meet deadline of fifth business day each month during the quarter enabling earlier monthly close deadline needed in accounting.
Travel developed successful workflow within the current s/w and manual process to provide reporting necessary for early monthly close.
Will continue to work with data and in Concur and JPMorgan to see how to make process less time consuming.
53

Q2 - Administration   

Objective: Develop Disaster Recovery plan

 

Objective Measure of success Status
Outline high level Disaster Recovery plan that complements Safety team work and documentation.  
Team assigned, outline of process and office wiki page outlined, perform critical services inventory with each department.
Office wiki page outlined, team assigned, completed department interviews documenting critical services inventory done.
Key learning was to format recovery methods into levels or tiers of emergency. Team is working toward creating three basic business interruption scenarios to develop and practice various disaster communication and recovery tasks. Cross org team with Facilities and OIT and Safety.
54

Q2 - Administration   

Objective: Evolution of Office space

 

Objective Measure of success Status
Outline process and launch team to determine critical needs for WMF work space; current lease terms September 2017
Real Estate broker and design team together with WMF will have outlined process, stakeholders and data needed to learn the critical priorities of our next working space. Six Leadership interviews complete, space utilization data analysed, staff engagement exercise set up on site, Office wiki page set up to inform and gather additional information.
Key Learnings:
Staff input data largely reflects what is best for individual needs, and challenging for folks to think about what is best for organization. Need to push on this, and solicit more with this vantage.
Developing a scorecard to help with weighted needs next quarter. CBRE and VITAL work well together and bring different strengths to this important process.
55

Editing

edit
Quarterly reviewQ2 — 2015/16
Approximate team size during this quarter: 31 FTEs
Time spent: strengthen 40%, focus 40%, experiment 20%

Key performance indicator;  * – N.B. data for Q2 is for the first two months only

 

Monthly active editors on all wikis 76.3k average in Q2* +1.5% from Q1 (75.1k) +1.0% YoY (75.5k)
56
All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

Collaboration Team

edit

Q2 - Collaboration Team  

Objective: Cross-wiki notifications

 

Objective Measure of success Status
Focus: Improve awareness of activity by providing a cross-wiki notifications feature
Team members involved: 7
Echo can deliver notifications between many wikis, allowing users to access their notifications from any content wiki on any other content wiki Done. Roll-out is a Q3 goal.

 

● Cross-wiki notifications now available as a Beta Feature on Beta Cluster and on test.wikipedia.org.
● Further roll-out to all wikis over the coming quarter.
● Most of this awesome work done by Kunal, Moriel and Pau; thank you.
Image is a screenshot of code in MediaWiki/OOUI/Echo licensed under MIT/GPL.
58

Q2 - Collaboration Team  

Objective: Flow opt-in beta feature

 

Objective Measure of success Status
Strengthen: Increase access to Flow by deploying and supporting an opt-in system for it
Team members involved: 6
Users have the ability enable Flow on their talk page Beta Feature was deployed to Wikidata and to Chinese, Urdu, Bosnian and Catalan Wikipedias.
● 462 users (~22%) opted in on the Chinese Wikipedia, 159 (~3%) on Wikidata, 31 (~6%) on Catalan, 10 (~23%) on Urdu, 6 (~22%) on Bosnian.

Rough percentages for scale only, expressed as a proportion of ‘active editors’ on each wiki.

● The Czech Wikipedia community asked for this, then changed their minds.
● The Chinese Wikipedia deployment exposed technical issues due to IP block exemptions.
● Spam attack early in the quarter required us to bring forward improved anti-spam measures.
59

Q2 - Collaboration Team  

Objective: Notifications prototyping

 

Objective Measure of success Status
Experiment: Increase engagement with notifications by prototyping and researching improvements to the notifications page
Team members involved: 2
We have an informed plan for the next steps for improving the notifications page Delayed.
● Completed one round of user testing, now refining design and planning second round in February.
● Research was delayed due to focus on cross-wiki notification research and delays on Design Research team’s side.
60

Language Team

edit

Q2 - Language Team  

Objective: User Engagement

 

Objective Measure of success Status
Focus: Improve user retention by adding additional 'suggestion' features such as customised lists and adding additional relevant notifications Increase in the number of translations per user Personalized suggestions integrated. Many enhancements for suggestions in Content Translation dashboard

 

  • New users have increased; percentage of returning users has also increased
  • Translators can create ‘Favorite list’ and discard suggestions
  • A campaign can be created using scripts
  • Suggestions are based on the previous translations
  • Teams involved: Language, Research
  • Tool is still a beta feature.
  • Blog post
Image is a screenshot of code in Content Translation licensed under GPL.
62

Q2 - Language Team  

Objective: Infrastructure improvements

 

Objective Measure of success Status
Strengthen: Improve reliability by complying with WMF infrastructure requirements as defined by Services, Tech Ops and Security teams Content Translation complies with WMF infrastructure requirements cxserver successfully migrated to service-runner. Uses uniform service architecture to make the maintenance, logging, monitoring, analytics easy.

 

API documented at https://cxserver.wikimedia.org/v1?doc

● Teams involved: Language, Services, Ops
Image is a screenshot of code in RESTbase/cxserver licensed under MIT.
63

Q2 - Language Team  

Objective: Reliability

 

Objective Measure of success Status
Strengthen: Improve reliability by fixing high-priority bugs that affect basic functionality Users can use Content Translation without interruption High priority bugs fixed for saving and publishing errors. AbuseFilter related errors are displayed nicely so that users can fix and retry publishing.

 

  • AbuseFilter-related errors are displayed nicely so that users can fix and retry publishing.
  • Comprehensive logging of errors and daily analysis
  • The reliability of auto save and translation restore improved - compressed data and restore algorithm improvements
Image is  licensed under CC-Zero and available on Wikimedia Commons.
64

Q2 - Language Team  

Objective: Collaboration

 

Objective Measure of success Status
Experiment: Increase visibility for third parties by collecting and sharing parallel corpora of translation modifications API will be able to access the parallel corpora Development completed, pending deployment
  • Parallel corpora (source-translation pair) API developed.
  • Deployment set to happen in late January 2016 on account of DBA scheduling.
  • Infrastructure also makes saving translations more reliable in long run.
  • Team involved: Language, Ops
65

Multimedia Team

edit

Q2 - Multimedia Team  

Objective: Drag-and-drop uploading

 

Objective Measure of success Status
Focus: Increase media contribution by integrating media upload into VisualEditor's media dialog and drag-and-drop system
Team members involved: 4
Users can upload media files directly within VisualEditor by browsing or dragging-and-dropping Done.
● Building on the work from the previous quarter and with support from the VisualEditor team, this additional feature was completed relatively early in the quarter.
● The cross-wiki upload tool to Commons is now used for approximately 1000 files by 527 users each day; this includes roughly doubling the number of uploads by first-time uploaders to Commons.
● The tool has roughly the same deletion rate for new users as other upload tools (see later slide).
● Thanks to the whole team – Mark, Prateek, and Bartosz – and to Ed from the VisualEditor team and Stephen from Legal for making this possible.
67

Q2 - Multimedia Team  

Objective: Improve upload funnel drop-offs

 

Objective Measure of success Status
Strengthen: Improve reliability by resolving UploadWizard bugs that cause users to start over
Team members involved: 2
Users can use UploadWizard to mass-upload without interruption. Done. Further improvements to come.
● Over the quarter we substantially reduced technical debt, refreshing the UI and fixing outdated code there, improving the error/warning system, and modernising archaïc server code.
● The new user error/warning system means fewer unrecoverable errors, and provides us with much better contextual information to fix rare cases where they occur.
● These changes will help us continue work towards our longer-term goal of allowing users to resume old uploads and go back and forth between steps.
68

Q2 - Multimedia Team  

Objective: Image editing prototype

 

Objective Measure of success Status
Experiment: Improve multimedia editing workflow by prototyping non-destructive image editing
Team members involved: 2
Users can make basic changes to images such as crop, rotation, etc. directly on the wiki. Done. Additional work forthcoming.
● We showed off a demonstration version of the prototype image editor to a lot of positive feedback.
● Our next step will be to make it available on Commons. We will assess real-world usage and needs, get feedback, and decide what next should be done.
● Many thanks to Prateek and Mark for their work on this.
69

Q2 - Multimedia Team  

Objective: Future content type research

 

Objective Measure of success Status
Experiment: Improve value of content by researching the learnability of content to inform future prioritisation of feature development
Team members involved: 0
We have an informed plan for next steps for multimedia contribution features. Delayed.
● We continued our lightweight engagement with Design Research to collaborate with academia to consider possible options, but no substantive work was completed in this quarter.
70

Parsing Team

edit

Q2 - Parsing Team  

Objective: Parsoid extension registration

 

Objective Measure of success Status
Focus: Improve editing experience and draw closer to Parsoid HTML for read views by implementing support for native Parsoid extensions
Team members involved: 2
Parsoid has an extension registration mechanism and native support for galleries
Native extension registration mechanism in place.
Support for <translate>, <tvar>, and other tags used in testing.
<gallery> deferred to next quarter (but have a prototype implementation from Wikia).
Required for supporting extensions that:
● … use wikitext internally (T110909)
● … need custom editing support in VE
Arlo did most of the work on this goal.
72

Q2 - Parsing Team  

Objective: Replace Tidy with a HTML5 parser

 

Objective Measure of success Status
Strengthen: Modernise parsing infrastructure to use modern Web standards by replacing Tidy with a HTML5 parser in MW core
Team members involved: 2
MediaWiki has an HTML5-compliant parser for fixing PHP parser output HTML5 parser in place. Reliance on Tidy bugs/features means incremental fixes. Last step will replace Tidy. Working on visual diff testing to enable this rollout.

Parsing changes (Tim has been working on a lot of this with input from rest of the team):

Done: Empty
  • items no longer stripped; rendering diffs hidden using CSS styles.TODO: More CSS fixes needed and will be rolled out incrementally after testing.TODO: Potentially some changes to parsing of wikitext - needs testing to identify impact on pages.TODO: Enable editors to fix pages and templates that might break when Tidy is replaced. Testing infrastructure (Tim and Subbu have been working on different pieces of this):DONE: Generalizations to visual diffing test framework to support mass visual diff tests.DONE: uprightdiff: image diffs based on (video) motion detection to get actionable diff metrics.IN PROGRESS: Puppetization and test mediawiki install
    73

    Q2 - Parsing Team  

    Objective: Prototype for balanced templates

     

    Objective Measure of success Status
    Experiment: Improve editing experience and take advantage of perf. opt. opportunities by prototyping some form of opt-in / opt-out system for balanced template output
    Team members involved: 3
    MediaWiki has a mechanism for templates to specify their output as balanced or not.
    Implementation sketch and RFC in place (T114445)
    Prototyping not yet begun.
    Proposal needs to go through the RFC process.
    ● Prototyping required in both the PHP parser as well as Parsoid.
    ● Will benefit from the visual diff testing framework.
    ● Replacing Tidy with a HTML5 parser will help with the PHP parser prototype.
    C.Scott has been driving this work with input from rest of the parsing team.
    74

    Q2 - Parsing Team  

    Objective: Multimedia support in Parsoid

     

    Objective Measure of success Status
    Strengthen: Improve mobile reading experience and draw closer to Parsoid HTML for read views by improving multimedia support in Parsoid
    Team members involved: 0
    Parsoid's HTML5 DOM spec and HTML generation have been updated for audio and video No progress beyond some discussions.
    ● Got de-prioritized over all the other work since it turned out that mobile content services didn’t need it right away (compared to other requests)
    ● More progress expected this quarter
    75
  • VisualEditor Team

    edit
  • Q2 - VisualEditor Team
    Objective: Wider deployment

     

    Objective Measure of success Status
    Focus: Increase use of the visual editor by releasing it to more accounts and anonymous users on English Wikipedia

    Team members involved: 6

    More editors use the visual editor. Goal changed after quarter start.
    In-edit switching work complete.
    Single edit tab development work still on-going, expected early Q3.
    ● Goal changed after quarter start for quality reasons: brought forward work from Q4 as new blockers.

    Overall uptake is roughly the same as the previous quarter at around 16% (now ~10.5k edits/day).

    Gradually increasing on the English Wikipedia; now at ~4.5%, up from ~3.9% last quarter.

    Proportion numbers reflect edits using the visual editor out of all article edits made by accounts & IPs, excluding registered bots, in the last week of December. Other tools (e.g. rollback, AWB) not excluded.

    ● In November we switched the Spanish Wikipedia back to default-on for accounts per their request.
    ● Particular thanks to Alex for his work on edit switching and single edit tab integration features.
    77

    Q2 - VisualEditor Team

    Objective: Formula editing

     

    Objective Measure of success Status
    Strengthen: Improve support for math editing by improving the UX of the Math extension and researching its usability

    Team members involved: 3

    It's easier to create and edit formulæ from within the visual editor. Done. Deployed in pieces through over the quarter.

     

    ● We brought on Thalia Chan to simplify formula editing for users.
    ● We now syntax highlight the LaTeX, show a rendering preview, and provide a clickable browsable tray of all fragments to insert.
    ● Particular thanks to the volunteer extension maintainer, Moritz, whose advice has been invaluable, and of course also to Thalia.
    Image is a screenshot of code in VisualEditor/OOUI/Math licensed under MIT.
    78

    Q2 - VisualEditor Team

    Objective: Chart editing

     

    Objective Measure of success Status
    Strengthen: Improve support for chart editing by improving the UX of the Graph extension and researching its usability

    Team members involved: 2

    It's possible to create and easier to edit charts from within the visual editor. Done. Deployed in pieces through over the quarter.
    ● We brought on Frédéric Bolduc from GSoC to continue their work improving chart editing for users.
    ● We now let users create new charts, syntax highlight the JSON, make it possible to resize charts, and support a new version of the ‘Vega’ back-end for additional forthcoming features.
    ● Particular thanks to the extension maintainer, Yuri Astrakhan of Discovery, for their helpful support and vision, and to Frédéric for their drive.
    79

    Q2 - VisualEditor Team

    Objective: Link suggestion experiment

     

    Objective Measure of success Status
    Experiment: Increase value of contributions by prototyping and researching integrating edit recommendations

    Team members involved: 1

    We have an informed plan for next steps for integrating edit recommendations. Development work done but late; deployment not yet undertaken, and so results not yet supplied.
    ● We worked with Research and Data to prototype a user tool to prompt adding cross-article links that were identified from reader behaviour but missing from the article copy. However, the work stalled and was not completed before the end of the quarter, which meant results to inform next steps are waiting.

    Lesson learnt: It was unclear who was leading this initiative, and so the drive to undertake the work was confused. Clarity on cross-team leadership should be established at the start of the work.

    ● Particular thanks for their design leadership to Nirzar, & to Ed for their rapid responsive prototyping of engineering solutions.
    80

    Q2 - VisualEditor Team

    Objective: Mobile macro-design prototyping

     

    Objective Measure of success Status
    Experiment: Improve the visual editor on mobile devices by prototyping and researching using visual editor on mobile devices

    Team members involved: 0

    We have an informed plan for next steps for using the visual editor on mobile devices. Some product market research done, but the bulk of product and design research work was delayed due to more pressing other issues.
    ● We worked with Design Research to consider mobile editing editor concepts and how they match with the user archetypes.
    ● A wider market analysis for design research and prototyping of possible designs for user testing based on this was delayed due to competing priorities during the quarter.
    ● Particular thanks to Nirzar for their design leadership, and to the Design Research team.
    81


  • Quarterly reviewQ2 - 2015/16
    Approximate team size during this quarter: 28 FTE
    (22 Reading, 4 Community Tech, 2 UX Standardization)
    Key performance indicator

     

    Global Pageviews (provisional*) 15.6 B / mo +1.3% from Q1 -8.8% YOY (estimate, ±2pp)
    82
    All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

    Q2 - Reading  

    Objective: Engagement

     

    Objective Measure of success Status
    Strengthen: Drive engagement by launching Link Preview on mobile web beta Engagement (5% increase in links clicked) in beta or on non-EN Wikis Put on hold
    • Android link engagement stats promising
    • Not strong enough engagement to push to mobile web Q2
    • However, we need to reconsider metric
    83

    Q2 - Reading  

    Objective: Engagement

     

    Objective Measure of success Status
    Strengthen: Drive engagement by launching Read More on web beta Engagement (5% increase in links clicked) in beta or on non-EN Wikis
    Mobile web beta:
    January 1-14: 19% engagement when panel seen.
    Internally referred pageviews as a percentage of total pageviews in mobile web beta climbed from around 52% (Th, Nov 19) to around 67% (Th, Jan 14), whereas stable channel internally referred pageview rates were roughly unchanged (approx. 25%).
    • Scalability
    • Co-design
    • Difficulties of bolt-on designing in desktop
    84

    Q2 - Reading  

    Objective: Apps Engagement

     

    Objective Measure of success Status
    Experiment: Prototype services based architecture for web and apps Working web app with feature parity of current mobile web stable reading experience (not scalability, quality). Done. The specific single page app approach used for the prototype will not be used, based on discussion at the summit (full preso). An alternative, iterative approach will be used.
    • Image loading should be optimized
    • Reference sections should be loaded smarter
    • Hidden navbox HTML should likely be disabled completely
    • A lot of mobile users don’t actually expand sections, although this begs additional questions
    85

    Q2 - Reading  

    Objective: Apps Engagement

     

    Objective Measure of success Status
    Experiment: Drive app retention via feeds and modern UX (notifications Q3)
    iOS App overhaul released

    7 day 15% of users retained (up from 10% in Sept ‘15) (Feeds T104415, Modernization T104510)

    Missed. However, the app was released to beta during the quarter. Currently in public beta and gathering user feedback.
    • Over 1,100 beta testers, with >30 responses on our form or via email
    • Very positive feedback on some changes (visual design, feed concept) as well as thoughtful concerns about some elements (navigation flow).
    • After project definition became clearer, pushed timelines by a few weeks to prioritize quality over schedule.
    86

    Q2 - Reading  

    Objective: Apps Engagement

     

    Objective Measure of success Status
    Drive engagement by integrating more types of content and media into the app (see appendix). General elevation of UX and usability to strengthen user engagement and retention. Done: Integrated Maps in Nearby, integrated with RESTBase, lookup of highlighted words from Wiktionary, playback of audio pronunciations, and full support for Android 6.0 (Marshmallow) design guidelines.
    • Led to the app being featured in the “Best of 2015” collection on Google Play, and resulted in a surge of new users.
    • Caused an appreciable increase in Maps traffic (see appendix), and improved engagement with the Nearby feature.
    87

    Q2 - Reading  

    Objective: API Engagement

     

    Objective Measure of success Status
    Focus: Measure MediaWiki API usage, and data dump loads. Availability of usage metrics  on Hadoop warehouse for analysis. (T102079) Missed. Progress has been made towards building the metric pipeline but tooling challenges blocked final implementation in November when there was time to implement. Work will continue in Q3.
    • Data pipeline from MediaWiki to Hadoop via Kafka is brand new tech at WMF
    • Discovery and Analytics have been working out best practices
    • Reading thought Discovery pilot project was done in Q1 but technical challenges extended through most of Q2
    • Still some discussion about "best" encoding to use for data being pushed through the pipeline
    88

    Q2 - Reading  

    Objective: API Engagement

     

    Objective Measure of success Status
    Strengthen: Investigate migrating API traffic to OAuth. Decision on if migration is desirable; then measurable impact based on existing usage (currently unknown) Done. Decision was that encouraging mass migration is not advisable at this time.
    • OAuth cannot be used by all API consumers as it needs "shared secret" protections. This means that consumers such as native applications cannot securely process an OAuth handshake.
    • OAuth is currently recommended for API consumers who need special permissions when interacting with the Action API (e.g. making edits, reading protected content)
    • Requiring OAuth for all Action API access is undesirable
    • Cacheable API requests will necessitate more anonymous API usage, not less and will be important for scaling the Action API just as it is important for scaling web request processing
    89

    Q2 - Reading  

    Objective: MediaWiki Security

     

    Objective Measure of success Status
    Strengthen: Release Auth Manager: MediaWiki plug-in security architecture Usage (# of services using new architecture) (T89459) Missed. Lengthy fundraising code freeze and security review backlog contributed, but ultimately the work was more than 1.5 FTEs could accomplish in the quarter.
    • Single user OAuth (useful for bots) in 1.27.0-wmf.10
    • SessionManager component in 1.27.0-wmf.11
    • "bot passwords" feature for bots that cannot use OAuth in 1.27.0-wmf.11
    • Created php-session-serializer library for use by MediaWiki and other FLOSS projects
    • AuthManager progressing. Current estimate for full deployment is late February with uncertainty narrowing.
    • Security interested in collaborating on follow on 2factor auth project in Q3
    90

    Q2 - Community Tech  

    Objective: Establish credibility
    with stakeholders

     

    Objective Measure of success Status
    Strengthen: Establish baseline for measuring community satisfaction with WMF tech support Complete Tech Support satisfaction poll across multiple Wikimedia projects and languages Poll completed Oct 22, with 467 participants from 10 wikis. Results posted on Meta Nov 4. [1]
    Focus: Include Wikimedia communities in goal development and prioritization Complete cross-project Community Wishlist survey, with community members voting on priority Community Wishlist survey completed Dec 14, with 634 community members participating. Results announced Dec 16. [2]
    Focus: Address current high-priority community technical requests Complete 3 mid-to-large sized requests from existing community wishlists 3 requests: Fix HotCat on 100+ wikis (Nov), Fix and improve Citation bot (Dec), Compile gadget usage statistics per-wiki and cross-wiki (Dec).

    1:  https://meta.wikimedia.org/wiki/Research:Tech_support_satisfaction_poll  2: https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey

    91


  • Quarterly reviewQ2 - 2015/16
    Approximate team size during this quarter: 12.5 FTE
    Time spent: strengthen 40%, focus 30%, experiment 30%
    Key performance indicators

     

    User satisfaction Start Q2: 15% End Q2: 28% (implementation change) -- YoY
    Zero Results Rate Start Q2: 33% End Q2: 26% (normal variance) -- YoY
    92
    All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

    Q2 - Discovery

    Objective: Improve search language support

     

    Objective Measure of success Status
    Improve language support for search.
    Team members involved: 6
    Run A/B test for a feature that detects language of user’s search query and adjusts results to match that language
    Determine from A/B test whether the feature is fit to push to production with aim to improve search satisfaction by 10% and reduce zero results rate for human searches by 10%
    A/B test was run

    An A/B test report was produced

    The test did not show that the feature had the desired impact

    After the A/B test showed disappointing results, the team re-focussed and implemented the completion suggester as a beta feature on all wikis (except Wikidata), which reduces zero results rate by ~10%. The beta feature was then rolled out. Thousands of users are now using the beta feature. User feedback has been overwhelmingly positive. The feature is in early stages; more work remains to be done before a production rollout. We’re targeting Q3 for a more full rollout.

    93

    Q2 - Discovery

    Objective: Improve www.wikipedia.org

     

    Objective Measure of success Status
    Make www.wikipedia.org a portal for exploring open content on Wikimedia sites.
    Team members involved: 4
    Measure usage of existing portal
    Perform A/B tests to improve portal
    Decrease time each user spends searching by Y ms (exact number pending logging implementation)
    Portal was migrated to more standard setup (git/gerrit); this took longer than expected
    Logging added to measure use

    Portal dashboard created

    First A/B test launched, but abandoned due to logging issues

    Migration of portal to git/gerrit setup is complete but took significantly longer than expected; see “Other successes and misses” section for more details.

    First A/B test was launched in December. Unfortunately, problems with implementation of data collection meant that the data from the test could not be used. The test was successfully relaunched in early January, and the initial analysis shows the test had a positive, statistically significant impact.
    94

    Q2 - Discovery

    Objective: Improve understanding of satisfaction

     

    Objective Measure of success Status
    Improve understanding of user satisfaction for search by iterating and improving on the search satisfaction metric.
    Team members involved: 3
    Design QuickSurvey to qualitatively assess user satisfaction with search
    Tie qualitative QuickSurvey results back to quantitative search satisfaction data
    Validate/invalidate current approach for measuring satisfaction
    Our QuickSurvey was designed
    Deployment of QuickSurvey extension (owned by Reading) was delayed due to technical obstacles with EventLogging and the extension
    Deployment freeze and annual fundraiser reduced time available to run survey
    Our survey was deferred to Q3
    In response to a request from the Head of Research and Vice President of Product, the Discovery Analysis Team dedicated one of its analysts at 33% capacity to the annual fundraising campaign to ensure its success. This reduction in anticipated capacity affected our ability to achieve this goal.
    95

    Q2 - Discovery

    Objective: Analyse WDQS and Maps service

     

    Objective Measure of success Status
    Support ongoing evaluation of usage of Wikidata Query Service and Maps services to decide on what’s next for these services.
    Team members involved: 2
    Continue maintenance of Wikidata Query Service and Maps dashboards
    Review user feedback on services
    Make decision on continued level of support for WDQS and maps service
    Maintained WDQS and Maps dashboards
    User feedback broadly positive
    Decided on modest continued investment in both services

    Q3 goals for WDQS and Maps posted

    Gradual adoption of maps service as standard continues. Now standard on many Wikivoyages (English, Russian, etc.), and on the Android Wikipedia app.
    Wikidata Query Service usage also continues steadily. Prominent third-party Wikidata consumers are planning switches over to the Wikidata Query Service. We continue to support the maintenance of the service.
    96

    Q2 - Discovery

    Objective: Create performance indicator for referrers

     

    Objective Measure of success Status
    Create recurring performance indicator on referrer traffic from the primary search engines and determine what features may have largest impact on referrer metric.
    Team members involved: 2
    Create dashboard which displays traffic to our sites broken down by major referrers.
    Based on the dashboard data, make a list of recommendations for features which could be implemented to increase traffic from prominent referrers.
    Referrer data analysed

    Dashboard created

    List of feature recommendations not completed due to analyst support being required for annual fundraiser, and lack of clarity around intended output
    In response to a request from the Head of Research and Vice President of Product, the Discovery Analysis Team dedicated one of its analysts at 33% capacity to the annual fundraising campaign to ensure its success. This reduction in anticipated capacity affected our ability to achieve this goal.
    97


  • Advancement and Fundraising Tech

    edit
  • Q2 - 2015/16

    Quarterly review

    Approximate team size during this quarter: 27 FTE
    Partnerships: 7 FTE; Online Fundraising and Operations: 8 FTE; Foundations, Major Gifts, and Endowment: 5 FTE;
    FR-Tech: 7 FTE
    KPI

     

    Funds raised $46.9m raised in Q2 $54.9m raised Fiscal Year-to-Date $13.4m needed to reach annual goal
    98
    All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

    Q2 - Advancement

    Objective: $34 million

     

    Objective Measure of success Status
    FOCUS- Raise $32 million in Online and Email campaigns in the US, Canada, U.K. Australia, New Zealand, Ireland, Italy, and France
    Amount raised
    $43 million raised (preliminary number).  We exceeded the quarterly goal to put us on target to reach the annual $68.2 million budget goal.  
    FOCUS - Raise $2 million through Foundations and Major Gifts
    Amount raised
    $4 million raised (preliminary number).
    STRENGTHEN - Complete the Endowment Plan and messaging; Prepare to launch Endowment Campaign in January Buy in from the board and key donors Board approved endowment plan. External announcement in conjunction with Wikipedia 15. Conversations with key donors to begin end of January.
    99

    Q2 - Strategic Partnerships

     

    Objective Measure of success Status
    Goal 1 - FOCUS
    Create pipeline of strategic partnerships
    Identify and tap potential partners with substantial impact around: reach, revenue, community, content, and/or other benefits List of key potential partners identified and in various stages of outreach
    Goal 2 - FOCUS
    Secure 10 new Zero deals
    Successful launches Launched 14 partners - 10 in Caribbean, and other countries included Serbia, Angola, Tunisia and Algeria. Total reach added in Q2 estimated at 28M subs
    Goal 3 - STRENGTHEN
    Identify app and mobile web needs for Global South and coordinate with Reading team on feature roadmap
    Deliver multiple feature descriptions for mobile web and app planning for Global South Roadmap features with Reading team established.  Features for Q3 include: better preload tracking, App support for new zero features, improved web page performance and more.
    Goal 4 - EXPERIMENT
    Refine syndication strategy, potential customer assessment, and business case
    Research Work still in progress.#
    Launched discussions with two largest syndicators - Google and Apple
    1. Developed insights from survey
    2. Started analysis of API usage
    100

    Q2 - Major Gifts, Foundations and Endowment

    Objective: $2 million

     

    Objective Measure of success Status
    FOCUS:Raise $2 million Amount raised $3.94 million raised (preliminary number as of Jan 12).
    STRENGTHEN:Hold successful fundraising event Host 200 people, Raise $200k, Strengthen relationships Goals achieved, except revenue target - event raised $120k
    STRENGTHEN:CRM discovery Decision on best CRM options Plan to make improvement to Civi in FY16-17.
    FOCUS:
    Finalize endowment terms and prepare for launch.
    Public launch of endowment. Ready to begin conversations with key donors. Announcement at Wikipedia 15.
    101

    Q2 - FR Tech

    Objective: $32 million

     

    Objective Measure of success Status
    FOCUS: Support the Big English fundraiser Fundraising creative reaches their donation goals in December Done - 3.46 Million separate online donations in Q2 2015
    STRENGTHEN: Get France campaign tech-ready Fundraising creative reaches donation goals in October (in France) Done
    EXPERIMENT: Support Banner History rollout Fundraising creative is able to see patterns and problems with banner consumption, before December Done
    EXPERIMENT: Start investigating SmashPig as its own code project Rough estimate of the work involved to create the open source project -OR- a decision not to move forward with independent payment libraries Didn’t Start. This evaluation was only going to proceed in Q2 if it didn’t interfere with mission-critical goals, and the preliminary conversations turned out to be extremely disruptive for all parties. Decision to halt progress on this goal until a less complicated time, was made early in the quarter (mid-October).
    102

    Q2 - Online Fundraising

    Objective: $32 million

     

    Objective Measure of success Status
    FOCUS:  Run and publicly post a reader survey on best banners of 2015 Survey posted in November with launch announcement for English campaign
    Survey is posted:

    https://upload.wikimedia.org/wikipedia/commons/2/25/Wikimedia_Reader_Survey_November_2015.pdf

    STRENGTHEN: Test & run campaigns with new World Pay and Amazon integrations
    • France campaign completed
    • Have Amazon tested and included as payment option for US during Big EN
    Done
    STRENGTHEN: Add a backup credit card processor Have backup tested during Big EN Done
    EXPERIMENT: Increase sales at the Wikistore Increase sales by adding the store link to the Thank You page Done. More info in the slides to follow
    103
  • Communications

    edit
  • Quarterly reviewQ2 - 2015/16
    Approximate team size during this quarter: 7 FTE
    Key performance indicator: See slides 6, 7, Scorecard appendix of department report
    104
    All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

    Q2 - Communications  

    Objective: Message platform

     

    Objective Measure of success Status
    Message Platform
    Produce clear message and proof points in support of organizational strategy.
    • Clear concept and phrasing for top-level strategy message
    • Supporting proof points and messages
    Postponed to Q3 based on strategy process timelines
    105

    Q2 - Communications  

    Objective: Brand strategy

     

    Objective Measure of success Status
    Brand Strategy
    Complete brand discovery process. Present synthesis of findings and recommended course of action.
    • Recommend course of action to develop effective, powerful brand strategy
    • Brand discovery summary for Board
    • Further work ongoing for Annual Plan

    Complete with all measures of success met.

    • Reviewed stakeholder statements, known challenges, comparable case studies, and external vendor services for problems
    • Brand brief including: Organizational philosophy; Vision & mission; Being effective; A reason to unite, Measures of success; How much change; Options of approach; Moving forward; Timeline draft
    • Finalized deck for Board presentation
    106

    Q2 - Communications  

    Objective: Wikipedia’s 15th birthday

     

    Objective Measure of success Status
    Wikipedia’s 15th birthday
    Create strategic messaging opportunity to engage new Wikimedia stakeholders and audiences.
    • Clearly defined WP15 campaign
    • WP15 microsite/AR staged by 12/30 for 1/15 launch
    • 15 AAA Wikipedia media pieces secured
    • 5+ global community events planned
    • ‘Knowledge is joy’ concept
    • 15.wikipedia.org launched
    • ~7.5M site impressions
    • 400+ press stories, trending positive
    • 149 listed community events

    Complete with all measures of success met.

    107

    Q2 - Communications  

    Objective: Core

     

    Objective Measure of success Status
    Timely, professional, first-rate support on core and reactive communications workflows.
    • Evaluative chart of core workflows and SLAs
    • See Core workflows and metrics (slide 20 of department report).

    Complete with all measures of success met.

    108
  • Quarterly review

    Q2 - 2015/16

    Approximate team size during this quarter: 11 FTE
    Time spent: strengthen 70%, focus 25%, experiment 5%
    *Temporary staff this quarter: 7 legal fellows/interns.
    109

    Q2 - Legal

    Objective: Public Policy Strategy

     

    Objective Measure of success Status
    Public policy strategy: address and support as appropriate key issues that could threaten or compromise mission and projects
    Team members involved: 4
    1 persuasive opposition to intermediary reform in EU; quality comments on mass digitization; develop baselines through censorship study for future measurement. All to ED/GC/community satisfaction.
    • Intermediary Reform Opposition submitted to European Commission.
    • Comments submitted to the US Copyright Office on mass digitization in early October.
    • Entered into contract with Berkman Center to begin censorship study and develop baselines for future measurement (ongoing).
    110

    Q2 - Legal

    Objective: Training

     

    Objective Measure of success Status
    Training: address and support as appropriate key issues that could threaten or compromise mission and projects
    Team members involved: 2
    Selection and implementation of training software that permits flexible presentations on multiple legal topics.  Q2 all staff legal presentation to 275+ employees.
    • Implementation of software training.  
    111

    Q2 - Legal

    Objective: Innovation

     

    Objective Measure of success Status
    Experiment:
    • Facilitate and provide bandwidth for unplanned innovative projects with no or minimal legal hurdles (5%).  
    • Ensure proper delivery of endowment analysis in support of Advancement team.
    Team members involved: 3
    • Satisfaction of ED and C-level peers.  
    • Deliver plan on new endowment, including a cost/benefit analysis of various structures to satisfaction of Revenue Officer, ED, and Board.
    • Provided support and detailed analysis of endowment to allow the Board’s approval of an endowment plan.
    112

    Q2 - Legal

    Objective: Core

     

    Objective Measure of success Status
    Core:
    Top notch, quick legal advice and support on wide host of issues constituting our 34 legal workflows.
    Categories include:
    • Transactional
    • Litigation
    • Privacy
    • Public Policy
    • Trademarks
    • Governance
    • Technology
    • Fundraising
    • Training
    Team members involved: 11
    KPIs
    • Turn-around rate for contracts exceeded KPI of 95% w/i 7 days (100%).
    • Turn-around rate for legal@ exceeded KPI of 95% w/i 7 days (100%).
    • Core legal advice and daily operations to the satisfaction of ED and C-levels per KPI.
    113
  • Research and Data

    edit
  • Quarterly reviewQ2 - 2015/16
    Approximate team size during this quarter: 4.5 FTE, 2 research fellows, 10 collaborators
    Time spent: strengthen 40%, focus 20%, experiment 40%
    114
    All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

    Q2 - Research and Data

    Objective: Data ownership map

     

    Objective Measure of success Status
    FOCUS
    Streamline ownership of data and research
    Team members involved: 1
    Publish a map for internal use of who is responsible for specific requests around research and analytics. (T112317) completed

     

    • Worked with individual teams on mentoring / backfilling / hiring of data analysts    
    • Published research/data ownership draft
    • Socialized division of labor with Audience and Technology teams, Legal, Comms, C-team

    https://office.wikimedia.org/wiki/Research_fingerpost

    115

    Q2 - Research and Data

    Objective: Revscoring integration

     

    Objective Measure of success Status
    STRENGTHEN
    Revscoring integration
    Team members involved: 1
    Collaborators: 9
    Bring revscoring to fruition to our users as a Beta Feature (score integration into RC feed) (T112856)
    missed
    Reason
    • blocked on code review for months
    • volunteer time went away during FR
    • we weren’t able to complete the design and deployment of this beta feature, due to limited resourcing and dependency on volunteers
    • revscores integration into edit histories / RecentChanges feed pushed to Q3
    • focus shifted on other priorities (massive deployments, impact analysis) see other accomplishments below

    https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service

    116

    Q2 - Research and Data

    Objective: Recsys productization

     

    Objective Measure of success Status
    STRENGTHEN
    Productize article and link recommender systems
    Team members involved: 2
    Collaborators: 3
    Article creation (T112321) and link (T112322) recommendations productized as services so they can be integrated with products and community tools.
    completed
    • APIs available on Labs
    • architecture and service implementation blessed by Ops
    • First 3rd party tools adopting the service (example)

    https://meta.wikimedia.org/wiki/Research:Increasing_article_coverage

    117

    Q2 - Research and Data

    Objective: Value added research

     

    Objective Measure of success Status
    EXPERIMENT
    Value added research
    Team members involved: 1
    Deliver research on historical data to quantify who adds value  to English Wikipedia
    missed
    Reason
    • completed part of the project (historical measurement of productivity on English WP)
    • Finished historical analysis of productive edits by anons vs registered users(presented at January 2016 Research Showcase)
    • Partnership with external Hadoop as a service provider (Altiscale) ended in Q3 due to limited funding (currently exploring potential pro bono extension of services)

    https://meta.wikimedia.org/wiki/Research:Measuring_value-added

    118

    Q2 - Research and Data

    Objective: Reader segmentation

     

    Objective Measure of success Status
    EXPERIMENT
    Reader segmentation research STRETCH
    Team members involved: 3
    Collaborators: 1
    Deliver research on reader segments and behavioral patterns (T112326)
    completed
    • completed 3 contextual surveys and identified robust categorization scheme for reader motivation and information needs
    • presented results to Reading team in preparation for Part 2 (Q3)

    https://meta.wikimedia.org/wiki/Research:Characterizing_Wikipedia_Reader_Behaviour

    119


  • Design Research

    edit
  • Quarterly reviewQ2 - 2015/16
    Approximate team size during this quarter: 4 FTE
    (+ 1TPGer, and a half time researcher who is on the Reading Team)
    Time spent: strengthen 60%, focus 30%, experiment 10%
    120
    All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

    Q2 - Design Research

    Objective: Design Research Workshop

     

    Objective Measure of success Status
    STRENGTHEN
    Design Research workshop to build collaboration capacity with product teams.
    Team members involved: 4 (plus 3 non team members and IT / Admin support)
    Workshop happens, and product teams learn more about collaboration with Design Research and vice versa.
    Workshop completed:
    40 participants attended
    (details on next page)
    121

    Q2 - Design Research  

    Objective: Mentoring

     

    Objective Measure of success Status
    STRENGTHENMentor people to do quality design research (those not on DR team) One person outside the design research team does a complete design research project to DR team standards, expanding the number of people who do design research well.
    Ongoing:
    Conceived of the prototype lab end of December.
    We initiated the first sessions on January 15. They will be ongoing every other week.
    Sherah has been participating in Design Research activities including being the lead in creating a methods menu for DR Workshop.

    Prototype labs is bi-weekly design research series. It is being implemented to enable consistent evaluative research in a regular cadence for the Reading team to iterate concepts and get to know users.

    Prototype Lab, besides being a great opportunity for mentoring, it is an attempt to address coordination difficulties with the multiple teams of the Reading team and provide a regular cadence of research and stronger understanding of readers.
    122

    Q2 - Design Research  

    Objective: Improve Participant Recruiting

     

    Objective Measure of success Status
    STRENGTHEN
    Improve participant recruitment process
    Implement changes to reduce steps in recruitment (of design research participants) process to make recruiting more efficient.
    Completed:
    (see next page for details)
    Provide recruiting and scheduling for designers doing design research within teams and for the Prototype Lab.
    Establish relationships with external recruiting firms to schedule international participants for deep dives.
    Participant recruiter taking notes for design research sessions as needed.
    123

    Q2 - Design Research  

    Objective: Design Research with Product Teams

     

    Objective Measure of success Status
    FOCUS
    Collaborate with product teams to implement design research projects.
    Design Research completes design research projects in collaboration with product teams to inform human centered product development. 11 research projects completed
    Learning:
    We potentially have too few QA professionals for the need within product teams. Design and Design Research end up doing QA before usability testing and before release.
    124

    Q2 - Design Research  

    Objective: Personas

     

    Objective Measure of success Status
    EXPERIMENT
    Persona interviews
    12 persona interviews completed
    5 interviews completed
    We noticed some patterns in Readers learning online, and what paths they take to learn.
    We are currently solving for the issue of participants who agree to participate in research and then don’t show up as scheduled.
    We deferred Persona research to get the Prototyping Lab rolling and because there were many unexpected and ad hoc research requests during this period.
    125


  • Analytics

    edit
  • Quarterly reviewQ2 - 2015/16
    Approximate team size during this quarter: 6 FTE, 1 PT

    Key performance indicator: Velocity

     

    Quarter:  911 (~906 last quarter) October: 426 (>10% MoM) November: 227 (<53% MoM)) December: 258 (>13% MoM)
    126
    All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

    Q2 - Analytics  

    Objective: Launch Pageview API

     

    Objective Measure of success Status
    Launch Pageview API

    Alpha API is operationalized and ready to be queried by outside world

    A blog post is published on Wikimedia’s blog
    Team members involved: 3
    Usage of new API (next slide)
    Community feedback (not so SMART but very relevant)
    The API was successfully launched
    Python, JavaScript and R clients written by community members

    Blog post published here

    Learning: It is great to work in projects that have very positive community feedback.

    Do capacity planning much before launch date, we had to drop hourly resolution as the system did not have capacity for it.
    127

    Q2 - Analytics  

    Objective: Replace wikistats pageview reports

     

    Objective Measure of success Status
    Use Hadoop infrastructure to replace computation of wikistats on http://stats.wikimedia.org (wikistats)
    Team members involved: All, at some point

    Replace 5 of wikistats most popular pageview reports defined here

    Reports updated:
    Monthly pageview reports for all wikis and projects, normalized/raw, mobile/desktop/combined.

    Available here

    We consolidated pageview definitions: we have only 1 pageview definition for pageview data since May 2015

    Pageview dumps have also been updated to match the new definition (daily/monthly per-article and per-project pageviews)

    Learning: New definition has less variability (good) and is better at detecting internal traffic and robots

    128

    Q2 - Analytics  

    Objective: Program Global Metrics using Wikimetrics

     

    Objective Measure of success Status
    Deploy to wikimetrics an easier way to collect program global metrics for Learning and Evaluation of our grants programs
    Team members involved: 2
    800 human hours saved by automation
    Feature adoption
    Missed
    Code complete but not deployed.

    Reason: Wikimetrics is deployed on labs and we need to refactor our puppet code to do away with the self hosted puppet master setup, in order to reliably deploy and keep the service running

    Currently deployed at: https://metrics.wmflabs.org/reports/program-metrics/create/

    129

    Q2 - Analytics  

    Objective: Build the infrastructure to support a generic EventBus

     

    Objective Measure of success Status
    Build infrastructure Scalable Event Bus
    Team members involved:2
    EventBus service up and running in production using shared schema repository and new Kafka cluster.
    Done.
    A stretch goal for Analytics was producing real events to this new service.  This work was done by the services team, and is ready to deploy.  However, the holiday deploy freeze kept the new MediaWiki code that produces to the service from being deployed before the end of the quarter.
    Learning:
    Anticipate the deploy freeze that happens at the end of every calendar year earlier. Allow more room when teams depend on your work.
    130
  • Performance

    edit
  • Quarterly reviewQ2 - 2015/16
    Team size during this quarter: 5 FTE
    Key performance indicator

     

    Time to first paint 819ms % from Q4 ~52.37 YoY
    131
    All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

    Q2 - Performance

    Objective: Availability

     

    Objective Measure of success Status
    Availability
    Make MediaWiki code ready for concurrent deployment across multiple data centers. Done with all high-level changes we anticipated for MediaWiki, but unknown add’l work remains.

     

    132

    Q2 - Performance

    Objective: Thumbnail do-over

     

    Objective Measure of success Status
    Solve all (5-6) thumbnailing problems on a VM. Move image manipulation (thumbnail rendering) out of MediaWiki by designing and implementing a standalone, horizontally-scalable imaging service based on Thumbor. Done. Vetted by TechOps; on the roadmap for initial deployment in Q3.
    (Some) problems with current stack:
    • Thumbnails of all sizes are stored forever, until the original gets deleted. Unused and rarely used thumbnails are never cleaned up.
    • Lots and lots of purge messages may be needed to clean up the thumbnails for a given media delete, to the point that the end-user request may timeout.
    What we want: far future-expires for images; webp support; operational simplicity; support for additional image transformations (rotations, cropping, etc.)
    133
  • Architecture

    edit
  • Quarterly reviewQ2 - 2015/16
    Approximate team size during this quarter: 0.5 FTE
    Time spent: strengthen 50%, focus 50%, experiment 0%
    Key performance indicator

     

    ?  - we may choose RFC flow, but no metrics gathered yet      
    134
    All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

    Q2 - Architecture  

    Objective: Prepare WikiDev '16

     

    Objective Measure of success Status
    Focus: Prepare WikiDev '16
    Demonstrate that we can develop a modern system in an inclusive, consensus-oriented, open manner.  Large number of diverse RfCs, with ample discussion prior to WikiDev '16
    Preparation happened, summit resulted :-)

    Learning: Sharpened collective thinking on good meetings and how to build consensus

    135

    Q2 - Architecture  

    Objective:  Strengthen: Improve ArchCom utility

     

    Objective Measure of success Status
    Strengthen: Improve ArchCom utility
    Set up Architecture Committee to succeed.  Maintain one public IRC meeting per week
    10 IRC RFC review meetings, plus 3 “Agenda bashing” meetings for WikiDev ‘16.  Most were at the usual time (14:00 PT)

    Phab query for meetings

    Learning: Quote: [With] archcom being effective now (Which is great btw, so don't get me wrong) - I have noticed a trend where decisions are made in irc meetings, with little to no public discussion on the mailing list”. -- Brian Wolff on T118932, November 30

    136

    Q2 - Architecture  

    Objective: Strengthen: Improve ArchCom utility

     

    Objective Measure of success Status
    Experiment: ArchCom naming Think through our loaded naming scheme, possibly renaming bits (e.g. “Architecture Committee” and “RfC”) I didn’t get around to proposing anything on this front.

    Learning: it’s ok that this one fell off.  “ArchCom” and “RFC” seem to work well enough for now.

    137
  • Operations

    edit
  • Quarterly reviewQ2 - 2015/16
    Approximate team size during this quarter: 17 FTE
    Time spent: strengthen 40%, focus 40%, experiment 20%
    Key performance indicator

     

    Availability 99.979% +0.037% from Q1  
    138
    All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

    Q2 - Technical Operations

    Objective: Encrypted PII

     

    Objective Measure of success Status
    Focus: Encrypt PII on cross-data center links
    Team members involved: 5
    Encrypt cross-data center traffic for MySQL/MariaDB, Kafka, Cassandra, (Varnish) HTTP
    Decommission udp2log (in favor of Kafka)
    Cross data center traffic for MariaDB, Kafka, Cassandra and between Varnish cache tiers is now encrypted.
    Legacy (and unencrypted) logging solution udp2log has been decommissioned.

    Learning: It has been confirmed by recent security vulnerability announcements (Juniper and Fortinet VPN backdoors) that we should not rely on transport link encryption by (proprietary) network hardware, as we have been hesitant to do since the start. We will continue to rely on FOSS implementations (TLS, IPsec) for encryption of PII data.

    139

    Q2 - Technical Operations

    Objective: User auth security

     

    Objective Measure of success Status
    Strengthen: Security: Strengthen infrastructure user authentication
    Team members involved: 3
    Establish org-wide workflow for creation and deactivation of user accounts in Production
    Implement (but don't enforce) two-factor authentication on SSH bastion hosts
    Migrate unmaintained OpenDJ LDAP to OpenLDAP
    Organization-wide on/offboarding process has been implemented.
    Two-factor authentication is implemented and under experiment by all root users (using Yubikeys).
    OpenDJ has been replaced with OpenLDAP.

    Learning: First results of using Yubikeys for ssh authentication are promising, however some usability and reliability issues remain. Further experimentation with the OTP setup will continue into the next quarter.

    140

    Q2 - Technical Operations

    Objective: Incident monitoring

     

    Objective Measure of success Status
    Strengthen: Incident monitoring infrastructure
    Team members involved: 2
    Upgrade incident monitoring and expand it to codfw
    Reduce monitoring check execution lag to 0
    Create new abstractions for incident monitoring in Puppet
    Software options were evaluated and we decided on a migration to Shinken (distributed monitoring). This migration has been prepared but didn’t complete in time.
    New monitoring abstractions have been created in puppet, work is still ongoing.

    Learning: FQ2 is a challenging quarter every year with reduced time available due to the holiday periods, relatively high amount of staff taking unused vacation days, and deployment freezes. We should take this into account for our goal planning for this quarter.

    141

    Q2 - Technical Operations

    Objective: Bare metal Labs

     

    Objective Measure of success Status
    Experiment: Support bare-metal servers in Labs
    Team members involved: 2
    Allow provisioning of physical servers within the Labs network, governed by Labs Puppetmaster and LDAP user accounts Labs infrastructure and Puppet manifests now support bare metal servers in the Labs context. One bare metal server has been provisioned.

    Learning: A separate small new test cluster was built for testing infrastructure changes to Labs itself, which proved very helpful for this goal and many projects in the future. OpenStack Ironic was also evaluated for this goal, but our current reliance on Nova-Network would imply a significant effort and create more technical debt until our migration to OpenStack Neutron completes.

    142
  • Release Engineering

    edit
  • Quarterly review

    Q2 - 2015/16

    Approximate team size during this quarter: … 6
    Time spent: strengthen 9%, focus 28%, experiment 0%, Maintenance/Other 63%
    Key performance indicator

     

    Time to merge in MW Core ~6 mins -45% from Q4 (still analyzing/making sense)
    143
    All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

    Q2 - Release Engineering  

    Objective: Reduce CI wait time

     

    Objective Measure of success Status
    STRENGTHEN
    Reduce CI wait time
    • CI cluster responds to spike in queued builds by starting and registering additional jenkins slaves (+)
    • Migrate majority of CI jobs to Nodepool (-)
    Nodepool is up and working.
    However, the majority of CI jobs are not yet migrated (waiting on a solution for caching dependencies).
    Learning: More collaboration is needed to address some of the more annoying problems (see: caching solution for dependencies).
    144

    Q2 - Release Engineering  

    Objective: Reduce number of deploy tools

     

    Objective Measure of success Status
    FOCUS
    Reduce number of deploy tools from 3 to 2
    • Migrate all Service team owned services and MW deploys to scap3
    Not completed. We are behind schedule in migration of services.
    The Dev Summit discussion, however, was very fruitful.
    Migrated some services on Beta Cluster to scap3 pre Holidays.
    Learning: After the slow beginnings to this project (slow due to other priorities taking time) we didn’t allocate enough time in Q2 for “the last 10%” of functionality needed by Services.
    145

    Q2 - Release Engineering  

    Objective: Retire Gerrit and Gitblit

     

    Objective Measure of success Status
    FOCUS
    Retire Gerrit and Gitblit in favor of Phabricator
    • Decommission Gitblit (-)
    • Code review RFC: creation, publication, discussion, feedback etc (+)
    Not completed.
    Gitblit is still live.
    Differential Migration RFC is published, has been discussed, and feedback being addressed (mostly from the Dev Summit).
    Learning: Happy surprise- The discussion at the Dev Summit about the migration was predominantly positive and there were no new blockers identified. It was a very encouraging conversation.
    146

    Q2 - Release Engineering  

    Objective: Release MediaWiki 1.26

     

    Objective Measure of success Status
    FOCUS
    Release MediaWiki 1.26
    • A quality MW 1.26 successfully released
    Released.
    Learning: Less of a learning and more of a perennial known: The releases we make for third-parties are still sub-par due to a number of reasons, one being internal priority setting (it is far less than 1 FTE per year).
    147
  • Services

    edit
  • Quarterly reviewQ2 - 2015/16
    Approximate team size during this quarter: 4 FTE
    Time spent: strengthen 40%, focus 40%, experiment 20%
    Key performance indicators

     

    REST API usage 248 req/s mean 15% up from Q4 ∞% YoY
    REST API HTML uptime (catchpoint) 99.97% 99.82% ∞% YoY
    148
    All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

    Q2 - Services  

    Objective: API build-out

     

    Objective Measure of success Status
    Strengthen: Build out the REST API
    Provide cached & purged REST API entry points for at least two high-traffic API use cases, and work with users to speed up page loads.
    New entry points: page summary, wiktionary definition, math SVG & MathML renders.
    Math used in production web; apps and hovercards (web) in process of switching over to summary & definition end points.
    Learning:
    Need a well-defined thumbnail API. Started discussion & proposed strawman.
    Broad support for maximizing caching with more REST end points at dev summit.
    149

    Q2 - Services  

    Objective: Event bus & change propagation

     

    Objective Measure of success Status
    Focus: Set up an event bus and change propagation solution
    Key change events like edits, renames, deletes available in a publish-subscribe event bus. Basic change propagation / pre-generation functionality for REST services operational.
    EventBus operational, MediaWiki event production just deployed.
    Change propagation service deployment delayed by dependency on event bus.
    Learning:
    EventBus collaboration between Services & Analytics initially suffered from an unclear division of responsibilities. Clarifying responsibilities and dependencies helped to make progress.
    150

    Q2 - Services  

    Objective: API-driven web front-end

     

    Objective Measure of success Status
    Experiment: Prototype an API-driven web front-end
    Gather information to inform our longer-term front-end architecture by prototyping a service-worker / node.js based front-end.
    Prototyped a service-worker front-end and established performance characteristics.
    Worked with Reading team & held a productive session on the topic at the dev summit.
    Learning:
    Session at the dev summit showed broad support for the direction & a preference for progressive enhancement.
    151
  • Security

    edit
  • Quarterly reviewQ2 - 2015/16
    Approximate team size during this quarter: 2.1 FTE (+Moritz)
    Time spent (strategic): strengthen 25%, focus 0%, experiment 75%
    Key performance indicator

     

    Critical and High Priority Security Bugs REDACTED REDACTED from Q4 n/a
    152
    All content of these slides is (c) Wikimedia Foundation and available under a CC BY-SA 3.0 license, unless noted otherwise.

    Q2 - Security  

    Objective: Static Security Testing

     

    Objective Measure of success Status
    Automated security static analysis of MediaWiki
    Automatically scan core and one extension weekly.
    Security team chose Veracode for scanning, and are submitting core and two extensions for each deployment branch.
    Learning:
    The quality of static security testing tools for PHP is still immature. The WMF could significantly contribute in this area, but the Security Team lacks the resources to pursue this.
    153

    Q2 - Security  

    Objective: Expand Training

     

    Objective Measure of success Status
    Expand developer training
    Present secure SDL training for community and staff.
    (stretch) Develop and present security training materials for DevOps and Mobile
    The team documented the optimal process for other teams to integrate security into their development process, but training has not been given.
    Learning:
    The team adapted to account for a security incident. Our goals should be more flexible to handle this in the future.
    154