Wikimedia Foundation Report, January-March 2015
and (for online presentation) the Google Slides version
You are more than welcome to edit this wiki version of the report for the purposes of usefulness, presentation, etc. |
Wikimedia Foundation
January — March 2015
Q3 of the fiscal year
Foreword
editWe are pleased to bring you the WMF Quarterly Report for Q3 of 2014/15. This is the second report since switching from a monthly cycle, to align with our quarterly goal setting process, and contains several changes that focus more on results and achievements.
One major change is that goals are counted as either a success (green) or a miss (red). This report is now a comprehensive overview of all the Q3 objectives rather than a subset of the quarter’s goals, as we delivered in the Q2 report. In a mature 90 day goal setting process the “sweet spot” is for about 75% of goals to be a success. Organizations that are meeting 100% of their goals are not typically setting aggressive goals.
We have also been able to reduce the amount of time and effort that goes into producing the report by flowing information straight through from the Quarterly Review Meetings.
In the report you will also find a couple of new pieces of data. The overview slide shows the status of all 130 goals from Q3 on a single page, broken down by organization. Additionally we have been able to include site speed metrics as part of our overall report card.
Terry Gilbey, Chief Operating Officer
Quarterly metrics scorecard (beta)
edit Participation
| ||
| Sign-ups | 1,507k | -1.4% from Q2 +53.3% y-o-y |
| New editors | 16.5k/mo | +6.9% from Q2 +1.3% y-o-y |
| Active editors (5+ edits/month) |
80.9k/mo | +6.8% from Q2 +5.5% y-o-y |
 Site reliability
| ||
| Read uptime (Eng. WP main page) |
99.99% | 100% in Q2 100% in Q3 2013/14 |
| Read latency Median first paint time |
1.4 seconds | N/A |
| Write latency Median page save time |
2.0 seconds | N/A |
 Readership
| ||
| Page Views Crawlers excluded |
18.5B/mo | +2.4% from Q2 +4.0% y-o-y |
| Visitors | Per-project unique visitors to come later in 2015 |
comScore desktop UVs (deprecated): 449M/mo in Q3 |
 Content
| ||
| New articles | 8.0k/day | +6.1% from Q2 +7.2% y-o-y |
| Edits (in WP articles) | 9.49M/mo | -1.7% from Q2 -4.0% y-o-y |
 Fundraising
| ||
| Amount raised | $9.9M (exceeded $3M target) |
Q2: $44M Q3 2013/14: $6.1M |
| On mobile | 21% | Q2: 16% |
Q3 2014/15: Successes/misses by team
edit
Q3 2014/15 objectives
- 130 objectives
- 32 teams
Misses include anything not delivered as planned, including “yellow” goals and those that were intentionally revised or abandoned due to changing priorities.
| Objective | Impact on Goal | Status |
| Top notch, quick legal advice and support on wide host of issues (e.g. 7/day turnaround of 85% contracts; fundraising legal review; litigation mgmt; trademark permissions; successful Board meeting; privacy protocols; product counseling) | Support of WMF mission by eliminating legal barriers and slow downs, by offering solutions to support users, and by stewarding values of WMF and community, including more knowledge. | Done (approx. 75% of capacity)
|
- 33 Core Workflows. We delivered on 33 core legal workflows - including contracts, trademarks, litigation & threats, privacy & transparency, governance, technology, fundraising, internal content review, public policy, grants & FDC, HR, and C-level support - despite resource challenges. We filled an open legal counsel position with an exceptional candidate.
- NSA lawsuit. We filed a complex lawsuit against NSA to fight for our community's values and to support our mission, resulting in positive global support. Details may be found here.
- Mobile trademark enforcement. We put a strong focus on mobile trademark enforcement by sending notices for apps that infringe Wikimedia marks. We are taking a supportive approach in an effort to encourage proper use of our trademarks. The positive result: large proportion of developers started to address infringements. We intend to continue these enforcement efforts as part of our ongoing trademark protection strategy.
Transparency report (Core). We released our transparency report late by three weeks, though the report showed strong pushback on censorship requests. This delay was due to our failure to anticipate competing pressures on dependent internal partners with resource challenges.
Learning - We need to streamline and simplify long-term, recurring commitments - like the 6-month transparency report - to ensure on-time delivery.
Action - Sensitive to others' capacity limitations, we must better manage expectations, obtain clearer buy-ins, and minimize high dependency when possible.
| Legal Scorecard | Q3 | QoQ (Q3/Q2) | YoY |
| Legal@ Requests | 149 | 23% ↓ (149/195) | TBD |
| Contract Requests | 93 | 3% ↑ (93/90) | 31% ↑ (93/71) |
| Trademark Permission Requests | 43 | 13% ↑ (43/38) | 14% ↓ (43/50) |
For more detail, see the slide deck and minutes from the quarterly review meeting of the Legal team
| Objective | Impact on Goal | Status |
| Purchase Order Implementation. | Standardized process for purchasing at WMF to incorporate the purchase of servers along with all other contract processes. | In progress. Delay in project, will be added to Q4 goals as it will not be completed in Q3. |
| 5th Floor Project | Update and prepare 5th floor for occupancy | Completed. |
| Office Wiki Makeover | Provide easier access to information for staff. Staff input on what is needed in Office Wiki. Survey at the end of the project. Test and verify one click access to most requested data. | Will be completed by April 30. |
| Metrics Report Implementation | Report for annual and quarterly metrics for decision making and progress mapping of WMF. | Project assigned to Terry with support from Garfield. |
5th Floor project was completed by the end of March. The project was completed on time due to good front end planning by Lynette, good vendor management and keeping all tasks on time for completion.
Purchase Order Implementation was delayed to Q4. Project failed to move forward as planned due to a limitation in the software which was not identified in the planning process. Software was unable to give us segmented approvals by department and dollar value. New software solution (Concur) was identified for this project and we are resuming implementation in Q4. Lesson learned is to verify twice, all key requirements to ensure software is able to implement current process or we are comfortable with the needed changes to our process.
Office Wiki Project was delayed to Q4. Is scheduled to be completed by April 30, 2015. Delay in project due to extra time taken for review by other departments. Need to build into project plan additional time for review by other departments. Also delayed due to policies on the Office Wiki which protect user privacy.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Finance & Admin team
| Objective | Impact on Goal | Status |
| Recruit key hires | Hire VP of Strategic Partnerships
Hire Director of Engineering Hire Legal Counsel |
Kourosh hired 3/30
Wes hired 3/30 Victoria accept 3/27 (4/20 start) Zhou accept 4/9 (4/27 start) |
| Rebuild Recruiting Team | Hire Director of Recruiting
Hire Technical Sourcer Hire Recruiting Coordinator |
Interviewing for Director
Carla hired as Sourcer 2/9 Maria hired as Coord 1/26 |
| Compensation Plan Update | Approve Y path comp grid for more options between SME and Mgr paths
Approve Radford comp matching method |
|
| International Payroll & Benefits | Pilot enrolled for April international payroll
Complete rollout meetings to Managers Complete rollout meetings to Employees Contract review by Legal |
5 person pilot group set for April
Manager meeting 3/6 Employee meeting 3/10-11 Contracts reviewed by counsel |
| Objective | Impact on Goal | Status |
| ADP Upgrade | Upgrade ADP to current version, to allow for performance review and analytic modules for Q4
Create interim org chart |
|
| All Hands | Review & publish All Hands survey data:
staff satisfied with event and feel connected |
Published 2/18 (60% participated) |
| Learning & Development | C2A Community related strategy support
Call to Action Support organizational changes & transitions (Community Engagement, Wikipedia Zero) |
Recruiting is making progress, with 12 req# hires in FQ3. This includes attending career fairs at Stanford and CCA for employer branding & sourcing technical candidates.
International contractor solution is launched with 5 staff in the pilot group for April, with 15 agreed for May & 2 for June. 73% of eligible staff confirmed so far (22 of 30). Work continues in Q4.
All Hands was the biggest yet with 210+ attendees.
Benefits has been working closely with the Admin team on a Safety Program. They have already created more clarity on the Worker’s Comp process and staff options. In Q4, they are planning to roll out more. Benefits has also researched an ACA compliance vendor in Q3 and is getting in place for Q4.
The FQ3 date for the compensation approvals was missed by a day, due to changes from the unexpected announcement of Gayle’s departure.
The ADP upgrade was pushed back due to ADP’s ongoing issues with customer service. Once the upgrade is done, if the software is still lacking, then we’d recommend getting a new HRIS & Payroll system. Garfield & Joady are doing a preliminary meeting with Workday on 4/30.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Talent & Culture team
| Objective | Impact on Goal | Status` |
| Staffing: Fill Jr Comms Mgr position; Develop WP0 marketing, internship program | Staff blog and core writing functions; develop WP0 regional marketing strategy. | - Movement Comms Manager - Fabrice Florin
- JCM intern (contract) - Sam Lien - WP0 marketing volunteer - Amy Bebbington - 6 volunteers/interns; Recruiting 6-8 summer |
| Reporting: State of the Wiki report, parts I & II of 3. | Deliver part 1 and 2 of State of the Wiki report; include report launch plan for community. | - State of the WMF published 4/2 - 1,400 PVs in first 90 days
- Rolled out via Signpost; wiki, PDF versions. - Parts II & III repurposed for WP15 campaign. |
| Product: Product & Comms workshop; product launch workflow | Co-develop product launch process with product team for more effective public product marketing and PR. | - Messaging and PR support for 5 features/product: WikiGrok, Gather, error pages, Share-a-Fact and lead images; VisualEditor
- Workshop completed 4/9 |
| Objective | Impact on Goal | Status |
| Brand & Content: Baseline brand and identity data; preliminary brand book; Content strategy for blog & social media | Deliver preliminary brand strategy exploration. Deliver content strategy. |
- Interim brand book. Planning in progress for brand project plan; WP15 anniversary - Collected content metrics, ran blog survey. - Prepared content strategy for the blog, started social media and video plan. |
| External collaboration: Workshop w/ PBS, NPR, chapter affiliates on future of knowledge and public media | Introduce WMF in content and media space, initiate discussion on new knowledge, explore possible brand partners. | - Workshop with PBS/NPR completed with Wikimedia NYC & Wikimedia DC
- Several immediate and longer term projects; incl. plans for edit-a-thons |
| Media/PR: Ongoing support for media events (60 Min, ACLU, emerging threats, etc.) | 100% max 24 hour response to inquiries. Reinforce positive brand perception. Advance brand values. |
- Wikimedia v. NSA launch (Mar 10) - Art + Feminism editathons (Mar 7) - Introducing SLA tracking in Q4 |
success
- Launched first-ever digital Annual Report, in easier to access format
- 13,688 views from March 7 to April 7
- State of the WMF community release
- Signpost exclusive introducing report to community, w/ many thoughtful responses (58)
- 1,000+ report views in first week
- Initiated support, collaboration, and process-building for Product teams
- Managed 2 integrated product launches; brand and messaging support for 3 product features
- Offsite workshop with representation from C-level, Product, UX, Engineering, CE, Communications
- Wikimedia v. NSA op-ed placement in The New York Times
- 300+ additional global media coverage
- Exceptional message delivery, almost all coverage pulled quotes and information from WMF material
- Drove campaigns for gender initiatives throughout Women’s History Month
- Visual design for Inspire Campaign and distribution support; campaign exceeded goals
- 30+ global media coverage on Art + Feminism editathons
- Original WHM content social and blog campaign with 10k+ impressions on blog alone
- Initiated development of integrated content strategy
- Introduced single-focus themed campaigns with integrated content across multiple channels (blog, video, social, external media); started content research for data-informed decision-making
- Developed baseline brand book for Wikimedia Foundation identity
- Supported 5 high-level executive and board announcements and transitions
- Held first team retreat to clarify mission, priorities, planning, and staffing
misses
- Executive Director global visits
- Developed calendar but had to cancel trip to South Africa due to lack of planning
- No further process in planning for remaining visits
- Social media strategy
- Did not develop strategy due to lack of resources (headcount, time)
- Did not successfully acquire social media management tool
- State of the Wikis and State of the Internet
- Reevaluated importance of these projects/formats and postponed development
- Aim to incorporate in major report for Wikipedia 15 campaign
- Reevaluated importance of these projects/formats and postponed development
- Wikipedia 15 planning delayed
- Coincided with Wikimedia v. NSA , annual planning, and leadership announcement support
- Annual Report challenges
- Made request for data too late for full capture, leading to incomplete statistics
- Production of print version of Annual Report took longer than planned
- Unsuccessful media outreach
- Open Access Policy - no significant coverage secured, possibly due to short timeline
- Funding free knowledge the wiki way - only negative coverage despite significant time investment; community response centered on report being commissioned by WMF
- Product/Comms offsite pushed back
- Underestimated amount of time required to align internally with stakeholders
For more detail, see the slide deck and minutes from the quarterly review meeting of the Communications team
| Objective:
Overall goal: ensure resources go to effective programs to increase participation and quality content |
Impact on Goal | Status |
| Objective 1.
Outreach for program effectiveness data Follow-up with grantees & non-grantees to fill gaps in data |
Increase L+E’s access to program data through direct data reporting from at least 35 program leaders on at least 58 implementations.
[“implementation” in this context is a specific time when a program has been carried out. For example, Wiki Loves Monuments is a program; Wiki Loves Monuments in Argentina in 2013 is an implementation.] |
Surpassed target by 150% for program leaders and nearly 400% for the implementations target.
Directly obtained reporting data from 49 different program leaders on 222 program implementations surpassing improvement targets. |
| Objective 2.
Capture better data (a) Targeted outreach for strong programs and key metrics. (b) Define and obtain metrics for data gaps |
- Increase program leader adoption of metrics and reportability (Increase the number reporting directly by 33%; prepare and update an overview of evaluation and reporting)
- Improve evaluation reports and identification of impactful programs (Identify 10 positive exemplars for writing contest toolkits investigation) - Assessment of quality and retention (Metrics pulls complete for 3 programs targeted for Q3 reporting) |
90% Complete
Behind schedule on pulling on-wiki user activity and content metrics due in part to some Wikimetrics issues in January as well as some time loss to department transition needs. We have completed for the majority (70%), but not all, programs data to be pulled. We have adjusted our timeline accordingly. |
| Objective:
|
Impact on Goal | Status |
| Objective 3.
Promote effective program design Complete 3 Programs Evaluation Reports: other Photo Events, and Editing Contests |
Increase knowledge about Wikimedia programs, their design, and impact through
- Publication on Meta of Reporting Overview (A review of state of reporting and the programs evaluation report series) - Publication on Meta of 3 Programs Evaluations (Wiki Loves Monuments, Other Photo Events, and On-wiki Writing Contests) to be followed by official announcements and RFC - Program toolkits development for Writing Contests and Photo Events* |
85% Complete
Behind on the third program evaluation report (On-wiki writing contests) due to shifting some of team members time to the director search and transition team needs. We have adjusted our timeline accordingly. *Initiating interviews outreach and process for toolkits mapping. |
Success
✯ Increased the number of implementations captured six-fold over last year’s data collection
✯ Doubled the number of program leaders who provided direct supplemental data reporting
✯ Quadrupled the number of implementations with primary data collection support to 222
~ 30% of the identified program implementations
Increased data access has been helpful to fill in gaps so that we can better map program impacts. For instance, we have username lists for nearly 50% of all implementations identified, compared to less than 20% last year, allowing us to better understand retention this round.
Behind schedule on:
- Obtaining user activity and content metrics: We have completed data queries for the majority (70%), but not all, programs with key query parameters available. The lag is also partly due in part to some Wikimetrics issues in January, as well as some time loss to department transition needs.
- 3rd program evaluation report: One of the three program evaluation reports due (On-wiki writing contests) is behind two weeks due to shifting some team members time to the director search and transition team needs.
We have adjusted our timeline accordingly.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department
| Objective | Impact on Goal | Status |
| Inspire campaign:
Create pipeline to source and support 20 new gender-focused projects w/ impact potential |
March campaign targets:
*500 participants (10x increase in IdeaLab‘s monthly) *100 ideas (20x increase in IdeaLab’s monthly) *At least 30 grant proposals |
*629 participants (13x IdeaLab monthly)
*266 ideas (50x IdeaLab monthly) *57 grant proposals total *43 proposals eligible for funding review (# of new grant-supported projects TBD April 31) |
| Emerging communities:
Improve capacity development for select Global South communities |
*Capacity Development Framework (CDF) published on-wiki |
*Changed plans: more GS community research was needed before CDF could be created. |
| Organizational benchmarking:
Build a more impact-focused culture by socializing benchmarking research |
2 research projects are socialized with key stakeholders (WMF staff, committees, and affiliates in the movement): |
*Organizational Effectiveness study finished and presented within WMF and to Affcom. Discussions w/ affiliate orgs scheduled for May at WMCon. *Participatory Grantmaking study finished and shared via blog post and Signpost. Discussions w/ grant committees scheduled for May at WMCon. |
Inspire: Campaign participation and ideas exceeded expectations (2x targets). Successful first step in broadening the pipeline for proactively distributing resources focused on a targeted strategic issue.
- First ever on-wiki “friendly space” expectations pilot in IdeaLab demonstrates potential to set higher tone standards. Community began to enforce these expectations, laying ground for more supportive engagement even on very controversial topics like gender. Survey outcomes on participant experiences coming in Q4.
- Alex Wang & Jonathan Morgan (CR), and Patrick Earley (CA) were particularly instrumental in preparing for and running this successful campaign
Emerging Communities: Interviews already surfacing insights with impact potential for less-known community practices and challenges in emerging communities
- Building trust as a foundation for impactful community health pilots with active emerging community members who are instrumental for community development. Several interviewees expressed happy surprise by the level of interest/attention from WMF.
- Sati Houston & Asaf Bartov for adjusting course to gather more qualitative data in over 30 hours of community interviews. No small effort to gather qualitative data on community health.
Emerging communities: Timeline for original objectives was ambitious.
- Learning: more research needed into emerging communities to validate community development hypotheses. Research on community health requires significant staffing investment. Action: prioritizing staffing in CR, L&E.
Organizational benchmarking: Time for socializing benchmarking research in the community was reduced in departmental transition, and some research has not yet been used as hoped.
- Learning: socializing research takes staff time and needs clear prioritization to have impact. Action: In Q4, prioritizing the Organizational Effectiveness work more highly again.
- Learning: Participatory grantmaking research sparked more community debate on how it was commissioned/reported than interest in the report’s substance. Staff transitions reduced our ability to address this narrative of concern w/ clear info. Action: future commissioned research must be more clearly planned/documented
Inspire: Many proposals developed in the campaign still require significant mentorship in order to become project plans with high impact potential
- Learning: significant staff time is needed to support development of impactful plans, in each round of grantmaking. In department transition, less staff resources were available to support this work for Inspire. 1 PO’s time reprioritized to lead new CR team, other PO also operating PEG’s biggest quarter ever (see slide 8). Action: backfilling Individual Grants PO
For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department
| Objective | Impact on Goal | Status |
| Actively mentor program leaders worldwide | Start tracking interactions with volunteers in order to establish a baseline during Q4 | Initial tracking numbers:
|
| Add content to Arabic Wikipedia through continuing and expanding education program | (No specific goals for Q3 because annual goal was achieved in Q2) |
|
| Add content to Spanish Wikipedia (and sister projects) | Targeted mentoring in Mexico and Argentina to help them run successful education programs |
|
| Objective | Impact on Goal | Status |
| Provide materials for program leaders and encourage community involvement in material development | Launch Education Toolkit by March 2015. Invite community engagement with toolkit | Launched Education Toolkit:
|
| Share lessons learned with broad education and Wikimedia audience and facilitate discussion and networking | Publish monthly newsletter jointly with community
Publish one blog post about education per month Start Facebook group, share one update per week. |
|
- Toolkit launch: Comprehensive overview of materials and best practices, organized logically, ready for international use. Joint L&E and Education project with huge community input, managed and written by Kacie Harold.
- Facebook group launched: Fills a networking need flagged by community members. Successful launch and daily management thanks to having a dedicated education communications intern, Samir Elsharbaty.
- Tracking team’s interactions with community: First attempt at tracking shows huge number of interactions on a range of topics. Tracking made possible through L&E guidance.
- Arabic: Achieved annual goal for bytes added in Q2, allowing us the opportunity to further diversify the Arab World program (continue support in Jordan, and explore an education program pilot in Oman).
- “Education Collaborative” challenges: Education Collaborative was intended to improve team effectiveness by helping community members help each other, but engagement was lower than expected. Targeting Q1 for improvements to the Collab model, based on learnings from other CE teams.
- Weak data for Spanish Wikipedia mentoring efforts: Spanish-language education program has been a focus, and anecdotal signs are encouraging, but promoting the use of WikiMetrics has not resulted in much data. WMF staff currently lacks time to move forward on this focus region; instead focusing on working through Iberocoop education coordinator to increase impact.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department
| Objective | Impact on Goal | Status |
| Integrate community engagement process with product development process | Create product development process that incorporated community feedback in structured, productive fashion | Process still in draft form; new methods of facilitation enacted within Editing Team |
| Support strategy consultation with Community Advocacy | Facilitate feedback from C-Levels to participating community members | Completed |
| Absorb the m:Tech/News newsletter | Take over publication of the TechNews newsletter from Erik’s team. | Initiative has been stalled until new team member onboarded at end of Q4 |
| Implement Single User Login (SUL) | Project manage necessary engineering work
Communicate change to community and resolve conflicting renames Rename 2.8 million accounts |
On track to rename affected accounts beginning 15 April |
- New mechanisms for collaboration: Sherry and Erica supported weekly bug triage meetings and a Qualtrics survey (“papercuts”) targeting five groups of editors that either use VisualEditor significantly or offered other feedback on VisualEditor in the past. This has broadened the base of community members giving product feedback in structured, measurable ways.The papercuts initiative has not surfaced any new Phabricator tasks.
- Strategy consultation: Nick and Moushira supported CA in the community strategy consultation by gathering feedback, identifying active community contributors and drawing attention to topics of interest. The consultation ended on March 16th with 2,216 unique participants.
- SUL: Keegan has project-managed Single User Login (SUL) since mid-Q2. Under his guidance, the necessary tooling was successfully written, and his collaboration with “Global Renamers” volunteers supported the renaming of 6,000 accounts with much lower-than-expected community friction. Kunal (Engineering) was key in supporting technical objectives, and User:DerHexer (Steward) played a vital role in handling difficult renaming cases and supporting broad communication.
- Product community engagement process stalled: The goal of an integrated product development/community engagement process reached only draft form, because of changed priorities (greater emphasis on Visual Editor and strategy consultation) and turnover in product leadership (Howie).
- Tech/News: We did not begin the planned transition of the Tech/News newsletter due to changes in priorities (e.g., strategy consultation) and delayed hiring. Staff being hired for end of Q4 may be able to take on some of this task.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department
| Objective | Impact on Goal | Status |
| Roll out community strategy consultation |
Create strategy consultation framework Publicize appropriately to community Provide basic translations where possible Clean up vandalism Summarize feedback on spreadsheet Engage community directly |
Achieved; community consultation wrapped on March 16th with 2,216 unique participants. |
| Protect communities through a global ban policy. | Create and implement a global ban policy
Staff and implement refined community protection processes |
In process. Global bans policy implemented and initial bout of requested bans deployed in Jan.
New CA staff, with part time focus on community protection role and other CA duties as assigned, anticipated on 4/20. |
| Objective | Impact on Goal | Status |
| Coordinate and facilitate visits to the Wikimedia Foundation office from the Affiliations Committee and Wikimedia Russia |
Coordinate with community members, travel, and administrative teams Coordinate with other WMF teams to plan and facilitate successful meetings |
Both meetings were successfully held in January, with two delegates of WMRU and nearly the entire body of AffCom (95%) + 2 Board liaisons in attendance. Across both groups, meetings lasted 5 days. |
| Support new initiatives in the Call to Action. | We committed to supporting the Call to Action, internally and externally, with community stakeholders. | The CtA was rolled out in stages, with the first round of presentations in January. CA has supported the process of developing the new CE team and has joined in discussing implementation of several initiatives, but staged rollout of the CtA has limited practical implementation. |
Strategy consultation: This engaged the entire team through Q3, with technical monitoring and community engagement from all, and Kim and the strategy team seem extremely satisfied with the outcome.
AffCom visit: January Affcom visit was highly successful, with 95% turnout, high attendee satisfaction, and reported improvement in committee alignment/motivation. Maggie, Janet Renteria and the travel team were particularly helpful in dealing with unexpected challenges.
Inspire: CA put considerable time into supporting the Inspire Campaign, particularly Patrick. We expect learnings from this process will be broadly applicable as we seek to build safer spaces.
Staffing: We made an offer to a new trust & safety hire towards the end of the quarter, but she will not start until mid-April, so little formal progress has been made on new initiatives. Long hiring lead times were further lengthened by the unexpected reformation of a new department. For future hires, we need to find ways to streamline this process.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department
| Objective | Impact on Goal (Q4 targets: no Q3 targets set) | Status |
| Add major partners (including non-English) and extend reach | * Add 5 major partners * Add 5 non-English partners |
* Added 12 partners, 3 non-English: |
| Start global satellite rollout | * Add 10 new global branches |
* Finished setup guide, reviewed with CE Global South and L&E |
| Objective | Impact on Goal (Q4 targets: no Q3 targets set) | Status |
| Train volunteers to run own branches |
* Onboard 15 global coordinators |
* Reached out to 3 WP language groups for new branches |
| Develop reference tech |
* Echo notifications built |
* Initial discussions with Engineering on Echo Notifications |
| Extend our network of influence |
* Present at 5 major conferences |
*Presented: ALA (Chicago), GLAM-US Consortium |
Already exceeded *Q4* goals for new partners, easily: Outreach to new partners is now very reliable, requiring little logistics and with recent partnerships coming in on their own with very low effort.
Volunteer recruitment and coordination growing scalably:Breaking large programmatic requirements into manageable volunteer responsibilities — Partner, Metrics, Outreach, Account coordinators — is helping us scale.
Leveraging successful networking: To help raise visibility in the library community, team has networked and spoken at conferences. Example: Invited to give Project MUSE Keynote speech through connection, introduced to us by OCLC at an ALA conference.
Building Interconnections w/ Community and across Foundation: Invited to serve on US GLAM-Wiki Consortium. Organized internal “Partnerships Brunch” — already seeing results. Sharing learnings with Kourosh, Edu, and APG. Teamed up with CA. Using L&E and GS guidance for program growth.
English-language Wikipedia signup saturation
- New partnerships not filling as rapidly. We suspect a lack of direct invitation, unbundled individual resources with delayed turnaround, resource unfamiliarity, and high requirements (1000+ edits and 1 year). Experimenting with lower baseline (500 edits and 6 months). Q4 onward should see raised awareness and amplify signups as new global branches start.
Tech Development Backburnered
- Team lacks strength in product development — we rely on outside consultation and coding. Signup management tool would scale and speed up distribution. Initial talks about Echo stalled without easy test-case. We likely need a technical coordinator (even if volunteer) and vastly increased tech capacity for superuser projects in either CE or Tech departments.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Community Engagement department
Advancement: Fundraising
edit| Objective | Impact on Goal | Status |
| Raise $3 million in Q3 primarily through:
1) Banner and Email campaigns in Israel, Hungary, Sweden, Denmark, Norway, and Spain. 2) Foundations, Major Gifts: Send out annual report; Send proposals to four new institutional donors and secure a multi-year commitment from one new funder |
Raise $58.5 million in FY 2014-15 | Done. All country campaigns were completed. Raised $7 million, exceeding the goal. The final number is still settling. |
| Research: Complete a study of our December fundraiser using a professional public opinion research firm | Develop a strategy for the long term financial health of the organization | Done; Released on Signpost |
| Objective | Impact on Goal | Status |
| Implement payment processor for China;
Prepare to implement payment processor for Latin America in early Q4 |
Raise $58.5 million in FY 2014-15 | Complete |
| Partnerships: 1. Hire a VP of Partnerships 2. Write a partnerships plan | Develop a strategy for the long term financial health of the organization | VP of Partnerships has been hired! We have an initial six month partnership plan to be refined further next quarter. |
The Advancement team has exceeded our annual fundraising goal by 14%. We have raised $66.4 million by the end of Q3 2014-15. Our annual goal is $58.5 million.
The Advancement team has exceeded our quarterly fundraising goal by 133%. We raised $7 million in Q3 2014-15. Our quarterly goal was $3 million.
All teams within Advancement have contributed to this success. Both Online Fundraising and Foundations and Major Gifts have exceeded their individual team goals.
We need to further develop our partnership plan.
Example:
We did not complete the partnership plan by the end of the quarter. We did hire our VP of Partnerships just before the close of the quarter. We will be working to develop a comprehensive partnerships plan next quarter.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Advancement department
Advancement: Fundraising Tech
edit| Objective | Impact on Goal | Status |
| PCI Compliance Assessment | Security | Done. Exceeded. |
| New payment method: China partner | Enable fundraising in China | Done. Disabled for legal reasons. |
| Start new 3rd party integration: LATAm partner | Enable fundraising in Latin America | Started coding. |
| Upgrade for nightly reconciliation file parsing with current vendor | Security | In progress. |
| Ongoing refactoring and code hygiene improvements in Donation Interface whenever possible. | More flexible (borrowed) and happier developers | Phase 1 complete. |
Reporting Improvements
|
Increase visibility | In progress. |
- PCI: The assessment (using an external vendor) is complete and we’ve begun implementing the recommendations.
- China Partner: We implemented our China Partner in January, but our account was disabled by the processor as soon as it became operational due to legal changes on their side.
- DonationInterface refactor: We accomplished the first phase of our goals, and are beginning the exciting second phase. See [1] for the roadmap.
- LATAm Partner: This goal was to “start”, according to the master goals doc [2], but this is squishy (and was misrepresented in the QR), hence a “miss”. We are now in initial phases of implementation and plan to finish this quarter. This is our top-priority Q4 goal.
- Nightly Reconciliation: Deprioritized due to sliding upstream deadlines.
- Reporting improvements: The subgoals are each in progress, but this goal as stated was too ambitious.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Advancement department
| Objective | Impact on Goal | Status |
| Release iOS app with lead images, image gallery, read more, and Tweet a Fact features | The features will be out in production | The release was pushed back to due regressions that were found by us during testing.
These regressions were fixed and the subsequent update has been approved by Apple and is awaiting release. The update is planned for Wednesday 15th April. |
| Release update to the Android app containing Tweet a Fact, search / featured article widgets | The features will be out in production | The update was released on 30th March 2015 |
| Trial single, more visually appealing “read more” result in Wikipedia Beta and select the best version for production | The new feature must drive more engagement than the old one in order to be released to production |
The new display was trialled in Wikipedia Beta on Android and found to not drive as much engagement. The new version was not deployed to production |
The Android app was listed as a featured app by Google in the Play Store. Downloads of the app increased by 2-3x as a result. Impact on monthly uniques not yet determined.
We are prototyping a new service to deliver HTML to the apps. Initial tests show it is performing well, reducing HTML payload by around 50% and significantly reducing the number of network round-trips required to load articles.
The Android team continues to reduce technical debt and improve performance of the platform. Memory overhead of the app has been significantly reduced. We have seen a significant reduction in the number of bug and crash reports.
Due to increases in staffing, the iOS team has seen a significant increase in velocity this quarter.
Despite increases in staffing helping to mitigate this, development of the iOS app continues to be slow due to technical debt, and the release had to be pushed to April. Wikiwand released its iOS version before we were able to offer a superior Wikipedia reading experience to iOS users.
Search is central to content discovery in the app, and users continue to report that our search experience is subpar. Data continues to show users type multiple queries to find the results they want. The mobile apps team has reached the limit of what they can do with the frontend to increase the relevance of results, and does not have the expertise required to improve the search backend.
Poor monitoring caused an API account creation outage which was only detected due to users giving feedback on the app store. The time that the API broke could only be figured out using manual funnel analysis on the client-side logging in the Android app. The API was broken for over 24 hours before the problem was detected.
Deployment of an experimental version of the mobile apps content service to production is currently blocked for a variety of operational reasons with unclear timelines. In the mean time, users suffer with inflated data usage and reduced performance.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Mobile Apps team
| Objective | Impact on Goal | Status |
| Baselines for WikiGrok campaigns on English Wikipedia | Measure response quality, engagement with feature, and short-term retention of WikiGrok campaigns on English Wikipedia. | Done. Data available here: https://meta.wikimedia.org/wiki/Research:WikiGrok |
| Unsampled WikiGrok campaigns | Run WikiGrok campaigns for 100% of readers, unsampled | Work on Wikidata backend was prioritized over this objective. |
| Send data to Wikidata | At least some of the data generated by WikiGrok campaigns should begin to migrate from the internal database we’ve been using to Wikidata | The objective was not met and is on hold pending community discussion. |
Despite many technical and social challenges, the team was able to generate new WikiGrok campaigns, create a data aggregation heuristic, and build a backend framework for an entirely new mental model for contributing to Wikimedia projects (aggregated reader data vs. one-to-one user-to-revision).
Even in its very early stages, this model shows the most promising engagement numbers of any existing contribution feature, or any new feature the Wikimedia Foundation has ever built.
Leila, Ryan, and Max worked overtime to make all this happen and are total rockstars.
We were unable to launch existing WikiGrok campaigns to 100% of readers. Sending data to Wikidata was prioritized over this objective, and we were not able to do both in one quarter.
The technical and social requirements of sending data to Wikidata also proved too challenging/in flux to tackle in one quarter. The latter (changing community requirements) are summed up by this user:
“(...) there were a lot of freedom in [Wikidata] at the beginning but now we have to know if we want to continue our lonely game of data import without reference which will lead to the reject by [Wikipedia].”
For more detail, see the slide deck and minutes from the quarterly review meeting of the WikiGrok team
| Objective | Impact on Goal | Status |
| Launch pilot of a new reader curation + sharing feature. | Identify issues to inform product direction | We launched pilot in beta. We did not launch with built-in share features |
| Begin collecting baseline usage statistics (creating, adding, consuming) and characterizing use cases to inform further development | Identify issues to inform product direction | The feature-set is well instrumented.
We have already identified and prioritized a few key issues that we will be addressing next. However, due to low traffic on mobile, we do not have enough metrics to drive other important decisions prior to roll-out on stable (moderation being one of them). |
(Cf. revised objectives from Q2 review meeting)
In just over a quarter: 2 weeks of research and 8 weeks of development, we were able to go from inception to launching an mvp in beta for a feature that takes important steps in key areas: article presentation, identity, social, and casual contributions.
We realized we would not exceed our pre-quarter expectations within the first 2 weeks of the quarter and communicated as much in Q2 Quarterly review.
When we made the emergency call for backend support, we were able to borrow Yuri from the Wikipedia Zero and had him on the team within 48 hours.
The team had outside support from design research, Aaron, Dario, and the rest of the mobile web team.
- Team set aggressive goals
- Did not start with necessary backend engineer
- Culture and practices are not conducive to rapid innovation:
- don’t have a great process in place for creating new extensions or making database changes
- fears of community reaction played a big role in determining scope, delaying key features (like sharing), and promoting others (like watchlist functionality)
- Mobile web beta population is not large enough for testing at scale
For more detail, see the slide deck and minutes from the quarterly review meeting of the Mobile Web/Gather team
| Objective | Impact on goal | Status |
| Develop deep dive approach. | Define plan (criteria & market, priorities, goals) | Analyzed countries, scoped |
| Preserve free access after https default. | Complete IP migration | 59 countries total
Remaining: 24% 7 in progress, 7 to do |
| Grow to 90M free page views/month. | Launch inbound requests
Manage existing partners |
74 M pvs as of 3/31
Launched 12 Digicels, Uninor (4/1). Lost Aircel and STC (March) |
| Provide for ongoing support for Zero with fewer tech resources. | Complete portal phase 1
Messaging on projects Messaging for termination |
In progress, will complete by April 20 |
| Objective | Impact on goal | Status |
| Ship Android preload | Preload app with min. 1 OEM | Preliminary yes 2 OEMs. Got de-prioritized by OEM. Needed marketing support. |
| Lay groundwork for industry coalition to promote open knowledge. | Develop materials
Identify participants Do initial outreach |
Identified key players, and had initial discussions with Mozilla and CIS India (4/9). |
Big push on IP migration (from URL-based zero-rating) to support HTTPS and editing. Big win was saving Beeline Russia after Ops defaulted to https (Carolynne, Dan). Adele and Dan actively engaged with the partners in Africa, where partnerships were at risk due to contractual challenges. So while not a “green,” we do see the progress and the engagement as a success.
Wikipedia Zero successes: 1) Carolynne and Manprit renegotiated the [operator] contract in line with operating principles, in signature process. 2) Smriti facilitated a big event in Bangladesh with WMBD and Grameenphone, where Jimmy spoke; resulted in substantial positive media coverage. 3) Smriti and Jorge engaged with partners at MWC, repopulating the active pipeline. 4) 8 partners have updated their IPs and contributed to keep the page views from declining.
We released the portal beta version to partners, who are now using it to set their test period directly. This reduces the partner testing time and allows quicker deployments.
We structured the framework for open knowledge coalition, identifying 45 key stakeholders to engage with. We started positive engagements with Mozilla and CIS India (April). We started socializing the initiative to community members (April).
Deep dive into one country to increase impact was put on hold while we worked through restructuring. The team completed the country analysis and selected two finalist countries. Adele organized a cross- functional team meeting for input and collaborative strategy. We’re ready to revisit the deep dive in Q4.
IP migration was not completed, because some partners were unresponsive, others in contract renewal. We will complete in-progress migrations in Q4, terminate unresponsive partners, and relaunch the ones engaged in contract renewal.
No growth in Wikipedia Zero page views. We lost Aircel India and STC Saudi Arabia, but still had a net gain of 1M in Q3. We intentionally did not pursue new outbound contacts. The team responded to inbound requests, but only Digicel launched. Managing the existing partners takes time away from new launches (especially the IP migration). The learning is valuable to understand the job demands and team capacity as we move forward.
Android preloads were a big disappointment. Carolynne had secured verbal agreement from [OEM], then they deprioritized. Same happened in Brazil. The Indian OEMs were unresponsive. We’re pitching mission, seeded content and search integration, but OEMs look for marketing or revenue. We will re-evaluate as a priority for Q4.
We did not complete the engineering work to set Zero up for low maintenance because Yuri was assigned to work on Gather. We have Eng time in April to complete (Ops dependency).
For more detail, see the slide deck and minutes from the quarterly review meeting of the Wikipedia Zero team
| Objective | Impact on Goal | Status |
| Fix functionality and correctness bugs that affect VisualEditor | Roundtrip testing accuracy stable at 99.8% (if false failures are fixed, would be closer to 99.9%) | All VE Q3 blocker + some Q4 bugs fixed |
| Reduce generated HTML size | 43% reduction in [[Barack Obama]] output | • Unused data stripped from template content
• Eliminated duplicated HTML from <ref>s |
| Work towards Parsoid-HTML read views (stretch goal) | Improved visual diffs
Language variants rendered Cite customized via CSS |
Marginal progress beyond whatever achieved as part of first objective. |
| Objective | Impact on Goal | Status |
| Initiate work to scope template output (stretch goal) | Parsing performance improvements | Wiki pages drafted; Wikimania talks submitted |
| Make the Parsoid cluster more stable | Stable memory usage graphs in Ganglia. Load spikes correlated with actual traffic. | • Handling more pathological pages
• Reliable timeout handling and process restarts • Eliminated memory leaks |
| Gather performance metrics | Performance regressions easier to detect. | wt2html and html2wt endpoints instrumented in detail + dashboards established |
Sustained focus on visual editor bug reports and blockers helped us fix reported issues as they cropped up.
Our fairly robust test infrastructure has been key to the smooth regular deployments and catching problems early. That said, we nevertheless identified a testing hole (now fixed) after VE’s migration to RESTBase. But, given that the bug could have caused serious page corruptions, it also highlighted the remarkable accuracy of our selective serialization strategy.
OPW intern (Christy Okpo) helped us tackle previously neglected task of production performance instrumentation.
Our goals and targets are more ambitious than what we can realistically achieve with our team size and composition. Learning to scale down ambitions. But, having Tim Starling join the rebranded parsing team helps here.
The long tail of small things to fix is long / endless (second system problem). It is equally important to (a) enable read views with Parsoid HTML (b) make forward progress enabling tools to fix broken wikitext on pages (c) fix wikitext / template processing model -- this more fundamental work on wikitext improvements keeps getting pushed back.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Parsoid team
| Objective | Impact on Goal | Status |
| RESTBase deployment | Production deploy of RESTBase with Parsoid HTML and metadata storage | https://rest.wikimedia.org/en.wikipedia.org/v1/?doc
- 15ms mean HTML revision load time - No SPOF, horizontal scaling, multi-DC |
| VE speed-up | - Store all current article revisions for low latency access
- Reduce HTML size by at least 20% - Provide low-latency public REST API |
VisualEditor HTML mean load time reduced by 36%.
Mean HTML size reduced by ~35%. Further significant speed-up coming with VE loading HTML directly from RESTBase (currently live on mw.org). |
| Section editing and retrieval API | Research and prototype section-level edit and view API in collaboration with Parsoid, Mobile and VE | Research and design work, but no implementation yet. |
RESTBase performs really well: ~12ms mean HTML revision load times in production; 10k req/s peak throughput on six boxes. This enables high volume use and consistently low latencies.
API versioning guidelines and Swagger API docs have been well received, and are being adopted by other teams. We now have a flexible and performant API proxy which lets us expose internal services in a uniform and well-documented external API.
We have made some progress on streamlining the service development & deployment pipeline. We developed shared service infrastructure (libraries like service-runner, a service template, standards for testing, metrics & logging), and support other teams in Citoid, Apps, Graphoid service development. Marko Obrovac is driving much of this work.
The section edit API work was slightly delayed by deployment-related work taking longer than expected & tying up limited team resources. We have researched issues and developed a plan for the first iteration with the Parsoid and VisualEditor teams, which is our top priority for Q4.
Service deployments still use more time and resources than we'd like. We have made some progress on shared infrastructure, but there are still long lead times and heavy dependencies on operations for configuration and setup. We hope to improve this situation in collaboration with Operations and Release Engineering in the coming quarter.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Services team
| Objective | Impact on Goal | Status |
| Modern authentication framework for MediaWiki | Re-design MediaWiki authentication process to allow more complex authentication flows | Implementation underway; ETA unknown as this will not be top priority in Q4 |
| Service-oriented architecture authentication | Document use cases for authentication and authorization by various non-MediaWiki consumers | Not worked on; a lighter solution was found for the immediate needs of RESTBase |
| Wikidata query service | Produce a working prototype of a graph search tool in the style of WDQ with a clear plan for production scaling | Implementation underway; progress delayed by unexpected need to select new graph database in Q3 |
| Multi datacenter MediaWiki | Document impediments to active write-active read multi-DC operation; Begin removing those impediments | Implementation underway |
| SUL finalization | Support the implementation of SUL finalization | On track |
Inter-team communications were greatly improved by having a full time interim Product Manager who was included in the PM team meetings. One PM for all of Platform will probably not be enough to successfully manage this communication for all teams in the Platform group but it's a great start.
The new Phabricator based process for signaling that an RfC is ready for discussion made scheduling discussions for both AuthManager and Multi-DC projects much easier than past workflows.
Changing the weekly meeting format from a report-up workflow to a topical discussion highlighted fragmentation of focus of the team members. This led to detailed discussions of possible solutions and the decision to reorganize into separate teams with more narrow focus starting in Q4.
Development across all projects was slowed because we took on too many projects, often assigning only a single full time developer. Reorganization into separate teams should improve focus but we still have some staffing challenges to overcome.
WDQ was set back significantly when the company developing the graph database was acquired. There are risks when working with bleeding edge technology.
The AuthManager project exposed some disconnects between the priorities of the team (developers and Product Manager) and those of management outside the team. Reaching agreement on goals prior to starting work on a project may reduce such tension going forward.
SUL Finalization should have been called out as a project for Q3 rather than being treated as something that would "just take running some scripts". Honestly it should probably be a Top N project for the whole WMF organization.
For more detail, see the slide deck and minutes from the quarterly review meeting of the MW Core team
| Objective | Impact on Goal | Status |
| Scale up HTTPS infrastructure | Scale up HTTPS infrastructure to be able to serve HTTPS-by-default. | Capacity expanded according to measurements.
Additionally, HTTPS performance improvements & SPDY were rolled out. |
| IPsec on cross-data centre links for private data | All PII (cache and logging traffic) between Wikimedia data centers is encrypted using IPsec ESP | Still under testing, stability problems found. Expected to complete near end of April. |
| Virtualization for miscellaneous servers at codfw | Virtualized cluster at codfw set up. At least 2 virtual services migrated to the virtualization cluster | Virtualized clusters at both codfw & eqiad are set up. Services on track to be migrated. |
| Objective | Impact on Goal | Status |
| Ability to serve core site services (wikis) at codfw | MediaWiki application servers deployed.
MediaWiki page loads functional. Capacity meets or exceeds eqiad’s application cluster. |
MediaWiki application servers and dependencies other than ElasticSearch and RESTbase deployed. |
| New external monitoring for performance and uptime metrics | Redefined availability metrics monitored from the start of Q4, and available and reported in the Q4 quarterly review. | Multiple providers evaluated. Contract with “winning” provider executed, monitoring has been setup. |
Our project of scaling-up our HTTPS infrastructure exceeded our stated goal. Not only was the infrastructure expanded to meet the required capacity, but also many additional performance improvements (including SPDY) were implemented. Brandon Black was instrumental in this work.
We were also able to complete the buildout of the MediaWiki application server cluster in the new data center (codfw), despite the large scale infrastructure and many components. Giuseppe Lavagetto led this project and coordinated with MediaWiki Core to resolve configuration and deployment challenges successfully.
We were unable to meet the deadline for full IPsec deployment. Insufficient subject matter expertise was available until later on in the quarter, so several technical blockers didn’t surface in time.
Our goal of deploying a virtualization cluster in codfw was also at risk due to delays at various stages of the procurement process, which resulted in the delivery of hardware very near the end of the quarter. We’ve started hiring for a Procurement Manager which will streamline this.
_Q3_2014-15.png)
These graphs will be replaced with data from our new external monitoring provider next quarter.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Tech Ops team
| Objective | Impact on Goal | Status |
| Beta Cluster stability |
| |
| Successfully re-integrate MediaWiki releases |
| |
| Isolated CI instances |
| |
| QA Support for Editing and Apps |
|
|
| Team process improvements |
|
|
The number of people involved with Continuous Integration maintenance has grown: special note to Kunal from MediaWiki Core Team and Timo Tijhof (as always).
WMDE is independently maintaining their jobs based on WMF templates.
The architecture review of the Isolated CI Instances work by Ops (especially Chase and Andrew) was very thorough and provided a lot of good suggestions.
We were unable to create the Staging cluster in just one quarter. This is mostly due to the unexpected issues that tend to come up. The support from Ops (especially Yuvi) was great, but sometimes things take longer than planned to do them right. And the goal is worth it (demonstrable cleanup which is better for everyone, and “Beta Cluster as a Service” - better name coming...).
Our goals from last quarter were predominantly all > 1 quarter long (hence all the red). Big projects are not inherently bad, but breaking them down into smaller sections can help gain traction/buy-in/support from others.
For more detail, see the slide deck and minutes from the quarterly review meeting of the RelEng team
| Objective | Impact on Goal | Status |
| Upload Wizard | Refactoring and unit testing
Tracking funnel metrics over time |
The objective is complete, a tremendous amount of cleaning up happened |
| JS error logging | Get visibility on which main errors/bugs are responsible for the Upload Wizard funnel drops | Completed the week after the end of the quarter and the disbanding of the team |
| Platform | Investigate a better solution for thumbnail storage | Not worked on |
Mark Holmquist’s relentless focus on UploadWizard helped us greatly reduce the technical debt this extension is affected by. Mark kept a sharp focus on that objective despite the distractions that came our way during the quarter.
Gergő broke new ground on JavaScript error logging. The expertise he’s developed in that area will prove invaluable for the future of Wikimedia engineering. He kept working to complete this task despite the team being disbanded.
Originally we expected to deliver JS error logging for everything, but that objective was revised mid-quarter to focus on UploadWizard. Even that was slightly too much to fit in the quarter. Our main mistake was to attempt to implement the ideal complete system perfectly instead of focusing on a minimum viable product.
This quarter saw us putting out a lot more unplanned fires than usual. One of us having to serve as substitute PM also reduced our bandwidth more than we anticipated. We overcommitted when we estimated our quarterly goals, which is why we left an objective completely untouched.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Multimedia team
| Objective | Impact on Goal | Status |
| Horizon dashboard proof-of-concept | OpenStack Horizon available read-only to Labs users. | Horizon dashboard has been made available with basic functionality. |
| Designate for Labs DNS | Replace our internal DNS (PowerDNS) code with OpenStack Designate. | Installed and being tested.
Due to an additional FQDN change, deployment was changed to opt-in by users, prolonging the migration. |
| Storage capacity & redundancy expansion | Expand storage by 18 TB of usable space. All data replicated to codfw. A documented and tested procedure for (manual) storage failure switchover in place. | Storage capacity was expanded, a cold spare deployed. Documentation and testing not yet complete. This will rollover into the next quarter goal (ToolLabs stability). |
The team’s goals were nearly met, despite a particularly difficult quarter for the infrastructure and team due to unexpected incidents and maintenance work (see next slide).
Yuvi Pandian’s collaboration with the Release Engineering team on its goal of Beta cluster stability was fruitful and resulted in significant improvements in the puppetization of Beta, addressing long-standing problems and technical debt.
While there was good progress on the team’s goals (both internal, as well as external commitments), limited extra effort was dedicated to longer-term improvements, as the team had to spend a significant amount of time fire-fighting.
The goal of deploying Designate for DNS, although nearly met, was modified slightly to take advantage of the opportunity to make an additional needed change. However this changed the migration to opt-in by users, delaying the completion.
Labs’ and Tools’ overall availability was considerably poor this past quarter with multiple outages for subsets of instances, and performance problems on underlying storage infrastructure (see the Appendix of the team’s quarterly review deck). This affected both foundation teams (e.g. Beta) and the engineering community (Tools).
This is being addressed by dedicating the next quarter to availability improvements and availability measurements, as well as engaging into discussions for increased staffing for the next fiscal year.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Parsoid team
| Objective | Impact on Goal | Status |
| MediaWiki Developer Summit | Successful organization of the first MediaWiki Developer Summit | A success, according to the participants' survey and our lessons learned. |
| Data & developer hub | A plan agreed and documented | We focused on prototyping first. We have a broad plan that needs to be written down and checked with the stakeholders. |
| Phabricator migration | Consolidate Phabricator as project management tool | Good progress overall. Some teams still using Trello. Scarcity of maintenance resources. |
| Engage with established technical communities | Identification of top 5 technical partners and first conversations established | Not started. |
Full ECT quarterly report: https://phabricator.wikimedia.org/T94165
After organizing the MediaWiki Developer Summit successfully, Rachel Farrand compiled our lessons learned with recommendations for the next edition that are being integrated in ECT's future event plans and the 2015-16 budget ask (i.e. 3 days event instead of 2).
S Page joined our team in January and broke the vicious circle of theoretical discussions around the Data & Developer Hub by putting together a prototype with real content. This is forcing stakeholders and open tasks to be more specific and less opinionated.
S Page volunteered a Trello - Phabricator migration script, and he has driven the migrations of the Collaboration and former Growth teams, extending an offer to the remaining teams using Trello.
Rachel Farrand and Quim Gil struggled getting clear objectives and main themes with Engineering and Product management, and we were late when announcing the basic plan to our developers. Correction: ECT to own budget and organization of the Summit and hackathons, starting public planning sooner.
S Page has been contacting the right people in Core, Design, and other areas in order to obtain quick tactical progress with the Data and Developer Hub. However, the lack of a written plan and formally agreed dependencies keeps the resourcing of this important project out of the official plans of these teams. Correction: agree the plan with these teams, including committed resourcing.
Quim Gil had proposed 20% Phabricator maintenance time for Chase Pettet (Ops) and Mukunda Modell (RelEng), but didn't follow up for confirmation. While Chase agreed this time with his team, Mukunda has been totally invested in RelEng work. Correction: transfer Phabricator ownership to Greg Grossmeier (RelEng).
For more detail, see the slide deck and minutes from the quarterly review meeting of the ECT
| Objective | Impact on Goal | Status |
|
Prepare for bringing VisualEditor to all new users on wikis where it is currently opt-in. We will get VisualEditor by the end of Q3 to be ready to be on by default ("opt-out") on the English Wikipedia and other wikis in desktop mode, subject to community happiness, for "new users" – anonymous users and logged in users who have never edited; doing nothing to existing accounts with edits. |
Any identified major stability issues fixed – no regressions or data corruptions. Improved performance, especially on load Existing features provided, plus in-flight UX and product changes:
Fit-for-purpose special character inserter; blocking ~50 languages which need it, like Navajo and Welsh |
GREEN |
| Objective | Impact on Goal | Status |
| Stretch goal
Test bringing VisualEditor to all new users on wikis where it is currently opt-in. |
We aim to run a trial for new and IP users getting VisualEditor on by default for a week or two to test the value before the end of Q3. | RED
Delayed for quality reasons until after the end of the quarter. To do in Q4 instead. |
We met our delivery criteria to improve the product greatly, notably including the auto-filled citations feature which has been very positively received.
We worked openly to burn down our work in partnership with the editing community to nominate blockers (top chart).
Load times hugely decreased: network load time down 41% (middle chart) and code load down 37% (lower chart).
For many users on most pages, VisualEditor load is now comparable to, or faster, than loading the wikitext editor.
Huge thanks to the entire Editing team & our partner teams for their great work, and particularly Ori, Marielle, Moriel, and Alex who made it happen.
We did not achieve the stretch goal of running an A/B test of the product for new users on the English Wikipedia. It was delayed for product quality reasons until after the end of the quarter, as we did not feel the auto-filled citations tool was sufficiently reliable to put in front of users, and held that it was so core a part of the user experience that we could not usefully test the effect without it.
We will undertake the test in the coming quarter, instead.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Editing team
| Objective | Impact on Goal | Status |
| French WP: Deployment on Help & some Project pages | Flow on all Help forum pages, and several Project pages in French WP | Held up on pre-filled text and move board, will do full Forum des Nouveaux & Drafts in April. |
| Portuguese WP: Deployment on significant number of Project pages | Flow on all Help pages, significant number of Project pages | On Contact WP page. Held up on various feature reqs. Talking about next steps. |
| Catalan WP: Deployment on all new pages on one namespace | Flow on Help, Village pump, and all new pages on either article talk or user talk | 3 of 4 Village pump pages. Held up on edit patrolling, will do 4th Village pump & Help in April. |
| An active non-WP wiki: Convert LQT pages to Flow | Flow used as the main discussion tool on Translatewiki, Mediawiki or Wikidata | Started converting LQT on Mediawiki; will be done by April. |
| Co-op project, English WP: Flow is a part of the pilot project at launch | Mentors and learners in the Co-op project use Flow to talk to each other, with a bot creating Flow boards for each learner | BAG wanted better board-deletion features. Co-op excited to include Flow in April. |
- VisualEditor integration in Flow, with a redesigned toolbar, including a custom “mentions” feature to support a core talk page use case. The toolbar can switch between VE and wikitext, and uses VE as wikitext’s “preview” mode.
- Some much-requested features and tweaks: Editing other people’s posts, Undo functionality, Indentation model.
- Built Special:EnableFlow, which makes deployments much easier.
- Hebrew is testing Flow on some user talk pages; this can be a wiki where we can try opt-in Flow for user talk.
- Made good progress with languages we didn’t expect: Chinese, Telugu, Punjabi.
.jpg)
- Our first quarter focusing on increased deployment on language Wikipedias. We were too optimistic about the timing.
- Great interest and engagement with Translatewiki in January, but lost touch suddenly & haven’t been able to engage since. Lost some time building features specifically for them.
- Shifting focus to LQT transition on Mediawiki increased feature requests.
- We realized too late that board deletion would be a crucial feature for the English WP Bot Approvals Group, which kept us from the Co-op pilot launch.
- Dependency on code review from the Search team.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Collaboration team
| Objective | Impact on Goal | Status |
| [CX] Respond to issues of initial deployment |
|
Talk page queries and Phabricator tickets promptly responded to.
Feature fixes and bigger bugs planned in sprints |
| [CX] Increase Language Coverage and support |
|
Increased coverage from 8 to 20 Wikipedias. Blocked on negotiations with Yandex for legal clearance to use the MT service for more languages |
| [CX] Improve link editing and creation |
|
Delayed due to urgent development needed for CX Analytics |
| Objective | Impact on Goal | Status |
| [CX] Facilitate finding articles to translate i.e. entry points |
|
Suggestions from interlanguage links, contributions menu & campaign messages during new article creation have been implemented. > 1200 articles translated |
| [Non-CX] Scoping for focus areas |
|
Preliminary statistics collected. Translate and ULS need priority scoping. |
- Increased precision in the new language activation process and follow-up on bugs and enhancements. 23 Wikipedias currently include CX. Collaborated with Tech Ops & Release Engineering to achieve this.
- Community requests helped prioritize language deployments and continuous interactions identified failures & special cases.
- Negligible deletion ratio of articles created with CX; only 2 articles deleted.
- Entry points improved discoverability. Sharp increase seen in the number of users activating the tool through new entry points. Design team will continue exploration.
.svg){kind=spacer}
- Collection of CX analytics data required additional developer overhead and out-of-turn planning. Increasing the scope of analytics support for the team would be helpful.
- As a fallout, restructuring the links framework had to be delayed to Q4
- Unable to complete the inclusion of new machine translation system - Yandex, due to pending legal clearance. This reduced the scope of supporting machine translation for more languages
- Need to better anticipate translation patterns for individual languages and help users translate regularly. Given the diverse factors we have not yet found a model to better explain the trends.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Language Engineering team
| Objective | Impact on Goal | Status |
| Fully instrument editing experiences across devices/platforms to support VisualEditor development | Collect and continually visualize key metrics including but not limited to:
- initialization time and save time (perceived and actual) - edit funnel from start of an edit action to successful save - save rates, abort rates, error rates across experiences: mobile, desktop, wikitext, VE. Prioritization of metrics will be driven by VisualEditor success criteria. |
First rev dashboard delivered early April.
Investigating data integrity issues with wikitext instrumentation. |
| Develop a prototype report and visualization on Unique Clients per project per day and month | Delivery of report to provide insights on readership trends | Still blocked on Ops support to integrate with Varnish |
Editing Dashboard and other VE analytics support
Close partnership with VisualEditor team this quarter has been key to developing consistent instrumentation and data visualization appropriate to the use case.
Unique Clients
The team designed a novel method to count unique visitors using a cookie and a last-access date; no pervasive tracking needed.
Editing Dashboard
Completed a week after Q3 due to lateness in getting wikitext editor data for comparison (VisualEditor-related work was prioritized more highly). Wikitext work landed late and there are still data integrity issues with the instrumentation.
Unique Clients
The project is blocked on prioritization with Ops. Need to establish clearer alignment around shared objectives early-on, possibly as part of goal-setting process changes.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Analytics Engineering team
| Objective | Impact on Goal | Status |
| Data services
• Prototype revscoring service |
Have a working prototype up on Labs of a public, query-able APIs of machine classified scores of Wikipedia revisions. | Released libraries the service depends on, tools for handcoding and project documentation. API up on Labs |
| Traffic reporting
• Finalize Unique Clients requirements • Support AnEng with the productization of Page Views reports • Continued responsibility for ad hoc traffic analysis and reports |
Clear all research dependencies for productizing traffic reports by Analytics Eng | Completed QA of data based on new pageview definition and supported AnEng with the design of UDFs. PV reports as of Q4 will be automatically generated from Hadoop.
Provided methodology and requirements for counting Unique Clients, implementation is pending. |
| Mobile research
• Continue research support to WikiGrok and microcontributions • Research support for Project Gather (aka Collections) |
Deliver analysis of an AB test for WikiGrok on stable, perform all supportive analysis needed by the team to decide on next iteration | Completed reports on WikiGrok test.
Provided ad-hoc support to Gather team prior to launch. |
| Objective | Impact on Goal | Status |
| Fundraising research
• Develop proposal to run FR as a minimization problem [ Better AB testing methods, Characterizing low value impressions ] |
Deliver and publish AB test methodological proposals | Completed analysis of Big English. Released tool with traffic projections.Published methodological reports: 1 - 2; #1 is now used as part of ongoing FR analysis |
| Formal collaborations
Wrap up Improving linking structure Kick off Increasing article coverage |
Complete report on improving link structure | Improving linking structure report in progress. Algorithm implementation being tested
Work on article creation recommenders started. |
Achievements
The team showed strong leadership in solving complex problems:
- Aaron took the lead of a team of grantees to do R&D work on first AI-as-a-service prototype.
- Leila performed research on microcontributions, led research with Stanford’s InfoLab team, and started a new collaboration with Stanford’s DeepDive team.
- Ellery redefined our testing methodology for FR and started work on big data services.
- Oliver completed work towards a new, robust page view metric
Other Q3 achievements beyond stated goals
- Completed 4 major open data releases
- Led and released org-wide Open Access policy for all WMF research collaborations
- Hosted a workshop on cross-platform research and a Wikipedia meetup at CSCW ‘15
- Developed with UX team a vision for content remix and reuse
- Organized a Wikipedia research workshop at ICWSM ‘15
(received great support from Team Practices, Legal, Analytics Eng, Ops, Mobile for all these projects)
The completion of link recommendation is delayed due to the need (identified through research in Q3) for an improved algorithm. There are edge cases in which the algorithm doesn't perform well. We are 1) assessing the quality of the algorithm, 2) doing experiments with improved versions. We commit to having a working version of the algorithm by 5/8, along with its quality assessment, a research page report and a paper submission and a release of the underlying data (aggregate browsing traces) for the community.
The team provided continuous responsibility for ad hoc reporting and querying (both at team- and org- level), lacking self-service reporting infrastructure for key metrics: while we substantially met our stated goals, ad hoc reporting slowed down our progress.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Research & Data team
| Objective | Impact on Goal | Status |
| Provide dedicated Team Practices Group support requested by product/engineering teams. | Hire an additional TPG resource to work with the MediaWiki Core team | Kevin Smith hired, started 3/9/2015 |
| Identify patterns and trends in team health across WMF product/engineering teams. | Publish Q2 health-check survey results | Data published and presented to EMGT 3/9/2015 |
| Measure and support engineering/product team health of WMF product/engineering teams. | Iterate on and deliver updated health-check survey to same teams as Q2 plus Editing and Community Liaisons | Q3 health check surveys completed week of 3/30/2015 |
| Support team development across WMF engineering | Conduct one structured team practices workshop | Deferred due to org prioritization of VE release |
| Support realization of top priority organizational goals. | Provide dedicated TPG support to the Editing team. | Joel Aufrecht hired, started 4/6/2015 |
- Mid-quarter pivot to support Call to Action and top organizational priority
- Built trust and laid groundwork for enabling dedicated TPG resourcing of Editing team
- Grace Gellerman
- Discovering capabilities and limitations of Phabricator as a PM tool
- Executive ask for burndown charts
- Streamlining organizational quarterly goals process
- Built trust and laid groundwork for enabling dedicated TPG resourcing of Editing team
- Launched cross-team initiative to decrease tech debt and increase automated testing
- Discovered trend of not enough automated testing/too much tech debt from team health checks
- Collaborating with Release Engineering to facilitate improvements
- Kristen Lans, Grace Gellerman, Kevin Smith, Dan Duvall (RelEng)
- Ability to support team health and facilitate changes in service to the Call to Action
- Cross-org and inter-team communication issues
- Exacerbating entropy, rumors, friction
- Increasing stress, causing drag on team health and sustainability, slowing our ability to facilitate change
- Phabricator is a better bug tracker than project management tool
- Burndown chart functionality limited, misleading
- Project management features lacking and require workarounds
- Collaboration with Release Engineering planned to facilitate improvements
- Cross-org and inter-team communication issues
- Ability to support product/engineering and the broader organization
- Could exceed expectations if we could move faster through the hiring process
- Friction experienced with: securing req numbers for out-of-band hires, lack of sourcing, recruiter turnover, candidate interviews with execs, offer letter approvals
- Could exceed expectations if we could move faster through the hiring process
For more detail, see the slide deck and minutes from the quarterly review meeting of the TPG
| Objective | Impact on Goal | Status |
| UI Standardization: Full coverage of iconography in core and WMF-deployed extensions with mw.ui icons | Standard iconography available to all projects | Completed (iconography) |
| Hire Sr. Designer with technical focus for partner projects and other feature areas | Accelerate UX standardization efforts | Not CompletedOne candidate reached final stage but did not pass; slow pipeline |
| Release Hovercards to selected wikis based on community requests | Completed**zh.wiki on hold pending review of API load analysis | |
| Extend MediaWiki theme and Living Style Guide to include all components used by WMF projects | Not Completed
High VisualEditor priority has impacted OOUI/OOJS devs | |
| OOJS controls for all in-use (core) controls completed | Not Completed
High VisualEditor priority has impacted OOUI/OOJS devs |
Achievements
Foundational work for UI Standardization
- Icon library completed and integrated into core/extension
- Technology for UI Standardization resolved to us OOJS, OOUI framework
- Final designs for core component libraries completed
User Research
- Extensive evaluative testing completed for top tier projects (VisualEditor, Flow, Apps)
- Infrastructure and test plan for REFLEX qualitative readiness testing completed
- Provisional persona content created and iteration workshop being defined.
Completion of UI Standardization library did not occur due, primarily to lack of engineering resources, which were redirected to VisualEditor.
Pilot tests for testing tools (Loop11 and UserZoom) to support implementation of REFLEX.
For more detail, see the slide deck and minutes from the quarterly review meeting of the User Experience team
| Objective | Impact on Goal | Status |
| Implement standardized product development methodology which includes community input across teams working on user-facing functionality |
|
Largely deferred:
Community Engagement group was moved out of Product in February (restructuring), and overall focus has shifted to prep for VisualEditor release. Product process overhaul is being subsumed into engineering/product re-org, which was jointly led by engineering & product management. New goalsetting/prioritization model for the organization is being developed under leadership of the COO. |
- Leadership team in engineering/product have collaborated on development of new org structure designed to increase velocity of engineering/product work and to build next level of highly autonomous product leadership.
- Clear focus on VisualEditor as top priority has galvanized teams to deliver a high quality product.
- Pending or completed reorgs in engineering/product/community made for a turbulent quarter with significant handover and re-negotiation of commitments mid-quarter; process improvement goals were backburnered.
- Product Management is under-staffed due to staff departures and delayed hiring on new PMs until the new organizational structure is in place. Teams are spending more time on technical debt / backend work due to lack of PM support.
For more detail, see the slide deck and minutes from the quarterly review meeting of the Product Management team