Grants:APG/Proposals/2016-2017 round 1/Wikimedia Deutschland e.V./Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$ 1,337,580	$311,208 (or 1,648,788 total with Wikidata funding)	+23.27%
Budget	$7,042,988	$7,579,222	+7.61%
Staff (FTE)	68.41	71.48	+3.9%

Overview of strengths and concerns

This section summarizes the strengths and concerns identified by staff, which are explained in the detailed narrative section of this assessment.

Strengths

• WMDE is a professional organization, with ample funding and staff expertise.
• Performance across global metrics appears to be improving.
• WMDE is playing an important role in advocacy work at the EU level.

Concerns

• Given ample unrestricted revenue, it is not clear why WMDE is requesting funds from the FDC process.
• A lack of targets makes past and future performance difficult to assess.
• Budget clarity is lacking for large line items, where a link to impact is not clear.

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

• Wikimedia Deutschland works with a challenging community. German Wikipedia is a well-established project with specific content standards and behavioral norms. This can make it an unwelcoming environment for new contributors, and resistant to interventions.
• There is an ongoing tension among some German-language contributors and Wikimedia Deutschland, since there is not universal agreement about what the chapter’s role is in supporting these contributors. As a large and strategic organization, WMDE also needs to balance strategic program design with the expressed needs of their community.
• WMDE is located in Berlin, which is a hub for software development. This has enabled them to build a talented team to execute their flagship project, Wikidata. This project is funded by a separate agreement with the Wikimedia Foundation, due to its international relevance.
• WMDE is able to maintain a steady revenue stream through donations, which means they do not rely on the WMF or other institutional donors to financially support their core operations.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

• Perhaps due to the restricted nature of last year’s grant, WMDE did not set any global metrics targets in their last proposal, although they are reporting on them in their progress report. This makes it difficult to track achievements and assess performance.
• WMDE reported on global metrics in their midpoint report for 2016, and they appear to be on track for exceeding 2015 achievements.
• Achievements are especially striking in the area of “articles added or improved,” where WMDE has already achieved 30,116 articles at the midpoint (compared with around 9000 in the impact report of 2015, although results are difficult to compare due to the lack of collated results in that report).
• Wikidata KPI’s evolution shows that apart from the number of editors and active editors (who are at at an increase of +2% and +3% respectively), the announced 10% increase across all KPI’s will likely be exceeded.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

• WMDE’s ED of many years abruptly left the organization in 2014. A new ED was hired in 2015. The transition appears to have gone smoothly and the organization has stabilized.
• In November 2015 WMDE’s general assembly adopted a change to the bylaws which allows the Board (Präsidium) to appoint two members (on top of the seven elected), a motion which will allow WMDE to address potential skill or representation gaps on the board. The next elected board (in November 2016) will be the first to decide to appoint two members outside of the election process. This change may be a good means to establish continuity and strengthen good governance.
• Wikimedia Deutschland has built strong capacity for Wikidata software development in their organization, and their ongoing work with Wikidata is being funded by a $1.2 million agreement with the Wikimedia Foundation. WMDE and WMF are collaborating effectively in this area.
• WMDE has always been active and innovative in contributing to the development of evaluation in the Wikimedia movement, for example in the production of useful learning patterns shared with the movement. Yet the proposed plan lacks clear targets in some areas that would allow them to effectively measure impact on a large scale.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

• WMDE works collaboratively with its members and the different communities it serves to iterate its annual Compass (yearly strategic document), which falls into the framework a five-year strategic plan that covers 2014-2018. We appreciate the dynamic and iterative nature of WMDE’s strategy process through its yearly implementation of strategy through the annual Compass.
• WMDE’s strategic plan outlines actions in five strategic areas, making a connection between program activities and WMDE’s larger strategic vision. The plan does not contain long term strategic goals or targets, and it may be difficult to track progress against this plan. It also does not include a risk assessment, or assessment of context, or a long term plan for resource allocation.
• The five areas outlined in WMDE’s strategic plan are supporting volunteer communities; relationships with cultural, educational and scientific institutions; conditions for free knowledge; developing new ideas, tools and projects; and exchange, scope and sustainability. The 2017 Annual Compass identifies three fields of action: support, attract and retain volunteers; software development; and strengthen conditions for knowledge.
• Strategy has been effectively implemented in WMDE’s 2017 annual plan, through their focus on volunteers, software, and advocacy.
• WMDE’s focal point “Attract and retain volunteers” is particularly relevant in the context of the decline of activity of editors on the German Wikipedia and the often complicated relationship between the chapter and the communities it serves. We are glad to see WMDE taking this on as part of their strategic vision and look forward to the implementation of this aspect of their strategy across the next several years.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

• The Community Support program continues to be Wikimedia Deutschland’s weakest program, while Wikidata is its strongest. In community support, the increase over the (partial) baseline of 258 individuals applications over the year seems low for a budget line of €550,000 (~$608,000). The target of 100% final response within three weeks of application for grant requests does not seem relevant to understanding the impact of this work. While WMDE includes a metric for the usefulness of volunteer support serves, it has not been developed in time to provide targets in this proposal. WMDE is also tracking the monthly increase of editors on the German Wikipedia that have 10+ edits as one of its grantee-defined metrics, although it is not clear how WMDE’s activities are linked to this metric or how this metric will be calculated (e.g. over what time period).
• WMDE hired the “Wikimedian in Brussels”, in an effort to streamline the actions of the Free Knowledge Advocacy Group EU (FKAGEU) operating in Brussels. This is an important move to ensure Wikimedia’s continued presence in decision-making circles at EU level and to strengthen the credibility of Wikimedia affiliates’ efforts in that domain.
• The policy themes defined by WMDE and the FKAGEU are aligned with Wikimedia’s overall goals and priorities and essential to the advocacy work to advance the Wikimedia mission; specifically, in Freedom of Panorama, Free government works, and Public domain actions. WMDE includes policy processes influenced by Wikimedia as one of its grantee-defined metrics.
• WMDE is also tracking the number of data uses from Wikidata and the number of uses of media items for GLAM-generated content, both of which are relevant metrics for WMDE’s software and GLAM work.
• WMDE’s goal to set up a plan for the migration of unstructured Commons data has potential for impact across all of Wikimedia’s projects. For this they plan to work closely with the Commons community.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

• Wikimedia Deutschland’s financial presentation is much improved in the current proposal, with clear and appropriate allocations across staff, operations, and program areas. It is especially useful to be able to differentiate between personnel and non-personnel expenses.
• WMDE has significant overhead in the form of administrative and office expenses, as well as personnel expenses, which is typical for a developed organization with a large staff.
• Membership services have been moved from the Fördergesellschaft to WMDE, and are integrated in the overall budget for Communication. It would have been useful to break down the costs in the €182k line for Communication.
• WMDE’s budget includes some line items that are not clearly related to impact. For example, nearly $1 million is allocated to “other program expenses.” There is no way to assess whether this large line item will lead to commensurate results. The same applies to the “risk liquidity” budget line including $128,000, which is not clearly linked to impact. While it has been understood last year that this money would be used to keep a certain flexibility in adapting programs across the year, it is not clear in the progress report what impact this flexible budget line has helped generate. It is not clear how the €550,000 (~$608,000) forecasted under non-personnel expenses for Volunteer Support will be used. Some of these line items are equal to the entire budget or several times the entire budget of other organizations in the FDC process.
• Wikimedia Deutschland is receiving 1.2 million US dollars to fund essential costs for Wikidata, through an agreement with the Wikimedia Foundation.
• Given that Wikimedia Deutschland has significant unrestricted revenue, they do not depend on APG funding for general operating support. It is therefore not clear what rationale underlies their application for the substantial sum of €300,000.

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Strength	WMDE has a long term strategic plan in place that is iterated on an annual basis through its Compass. The plan makes links between programs and activities and WMDE's larger vision clear. The plan is well-implemented in the 2017 proposal, which focuses on volunteers, advocacy, and software. WMDE has increasing alignment with WMF's strategy, especially in the area of software development.
P2. Potential for impact at scale	Strength	WMDE is showing their ability to achieve results related to online impact, through articles and media added, and their work on Wikidata may achieve broad impact throughout the movement and beyond. We also appreciate WMDE's advocacy work, which could have impact at the EU level. Some approaches, like community support, are not likely to have impact at scale.
P3. Objectives and evaluation methods	Neither	WMDE clearly has strong capacity in the area of evaluation, and this proposal demonstrates improvements in their willingness to set targets. At the same time, targets are still being developed in some key areas, which makes evaluation of the plan difficult.
P4. Diversity	Concern	WMDE does not address the topic of diversity. For an organization this size and with its capacity, it is unfortunate that they are not leveraging their skills and competence to work on this programmatic area.
Organizational effectiveness
O1. Past results	Strength	Past results remain difficult to track due to a lack of targets, but WMDE is on track to exceed their 2015 results for global metrics. In areas such as contribution to content (images or articles added) the increase is very significant. Wikidata KPI's are also on track to reach the 10% increase forecasted last year, except maybe in the number of editors involved.
O2. Learning	Strength	WMDE values learning, and has created many learning patterns in diverse areas, programmatic as well as organizational or technical. They have a long tradition of sharing their experience with the wider movement.
O3. Improving movement practices	Major strength	WMDE has a long tradition of working on movement wide activities and sharing their findings, as well as supporting movement wide events, such as Wikimedia Conference. They share their learnings through learning patterns in diverse areas such as GLAM but also how to work with volunteers. Their work in helping streamlining the Advocacy efforts at EU level through hiring the Wikimedia in Brussels is also to be commended.
O4. Community engagement	Neither	WMDE has always had a complicated relationship with the communities it serves, at times conflictual, at others synergetic. In the past years, WMDE has managed to foster community involvement particularly around the technical wishlist, which has allowed them to answer community needs in the technical field, and the local places and hubs, which have successfully fostered local and regional presence. However, it is not clear through this plan how WMDE really engages community on a large scale around its work. Volunteer support is a big part of this proposal, and we are curious to see how this pans out in achieving real impact in the community engagement field.
O5. Capacity	Strength	WMDE's executive leadership is now stable, and the board has made improvements to its governance structure. WMDE's talented staff has expertise in software development, fundraising, event management, and other important capacities.
Budget
B1. Past budgeting and spending	Neither	Functioning within the FDC process has been a challenge for WMDE, which has a budget and staff structure that are significantly different from other organizations in the process. It has been difficult to achieve an appropriate balance in budget detail and evaluation which makes it difficult to assess the accuracy of the budgeting process. WMDE has been underspending against their planned budget in 2014 and 2015, while this year, where they project to spend more than planned, in line with their increase in revenue.
B2. Budget is focused on impact	Neither	We appreciate that WMDE is offering increased clarity in its budget, which gives a better understanding of how expenses may be related to outcomes in some program areas. A lack of targets in key areas sometimes makes this association with expenses and outcomes difficult to assess. While some of the programs, despite the lack of clear targets in this proposal have real potential for impact, among which Wikidata, we are still worried that the Volunteer Support program does not clearly outline outcomes that match the budget allocated. While we understand that WMDE's APG request this year may allow some programs to go above basic outcomes, this is not clearly outlined in the plan.

This staff proposal assessment is the work of FDC staff and is submitted by: Delphine (WMF) (talk) 21:38, 8 November 2016 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.