Grants:APG/Proposals/2016-2017 round1/Wikimedia Ukraine/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$75,000	$87,000	+16%
Budget	$76,200	$88,200	+15.75%
Staff (FTE)	1.5	2.25	+50%

Overview of strengths and concerns

This section summarizes the strengths and concerns identified by staff, which are explained in the detailed narrative section of this assessment.

Strengths

WMUA targets culturally relevant content through its GLAM work.
WMUA has contributed to movement learning and coordination through WLE
WMUA successfully leverages volunteer initiatives to achieve impact.

Concerns

WMUA does not have a long term strategy
WMUA’s budget growth is not linked with impact

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

WMUA functions within a challenging and sometimes unstable political environment. At times, the political situation in Ukraine has made maintaining activity challenging, although WMUA has managed to grow and thrive despite this difficult atmosphere.
WMUA is located in Eastern Europe, offering opportunities for collaborating in the regional CEE group. WMUA continues to be a leader and major contributor in CEE and the broader Wikimedia movement.
Ukrainian is one of the top 30 languages of the world by number of speakers, with 39 million speakers worldwide, although less than half of people in Ukraine have Internet access. This may present a challenge when working with new or existing contributors, but at the same time may be an opportunity for growth and impact in the long term.
WMUA has partnered with its communities and the Wikimedia Foundation through the Community Capacity Development initiative, which has focused on conflict management and included a two-day in-person training this year.
Wikimedia Ukraine prioritizes volunteers in their work, and considers how community involvement is crucial to the success of programs like GLAM and education. Programs are not staff-led, but rather community-driven. WMUA collaborates with contributors both online and offline, and appears to be developing a pipeline of active volunteers to ensure that volunteer contributions to the organization will be sustainable over time.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

This is WMUA’s first year reporting in the FDC process, and so yearly patterns of activity have not yet been established. In some ways, this makes it difficult to understand how achievements may predict future performance.
WMUA already reports 3,489 articles created or improved by mid-2016, against a target of 3,550. Most articles were generated through WMUA’s thematic content focus, including 1,672 articles generated through competitions. WMUA’s wikiworshop activities seemed to be exceeding expectations, with 1000 articles created through the WikiFlashMob (of 250 targeted), and better than expected participation achievements for these activities.
Despite a large number of images uploaded, WMUA is underperforming against their targets in the area of images in use with 412 achieved at the 2016 midpoint out of 10,605 targeted. WMUA has mentioned that motivating contributors to work on adding images to articles has been challenging for some activities.
While not on track to meet their ambitious targets in these areas, WMUA has involved 580 active editors and 380 new editors in their work by mid-2016.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

WMUA is going through a period of organizational transition due to the departure of their long time volunteer ED. Rather than replacing the role of ED, the board is taking on staff management responsibilities. WMUA has 1.5 FTE in 2016 and plans to increase to 2.25 FTE in 2017. The increase will lead to more hours for each of WMUA’s staff positions, rather than a change to the staffing structure or staff roles.
WMUA has staff in two cities (Kiev and Lviv) in order to reach its community, which is dispersed over a large geographic area.
WMUA has a committed board, and a pipeline of volunteers who are working on various projects. WMUA has a group of highly engaged volunteers that are increasing its capacity.
WMUA has not yet created a strong annual plan, and a list-style proposal may be difficult to effectively measure and implement, especially while managing the rigorous requirements of the FDC process.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

WMUA has no strategic plan, and is conducting planning on an annual basis. WMUA explains that they plan to develop a strategic plan in the coming year. Multiyear strategic planning is expected of all organizations in the FDC process, because it is necessary for effective annual planning and prioritization of resource allocation needed to manage general support funding.
Strategy will also help WMUA and the FDC to understand WMUA’s longer term program plans, and to put program outcomes into a larger strategic context. This will also help WMUA and the FDC understand the long term vision for resource allocation to these plans.
In general, WMUA’s programs have a strong focus on content, which is well-aligned with the WMF’s strategy and the strategy of many other Wikimedia chapters.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

WMUA has divided its activities into three programs: Outreach & Partnerships, Thematic Content, and Community Support. Despite this high level organization, this proposal still reads like a list of many projects and activities. Projects included in each program do not appear to be linked by a similar theory of change, and distinct targets and objectives are presented in each project area rather than provided for each program overall.
We appreciate that WMUA is building on its expertise in photo competitions, and continuing the coordination of the international WLE initiative, which is an important contribution to the movement.
In GLAM, we see that WMUA is continuing their focus on culturally relevant content, and are working to change attitudes among institutions in an environment where this is still an uphill battle. Work with journalists is particularly interesting, and something that others in the movement can learn from, although we don’t see any relevant targets included for this work.
We do not see WMUA tracking many relevant outcomes for its Community Support work, except for measuring the survey responses of training participants. Although we realize that community work is core to what WMUA does, it is difficult to link this program with online impact.
In line with their content focus, WMUA has selected media files in use as a grantee-defined metric. Their second grantee-defined metric is geographic diversity; this may be particularly relevant in WMUA’s context and yet the metric is not well-defined enough, since it does not take into account the nature and depth of work in particular regions.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

WMUA’s budgeting and evaluation practices are still stabilizing, as they enter their second year of annual planning in the FDC process. WMUA expects to spend on target in the current year.
WMUA relies on the APG process for 100% of their funding, and raising funding from other sources in Ukraine would be difficult. We recommend that WMUA begin to track in-kind donations, to show evidence of other forms of support.
WMUA is increasing their budget and grant amount by $12,000, and it is not clear that this increase is linked to the potential for increased impact. For example, there is a $10,000 increase for contests in 2017, included in the thematic content area. Amounts allocated for prizes ($9,000), scholarships ($8,000), merchandise ($7,000) microgrants ($3,000) would be likely to receive more scrutiny in Simple APG, to ensure a link to impact. Participation in Simple APG may have other benefits for Wikimedia Ukraine, since application and reporting processes are lighter weight and less resource-intensive.
Wikimedia Ukraine does not keep any reserves.

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

Major strength
Strength
Neither a strength nor a concern
Concern
Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Concern	WMUA is entering its second year in the FDC process, and does not have a strategic plan in place. However, it is launching community consultation to serve as a basis for their upcoming strategic plan.
P2. Potential for impact at scale	Neither	WMUA's work on the international coordination of the Wiki Loves Earth photo competition (about $12000 of their current budget) is likely to support results outside of WMUA's borders, although the impact of most other initiatives is likely to remain within Ukraine. Conducting work on a national scale and gaining traction with GLAM partnerships has been an ongoing challenge for WMUA.
P3. Objectives and evaluation methods	Neither	WMUA provides relevant metrics for its content-based programs, but does not provide relevant metrics in areas like Community Support or their work with journalists. WMUA occasionally provides inconsistent numbers in reports, which may indicate that it is challenging for them to evaluate their work across so many activities with distinct objectives.
P4. Diversity	Neither	While they work in a country with an emerging opportunity, they are not addressing diversity through their program activities or including diversity-related metrics (except for one participation-related target in their outreach program).
Organizational effectiveness
O1. Past results	Neither	WMUA set ambitious targets for 2016 and is not on track to achieve targets for images in use, although activity may increase in the last six months of 2016. WMUA is new to the FDC and annual planning process and a pattern of program activity has not been clearly established. WMUA is on track to exceed their targets for articles added or improved in 2016.
O2. Learning	Neither	WMUA performs adequately in this area, but we have not seen significant improvements since last year.
O3. Improving movement practices	Major strength	WMUA receives one of the smallest grants in the FDC process, and yet is an active contributor to the international movement. WMUA continues to have a leadership role in the regional CEE group, and is coordinating WLE internationally. They are also participating in the pilot Community Capacity Development initiative.
O4. Community engagement	Strength	WMUA is doing a good job of cultivating a pipeline of new volunteers who are becoming more involved in its work and its leadership. WMUA is partnering with WMF and its community on the Community Capacity Development initiative to improve conflict management skills in the community.
O5. Capacity	Neither	WMUA has more experience managing staff and we do not have concerns with WMUA's current staffing model. WMUA is currently undergoing a leadership transition, as their volunteer ED has left her position in the organization.
Budget
B1. Past budgeting and spending	Neither	WMUA is new to annual budgeting at the FDC process, and spending for 2016 appears to be on track. At the same time, budgeting practices are somewhat inconsistent (e.g. level of detail), and relatively large amounts of funding are allocated to prizes and merchandise.
B2. Budget is focused on impact	Neither	WMUA has a reputation for doing effective work with a small budget, and costs in Ukraine are minimal compared with other locations where FDC organizations operate. At the same time, large amounts of money are devoted to expenses that remain unscrutinized as part of the FDC process, such as more than $9,000 in prize money and $7,000 in merchandise. Based on the current year's results, it is not clear that the $10,000 increase in contest costs will lead to more impact in this area.

This staff proposal assessment is the work of FDC staff and is submitted by: Delphine (WMF) (talk) 21:46, 8 November 2016 (UTC)

Staff proposal assessment framework

Major strength. This is something the organization does very well, and this is a strong indicator of future success.
Strength. This is something that the organization does well, and this could indicate future success.
Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
Concern. This is something that the organization does not do well, and this could make funding the application a risk.
Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.