Grants:APG/Proposals/2016-2017 round 2/Wikimedia Norge/Staff proposal assessment

The staff proposal assessment is one of the inputs into the Funds Dissemination Committee (FDC) proposal review process, and is reviewed by the FDC in preparation for and during the in-person deliberations each round. The purpose of the staff proposal assessment is to offer the FDC an overview of the expert opinions of the FDC staff team about this annual plan grant proposal, and includes (1) A narrative assessment; (2) An assessment that scores each applicant according to specific criteria in program design, organizational effectiveness, and budgeting.

Overview

Summary

	Current (projected)	Upcoming (proposed)	Proposed change (as a +/- percentage)
FDC or other relevant funding	$191,590	$220,150	+14.91%
Budget	$204,085	$238,595	+16.91%
Staff (FTE)	1.7	2.3	+35%

Overview of strengths and concerns

This section summarizes the strengths and concerns identified by staff, which are explained in the detailed narrative section of this assessment.

Strengths

• WMNO’s grantee-defined metrics, diversity and volunteer hours, are highly relevant.
• WMNO is improving its focus on engaging volunteers and communities in its work.
• WMNO has achieved good results in terms of images in use, through a donation from The National Library.

Concerns

• Despite improvements, results are not commensurate with past funding.
• WMNO’s Northern Sami focus is a large investment in a community with no active editors.
• The lists of activities included in WMNO’s programs are not linked by a common thread.

Staff proposal assessment narrative

This section takes an in-depth look at this organization's past performance and current plans.

Context and effectiveness

This section takes a close look at this organization's context. Here are some questions we will consider:

Environment

How does this organization's environment (including its local context and its community) enable this plan to succeed and have impact?

WMNO’s staff and operating costs will be high in Norway, and so securing funding from other sources will be necessary for WMNO to continue to operate with a budget of this size.

WMNO is continuing to cultivate opportunities for GLAM partnerships that are present in their environment, including expanding their work with libraries, which is a high potential area where they have been deepening their engagement in the current year.

WMNO is continuing to improve their approach to engaging more online contributors in their work and we hope this leads to improved engagement with these communities. However, Norwegian Wikipedias have small communities that are stagnant or in decline and work on Northern Sami will not have impact at scale. The Northern Sami Wikipedia currently does not have any active editors, as WMNO notes. Although WMNO has noted in their progress report that there may be opportunities for their GLAM work to benefit language projects beyond the Norwegian and Northern Sami Wikipedias, they have not yet been able to leverage such opportunities to offset their limited impact overall.

Past performance

Will this organization's past performance with program implementation enable this plan to succeed?

WMNO has not been producing significant results, despite receiving large amounts of funding year after year. This year, we see improvements in the area of images in use. Yet, outcomes in other areas, such as participation and articles created or improved, remain low with respect to the funding received.

The bulk of images in use (4,894 of 5,559 images in use) were contributed through a single content donation from The National Library. About half of these images in use were used on no.wikipedia, and images were also used on diverse projects such as Nynorsk Wikipedia, French Wikibooks, English Wikimedia, Swedish Wikipedia, Russian Wikipedia, German Wikipedia, and Armenian Wikipedia.

The good results from this one substantial content donation show that cultivating GLAM relationships in the long term is starting to pay off. WMNO has reported that their relationship with The National Library may also open doors to collaboration with other libraries and archives throughout Norway. This is an important qualitative achievement, even if aside from this large content donation from The National Library, achievements for images in use and articles created or improved are low.

Academia and Gender Gap programs have achieved minimal measurable impact.

In the current year, WMNO is on track to meet or exceed their own targets because they set very low targets across the board. It looks like WMNO is improving in their ability to set more realistic and ambitious targets in the current year. It seems likely that they will achieve their targets for images uploaded based on their current achievements, yet unlikely that they will achieve their target for other content types (such as articles). WMNO has set realistic but modest targets for participation that they are likely to meet, but has not aimed for major increases in this area, despite the request for more funding to increase the hours of their community manager.

Organizational effectiveness

Will this organization's overall effectiveness enable this plan to succeed?

WMNO has not yet been effective in raising a large enough portion of their budget from institutional donors or other non-Wikimedia sources. Given the costs of maintenance and growth for WMNO, this is a serious barrier to their sustainability. WMNO must increase their effectiveness in fundraising in future years, or consider reducing their budget and scope of activities.

WMNO has done a considerable amount of work to build better processes to engage its communities, and it shows. For example, WMNO received a large number of responses to the community survey they conducted relative to the size of their community, showing that contributors are willing to engage with the organization in discussions about WMNO’s priorities. WMNO’s choice of volunteer hours as a global metric demonstrates their commitment to improving and prioritizing this, and WMNO has also included several community-focused activities in this year’s plan, based on feedback from the survey. While WMNO is laying good groundwork in this area, many program activities are still heavily reliant on WMNO’s paid staff.

WMNO is continuing to improve their governance and financial systems. Hiring an external accountant was an important step toward stabilizing their finances, and they are dedicated to improving board transparency and preparing for effective board and staff transitions. We appreciate that WMNO has been continuing this important work in the current year.

Strategy and programs

This section takes a close look at this organization's programs. Here are some questions we will consider:

Strategy

Does this organization have a high-quality strategic plan in place, and are programs well-aligned with this strategic plan?

WMNO’s strategic plan is minimal, and does not include long term targets or an adequate assessment of risks and opportunities. WMNO’s strategic plan wisely emphasizes WMNO’s GLAM work, diversity, and supporting volunteers, but this strategy is not fully reflected in the structure of its annual plan. While the main program areas described in the annual plan may fit nicely within WMNO’s strategic vision, the activities grouped under each program do not create a coherent whole. Rather, this proposal reads as two separate lists of activities, mixed with daily tasks. For example, taking part in Art+Feminism, the education program, and an international contest with Wikimedia Armenia are all included in the same program, but seem like distinct approaches with distinct goals. Communications is included as part of the outreach program, although these activities are operational.

WMNO has emphasized in the current proposal that they are seeking to extend their impact beyond Norwegian-language projects, yet this is not reflected in the strategic plan, which focuses on Norwegian language projects and Northern Sami. While the strategic plan emphasizes the quality of content on the Norwegian Wikipedias, WMNO is not measuring quality as part of their work.

Programs

Do proposed programs have specific, measurable, time-bound objectives with clear targets, and are program activities and results logically linked? Are there specific programs with a high potential for impact?

With the information included in this proposal, it is not possible to link the targets listed for each program to the specific activities listed, nor is it possible to understand how the activities listed will achieve the stated goals for each program. This makes it impossible to understand if or how specific program activities and spending will lead to impact.

Given that GLAM is WMNO’s highest performing program, we would have expected this program to be more of a focus of the current proposal. It is not clear why GLAM and Academia were combined into a single program, since they do not seem to share a common theory of change. WMNO plans to continue its GLAM work in the coming year, but there is no description of the specific work that will be done in the area of GLAM, or an explanation of how WMNO will prioritize various opportunities for GLAM partnerships. WMNO has implemented its localized version of #1lib1ref with good results.

Last year and the year before, we offered feedback that the Academia program did not seem like a good way for WMNO to focus its energy because it is unlikely to generate impact at scale, and yet it appears this work is continuing in the upcoming year. With the information provided, it is difficult to understand to what extent WMNO is focusing organizational resources in this area.

We are concerned about the increasing investment in Northern Sami, which has no active community. This seemed to be a major theme of this year’s proposal, eclipsing the emphasis on gender gap work that we have appreciated in past years. While diversity is emphasized in WMNO’s grantee-defined metrics, specific activities that address this theme are scarce. For example, WMNO is organizing one contest with Wikimedia Armenia to focus on biographies of females, and is also participating in Art+Feminism. Other than this, there do not seem to be activities devoted specifically to gender gap work.

Wikimedia Norge’s plan includes some interesting ideas for supporting their communities, including microgrants, press accreditation, streaming meetups, and helping contributors gain access to more online resources. This is a nice complement to their other community-focused approaches. We hope these activities yield good results in terms of involving the editing community in WMNO’s work.

Budget

Is this plan and budget focused on the programs with the highest potential for online impact?

The bulk of WMNO’s budget is focused on staffing, including their ED and program staff. The large proportion of staff costs is due to the cost of labor in Norway, which makes the relationship between actual results and amount of funding all the more critical. WMNO has been setting low targets, and while it has managed to improve some of its results in the past year, it is still difficult to tie adequate and direct impact to the Wikimedia projects with the costs associated with maintaining and growing the organization.

WMNO is requesting a staffing increase in the upcoming year, in order to focus staff resources on community support. While this has the potential to contribute to their progress in an important area, WMNO is not projecting results that will justify this spending increase. A great portion of the budget is weighted towards the Outreach Program, which as described in the annual plan seems to be mostly operational.

WMNO projects that they will be 90% reliant on APG funding for the upcoming year’s revenue, and does not have a specific plan for raising significant external funding in the current year, which would be necessary to sustain their maintenance and growth.

We understand that the Northern Sami project may bring external funds in the longer run, but its limited potential for impact may not justify the investment in staff resources WMNO seems to be planning for it in this funding period (for example, Phase 1 of the project leads to no direct impact in the Wikimedia projects).

Summary of expert opinions (if applicable)

This section will summarize any expert opinions or other research.

N/A

Staff proposal assessment framework

This framework assesses annual plan grant proposals across the three dimensions of (1) Program design, (2) Organizational effectiveness, and (3) Budgeting. To complete the assessment, we identify whether each criterion is a strength or a concern:

• Major strength
• Strength
• Neither a strength nor a concern
• Concern
• Major concern

Criterion	Assessment	Description
Program design
P1. Strategy	Neither	Wikimedia Norge has a strategic plan in place since last year, but the structure of this proposal does not reflect this strategic approach. For example, activities are grouped into two program areas but it is difficult to link these long lists of activities to the program objectives and strategic priorities described.
P2. Potential for impact at scale	Concern	While WMNO's work may touch a variety of Wikimedia projects beyond no.wikipedia to some extent, its impact is still primarily seen on no.wikipedia, which involves a relatively small number of readers and contributors. Work on Northern Sami will also not generate impact at scale.
P3. Objectives and evaluation methods	Neither	WMNO has chosen relevant grantee-defined metrics. Volunteer hours are important to measure how well their approaches to working with the community are working. However, targets are low for the amount of funding requested and it is difficult to link the activities described in this plan with outcomes.
P4. Diversity	Neither	One of WMNO's grantee-defined metrics is diversity, which shows a will to focus on this across programs. Yet WMNO is not planning significant activities to tackle this directly. WMNO is embracing minority languages through its focus on Northern Sami, and is increasing images in use on different language projects.
Organizational effectiveness
O1. Past results	Neither	While results for participation and articles improved are low in the current year, WMNO has made significant achievements in the area of images in use generated through their partnership with The National Library. On the qualitative side, this partnership is generating more opportunities for work with libraries throughout Norway. Results from Gender Gap and Academia programs are minimal.
O2. Learning	Strength	WMNO has thoughtfully considered the advice offered by the FDC in previous rounds, and is implementing improved approaches such as attempting to strengthen ties with the community. WMNO is improving in its ability to describe and evaluate its work.
O3. Improving movement practices	Neither	WMNO collaborates with other Wikimedia affiliates in their program work, which is impressive considering their limited resources. Yet, WMNO is not making significant contributions to the global movement such as programs, approaches, or practices that are being adopted by other Wikimedia affiliates.
O4. Community engagement	Neither	WMNO is working to significantly improve its relationship with the community, and has managed to engage them more deeply in their activity, demonstrated by the very good response rate to their survey. We look forward to the results of this approach.
O5. Capacity	Neither	WMNO's investment in a skilled community manager position has brought good results so far, and increasing capacity in this area has the potential to further better the results. However, it will be difficult to achieve results that justify the increased cost of this position.
Budget
B1. Past budgeting and spending	Concern	While WMNO's results have improved in the current year, past results are still not commensurate with the funding WMNO has received.
B2. Budget is focused on impact	Major concern	Increase in budget is due to projected increase in staffing, but their projected results do not increase proportionally. The budget is weighted towards outreach, which as described is mostly operational. WMNO must secure more external funding to support its maintenance and growth.

This staff proposal assessment is the work of FDC staff and is submitted by: Delphine (WMF) (talk) 18:56, 8 May 2017 (UTC)

Staff proposal assessment framework

• Major strength. This is something the organization does very well, and this is a strong indicator of future success.
• Strength. This is something that the organization does well, and this could indicate future success.
• Neither a strength nor a concern. This is something that does not indicate future success or make funding the application a risk, or aspects of this criterion conflict.
• Concern. This is something that the organization does not do well, and this could make funding the application a risk.
• Major concern. This is an area where the organization is not strong, and this could make funding the application a serious risk.

Criterion	Description
Program design
P1. Strategy	The organization has a quality strategic plan in place, programs are aligned with this strategy, and this strategy is aligned with online impact.
P2. Potential for impact at scale	Programs could lead to significant online impact at scale, and corresponding to the amount of funds requested
P3. Evaluation methods	Programs include a plan for measuring results and ensuring learning, and employ effective evaluation tools and systems. Programs include SMART objectives, targets, and logic models.
P4. Diversity	Programs will expand the participation in and reach of the Wikimedia movement, especially in parts of the world or among groups that are not currently well-served.
Organizational effectiveness
O1. Past results	This organization has had success with similar programs or approaches in the past, and has effectively measured and documented the results of its past work.
O2. Learning	This organization is addressing risks and challenges effectively, is learning from and documenting its experiences, and is applying learning to improve its programs.
O3. Improving movement practices	This organization effectively shares learning about its work with the broader movement and beyond, and helps others in the movement achieve more impact.
O4. Community engagement	This organization effectively engages communities and volunteers in the planning and implementation of its work.
O5. Capacity	This organization has the resources and ability (for example, leadership, expertise, staff, experience managing funds) to do the plan proposed.
Budget
B1. Past budgeting and spending	This organization has a history of budgeting realistically and managing funds effectively in the past.
B2. Budget is focused on programmatic impact	Based on past performance and current plans, funds are allocated to programs and activities with corresponding potential for programmatic impact.