Grants:APG/Feedback and continuous improvement of the FDC process

Summary of this Section

Feedback and continuous improvement are a critical part of the FDC process. Because the FDC is an experiment with little precedent within Wikimedia or in other organizations, it will be critical to continually assess how the process is working and make adjustments and improvements as needed
There are four main mechanisms for collecting feedback and continuous improvement:

1. The ongoing feedback process collected by the Ombudsperson

2. The FDC recommendation complaints process

3. The annual assessment of the FDC process, which will include:

A survey for involved parties after each funding round
Feedback gathered in the annual editor survey
A summary report from the Ombudsperson from all feedback collected over the course of the cycle
An assessment of the time and financial costs of the FDC
An objective assessment of the effectiveness and outcomes of funds allocated in the cycle
A self-assessment of the FDC's performance by FDC members

4. An annual FDC report summarizing learnings and reflections on past grants

For the first two years, the FDC will be in a trial period with special oversight and evaluation in addition to the regular process outlined above. The objective will be to assess whether the FDC is successful and whether/how it should continue in the future

Ongoing feedback to the Ombudsperson

The primary role of the Ombudsperson is to gather input and feedback about the FDC process to ensure it is effective and efficient in disseminating funds in pursuit of the mission goals of the Wikimedia movement. Individuals with input, feedback, or complaints about the FDC process should add comments to FDC Ombudsperson's comment page by creating a new discussion header and describing their input, feedback, or complaints..

If the matter requires immediate attention, the Ombudsperson will work with the person or entity submitting the feedback as well as other people or entities that may be involved to address it. If the input is more general, the Ombudsperson will gather this input and develop a report for each funding cycle cataloging feedback submitted by the community about the FDC process and identifying key pain points or opportunities for improvement that this feedback highlights. This report would be shared publicly on-wiki and be used by the FDC and the Board as an input to assess the effectiveness and efficiency of the FDC process.

Note: If an entity has a complaint about an FDC allocation recommendation, they should not submit this complaint to the Ombudsperson: they should submit it directly to the two Board representatives on the FDC within seven days of the publication of the FDC recommendation in question. The Board may solicit the help of the Ombudsperson to investigate the issue if further information is required. For more information about this process, see the FDC recommendation complaints process below.

FDC recommendation complaints process

Entities who submit proposals to the FDC may submit any formal complaint about an FDC allocation recommendation in a summary of up to 500 words to the two non-voting WMF Board representatives on the FDC, who will in turn present the complaint to the WMF Board at the same time it considers the FDC recommendation.

Formal complaints can be submitted only by the Board Chair of a funding-seeking entity;
Formal complaints must be filed within seven days of the submission of the FDC slate of recommendations to the WMF Board (November 22 for Round 1; May 8 for Round 2);
Any planned or approved disbursements to the organization filing a complaint will be put on hold until the complaint is resolved;
If the WMF Board's consideration of the complaint results in an amendment of the FDC's recommendations (which is expected only in extraordinary circumstances), the WMF Board may choose to release extra funds from the WMF reserves to provide additional funds not allocated by the FDC's initial recommendation.
Other members of the WMF Board may participate in the investigation if approved by the Chair of the WMF Board.

To submit a complaint, regarding the FDC recommendations to the board, please discuss them with the board representatives on the FDC.

Annual FDC assessment process

The hypothesis underpinning the FDC is that a global body of committed, experienced Wikimedians, supported by a well-designed process and dedicated staff, can make decisions about where the Wikimedia movement should spend its money that will:

Create better alignment between the Wikimedia movement's strategy and its spending;
Support greater impact and greater progress towards achieving shared goals than would otherwise be the case; and
Enable the movement to get smarter faster about how to achieve shared goals than otherwise would have happened.

A secondary hypothesis is that the FDC process will be seen by the global Wikimedia community (including but not limited to fund-seeking entities) as more fair, transparent and accountable than the funds dissemination practices that predated it. Creating the FDC process will be expensive: it will cost a significant amount of money, staff time and volunteer time, and it will require Board of Trustees involvement and oversight.

Given these objectives and considerations, the purpose of the FDC assessment process will be to determine the extent to which the FDC process is creating the expected value for the Wikimedia movement (as defined in the first paragraph of this section), and to assess its costs to the movement, so that the Board of Trustees can decide whether its value warrants its costs.

How the annual assessment process will work

To ensure the FDC process as efficient and effective as possible (as defined by the objectives above), the FDC staff will be responsible for facilitating a regular process of collecting feedback and assessing the FDC process, to be published on-wiki. Each involved party will be responsible for improving its own practices in response to this feedback.

FDC assessment dimensions

To understand whether the FDC process is achieving its established objectives, it will be evaluated on the following dimensions:

Ability to fulfill its basic purpose, as set out in the Board's March 2012 resolution, to make recommendations to the Board of Trustees for funding activities and initiatives in support of the mission goals of the Wikimedia movement;
Adherence to the Board's principles for funds distribution (as laid out in the Board's January 2012 resolution: to protect core operations, assess impact, promote transparency and stability, support decentralized programs, promote responsibility and accountability, and make decisions openly and collaboratively;
Effectiveness in enabling increased alignment between the Wikimedia strategy and its spending;
Effectiveness in enabling greater impact and progress towards achieving the movement's shared goals than might otherwise have been the case;
Effectiveness in enabling the Wikimedia movement to get smarter faster about how to achieve the movement's shared goals than might otherwise have been the case;
Whether the global Wikimedia community perceives the process to be more fair, transparent and accountable than the funds dissemination practices that predated it;
Whether the costs of the process, as measured in financial costs, volunteer time, staff time and Board of Trustees time, are merited by the effectiveness measures outlined above.

Inputs to the assessment process

To evaluate these dimensions, the following inputs will be collected as part of the regular assessment process for the FDC:

The FDC staff will administer a survey of involved parties after each funding cycle to gather:

Perceptions of the FDC process's effectiveness based on the evaluation dimensions outlined above;
Perception of the costs (money and time) of the FDC process; and
Input on potential process improvements (e.g., what should continue, what should be stopped, what should be started).

Link to FDC stakeholder assessment survey

The FDC staff will administer a cost-benefit survey to primary applicants after each funding cycle to gather:

The costs (money and time) of the FDC process; and
Suggestions for improving efficiency of the FDC process

Link to FDC stakeholder cost-benefit survey

The annual editor survey will include questions to assess editor perception of the FDC process, including how well it has created increased alignment between spending and strategy, whether it has enabled more impact and better progress towards Wikimedia global goals, and whether it is perceived by respondents to be fair, transparent and accountable.

Link to annual editor survey (to be included)

The Ombudsperson will develop a report after each funding cycle cataloging all feedback submitted by the community about the FDC process including official complaints from fund-seeking entities disputing the FDC recommendations.

Link to current Ombudsperson report

The WMF finance staff will issue an on-wiki statement of FDC financial costs to understand what it takes to conduct the overall process.

Link to FDC financials, Year 1

Based on the impact reports submitted by funds recipients, the FDC staff will develop an annual objective summary of past funds given and the outcomes of those funds.
Based on all of these inputs, the FDC Chair will coordinate the FDC members in developing an annual on-wiki statement assessing the FDC's performance in fulfilling its overall purpose, adhering to the Board principles, increasing alignment between spending and strategy, and enabling more impact and better progress towards the global Wikimedia goals. Link to comprehensive list of prior years' summative assessment, which will contain a link to this year's assessment (to be included)

Initial two-year trial period

For the first two years, the FDC will be in a trial period with special oversight and evaluation in addition to the regular process outlined above.

The FDC Advisory Group will play a special role in the development and assessment of the FDC during this period:

Until September 2012, the Advisory Group will continue to advise the Executive Director of the Wikimedia Foundation in the formation and on-boarding of the first FDC;
Once the inaugural FDC is up and running, the Advisory Group’s formal role will be suspended. However, its members will continue to closely observe the process, and to participate in it as community members as they see fit.
The Advisory Group will return in a formal capacity at two points before disbanding in September of 2014:

1. After the first round of funds dissemination is completed and has been assessed (roughly, in mid-March 2013), the Advisory Group will use the assessment and their own observations to advise the WMF Executive Director on how to improve the process. Process improvements may include adjustments to:

Timelines;
Eligibility requirements for grantees;
Membership, eligibility requirements, and workload for the FDC;
Proposal process, including the role of FDC staff; and
Decision-making processes.

2. After 1.5 years (three rounds) of funds dissemination has been completed and assessed (roughly, in mid-March 2014), the Advisory Group will meet again to advise the WMF Executive Director on improvements to the process based on updated assessment and learnings.

The WMF Board of Trustees will also play a special evaluation role during this period:

At the end of the FDC's first year of operation, in roughly mid-August 2013, the Executive Director will create a report for the Board of Trustees documenting the state of the FDC process and informing the Board about changes that will be made to it as a result of the assessments to date, and the counsel of the FDC Advisory Group.
At the end of the two-year trial period, in roughly mid-August 2014, the Executive Director will make a recommendation to the Board of Trustees, rooted in the assessments done to date as well as the counsel from the FDC Advisory Group, recommending one of the following outcomes:

To continue the FDC process as is;
To continue the FDC process with modifications;
To eliminate the FDC process and replace it with a different process designed to achieve some of the same goals (alignment of spending with strategy, greater impact, better learnings, perceived as high quality) but in a more effective fashion; or
To eliminate the FDC process and request the Executive Director to assume responsibility for funds dissemination for the Wikimedia movement.

Annual FDC report summarizing learnings and reflections on past grants

Because of their role as promoters and enablers of high impact activities, the FDC will become a center for thought leadership about what works to support the mission goals of the Wikimedia movement. To share this wisdom, the FDC will publish an annual "report" providing reflections and learnings from the grantmaking work they do. This report will draw from annual funds reports that each FDC funds-recipient will complete. This may include:

Promising practices and lessons learned in movement entities' programs and activities
Trends in the progress, activities, and challenges of movement entities themselves
Observations about where the movement is over- or under-investing to achieve the mission goals of the Wikimedia movement
Recommendations for movement strategic priorities and goals

Not only will this annual report help movement entities enhance and refine their own plans to support the mission goals of the Wikimedia movement, but it will also be a critical input into the movement's broader strategic planning process.

In addition to the annual report, FDC members will participate in the broader discussion about how to best achieve the mission goals of the Wikimedia movement in other ways as well, seeking to disseminate lessons widely through any means available (e.g., presenting at Wikimania, chapter meetings, and other movement conferences and events, maintaining a blog or other medium for sharing reflections electronically).