Learning and Evaluation/Archive/Participation Support Program revamp

Fall 2013 Revamp Research

Situation

The Travel & Participation Support Program (formerly "PSP") is going through some self-reflection! We (WMF Grantmaking) haven't done a serious reflection on the successes and challenges of the programs, nor have we verified that this program is meeting the needs of our broader community. PSP was launched in 2011 to fill a void in the WMF Grants Program - what was then the only form of grants distribution. This particular program in the WMF Grantmaking portfolio has not gone through careful reconsideration since the launch of the different grantmaking streams in late 2012. We have never taken systematic feedback of the participants.

Now that the other grants streams are more settled in their scopes and target audiences, we should take the time to think about what needs the Travel & Participation support grants are meeting, and how we could optimally define its scope for the future!

TPSP grantmakers will be redesigning the program beginning in October 2013 and plan to use the outcomes of this research to help guide choices.

Key questions

• Who has the TPSP served in the past (Wikipedians? Wikimedians? Women? Men?)?
• What needs are being met through TPSP?
• Is the process (all stages: submission -> approval -> reporting) and grantee experience working well? If not, what can be improved? Could we simplify anything without losing value?
• Should the scope of the program stay specifically tied to Wikimedians to attend non-Wikimedia events? What other needs are out there that this program should consider meeting?
• How do people become aware of this program? How can we raise awareness?
• Are participants in TPSP increasing contributions on Wikimedia projects following their funded event(s)?

Scope of work

1. Light demographic study of past recipients

• Using the Grants Tracker, put together some snapshots of the TPSP since its inception. Among other things, this should include:
  1. # of PSP grants by fiscal year
  2. Acceptance rates by fiscal year
  3. # of unique PSP recipients (total)
  4. range of amount of funding distributed (and average & median)
  5. Distribution of PSP recipients by location
  6. Male:Female ratio of recipients

• Things to be looking for:
  • Are the same people always applying for these grants?
  • Where are the grant requestors coming from? Are they from countries without chapters doing microgrant programs (see #4 task below!)
  • Is the demand for these grants growing over time? Are they becoming more competitive?

2. Survey past PSP participants

The main crux of the research will be the feedback we receive from the past participants of the TPSP!

Target Audience: Any user who has edited a page in the Participation namespace on meta! This would include all applicants, all recipients, all committee members, and any interested community members who have endorsed/commented.
Purpose:

1. Evaluate the effectiveness of the TPSP process (Submission -> Approval -> Report)
2. Identify areas for improvement in program and page designs
3. Identify successes and challenges of TPSP
4. Find out how people became aware of the program
5. Get crazy ideas for what TPSP could look like!

Sample questions

1. How have you participated (checkboxes so we can disambiguate perspectives from committee, endorsers, applicants, grantees, etc)
2. How did you learn about this program?
3. Were the criteria for selection clear?
4. Were all reporting requirements clear?
5. Would you apply for funds from the Travel & Participation Support Program again in the future? Would you recommend it to others?
6. How satisfied are you with your experience with the Travel & Participation Support Program?
7. Suggestions for improvement... (suggest reusing some of the IEG survey questions where applicable...)
8. ... etc.

• The full questions of the survey.

3. Study of contribs history of past participants

Using the list of usernames of past TPSP participants, run some brief analysis on any fluxes in user activity around the timing of the TPSP. Particularly, look at:

1. Avg monthly contribs BEFORE applying to TPSP
2. Avg monthly contribs AFTER applying to TPSP
3. Avg monthly contribs AFTER the event funded by TPSP
4. Avg. monthly contribs AFTER the final TPSP report is due

4. Brief overview of other microgrants programs

At a minimum, we'll want to look at which countries are providing microgrants for non-Wikimedia events. This way, we can overlay where we are not receiving grant requests to potential countries that are having this need serviced elsewhere.

Could also be interesting to do some lightweight benchmarking on process for different microgrants programs: are there some that are working really well? Why?

• One way that we could collect this rather than full interviews is inviting them to participate on the proposal pages on meta.