Learning and Evaluation/Evaluation reports/Learning Days Wikimania 2019 Evaluation
Learning Days at Wikimania 2019 was the first Learning Days workshop to be led by the new Community Development team at the Wikimedia Foundation. This iteration of the workshop was also the first to separate participants into three tracks or cohorts based on their prior organizing experience, in order to better tailor the workshop to the needs and capacities of attendees.
Key Findings for Consideration
edit- Most participants (92%) found the Learning Days environment to be safe, respectful, and comfortable. Some reported problems with temperature and noise.
- Most participants (83%) liked the Track structure, especially the Track 3 cohort (94%).
- More than 90% said they gained knowledge (91%) that they intend to apply (93%) in their communities.
- Track 1 and 2 cohorts reported increased confidence in movement knowledge and organizing skills across the board.
- Many expressed a desire to learn more practical organizing skill-sets.
Data Overview
editAttendees were surveyed prior to Learning Days about confidence in movement knowledge and various organizing skills, their organizing experience, and expectations of the workshop. They were also surveyed in the week following the event to measure their responses to the workshop environment, confidence in organizing skills following the workshop, and the relevance and applicability of the workshop content. Response rates to the post-workshop survey were lower than for the pre-event survey, but still at or above 60% for each cohort.
| Cohort 1 | Cohort 2 | Cohort 3 | |
|---|---|---|---|
| Total Attendees | 15 | 27 | 21 |
| Completed Pre-event Survey (%) | 12 (80%) | 25 (93%) | 16 (76%) |
| Completed Post-event Survey (%) | 9 (60%) | 20 (74%) | 16 (76%) |
The Learning Days Environment
edit
The vast majority of attendees reported that Learning Days provided a friendly and supportive environment, with most indicating that the Friendly Space Policy was both followed (92%) and enforced (92%), that feedback was constructive (92%), and that they felt comfortable participating (92%). Cohort 1 participants almost universally found the environment to be positive, while the small number of participants who found the training environment unpleasant were all participants in Cohorts 2 and 3.
Attendees were slightly less likely to report being able to communicate about organizing in a manner comfortable to them (85 to 87 percent). They reported the most difficulty with the physical environment of the training, with only 80 percent reporting that they were able to fully see, hear, and participate. Cohort 3 in particular reported that the room was too cold, small, and loud during group discussions. One Cohort 2 participant suggested a social event or field trip "break" from the classroom-like setting.
Most attendees reacted positively to the new "Track" cohort structure, with a majority mostly or completely satisfied. Cohort 3 participants were much more likely to be mostly or completely satisfied (94 percent) with the Track structure than other participants.
Cohort 1: What Did Participants Gain?
editThe workshop content for Cohort 1 was designed for participants newer to organizing within the Wikimedia movement. All respondents (100%) reported having learned new or different ideas that they intended to apply to their home communities. Some added that they specifically planned to build and strengthen partnerships (n=2) or plan or improve events (n=2). Most (78%) indicated that they had learned practical skills and knowledge relevant to their Wikimedia movement work.
The majority (more than 75%) found the content of the curriculum to be interesting, practical, and relevant. Most also reported making new connections with other organizers at the workshop.
On average, the Track 1 Cohort increased their movement knowledge across the board, with the biggest knowledge gains in what the Foundation does and what key Wikimedia projects are. The smallest gains were seen in knowing "what the free knowledge movement is."
[The Tracks] allowed for a more intimate learning environment.
Similar gains were also seen in Cohort 1's confidence in their basic organizing skills, though especially in motivating and supporting other volunteers, recognizing those volunteers' unique needs, and event planning.
I thought the facilitators and the other participants did a great job.
Generally, Cohort 1 was positive about what they had learned. They responded especially well to gaining and developing concrete skills (such as action planning, human-centered design, building partnerships, and collaboration and communication skills).
In fact, when expressing dissatisfaction they asked for the opportunity to build even more concrete skills, including grant writing, operating an affiliate, and event organizing.
The amount of content related to practical skills (such as "how to write a grant application" or "how to organize an event") is insufficient.
Cohort 2: What Did Participants Gain?
edit
The workshop for the Track 2 cohort was designed for organizers with intermediate levels of experience. Similar to the first cohort, most Track 2 participants reported having learned new or different ideas that they intended to apply to their home communities. In particular, several indicated that they intended to apply their knowledge to organizing events (n=5) and sharing what they had learned with other organizers (n=5). They also planned to use these skills to improve presentations (n=3), build partnerships (n=3), and develop action plans (n=2). The majority found the content of the curriculum to be interesting (91%), practical (82%), and relevant to their activities in the movement (95%). The Track 2 cohort also gained confidence in every movement knowledge indicator, especially how movement grants and partnerships work and what the Wikimedia Foundation does. Smaller gains were seen for other indicators, particularly knowledge of the Wikimedia ecosystem of contributors, affiliates, organizers, and regional networks, where average pre-event knowledge was already high (3.33 out of 4).
This cohort's confidence in basic organizing skills increased on every indicator as well, with the biggest confidence gains in designing a new program or initiative and motivating and supporting other volunteers.
The same increases were seen for more advanced organizing skills, particularly mentoring other organizers, public speaking, and program evaluation. Smaller yet notable gains were also seen in building partnerships, event facilitation, long-term strategizing, and managing group conflict.
I like the way facilitators freely connected and engaged participants.
Prior to Learning Days, 44% of Track 2 respondents could identify an element of organizing they felt comfortable mentoring other organizers in. Following the workshop, 75% could do so. This cohort specifically cited event planning and execution (n=6), teaching, facilitation, and public speaking (n=6), and managing or building partnerships (n=3).
I think this year's structure was properly tailored for the skill sets of participants, it was like everyone in our track (were) actually speaking [about] the same things but with different contexts.
Cohort 2 was very positive about the structure and facilitation of the workshop, and found the modules on evaluation (n=5), public speaking (n=4), and building partnerships (n=4) especially valuable, though some indicated that the module on building partnerships was least valuable to them (n=2) due to their previous experience. Two people mentioned that they were hoping for more information on grant writing and fundraising.
Learning Days could offer more substantive courses for experienced organizers, such as technical know-how on current Foundation policy and others.
Cohort 3: What Did Participants Gain?
edit
The curriculum for the Track 3 cohort was designed for more experienced organizers to reflect on their own movement experience and organizer expertise to workshop ways to support, engage, and include others in the Wikimedia movement.
Track 3 participants showed little to no change in movement knowledge or organizing skills, which is to be expected given the Track's curricular focus on reflection on leadership and movement growth rather than concrete organizing skills.
I liked the fact that every module worked as a path to a better understanding of my history in the community, and of how I can correct certain behaviors and "think outside my box" when helping out my community.
This cohort was most likely to report making connections with other organizers (94%). They also largely reported that the curriculum was appropriate (88%), interesting (88%), and relevant to their activities (94%). Many (75%) reported gaining new or different ideas about future approaches to organizing. One key element of the Track 3 curriculum--engaging with community growth frameworks--was said to be a productive activity (87%), but Track 3 was somewhat less sure about applicability to their own communities (60%).
I'm planning to share this knowledge with my [organization] and amend some of our policies to fit in with the tools I've learned as an organizer.
Six Track 3 participants indicated that they found the second day to be more valuable to their development than the first, and two suggested shortening the training to one day in the future to accommodate the busy schedules of experienced organizers. This group also appreciated the modules on Movement Leadership (n=3), Using Research To Plan (n=3), and Engaging and Building Your Community (n=2).
Critical comments often centered on a desire to learn more practical skills, primarily in leadership training (n=4), conflict management (n=3), grant writing and fundraising (n=2), financial management (n=2), and evaluation (n=2).
Some of the discussions [were] very vague. I don't see any practical yield or retrospection of what we have been doing or learning something new, out of these discussions. The direction of the discussions should [be] very specific.
Overall, the Track 3 cohort found the collaborative and reflective nature of the curriculum productive for their development as organizers, and found the content interesting and relevant to them. They did express a desire for more grounded discussions and an opportunity to learn or develop more practical skillsets.
Presentation Slides for This Report
edit