User:JAnstee (WMF)/Sandbox/Draft/Reports/CEE 2015

CEE L&E Sessions Evaluation

The Learning and Evaluation team hosted a series of workshops at the CEE Meeting, September 10 - 13, in Tartu (Estonia) [1]. Find the following workshops and join us for these new learning experiences:

• Wikimedia Program Toolkits: Overview workshop (and practical session) for using program toolkits for education, writing contests, and photo events and how to help them grow.
• Tools to Measure & Track (Tools Rotations): A roundtables session with three break-out group choices to learn about a handful of tools available for measuring work across the wikis. Tools include: GLAMorous, CatScan, Education Extension, Wikimetrics and Quarry.
• Fail Fest – Learning Together: A very brief overview about learning from failure including a useful strategy for framing failure stories.
• Storytelling to Advance the Wikimedia Movement: A workshop with brief presentation followed by applied group learning activity for telling a story. The workshop will focus on the key types and aspects of storytelling as well as provide a framework for developing stories so that we might learn together from participants experiences.

Pilot of Workshop Toolkits

Two of these sessions involved coordination with community presenters and our pilot of two new train-the-trainer style Workshop Kits on Storytelling and Fail Fest

These new Workshop Kits aimed at helping community leaders lead peer and volunteer training. In each kit you will find all the steps you need to take to host a Learning and Evaluation workshop, as well as infrastructure needed, and all the resources to present. All kits have a learning journal, for workshop hosts to be able to document their experience and share with other community members across the movement. We hope to expand this collection soon, with other workshop kits forthcoming. If you are interested in hosting one of these workshops, get in touch with the Learning and Evaluation team. Train the Trainer sessions will be hosted at regional and global conferences and via Google hangouts. We look forward to working with community leaders in workshop kit updates, facilitation and translation as we work to build specific examples and case studies from across the movement.

Evaluation of the L&E Sessions

85% indicated at exit that the workshop objectives had been met and core concepts were understood to a moderate (35%- 44%) or "high" (42-50%) extent.

Session by Session

Program Toolkits

• The majority of participants understood the importance of connecting with other movement leaders to share best practices for programs
• While most had at least a moderate understanding of the toolkits purpose there was some concerns about the language and multiple requests for translation. For those who explored the Education toolkit there was much too dense information in the pages they found.
• Most did not understand very well how to use and/or contribute to the toolkits - For those who explored the Education toolkit there was much too dense information in the pages they found. Others shared they had no personal connection to use cases for the toolkit to be relevant. In addition, many desired for better instruction in how to add to the toolkits and a guide or learning pattern on just that.

• Based on debrief with a number of participants and the teaching team it is clear there was too much focus on learning from the participants than on sharing the resource and ways to interact with it which would have been a better fit.

• Still, when asked for show of hands which participants would likely use the toolkits in the near future, 15 raised hands

Session Stats:

• 48 participants
• 34 surveys (71%)
• 19 shared a contact

Expand data table

Program Toolkits Session Concepts	Understanding at Exit	Understanding at Exit
	% Moderate or High	% High
	Target 80%+	Target 50%+
I understand the importance of connecting with other movement leaders to share best practices for programs	83%	60%
I understand how Wikimedia program toolkits can help with planning, running, and evaluating programs.	83%	47%
I know how to use and/or contribute to the toolkits.	68%	32%

Tools Rotation

• 11 shared their contact for staying tuned about category metrics and 1 indicated they would look at the metrics for interest
• 7 shared their contact for staying tuned and 2 participants indicated they would begin using Wikimetrics in the next 30-45 days
• 4 participants indicated they would begin using the Extension in the next 30-45 days

Session Stats:

• 24 participants
• 10 surveys (42%)
• 18 shared a contact to stay tuned.

Expand data table

Tools Rotation Tools	Understanding at Exit	Understanding at Exit
	% Moderate or High	% High
	Target 80%+	Target 50%+
Category Tools (GLAMorous & CatScan)	88%	38%
User Metrics (Wikimetrics & Quarry)	91%	45%
Group pages (Education Extention)	90%	50%

Fail Fest

• The Failure Always Invites Learning acronym and concept was well understood by most participants.
• Nearly all respondents understood the value of sharing failure stories to prevent repeating mistakes, however, some left without a clear understanding of learning patterns as an easy way to share learning.
• In the end, six learning pattern stubs were produced in hard copy at the session and nearly all hands were raised that they had an enjoyable time sharing about failures during the session.

Session Stats:

• 18 participants
• 1 Community Facilitator
• 10 surveys (55%)
• 6 Learning pattern stubs

Expand data table

Fail Fest Session Concepts	Understanding at Exit	Understanding at Exit
	% Moderate or High	% High
	Target 80%+	Target 50%+
Failure always invites learning	100%	78%
Sharing failure stories helps others to not repeat the same mistakes	89%	89%
Learning Patterns are an easy way to share learning about problems and their solutions	78%	33%

Storytelling

• Most participants understood quite well that stories are important to advancing Wikimedia work.*
• While some struggled with understanding the importance of identifying different channels for effective storytelling the majority understood well how they could use the ABCD model and template to craft better stories.
• The storytelling workshop was mentioned by multiple people in the closing session and two additional program leaders indicated in next steps call outs that they would share the ABCD's with their colleagues back home.

Session Stats:

• 40 participants
• 1 Community Facilitator
• 13 surveys (33%)

Expand data table

Storytelling Concepts	Understanding at Exit	Understanding at Exit
	% Moderate or High	% High
	Target 80%+	Target 50%+
Stories are important to advancing Wikimedia work.	100%	69%
It is important to identify different channels for effective storytelling.	100%	33%
It is useful to craft a Wikimedia story using an ABCD model and template.	100%	55%

At exit, 54% felt "mostly" or "very" prepared to implement what they had learned in the workshop in their work