Brazil Program/Education program/Learning/Brazil Pilot 1/Meeting goals

We had various Factors outlined for success of the program.

Overall notes

The two categories of goals we did not meet were:

1. Impact on Portuguese Wikipedia, and
2. The road ahead

For some of that, we recognize that we made intentional concessions during the course of the semester which resulted in lower editing counts. We decided to prioritize Goals 3 & 4: positive community reception and constructive classroom experiences. We succeeded in this!

That said, we can look at the categories we did less positively on and as we ponder why, decide (a) whether the goal itself was wrong, or (b) reconsider how we should approach the goal next semester.

Impact on Portuguese Wikipedia

Given the history of the Brazilian WP-PT communities, we wanted to take time to socialize the program and let people feel like they were being heard and trying things for themselves. As a result, we didn't push quite so hard or try to professionalize or over-hire: we wanted to develop ownership and buy-in, so that it wasn't just the "big-bad WMF" coming in to control. We have found in past work with Brazil that this is essential. Also, given the WP-PT community, we wanted to make sure that they have time to get acquainted with the program since any rejection of it on their part would result in extremely frustrated students and faculty. All these things are an ongoing process, and we will need to continue to work on them going forward. This past semester, though, we saw great success in these areas, as there has been positive reception from all Wikipedian/Wikimedian participants.

That said, it is clear that there is more room for being bold in pushing for editing on Wikipedia, and in order to make a bigger impact on the Portuguese Wikipedia, we need to provide more support in making this editing happen. Some thing we can do to help with this next semester:

• More carefully working with professors in incorporating editing milestones into lesson plans
• Requiring editing of Wikipedia in our supported courses
• Increasing number of ambassadors involved
• Providing more training to ambassadors to help them be more hands-on in helping students and incorporating content into the main namespace
• (Potential) Hiring more support (e.g., fellowships, additional BR-WMF contractors)

The Road Ahead

It is really difficult to prioritize documentation when there is a lot of physical work that needs to get done. That said, clear documentation helps the team in San Francisco track with the team in Brazil and it also helps the community to understand on which projects the staff is spending time and resources. Some suggested ways to help support this:

• Write all reports in Portuguese
• Solicit volunteers to write blog posts throughout the semester (one from a student, one from a professor, one from an ambassador) to take the load off of the project coordinator

Goal achievement: 1st semester 2012

Key

Fully met the original goal	Partially met goal / met modified goal	Missed goal (reevaluate!)	Not enough information to assess

Table

Areas of impact	Goal Set, 1-2012	Actual result, 1-2012	Goal achievement
1. Impact on Portuguese Wikipedia	At least 75% of the students participating in the pilot add 500 words or more to the Wikipedia article namespace	Exact numbers TBD, but will not meet this goal. This is partially due to the class structure (lots of group editing) and the lack of direct editing to the main namespace
	At least 75% of the content contributed by students "survives" (i.e., is not deleted from the Wikipedia namespace).	100% of the articles moved from sandboxes to the mainspace survived, but 67% of the articles with contributions (20 articles out of 30) were moved to the mainspace. All courses were oriented to start working on sandboxes and then move to the mainspace, but some professors have decided to work offline and then at the end of the course to start moving to Wikipedia.
	At least 50 students in the Education Program part of the pilot start editing	TBD. This number, though may be hard to measure based on students contributions: we had a lot of students who worked on group projects and therefore did not edit directly (rather, contributed offline)	TBD
	At least 10 people being recruited through the general university outreach part of the pilot start editing	Chose not to pursue alternate outreach given resource constraints.
2. Structures for success	The Brazil Pilot starts on-time (March 2012).	Started!
	Training sessions are conducted by volunteers for the professors, campus ambassadors, and online ambassadors	Held training sessions, but they were poor. Did not host a comprehensive one for ambassadors, which was needed.
	Surveys are created and distributed to professors and students at the beginning and end of the semester	Created and distributed
	Feedback from training session is gathered via a post-training survey	Created and distributed
	Clear documentation is created showing different options for assignment (e.g., editing existing articles, creating new articles, translating articles)	Created ultimately by the broader education team, and localized to Portuguese; it was not available for the professors at the beginning of this semester.
	Clear documentation is created showing different options for Ambassador models, and pros/cons	Not documented: this is something we should do to help ambassadors understand where to plug in better.
	We will have numbers that clearly show which model for in-classroom activities is more successful (in terms of both amount and quality contributed)	TBD - we know that we have to have professors that are engaged in milestone projects, and we also saw that a dedicated professor makes the biggest difference. We are still waiting for final numbers, but they will not be best situated to give us the full story of what went on behind the scenes, given the number of students who edited offline.
3. Community impact and reaction	Members of the Portuguese Wikipedia community are supportive of the pilot and volunteer to participate in supporting it, both online and offline.	A few community members engaged in the discussion and strategy, but many more engaged in the training materials and support pages on the PT.WP.
	We start general outreach activities on campus in the first semester in 2012	We de-prioritized this (see above) given resource constraints, but we did see one event occur on campus by CAs, and also we spoke at other campuses to spread the world
	We fill all volunteer positions; 50% of the volunteers are Wikipedians	Yes!More than 50% of the ambassadors that ended up participating in the project were already Wikipedians
	More than 75% of the community volunteers involved in the pilot indicate that it was worth their time	Didn't ask this question directly, but ~67% had "Good" or "Very good" experiences, and 100% want to help next semester
	Members of the Portuguese Wikipedia indicate support for continuing to build an education program in Brazi	All those volunteers involved with the project agree it was and is useful, though many indicate needs for improvement if it will be successful long-term.
4. Professor and student reaction	We fill all slots for professors in 5 classes	We worked with 5 professors in 6 courses this semester, though a few did not complete Wikipedia assignments
	Two (2) prominent universities participate in the program	We worked with three of the top universities in the country: USP, UNESP, and UFRJ
	100% increase in the level of comfort in editing	After orientation, all the professors indicated increases in their comfort on Wikipedia. For students, they moved from being mostly uncomfortable editing to planning to edit more in the future.
	75% of the participants indicate happiness with the outcome.	Yes! 100% of students were "Very Likely," "Likely" or "Somewhat Likely" to recommend the program to a friend, and 93% of students "Agree" or "Strongly Agree" that the program was a beneficial experience.
	More than 50% of the students indicate that they would prefer a Wikipedia assignment type course to a traditional course in the future.	50% of students "Agree" or "Strongly Agree" that they would prefer a Wikipedia assignment toa term paper assignment.
	More than 50% of the teachers indicate that they would use Wikipedia as a teaching tool again	100% of the teachers surveyed indicated they would want to participate in the program again. Two, though, indicated they would want a less restrictive version of the program
	More than 50% of the teachers indicate that they would recommend using Wikipedia as a teaching tool to their colleagues/peers	100% of the professors surveyed indicated they would recommend this to friends or colleagues
5. The road ahead	Bi-weekly updates on the program are shared with the PT:WP community; we publish at least 3 interim reviews with stories of success and lessons learned on the WMF blog	We were able to publish 3 blog posts. We were not, though, as diligent in publishing comprehensive reports on where we were throughout the program., and we got some feedback about not being quite transparent enough in the activities.
	An outcome / lessons learnt / the road ahead document will be completed by July 2012.	Late, but being worked on it!
	We will identify which in-classroom activities are most successful (in terms of both amount and quality contributed)	Should be incorporated into the above report