IRC office hours/Office hours 2013-10-30

Chat on US/Canada Education Program and the transition to the Wiki Education Foundation
Wednesday 30 October 2013
23:00 - 24:00 UTC

[20:01:13] <jmathewson> Hi, guys! Jami here (with the US and Canada program)
[20:01:14] <ldavis> we'll start by introducing ourselves
[20:01:30] <ldavis> i'm LiAnna Davis, the communications manager for the Wikipedia Education Program globally
[20:01:45] <jmathewson> we wanted to put this together to get a chance to answer some questions about the transition that is currently taking place from the WMF-led program to the new nonprofit (Wiki Education Foundation)
[20:01:49] <Crtew> Hello, I'm Chad Tew from USI in Indiana, USA
[20:02:17] <jmathewson> Pharos is here on behalf of the "WEF"
[20:02:42] <Pharos> i'm Richard (User:Pharos), one of the regional ambassador and theoretically a board member of "WEF" :)
[20:03:25] <ldavis> i can give some strategic background on why we formed this group
[20:03:26] <Pharos> also, the official webmaster for now :)
[20:03:34] <PauloSlomp> I'm Paulo Francisco Slomp from Porto Alegre, Brazil
[20:03:34] --> PJ___ (~pj@c-69-138-7-138.hsd1.dc.comcast.net) has joined #wikimedia-office
[20:04:09] <Pharos> PJ___ is also on the WEF board
[20:04:22] --> jps (d8a9891d@gateway/web/freenode/ip.216.169.137.29) has joined #wikimedia-office
[20:04:45] <Crtew> greetings PJ
[20:04:46] <ldavis> as part of the narrowing focus plan, we at the wikimedia foundation wanted to have more programs run by local organizations and supported through grant making, so we've been working to spin out some of the programs we were currently running
[20:04:46] <-> jps is now known as Guest80589
[20:04:58] <-> Guest80589 is now known as jsalsman
[20:05:24] <ldavis> with the US/Canada education program in specific, we wanted to be sure that the group represented the two core constituencies of the program: academics and Wikipedians
[20:05:27] --> MikeChristie (6c36ca57@gateway/web/freenode/ip.108.54.202.87) has joined #wikimedia-office
[20:05:38] <PauloSlomp> Federal University of Rio Grande do Sul http://www.ufrgs.br https://pt.wikipedia.org/wiki/Wikip%C3%A9dia:Wikip%C3%A9dia_na_Universidade
[20:05:57] <ldavis> so we formed a working group of both academics and wikipedians to figure out the best way to approach this
[20:06:24] <ldavis> the working group decided to form a new nonprofit organization, called the Wiki Education Foundation, to take over running the program in the US and Canada
[20:06:40] --> drdee (~diederik@mediawiki/drdee) has joined #wikimedia-office
[20:07:19] <ldavis> that group is up and running now, and there will be an official hand-off in november
[20:07:36] <Crtew> * applause
[20:07:50] <ldavis> and MikeChristie, Pharos, and jmathewson can talk more about the direction the new organization is taking
[20:08:18] <jmathewson> Sure, no problem.
[20:08:21] <ldavis> any questions about the background?
[20:08:23] <jmathewson> First, do any of you have questions at all?
[20:08:25] <jmathewson> :)
[20:08:32] <aschmidt> I would like to ask why the WEF will not become a Thorg for the time being?
[20:08:47] --> lizzard (~lizzard@172.56.17.85) has joined #wikimedia-office
[20:08:50] <Crtew> what is a thorg?
[20:08:51] <aschmidt> I already asked this on Meta
[20:08:59] <MikeChristie> There's a note in the ENB that explains this; I'll find the link and post it here.
[20:09:04] <ldavis> a thorg is a thematic organization
[20:09:04] <aschmidt> sorry, a thematic organisation
[20:09:12] <-- PauloSlomp (c8ec27a5@gateway/web/freenode/ip.200.236.39.165) has left #wikimedia-office
[20:09:13] <ldavis> it's one of the affiliate models
[20:09:14] <Crtew> oh one of those .....
[20:09:26] <aschmidt> MikeChristie: that would be fine, thx
[20:09:44] <ldavis> while MikeChristie looks that up, any other burning questions?
[20:10:13] <jsalsman> on page 23 of the new Annual Plan ( http://wikimediafoundation.org/wiki/File:2013-2014_WMF_Plan_As_Published.pdf ) it says "Education Program students now make up around 10% of the active and very active editors on the project." How does that measure up to other editor recruitment initiatives?
[20:10:23] <aschmidt> How will WEF be financed? Will there be grants?
[20:10:44] --> Dcoetzee_ (kvirc@wikimedia/Dcoetzee) has joined #wikimedia-office
[20:10:53] <ldavis> jsalsman: that question's just about the arabic wikipedia initiative, which isn't the topic for discussion here :)
[20:10:55] <Crtew> that number sounds high
[20:11:02] <jmathewson> aschmidt: We currently have a grant from the Grants Approval Committee from the Wikimedia Foundation
[20:11:11] <ldavis> the 10% number is only for the arab world program, yes
[20:11:21] <Crtew> oh
[20:11:21] <aschmidt> jmathewson: thanks
[20:11:25] <ldavis> the arabic wikipedia is much smaller than the english wikipedia
[20:11:25] <jmathewson> that will fund us to start. We are soon going to be fundraising (and hiring an Executive Director who has fundraising experience) to seek funding past the…8 month period
[20:11:31] <jmathewson> (PJ_ is it 8 months?)
[20:11:40] <Pharos> there will be some WMF, but the plan is for more outside grants in future
[20:11:56] --> PJ____ (~PJ@c-69-138-7-138.hsd1.dc.comcast.net) has joined #wikimedia-office
[20:12:06] <-- PJ___ (~pj@c-69-138-7-138.hsd1.dc.comcast.net) has quit
[20:12:15] <Crtew> who are the sources of these grants?
[20:12:22] <jmathewson> precisely. We're hoping we can seek funding outside of the "Wikimedia movement" and more specifically in the education realm
[20:12:24] <PJ____> Hey, sorry about that. I'm back.
[20:12:34] <MikeChristie> Here's the link to the announcement that we were suspending our thorg application: https://en.wikipedia.org/wiki/Wikipedia:Education_noticeboard/Archive4#Wiki_Education_Foundation_update_August_update
[20:12:45] <PJ____> The grant we received from the GAC is budgeted to fund us for 7 months.
[20:12:48] <jsalsman> I hope we have been tracking and publishing that percentage for all projects.
[20:13:00] <Crtew> thank you for looking that up MikeChristie
[20:13:14] <Crtew> What is GAC?
[20:13:29] <Pharos> Grants committee
[20:13:31] <jmathewson> (Grants Approval Committee)…way to seek funding from the WMF
[20:13:39] <Crtew> ok
[20:13:43] <jmathewson> so they approved our proposal and gave us the "seed funding"
[20:13:53] <-- qchris (~xanthalan@90.146.67.180) has quit (Quit: Verlassend)
[20:13:57] <Crtew> *applause
[20:13:58] <aschmidt> MikeChristie: thanks for the link
[20:14:02] <-> gwicke is now known as gwicke_away
[20:14:20] <ldavis> jsalsman: see more information about evaluating and tracking all our programs here: https://meta.wikimedia.org/wiki/Programs:Evaluation_portal
[20:14:26] <jsalsman> thank you!
[20:14:31] <Dcoetzee_> When you say funding in the education realm do you mean from universities themselves or from charities that give money to educational efforts?
[20:14:39] <-- tfinc (~tfinc@wikipedia/Tfinc) has quit (Remote host closed the connection)
[20:14:44] <Pharos> probably the student % is rather smaller on English than Arabic, not sure if it's been calculated
[20:14:58] <PJ____> More likely grant-making foundations.
[20:15:11] <jmathewson> I think mostly from charities
[20:15:25] <PJ____> We plan to seek other relationships with universities (e.g. fiscal sponsorship), but grants will likely come from foundations.
[20:15:34] <aschmidt> what will become of the education programme with the WMF? will it continue or will all education activities eventually be handed over to other groups?
[20:15:38] <Pharos> probably more cash from foundations, and perhaps more "in-kind" from univerities
[20:15:42] <Crtew> philanthropic organizations?
[20:16:00] <Pharos> yes, Ctrew
[20:16:04] <jmathewson> yes, and I think we will want to work more closely with faculty at universities, though not directly through funding…maybe through job roles
[20:16:18] <jmathewson> ldavis can talk about the global program
[20:16:31] <Crtew> what do you mean mathewson?
[20:16:42] <Crtew> about working with faculty?
[20:16:43] <ldavis> aschmidt: the wikipedia education program team within the wikimedia foundation is working more to support all the education program activities worldwide instead
[20:16:52] <Dcoetzee_> I assume the implication is that faculty could assist with other courses by faculty who are newer to the program than they are
[20:16:54] <jmathewson> Well, I think we'd like to work more closely with faculty members whose job descriptions (and interests) make them great candidates to advise professors and work with students
[20:17:10] <jmathewson> and what Dcoetzee_ said
[20:17:32] <Crtew> ok
[20:17:40] <aschmidt> ldavis: so there will still be activities directly run by the WMF outside North America?
[20:17:40] <ldavis> there are education program efforts underway in more than 60 countries right now; the WMF team will work to support things like brochures, the mediawiki extension, development of a mentoring structure, etc. to share learnings and resources across the different programs, rather than running programs ourselves
[20:17:49] <jmathewson> crtew: librarians, instructional designers, technologists, etc. We've had a lot of success with the few who have been Ambassadors so far, and I think we want to explore that more!
[20:18:06] <ldavis> we are still running the program in the arab world, but we are hoping to move that to a more volunteer-driven initiative as well
[20:18:32] <ldavis> but the arab world program will be the only one we are directly running after the official WMF->WEF transition of the US and Canada program
[20:19:07] <Dcoetzee_> Do we have any data on number of volunteers over time? Is that remaining sustainable?
[20:19:43] <aschmidt> ldavis: thx
[20:19:46] <MikeChristie> Dcoetzee: do you mean in en.wp or globally? Do you mean ambassador counts?
[20:20:19] <Dcoetzee_> Yeah ambassador counts. I'm not sure if I'm asking "are they keeping up with demand" or "are they dying out", maybe both
[20:20:28] <Dcoetzee_> I mean mainly in en.wp
[20:20:32] <jmathewson> Dcoetzee_: Here's what I see happening:
[20:20:49] <Crtew> Do we have data on how many professors are continuing / repeating their experiences?
[20:20:51] <jmathewson> We have about the same number of Ambassadors that we've had (active, at least), but we've moved to where we're able to support more classes with those Ambassadors
[20:21:01] <Dcoetzee_> I see, efficiency
[20:21:17] <jmathewson> Though the WEF has talked at length about the importance of reinvigorating our volunteer community
[20:21:27] <jmathewson> both on-wiki and at universities
[20:21:46] <-> kaity|away is now known as kaity
[20:21:55] <Pharos> Crtew: I don't have statistics, but lots of profs I've worked with have been repeaters
[20:22:07] <Dcoetzee_> All the profs I've worked with are repeaters
[20:22:08] <PJ____> We have also had discussions about options for more systematic data collection.
[20:22:18] <jmathewson> crew I do have some retention numbers, pulling it up now
[20:22:33] <jmathewson> crtew*
[20:22:37] <Crtew> lol
[20:22:46] <jmathewson> :)
[20:23:27] <Dcoetzee_> If I can make a suggestion about recruitment: as far as I know there hasn't been a lot of individual reach-out to Wikipedians in a known location
[20:23:32] <jmathewson> so we looked at it through a few different lenses. In a survey, we asked profs if they "would" return if it fits in the class, and 78% said yes
[20:23:41] <-- rfarrand (~rfarrand@108-235-115-48.lightspeed.sntcca.sbcglobal.net) has quit (Quit: Computer has gone to sleep.)
[20:23:42] <jmathewson> (it gets tricky because a lot of profs only teach in the spring or fall)
[20:23:48] <Crtew> * applause
[20:25:27] <jmathewson> Each semester, approx. 30% classes are taught by return professors.
[20:25:27] <Crtew> yes, i understand
[20:25:27] <Pharos> Dcoetzee_: I'm a provincialist myself!
[20:25:27] <jsalsman> DarTar: do you have a tabulation of the editor recruitment efforts by editors recruited per program dollar spent?
[20:25:27] <jmathewson> of our best professors (defined as the ones who added highest quality from spring 2012 research), 52% have returned
[20:25:27] <jmathewson> those are the numbers we have from that
[20:25:27] <-- MelBee (~MelBee@wikimedia/MBrown-WMF) has quit (Ping timeout: 245 seconds)
[20:25:27] <Crtew> I've been having a hard time developing/recruiting a campus ambassador
[20:25:27] <PJ____> One of the discussions we've started to have is about the role that regional ambassadors currently play in the program and how that could be improved. Do you all have some ideas about that?
[20:25:37] <Pharos> of course, I guess prvincialism works better in high density areas
[20:25:47] <Dcoetzee_> My only comment on regional ambassadors is that they need to be drawn from volunteers with strong management abilities if they're going to fulfill that role
[20:26:24] <Dcoetzee_> People skills aren't universal among Wikipedians :-)
[20:26:33] <Crtew> lol
[20:26:36] <PJ____> haha good point. We have talked about making them subject-based rather than location-based since much of the interaction happens online.
[20:26:51] <jmathewson> Crtew, I've found that a lot of experienced professors don't really *need* an Ambassador, though of course you could train someone to really support your students. I wonder if the library is a good place to look for that, since you don't particularly need support in the traditional way. Your students might need help with research/references/writing, though.
[20:26:57] <jsalsman> I hope we are documenting ambassador best practices and have some kind of award for role model ambassadors
[20:27:23] <jmathewson> Dcoetzee_: I agree. That's traditionally been a main criterion…though it doesn't always work out long-term
[20:27:23] <PJ____> So you'd have a regional ambassador working with ambassadors in history classes, for example. Another for biology, etc.
[20:27:48] <Crtew> thx jmathewson
[20:28:03] <Dcoetzee_> I think training TAs as CAs and hooking them up with OAs is a good model, because they're already committed to support the class
[20:28:11] <Dcoetzee_> To some extent we've already done this
[20:28:44] <PJ____> I know the WMF has been putting together ambassador training materials and practices for awhile, but I'll defer to Jami and Lianna on that.
[20:29:02] <Dcoetzee_> Another unexplored option is to involve students in the classes who have Wikipedia experience
[20:29:13] <Dcoetzee_> Maybe for extra credit or whatever
[20:29:27] <PJ____> @Dcoetzee - that is an option in situations where the class has a TA. Not always the case, esp. at smaller universities
[20:29:29] <MikeChristie> jsalsman: I'm not sure what we can point to as ambassador best practices; Jami might have something to say. One difficulty is that their work is not always out on noticeboards; it's on talk pages and is not necessarily monitored by others.
[20:29:55] <jmathewson> dcoetzee, I wonder what you think about the reactive/proactive issue that i've seen with a lot of ambassadors. I think we need to move toward a system where we have various roles that ambassadors can fill (based on their interests/skills). i don't think students ask as many questions (or think they need help) as we want them to
[20:30:05] <Pharos> Dcoetzee_: not unexplored!
[20:30:17] <Dcoetzee_> Pharos: Oh? I did not know :-)
[20:30:21] <PJ____> haha, we'd recruit them to be CA's, I think!
[20:30:29] <jmathewson> so maybe we could have some 'best practices' like that, outside of the training, that actually provide various helpful tasks for professors, students, and assignments in general
[20:30:29] <ragesoss> We've tried to make the trainings we've been developing into the main collection of best practices, for Ambassadors, instructors and students.
[20:30:45] <Pharos> I recruit everybody :)
[20:30:46] <ragesoss> https://en.wikipedia.org/wiki/Wikipedia:Training
[20:31:14] <jmathewson> pharos is very inclusive
[20:31:18] <jmathewson> which i adore
[20:31:27] <Dcoetzee_> jmathewson: I believe the best way to encourage proactive ambassadorship is through syllabi that involve checkpoints with explicit ambassador duties attached to them
[20:31:48] <Dcoetzee_> e.g. your first draft is due Aug 13, and by Aug 15 your ambassador will leave you feedback on the talk page
[20:31:53] <Crtew> that's a good idea
[20:32:00] <ragesoss> In terms of Ambassador best practices, the training doesn't go into a lot of detail beyond framing how to interact with professors and students, and laying out the same best practices for assignments that it presents to professors.
[20:32:24] <jmathewson> Dcoetzee_ yes, that could definitely be the route to take. the WEF also talked about just having "Wikipedia Ambassadors", some who may help with formatting, some who may advise on reliable sources, some who may monitor for plagiarism
[20:32:46] <jmathewson> Dcoetzee_: I think that deadline style will be *really tough* with volunteers :(
[20:32:46] <MikeChristie> Dcoetzee_: what do you do when an ambassador goes awol during the class? It has certainly happened, though I don't know how often, and I don't think I'd feel comfortable telling a professor to rely on a volunteer for their course structure.
[20:32:53] <ragesoss> Because, as most of you know, the day-to-day activities of ambassadors and how they work with professors and/or students can vary a lot, depending on the needs of the class, and their own communication styles and expertise.
[20:33:04] <aschmidt> what is your experience with copyright violations? I remember the problems in India...
[20:33:09] <-- Maryana (~Maryana@wikipedia/Accedie) has quit (Quit: Maryana)
[20:33:20] <jmathewson> MikeChristie: this is why we should move toward a crowdsourced Wikipedia Ambassador model, maybe?
[20:33:29] <MikeChristie> aschmidt: there have been a lot of discussions -- any particular question?
[20:33:50] <Dcoetzee_> MikeChristie, jmathewson: I agree that it can be difficult with volunteers to maintain deadlines, but if they're AWOL we could have a pool of backup people to step in
[20:34:05] <MikeChristie> jmathewson: that would help but I doubt if that would make it possible to build ambassadors into course structures.
[20:34:06] <ragesoss> aschmidt: It's also been an issue with US and Canada classes. Not nearly to the extent as the India pilot, but significant enough not to ignore.
[20:34:07] <aschmidt> MikeChristie: just an overview from your side, please. any problems the really made it hard for the wp community?
[20:34:07] <jmathewson> Dcoetzee_: give me your pool of volunteers and i will give you a kitten banner
[20:34:28] <PJ____> I'd like to get thoughts from professors and students re: crowdsourcing ambassadors. I wonder how important the direct connection to one person is to them.
[20:34:52] <ragesoss> aschmidt: we recently did a study to try to get a better understanding of plagiarism among student editors, and how that compares with other editor groups.
[20:34:56] <MikeChristie> A couple of points -- one is that it seems to be very variable, both by class and topic. The editors working on medical and psychology topics report a lot of problems for example.
[20:35:17] <jmathewson> PJ____: that'd be a really good idea
[20:35:36] <aschmidt> ragesoss: could you link to that study, please?
[20:35:41] <Dcoetzee_> I think part of the separation is that CA/OA roles by their nature require commitment, while Wikipedia editing in general really doesn't. The program currently does not tap the people who would like to help but would not like to commit to anything.
[20:35:52] <ragesoss> aschmidt: just fetching the link... https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Education_Program/Research/Plagiarism
[20:35:53] <MikeChristie> Another is that despite a lot of effort by educators to teach the students about plagiarism and copyvio, there are always a few who think that citing a source makes it OK to quote that source.
[20:36:26] <MikeChristie> On the other side of the coin, I've heard feedback that working on Wikipedia is a good way to teach the students about plagiarism and copyvio.
[20:36:32] <aschmidt> ragesoss: thx
[20:36:35] <jmathewson> Dcoetzee_: exactly, which is why i also think we could have much "smaller" roles, though no less important. it could work for skill sets, too, like you mentioned earlier
[20:36:44] <Pharos> Dcoetzee_: maybe we should make a template for {{needsitinerantambassaor}}
[20:36:45] <-- J-Mo (~jtmorgan@wikipedia/Jtmorgan) has quit (Quit: Leaving.)
[20:36:50] <Dcoetzee_> Yes, I agree that's a great idea - they might not even be formal roles
[20:36:58] <Dcoetzee_> Just, leaving notes on talk pages "can you look at this please"
[20:37:25] <Dcoetzee_> A lot of people will volunteer for an article review if someone is in need.
[20:37:33] <aschmidt> MikeChristie: that's what I think, too. it trains students to use source material and to quote correctly
[20:38:06] <jmathewson> Dcoetzee_: ragesoss and I talked extensively about ways to utilize the notifications. people could even just sign up to get a specific type of student notification
[20:38:25] <ragesoss> aschmidt: as a followup (after seeing exactly the kinds of plagiarism that are common, for hours and hours per day for several weeks while working on that study) I also made this video to help curb the problem better: https://en.wikipedia.org/wiki/Wikipedia:Training/For_students/Copyright_and_plagiarism
[20:38:25] <Dcoetzee_> That could be neat, although signing up to receive them in itself represents a commitment :-)
[20:39:13] <ragesoss> We have (finally) a developer working on the course pages extension again. And he's keen to start integrating some Notifications.
[20:39:20] <Dcoetzee_> I think the best way to deal with copyvio and plagiarism is, again, to have ambassadors who catch the problem early
[20:39:42] <-- drdee (~diederik@mediawiki/drdee) has quit (Quit: drdee)
[20:39:47] <ragesoss> Instructors and ambassadors also should soon have the ability to add articles to students, and students to classes.
[20:40:00] <Dcoetzee_> They need to review early drafts and be aware that many students are plagiarism ignorant
[20:40:01] <aschmidt> Dcoetzee_: good point, I think
[20:40:19] <Crtew> The video above was very helpful for my students!
[20:40:34] <Dcoetzee_> And we need to have good copyvio bots running over all student pages, all the time
[20:40:42] <ragesoss> Crtew: :-)
[20:41:01] <jmathewson> Dcoetzee_: ignorant is a good word for it. Because they've heard about it and know they shouldn't do it, but they genuinely don't know what only plagiarism is
[20:41:07] <ragesoss> Dcoetzee_: you mean all pages, all the time.
[20:41:09] <jmathewson> by "they" I mean "some students"
[20:41:26] <Dcoetzee_> ragesoss: There are scalability limitations due to query limits and other issues
[20:41:49] <Dcoetzee_> That is the ideal, but the number of student articles is much much smaller and easier to handle
[20:41:54] <ragesoss> true.
[20:42:09] --> tfinc (~tfinc@wikipedia/Tfinc) has joined #wikimedia-office
[20:42:35] <ldavis> so we have about 15 more minutes… anyone have any additional questions?
[20:42:36] <Dcoetzee_> If a copyvio bot were running during the IEP it would've been detected very early before it blew up. I think it's a critical component.
[20:42:43] <ragesoss> although, if the results of the plagiarism research hold, general new editors plagiarize at a higher rate than student editors.
[20:42:52] <Dcoetzee_> I believe that as well :-)
[20:43:11] <ragesoss> (student editors in the US/Canada, at least)
[20:43:22] <-- tfinc (~tfinc@wikipedia/Tfinc) has quit (Remote host closed the connection)
[20:43:22] <Dcoetzee_> On the other hand students are easier to hold accountable for plagiarism and stop them
[20:43:23] <MikeChristie> I think course design is likely to be a predictor of plagiarism too.
[20:43:29] <aschmidt> are there already copyvio bots, or are you planning to introduce them?
[20:43:36] <PJ____> To throw another question out there. What are your thoughts on appropriate metrics for the program? Basically the quantity vs. quality debate.
[20:43:46] <Crtew> How are we all going to make WEF more visible to get grants and to keep up the growth?
[20:43:58] <Dcoetzee_> There were copyvio bots. Changes to search engines killed them. But they can be revived on a smaller scale.
[20:44:09] <MikeChristie> If a course asks a large group of students to add one or two sentences, each to a different article, you will see a lot of plagiarism, I suspect.
[20:44:26] <-- lizzard (~lizzard@172.56.17.85) has quit (Quit: lizzard)
[20:44:39] <PJ____> @Crtew: A few things. We will be hiring an executive director who will spend much of his/her time focusing on securing resources (monetary or otherwise) to sustain WEF.
[20:44:46] <jsalsman> Ironholds: can you ask your friends at Google to let the GEP run a copyvio bot?
[20:45:11] <MikeChristie> Dcoetzee_: would it be worth approaching Google and asking if a copyvio bot could be written as a donation to the Wikipedia movement? Has anything like that been discussed?
[20:45:15] <PJ____> We will also leverage relationships of Board members to secure additional funds.
[20:45:17] <-- DarTar (~DarTar@wikimedia/DarTar) has quit (Quit: DarTar)
[20:45:22] --> drdee (~diederik@mediawiki/drdee) has joined #wikimedia-office
[20:45:37] <Ironholds> jsalsman, GEP? and, I have friends at google?
[20:45:46] <Ironholds> I mean, some people from the search team came by the office. that's about it.
[20:45:47] <Dcoetzee_> MikeChristie: That hasn't been discussed (or at least not acted upon) but I think it's a promising option.
[20:46:07] <jsalsman> Ironholds, didn't you offer to put people in touch with them if they could think of ways they can help the projects?
[20:46:09] <ragesoss> Ironholds, the GEP he means is Global Education Program
[20:46:25] <Crtew> I still think we have a long way to go in changing the mindset on campuses about Wikipedia being a negative
[20:46:27] <ragesoss> (which is not the name, but close)
[20:46:39] <Dcoetzee_> PJ____: Re quality vs. quantity: I think, realistically, any course that makes the Wikipedia assignment worth real points and grades on quality is going to turn out reasonably good quality.
[20:46:55] <jsalsman> if copyvio bots aren't running because of something the search engines did, then maybe they can get an exemption for Labs IP addresses or such
[20:47:08] <Ironholds> jsalsman: oh, gotcha. yeah, that might work, but I don't know why the bots aren't running, so...
[20:47:16] <Dcoetzee_> The bots were stopped due to query limits
[20:47:20] <Dcoetzee_> A few years ago
[20:47:20] <Ironholds> aha
[20:47:35] <Ironholds> possibly something that can be discussed; I dunno. Who maintains em?
[20:47:38] <Dcoetzee_> Also TOS violations as they were scraping I think
[20:47:41] <Ironholds> or, maintained
[20:47:51] <Dcoetzee_> Let me check
[20:48:21] <Dcoetzee_> Ah my info is out of date
[20:48:26] <Dcoetzee_> We have at least one operational bot
[20:48:33] <ragesoss> (On the bot topic there's https://en.wikipedia.org/wiki/User:MadmanBot )
[20:48:35] <Dcoetzee_> https://en.wikipedia.org/wiki/User:MadmanBot
[20:48:38] <PJ____> Dcoetzee_: True. Emphasizes the importance of course design. I don't think all professors really know how to grade WP contributions since they could be edited by others.
[20:48:51] <PJ____> But I know we have put together information on that.
[20:49:13] <jmathewson> PJ____: your metrics question was a good one. I think we should lean toward the program evaluation and design team's metrics
[20:49:27] <jmathewson> (which i think they're still developing metrics, since they're new)
[20:49:37] <-> kaity is now known as kaity|away
[20:49:42] <PJ____> Are they posted somewhere?
[20:49:43] <jmathewson> not that we can't have more metrics, but it would make sense to work within the wikimedia community
[20:49:47] <Dcoetzee_> I don't know exactly how CorenSearchBot's code works, but I imagine it could be modified to target pages in a specific category
[20:49:53] <jmathewson> I think ldavis linked somewhere above…it's frank's team
[20:50:01] <PJ____> kk I'll look
[20:50:12] <ldavis> https://meta.wikimedia.org/wiki/Programs:Evaluation_portal
[20:50:21] <ldavis> they're focusing now on what can be measured
[20:50:25] <jmathewson> they are currently doing some research based on feedback from a bunch of global programs and activities
[20:50:29] <ldavis> which is essentially anything we can get out of wikimetrics
[20:50:41] <ldavis> which is only quantitative and not qualitative
[20:51:01] <ragesoss> Those of interested in course design, FYI, building in some better and more elegant assignment design guidance into the course pages extension is something that our developer and I have started brainstorming.
[20:51:11] <PJ____> Ldavis: And I think that's a major issue.
[20:51:15] <jmathewson> that's awesome ragesoss
[20:51:25] <ldavis> yes but one that doesn't have an easy answer, PJ____
[20:51:29] <ragesoss> If you have an interest in contributing to that brainstorming and design process, please contact me.
[20:51:34] <ldavis> qualitative evaluation takes a LOT of volunteer time
[20:51:38] <PJ____> Oh agreed. Hence my question :)
[20:52:04] <PJ____> I've raised that precise issue on the ENB when we've been criticized for emphasizing quantity contributed too heavily.
[20:52:04] <Dcoetzee_> I think peer grading with a good rubric is an adequate substitute for volunteer evaluation
[20:52:04] <ldavis> so yeah, they're starting with the easy ones
[20:52:05] <jmathewson> so my answer is: well, quantitative metrics are about as realistic as we're going to get for now
[20:52:38] <Dcoetzee_> Oh ignore my comment, I misunderstood
[20:52:39] <jsalsman> I have one request, that ambassadors consider http://toolserver.org/~dispenser/cgi-bin/categorder.py/Category:Wikipedia_articles_in_need_of_updating when suggesting assignments
[20:52:55] <MikeChristie> I don't think it's enough to have purely quantitative metrics though; we need at least sampling for quality.
[20:53:11] <Dcoetzee_> Random sampling is a good idea I think
[20:53:19] <Dcoetzee_> We already highlight the superstar articles but they're not representative
[20:53:41] <jsalsman> ...when it is working... (sorry...)
[20:53:41] <Crtew> There are ways to deploy qualitative methods that are not so intrusive and can limit volunteer time
[20:53:42] <jmathewson> we have 2 quality studies
[20:53:46] <jmathewson> both have had similar results
[20:53:53] <jmathewson> (and both took a lot of volunteer time)
[20:54:14] <jmathewson> with limited resources, i think you have to stop repeating the same thing and getting the same results. hopefully we can pick that back up down the line
[20:54:22] <jmathewson> but i don't think it's the right time when we're starting a new organization
[20:54:40] <aschmidt> do i get you right that editor engagement is not your primary concern, but rather updating WP articles and good content?
[20:54:53] <ldavis> yes, exactly, aschmidt
[20:54:56] <Crtew> do we have access to those two studies?
[20:55:02] <aschmidt> ldavis: i think that's a good thing
[20:55:05] <MikeChristie> In addition to quality and quantity, people have asked what burden students place on regular editors. https://en.wikipedia.org/wiki/Wikipedia:Ambassadors/Research/Spring_2012_burden_analysis is a page that tries to answer that question for a sample.
[20:55:07] <PJ____> By editor engagement do you mean editor retention?
[20:55:10] <jmathewson> ragesoss is grabbing them
[20:55:32] <aschmidt> PJ____: yes
[20:55:43] <Dcoetzee_> I think there are a lot of other interesting questions to ask about student articles other than quality, like e.g. how they develop after the student departs, their long-term impact, problems that were identified and fixed later on
[20:55:52] <Dcoetzee_> Kind of the afterstory
[20:55:55] <ragesoss> Here's the second quality study: https://en.wikipedia.org/wiki/Wikipedia:Ambassadors/Research/Article_quality/Results
[20:56:02] <PJ____> That has been the WMF's approach.
[20:56:14] <ragesoss> And the first: https://outreach.wikimedia.org/wiki/Student_Contributions_to_Wikipedia
[20:56:20] <jmathewson> Dcoetzee_: I often look at page views of new articles from students. it's amazing how many people find those articles, especially when students rarely integrate them well into the rest of WP
[20:56:43] <PJ____> +1 dcoetzee
[20:56:56] <MikeChristie> Dcoetzee_: yes; and I also think that there's a role for professors in "curating" a topic area -- if they get students to work in the same area, semester after semester, then the articles should improve each semester.
[20:57:01] <Dcoetzee_> jmathewson: It makes sense to me: if students are writing about something, it's relevant to their course's material. If it's relevant to their course it's relevant to similar courses around the world.
[20:57:31] <jmathewson> Dcoetzee_: exactly. that's why I think it's great. the education program adds a lot of diversity (in editors and topics)
[20:57:42] <jmathewson> which tends to mean branching out into new territories on WP, if you will
[20:57:45] <-> rmoen is now known as rmoen|away
[20:57:48] <Dcoetzee_> Yep fighting systemic bias
[20:58:19] <Dcoetzee_> The downside is, it can be hard to maintain and fact-check articles in topic areas where all the original knowledgable authors have gone MIA
[20:58:23] <Crtew> Agree on improvement each semester, and we professors learn how to tweak it each time too
[20:58:29] <aschmidt> where will you report on the developments of the programme and metrics?
[20:58:36] <jmathewson> we have just a few more minutes. does anyone else have any other questions? this has been really interesting and a great place to start!
[20:58:52] <ldavis> aschmidt: you can see our theory of change here: https://outreach.wikimedia.org/wiki/User:Ldavis_(WMF)/Starting_an_education_program#Slide_4:_Thinking_beyond_editor_retention (ignore the rest of the content; that's just a training i'm developing for other countries who are looking to start education programs)
[20:59:27] <jmathewson> I know we have a lot of questions for editors, professors, and ambassadors, so maybe we can host another office hour in the near future (with more advance notice!)
[21:00:00] <Dcoetzee_> Just one quick question: when will you be doing recruiting for the upcoming term?
[21:00:26] <PJ____> I would also encourage you all to participate in discussions on the ENB. It's important to demonstrate that there are WP-ians that support the program and want to see it continue to improve and grow.
[21:00:38] <jmathewson> I wouldn't say "recruiting" since we aren't planning to expand beyond the folks who are already planning their assignment. I've already spoken with many professors who have been working on their assignments since last spring!
[21:00:47] <jmathewson> so any time basically
[21:00:48] <aschmidt> ldavis: thx, I'm looking forward to following events, interested in co-operating in the future for Germany
[21:01:04] <Dcoetzee_> I meant ambassadors actually
[21:01:10] <jmathewson> ah!
[21:01:38] <jmathewson> that is definitely an on-going process, too. I don't think I have a lot of time in the near future to actively search for new ambassadors, but it would be awesome if some other ambassadors did so
[21:01:39] <Dcoetzee_> PJ____: I agree re ENB representation, the more transparency the better
[21:01:54] <Dcoetzee_> jmathewson: Okay got it :-)
[21:01:56] <ldavis> ok, we're over time now so we'll wrap up
[21:02:02] <ldavis> but these were some great questions
[21:02:10] <ldavis> sounds like we should have another office hour in the near future
[21:02:13] <jmathewson> and the professors usually recruit their on-campus support these days, but i'd definitely like to get a resurge in wikipedians into the mix!
[21:02:17] <Crtew> *applause
[21:02:24] <jmathewson> yes, definitely
[21:02:24] <jmathewson> thank you to everyone who joined!
[21:02:27] <aschmidt> thank you very much! I's been a pleasure!
[21:02:34] <Crtew> thank you!
[21:02:38] <jmathewson> now to go watch some baseball :)
[21:02:41] <ldavis> thanks for the great questions, all!
[21:02:44] <Dcoetzee_> Enjoy :-)
[21:02:45] <Crtew> woot woot