IRC office hours/Office hours 2012-07-07

[13:58:02] <Maryana> hi stevenw. everyone's tearing to go, it seems
[13:58:13] <Maryana> do your voodoo IRC magic
[13:58:13] *** ChanServ sets mode: +o StevenW
[13:58:16] <StevenW> Sweet
[13:58:37] *** ChanServ sets mode: +v Maryana
[13:58:49] <StevenW> Okay
[13:59:08] <StevenW> So who's around for the office hours? :)
[13:59:20] * StevenW is maybe going to spam the other channels
[13:59:39] <Maryana> also, hands up if you're pumped for wikimania :)
[13:59:57] * StevenW raises hand
[14:00:13] <Maryana> ^
[14:00:32] <WereSpielChqrs> Here for office hour not doing Wikimania this year
[14:01:07] <StevenW> Nice. Thanks for joining WereSpielChqrs
[14:01:14] *** Joins: thekaryn (~kgladston@50-0-133-213.dsl.static.sonic.net)
[14:01:21] <Maryana> hey, it's thekaryn
[14:01:22] <Mh7kJ> Here for the office hour, but I'm here all the time anyway :P
[14:01:25] <Maryana> greetings from mississippi
[14:01:28] *** ChanServ sets mode: +v thekaryn
[14:01:34] *** Joins: linnea (linnea@wikipedia/linnea)
[14:01:48] <thekaryn> Hello Maryana. Greetings from SF
[14:02:08] *** Joins: Platonides (~Platonide@wikipedia/Platonides)
[14:02:32] *** Joins: Dragonfly6-7 (~test@bas1-montreal48-1176431841.dsl.bell.ca)
[14:02:47] *** Joins: gmaxwell (~greg@wikimedia/KatWalsh/x-0001)
[14:02:50] <StevenW> So is everyone here for the office hours aware of who we are/what we do? Or should we maybe introduce what we do?
[14:03:41] *** StevenW changes topic to 'IRC office hours with the Editor Engagement Experiments team -- https://en.wikipedia.org/wiki/Wikipedia:Editor_engagement_experiments'
[14:03:53] <Maryana> i take it that's a "who are you and what's happening here?"
[14:03:58] <StevenW> Yeah
[14:04:36] *** Joins: ToAruShiroiNeko (~admin@wikimedia/ToAruShiroiNeko)
[14:05:02] <StevenW> Ok I am going to take that as a no, we don't need to make an introduction
[14:05:04] <StevenW> :)
[14:05:30] <linnea> could you do brief one minute introduction?
[14:05:34] <StevenW> Sure
[14:06:35] <StevenW> So the Foundation is currently working on some very big projects to try and make Wikipedia editing a better experience for newbies and experienced users.
[14:06:56] *** Joins: apergos (~ariel@wiktionary/ArielGlenn)
[14:07:01] <StevenW> Stuff like a WYSIWYG editor (aka the VisualEditor) and other projects
[14:07:25] <StevenW> But those projects take a long time to build and deploy to the site, because they're so huge
[14:07:45] <StevenW> What we do, as the experiments team, is take the opposite approach
[14:08:05] <StevenW> We're interested in testing many ideas in a more rapid fashion, say a new experiment every two weeks
[14:08:25] <Maryana> yeah, so stevenw, thekaryn, and i are all part of this team. we're sort of the scrappy throw-stuff-at-the-wall unit
[14:08:31] *** Joins: Sven_Manguard (~~~@wikipedia/Sven-Manguard)
[14:08:39] <Sven_Manguard> Office hours?
[14:08:47] <Maryana> indeed
[14:08:49] <StevenW> Yeah
[14:08:52] <Sven_Manguard> Oh good.
[14:08:58] <Maryana> just doing brief intros
[14:08:59] <StevenW> We were just introducing the team etc. Sven
[14:09:01] <StevenW> :)
[14:09:04] <Sven_Manguard> Sorry.
[14:09:06] <StevenW> No worries
[14:09:30] <StevenW> Does that give you better idea, linnea et al?
[14:09:52] <Maryana> so, we're fairly new, but we've already gone through a couple iterations as a team. if anybody's interested in that, please speak up :)
[14:10:06] <Sven_Manguard> you inviting people to join the team?
[14:10:07] *** Joins: worm_that_turned (~worm_that@wikipedia/Worm-That-Turned)
[14:10:14] <Sven_Manguard> because I have lots of interesting ideas
[14:10:20] <Sven_Manguard> some are even safe/legal
[14:10:24] <StevenW> heh
[14:10:25] *** Quits: Jyothis (~Jyothis@wikipedia/Jyothis) (Ping timeout: 244 seconds)
[14:10:28] <Maryana> ha
[14:10:45] <StevenW> Actually we do have a couple open jobs on the team, for a senior engineer and a backend engineer.
[14:10:48] *** Joins: Emw (~emw@wikipedia/Emw)
[14:11:14] <Maryana> as for ideas, you're always welcome to throw them our way
[14:11:19] <linnea> StevenW: yeah
[14:11:24] <StevenW> Great.
[14:12:12] <Maryana> oh, also, this happened before we officially started this team, but a related project stevenw and i worked on was template A/B testing - some of you might have seen our RfC to change the level one user warnings
[14:12:56] <Maryana> thogo is aware of it, i believe :)
[14:13:01] <StevenW> In addition to that, we just wrapped up our first experiment with a new feature.
[14:13:07] <Thogo> indeed
[14:13:59] <Thogo> which new feature?
[14:14:04] <StevenW> https://meta.wikimedia.org/wiki/Research:Timestamp_position_modification
[14:14:13] <Thogo> ah
[14:14:20] <StevenW> Looks like https://commons.wikimedia.org/wiki/File:Screen_Shot_2012-06-21_of_LastModified_experimental_feature.png
[14:14:42] <Maryana> the tl;dr is, it's a more prominent, human-readable timestamp
[14:15:25] <Maryana> it's not exactly an earth-shattering breakthrough, but it was a good first test of our testing infrastructure and development cycle
[14:15:50] <StevenW> This was built during a hackathon a while back, so it allowed us to test our our experimental methods, like how we can bucket people into test and control groups, but still maintain an appropriate level of anonymity for them, etc.
[14:16:04] <linnea> Maryana: is that going to be activated in wikipedias soon?
[14:16:26] <ToAruShiroiNeko> can we have a list of these features on meta?
[14:16:35] <StevenW> https://meta.wikimedia.org/wiki/Editor_engagement_experiments
[14:16:37] <Maryana> linnea: so, we ran the test a couple weeks ago
[14:16:38] <ToAruShiroiNeko> and when the switches would be turned on and off?
[14:16:40] <ToAruShiroiNeko> ah nice
[14:17:02] <Maryana> just for a short time, and only on a sample of articles
[14:17:06] <StevenW> Yeah, and we announce on the technical Village Pump for whenever there is a new experiment being turned on
[14:17:50] <thekaryn> linnea - we're likely to iterate on the timestamp experiment to see if we can make it even more effective before we make any recommendation about whether it should become part of the encyclopedias
[14:18:24] <StevenW> Yeah, what thekaryn said.
[14:19:07] <Maryana> yep. one idea was using it in a more explicit call to action for articles that are really out of date
[14:19:14] *** Joins: cndiv (~textual@adsl-71-135-43-244.dsl.pltn13.pacbell.net)
[14:19:41] <StevenW> Due to the experimental nature of most of our work, it's unlikely that we're going to enable anything we try project-wide immediately after testing. Unless it's a wildly unexpected success. ;-)
[14:20:39] <Sven_Manguard> and even then
[14:20:40] <linnea> can these be implemented as gadgets?
[14:20:46] *** Joins: glavkos (~glavkos@wikimedia/glavkos)
[14:20:46] <Sven_Manguard> you need to tell us first StevenW
[14:20:48] <WereSpielChqrs> I like the 1k barnstar test, you might want to look at the history of 100k awards, its probably been running for a year or so, but again no control sample
[14:20:49] <StevenW> Right
[14:20:54] <Sven_Manguard> because when you all turn stuff on without telling us
[14:21:03] <Sven_Manguard> pitchforks and torches every time
[14:21:06] <Dragonfly6-7> 1K/
[14:21:08] <Dragonfly6-7> ?
[14:21:08] <StevenW> Indeed.
[14:21:10] <Sven_Manguard> even the good stuff
[14:21:22] <StevenW> Maryana: do you want to introduce the 1k editors experiment?
[14:21:36] <Emw> i'd be interested to see how 'History tab CTR' results in http://meta.wikimedia.org/wiki/Research:Timestamp_position_modification#Results change over time for articles with the new timestamp feature.
[14:21:37] <Maryana> sure!
[14:21:54] <Dragonfly6-7> sorry, what do you mean by 1K and 100K awards?
[14:22:08] <Maryana> so, for a few months now we've been thanking every editor who completes his/her 1,000th edit to articles on english wikipedia
[14:22:08] <Thogo> I actually wonder how increased clicking the history tab helps the article...
[14:22:25] <worm_that_turned> is that article edits or general edits?
[14:22:31] <StevenW> article
[14:22:46] <StevenW> https://blog.wikimedia.org/2012/04/09/an-experiment-in-finding-and-thanking-wikipedians/
[14:23:09] <Maryana> last month, we did an A/B test - thanked half of the folks who reached the milestone, and didn't do anything for the other half
[14:23:38] *** Quits: Jan-Bart (~Jan-Bart@216.131.120.122) (Quit: Jan-Bart)
[14:23:41] <StevenW> Thogo: Yeah, one of the reasons we'd want to test further before implementing is that while increased use of history is good for teaching readers about how editing works, it doesn't directly benefit the article.
[14:23:46] <WereSpielChqrs> The 100k program is similar, I think a little older, but probably less than a 100 editors have completed their 100,000 edit while its been running
[14:24:00] <Emw> Thogo: from the link: "Our hypothesis is that this will increase user knowledge of how Wikipedia works, increase reader trust in articles, and encourage updating of articles which have not been edited for months or years by highlighting their outdated state."
[14:24:12] <Dragonfly6-7> wait
[14:24:13] <Sven_Manguard> Maryana: what was the result with the A/B test?
[14:24:21] <Dragonfly6-7> does that take into account edits made to articles that have been deleted?
[14:24:25] <Thogo> hm ok
[14:24:25] <StevenW> Yes
[14:24:31] <Maryana> sven_manguard: we're looking into the data now. so far it looks like in the 0-3 day period afterwards, there doesn't appear to be an effect. but i think it'll be interesting to see if there are any middle- or long-term bumps in editing.
[14:24:34] *** Joins: Jan-Bart (~Jan-Bart@216.131.120.122)
[14:24:52] <Maryana> but we have to wait to let everybody reach an equal maturity post-1k
[14:24:52] <Dragonfly6-7> so someone could hypothetically get the 1K barnstar while having no edits to articles readable by non-admins?
[14:25:04] <Sven_Manguard> Maryana: if people need to be thanked for their work in order to continue to edit, they ain't gonna last
[14:25:18] <Maryana> dragonfly6-7: highly unlikely. they'd most likely be banned if they'd made 1,000 deleted edits
[14:25:23] <Sven_Manguard> English Wikipedia, at least, is openly hostile to new users and abrasive to the rest
[14:25:43] <Dragonfly6-7> Sven_Manguard - don't know about that
[14:25:51] <StevenW> Hypothetically, but since we thanked people by hand and checked their contribs beforehand, we would not thank people doing that. ;)
[14:26:14] <worm_that_turned> Not quite true Maryana. Editors who work in articles for creation can rack up pretty high deleted edits.
[14:26:26] <Maryana> dragonfly6-7: regardless, i only thanked users who weren't banned/in arbitration/using AWB or alt accounts, etc. and those only made up about 1-3% of the sample (of hundreds)
[14:26:44] <Thogo> but they will most likely also have edits in non-deleted articles, worm_that_turned
[14:26:46] <ToAruShiroiNeko> StevenW can I propose something?
[14:26:50] <StevenW> Sure
[14:26:51] <Maryana> worm_that_turned: that's a bit of an edge case. i haven't seen it so far.
[14:26:51] <worm_that_turned> very true
[14:27:20] <ToAruShiroiNeko> (MediaWikiwiki:Lua scripting MediaWikiwiki:Extension:Scribunto
[14:27:24] <ToAruShiroiNeko> -(
[14:27:30] <ToAruShiroiNeko> these two are of interest to me
[14:27:42] <Maryana> sven_manguard: so why do you edit wikipedia?
[14:27:44] <ToAruShiroiNeko> I wonder if these can be experimented on at some point
[14:27:44] <Maryana> :D
[14:27:57] <Sven_Manguard> Maryana: addiction
[14:27:58] <StevenW> We haven't been involved with those projects personally. But I can make an IRC office hours about those happen ToAruShiroiNeko
[14:28:10] <Sven_Manguard> Maryana: real answer: I don't know, really.
[14:28:13] <Maryana> oh! are you going to be at wikimania, sven_manguard?
[14:28:17] <Sven_Manguard> Yes.
[14:28:23] <Maryana> stevenw and i are giving a talk on editor motivations
[14:28:25] <Maryana> you should come
[14:28:30] <Dragonfly6-7> I'm not
[14:28:31] <Maryana> everyone should come :)
[14:28:32] <Dragonfly6-7> I'll be working
[14:28:53] *** Quits: Emw (~emw@wikipedia/Emw) (Ping timeout: 245 seconds)
[14:28:55] <Maryana> dragonfly6-7: the talks will be taped, if you're interested in seeing them :)
[14:28:59] <linnea> hey i just realised something. is this office hours because this is during your office day?
[14:29:12] <StevenW> No, none of us are in the office now.
[14:29:20] <ToAruShiroiNeko> StevenW I was hoping they would be ready by summer 2012 they are pushed to 2013
[14:29:27] <ToAruShiroiNeko> I died a little when I heard that :p
[14:29:31] <Maryana> linnea: yeah, it's actually saturday for us, and i'm technically on vacation to boot :)
[14:29:45] <linnea> oh right yeah today is saturday :D
[14:29:47] <worm_that_turned> there's dedication
[14:29:49] <Sven_Manguard> Maryana: I am uncomfortable with the community thouugh. Even before they chased off one of my friends, I've found Wikipedia's community as a whole to be unpleasant
[14:29:58] <StevenW> ToAruShiroiNeko, I would encourage you to speak up and say you really want that to happen. Whether on something like wikitech-l or onwiki.
[14:30:21] <Maryana> sven_manguard: the great thing about the 1k editor project is that it introduced me to a lot of awesome people who are flying under the radar, so to speak
[14:30:22] <Sven_Manguard> So I can give you a few good reasons why I've cut my contributions by 66%
[14:30:35] <Sven_Manguard> but none about why I'm still doing the other 33%
[14:30:36] <linnea> Sven_Manguard: but I would think that encouraging good manners (welcoming and congratulations) would change the community
[14:30:45] <Sven_Manguard> linnea: not a chance
[14:30:47] <apergos> Is the editor engagement work also about converting readers to editors, or is it primarily focused on people who have already done their first edit?
[14:30:55] <StevenW> Both apergos
[14:31:08] *** Joins: Emw (~emw@wikipedia/Emw)
[14:31:08] <Sven_Manguard> because unless we get rid of the troublemakers, and there are dozens, we're tossing new users into a nasty existing problem
[14:31:19] <linnea> I quitted wikipedia years ago because of death threats.. but now I'm back.
[14:31:19] <Sven_Manguard> which makes them leave
[14:31:25] <Sven_Manguard> or become nasty
[14:31:36] <WereSpielChqrs> @Sven, Being nice doesn't hurt, doesn't cost and can be catching
[14:32:07] <Maryana> yeah, i don't think it's realistic for us to say we're going to totally change the social norms of an online community in one go
[14:32:16] *** Joins: BirgitteSB (~birgittes@24-217-149-193.dhcp.stls.mo.charter.com)
[14:32:19] <StevenW> If people are interested in fixing more community-oriented problems instead of the kind of experiments we are working on, I would poke you to take a look at the new WikiProject on Editor Retention (if you're an English Wikipedian)
[14:32:28] <apergos> what are you thinking of doing to get readers over the hurdle of their first edit? Are there any experiments planned? (A bad idea: a popup window "people who loked at this article also edited X" :-P But maybe there are some good ideas out there)
[14:32:44] <StevenW> Yeah, we've got some ideas sort of like that in our backlog.
[14:32:44] <thekaryn> making the editing experience more gracious and pleasant is something we want to factor into our experiments wherever feasible
[14:32:54] <apergos> ok cool, thanks
[14:32:57] <WereSpielChqrs> @Sven Yes the Dramah boards cn be messy at times, but they and the contentious articlse only involve a smal proprtion of the community
[14:32:59] <StevenW> Before we dive too deeply into converting readers into bonafide Wikipedians, we want to take a look at testing improvements to the account creation process.
[14:32:59] <linnea> apergos: that would be really interesting :P like amazon style "people who bought this also bought this"
[14:33:22] <Thogo> which would be annoying for many people. ^^
[14:33:58] <StevenW> Yes, recommender systems and the like can be annoying or useless when implemented poorly.
[14:34:14] <Sven_Manguard> linnea: "People who edited Saturn also edited Auto-erotic asphyxiation"
[14:34:16] <Maryana> heh. i think even before that, we need to do a better job of surfacing that wikipedia is edited by other humans
[14:34:23] <apergos> well there is a study from last year I think that showed (boy is this vague cause I don't remember the details) that the pirmary motivator for people to do something was to be told other people had done it, like leaving the towel on the shower rod at the hotel if you don't need it to be washed, "everyone else is doing it"
[14:34:32] <Thogo> I would be interested in culturally separated tests, so for example, do US-americans react differently than let's say Italians, on things that you test.
[14:34:38] *** Joins: Krenair (~Krenair@wikimedia/Krenair)
[14:34:40] <apergos> so something like that might have traction, dunno
[14:35:28] <Maryana> apergos: interesting. can you send me or stevenw the link if you remember the title? i think people's motivations for contributing to WP are pretty different from other general motivations, but it's always good to know the literature :)
[14:35:44] <apergos> heh, I am sure they are, lemme dig around
[14:35:51] <Maryana> thogo: that would be tricky to do without dipping into private data about user location
[14:35:56] <WereSpielChqrs> I've appointed a hundred or so Autopatrollers over the last couple of years, and there are plenty of prospects out there. I think that such editorrs appreciate it and I get a lot of feedback, but it would be interesting to know whether it really made a difference to their subsequent editing
[14:36:11] <Thogo> uhm, you could use anon users only.
[14:36:16] <StevenW> Right
[14:36:34] <apergos> http://www.beyondthepurchase.org/blog/06/a-room-with-a-viewpoint-or-who-%E2%80%9Cwe%E2%80%9D-are-depends-upon-what-i-think-you-know/ found a pointer, that should get you started
[14:36:37] <ToAruShiroiNeko> StevenW me? speak up?
[14:36:45] <StevenW> Even just usability tests with a handful of people carefully chosen from different backgrounds would be useful I think, Thogo.
[14:36:46] <ToAruShiroiNeko> I would get shot :p
[14:36:52] *** Joins: Isarra (~urgh@wikimedia/Isarra)
[14:36:55] *** Joins: Risker (~chatzilla@wikimedia/Risker)
[14:37:06] <StevenW> Hi Isarra, Risker
[14:37:13] * Risker waves
[14:37:19] * Isarra passes out on StevenW.
[14:37:24] <StevenW> LOL
[14:37:38] * ToAruShiroiNeko passes Isarra around
[14:37:40] <Thogo> indeed, but it's a different study then. What I wonder is if it would be necessary to show certain things different for people from different cultures to make sure everyone feels good editing.
[14:38:21] <StevenW> One way to take a look at this would be to compare WikiLove on Wikipedias where it has been localized by the community to fit their culture, versus ones where it has not been.
[14:38:42] <Maryana> werespielchqrs: yes! that would actually be an interesting A/B test. you should try it for a month.
[14:38:43] <StevenW> For instance, Portuguese Wikipedia insert a different icon instead of the heart, and uses Brazilian snacks
[14:39:12] <Thogo> right... that would be interesting.
[14:39:33] <Maryana> werespielchqrs: i'd be really interested in the results. you could make a list of all users you'd trust as autopatrollers, then split them into two groups, give one group the user right and keep the other as a control.
[14:40:27] <StevenW> We've sort of covered the timestamp experiment. I just want to make sure: does anyone have any more questions about it?
[14:41:15] <StevenW> If not, maybe we should talk about the next experiment we're planning...
[14:41:27] <Maryana> it's very exciting
[14:41:36] <Maryana> although i bet sven_manguard is going to hate it :-P
[14:41:44] <Emw> is there any plan for a more longitudinal analysis with the timestamp feature?
[14:41:56] <ToAruShiroiNeko> Brazilion many snacks?
[14:42:01] <Sven_Manguard> Maryana: if you say badges I'm going to moan
[14:42:05] <StevenW> Nope
[14:42:08] <Maryana> haha, no.
[14:42:27] <StevenW> Emw: I think we should add that as one idea to the next iterations, for sure.
[14:42:32] <Dragonfly6-7> Maryana - you know what the benefit is of autopatrolled?
[14:42:33] <ToAruShiroiNeko> StevenW right now I am trying to create a template range that deals with international membership of countries
[14:42:35] <Dragonfly6-7> to keep them out of *my* way
[14:42:37] <Dragonfly6-7> me specifically
[14:42:44] <Dragonfly6-7> I'm the one who pressured to have it implemented on en.wiki
[14:42:45] <ToAruShiroiNeko> a problem we get is the absence of a "in between" parser
[14:43:04] <Dragonfly6-7> if you don't get them out of my way while I'm trying to work, I get annoyed.
[14:43:10] <Maryana> dragonfly6-7: yeah, i bet. i remember looking at the patrolling data during the summer of research. we all thought you were a bot :-P
[14:43:12] <ToAruShiroiNeko> and parsers themselves are far too slow
[14:43:28] <StevenW> Our next experiment though, is one that only brand new registered editors will see.
[14:44:00] <Dragonfly6-7> Maryana -you have no idea how much patrol I do that doesn't get registered because the pages have already expired
[14:44:10] <WereSpielChqrs> What I know makes a difference to retention is making people admins. Admins stick around much longer than other editors, I beleive that RFA being broken is a big part of the problem in declining retention of longterm editors
[14:44:11] <linnea> how you can have these experiments on "your" wikipedia? (i mean other than english)
[14:44:54] <Maryana> linnea: it's tricky, because we don't have as much access to data on other wikis
[14:45:03] <linnea> WereSpielChqrs: but also they can change the edit types. people no longer expand articles, instead they are doing admin duties
[14:45:44] <Sven_Manguard> WereSpielChqrs: +1
[14:45:45] <Maryana> werespielchqrs: have you checked out the editor retention wikiproject? that strikes me as a great opportunity to extend the RfA reform conversation
[14:45:47] <StevenW> Yeah, we're not currently set up to fully test new features outside English. However, if people are asking for us to do that, this gives us leverage to ask for more technical resources in that area and the time to set something up.
[14:45:50] <Dragonfly6-7> linnea - that's damn right
[14:46:13] <WereSpielChqrs> @Maryana and Dragonfly. What I might do is test telling people I've made them an Autopatroller against not telling them. That should please both of you
[14:46:20] <Sven_Manguard> Maryana: this is going to sound harsh, but how much time have you actually spent in the trenches editing/being part of the community at Wikipedia?
[14:46:42] <Maryana> heh, not at all
[14:46:44] <Maryana> about a year
[14:46:50] <Maryana> (not harsh at all, i mean)
[14:46:53] <Sven_Manguard> Because there have been in the past 18 months, at least three major RfA reform pushes and a dozen major discussions, at least
[14:46:54] <Maryana> i'm user:accedie
[14:46:58] <Sven_Manguard> it never gets anywhere
[14:47:00] <Dragonfly6-7> WereSpielChqrs- uh, doesn't it turn up in your watchlist if your userrights get changed?
[14:47:11] <Dragonfly6-7> which means that people *will* know when they get autopatrolled?
[14:47:12] <worm_that_turned> never gets anywhere Sven?
[14:47:20] <Sven_Manguard> it never gets anywhere because everyone knows it's broken but disagrees with how to fix it
[14:47:34] <worm_that_turned> everyone disagrees about what's broken about it.
[14:47:35] <Sven_Manguard> the closest we got was a proposal to create apprenticeship
[14:47:39] <linnea> Dragonfly6-7: I don't think it shows on your watchlist
[14:47:54] <StevenW> Yeah thankfully RFA reform is not a part of our experimental work. ;)
[14:47:58] <Sven_Manguard> but it failed because the person running it changed it each time soneone had an objection
[14:48:04] <Dragonfly6-7> it shows *somewhere*
[14:48:19] <linnea> Dragonfly6-7: it shows on your settings if you look carefully enough
[14:48:56] <StevenW> So the next experiment we're setting up (for after Wikimania) is https://www.mediawiki.org/wiki/Post-edit_feedback
[14:48:57] <Maryana> sven_manguard: one thing that russian wikipedia has done is unbundled adminship and created different "social" user rights, as opposed to technical ones
[14:49:09] <WereSpielChqrs> @Dragonfly, yes it shows up somewhere, but not everyone uses their watchlist, and there is a difference between merely flipping a bit and doing so with a friendly message
[14:49:16] <Dragonfly6-7> true enough
[14:49:20] <Sven_Manguard> Maryana: suggested, shot down, suggested again, shot down again
[14:49:30] <Sven_Manguard> possibly more than twice
[14:49:31] <Isarra> It does show in your watchlist.
[14:49:32] <StevenW> For brand new editors (i.e. only people who registered after the start of the experiment) we're going to try giving them some kind of immediate positive feedback after they edit.
[14:49:57] <Isarra> Most new editors make bad first edits.
[14:50:08] <Maryana> isarra: untrue, actually
[14:50:09] <WereSpielChqrs> Rollback and Filemover have both been unbundled - so RFA reform is possible on EN wiki, just rare
[14:50:17] <Maryana> we has the data to prove it :)
[14:50:19] <Isarra> Okay, they don't make good ones.
[14:50:20] * worm_that_turned agrees with Maryana ;)
[14:50:48] <StevenW> Positive in this case doesn't necessarily mean telling them their edit was perfect and without flaws. ;)
[14:50:53] <worm_that_turned> there's a difference between "everyone makes mistakes" and new editors make bad (or not good) first edits
[14:51:31] <Maryana> worm_that_turned: exactly. isarra: it depends on what you mean by "bad." but the majority of new editors are trying to contribute constructively to the encyclopedia. we did research on this, and there's a blog post somewhere
[14:51:38] <BirgitteSB> Positive only means pointing out the best of their edit. Not saying their edit was the best.
[14:51:39] <StevenW> For the first iteration, it basically will just test a feature that many major websites do but we don't, which is confirming that you successfully submitted something.
[14:52:16] <Maryana> https://blog.wikimedia.org/2012/03/27/analysis-of-the-quality-of-newcomers-in-wikipedia-over-time/
[14:52:21] <Isarra> That makes a little more sense.
[14:52:48] <WereSpielChqrs> There are some things that newbies will inevitably get wrong and we should help them with that. I'd love to see an auto sign facility that regulars could opt into but all new accounts started with
[14:53:04] <StevenW> That is a good idea too, methinks.
[14:53:05] <linnea> Isarra: I don't have data, but in finnish wikipedia most ip editors corrects typos etc
[14:53:20] <Isarra> Neat.
[14:53:24] <Maryana> another possible benefit of immediate post-edit feedback is that many first-time editors probably aren't sure if their edit went live or not
[14:53:35] <Maryana> since there's nothing that tells them their save was successful
[14:54:01] <Isarra> How will that work with edits that really haven't gone live due to pending changes?
[14:54:04] <Krenair> FlaggedRevs would make that a lot clearer :)
[14:54:12] <Isarra> Might be a good place to explain that, though.
[14:54:26] <Maryana> isarra: this will only be tested on english wikipedia
[14:54:33] <Maryana> which won't have PC for awhile
[14:54:43] <Isarra> But it will. So it might be very useful for that.
[14:54:52] <Isarra> This might, that is.
[14:55:18] <Maryana> ah, i see what you mean
[14:55:22] <Isarra> And other languages have full flagged revisions, so seeing how it interacts with that sort of thing would probably be useful anyhow, no?
[14:55:30] <apergos> I guess after the save the page could say (for new editors) "Congratulations, your edit is now live" .. why not?
[14:55:37] <apergos> could be tested anyways
[14:55:54] <Dragonfly6-7> it also helps to hang out on #wikipedia-en-help and see the people who come begging for help
[14:56:00] <Dragonfly6-7> WHY ISN'T MY PRESS RELEASE ON GOOGLE YET
[14:56:37] <StevenW> After we test that, depending on the results, we want to try only giving people that kind of notification when they reach certain editing milestones, like 1, 10, 50, 100 edits etc.
[14:56:48] <Maryana> dragonfly6-7: i wouldn't take the tiny group of people who use IRC as a representative sample of internet users ;)
[14:57:01] <StevenW> This is similar to the experiments thanking people at the 1,000th edit mark
[14:57:21] <Dragonfly6-7> Maryana - you do know that the AfC project offers a web-based IRC client for Live Real-Time Help , yes?
[14:57:21] <worm_that_turned> Maryana: It's a depressing truth though. A significant portion of AfCs fit into the category too
[14:57:35] <WereSpielChqrs> What the graphs of new editors by quality aren't showing are the new editors screened out by the edit filters.
[14:57:52] <apergos> hmm you know wordpress (I think?) sets milestones for the new user: first post, first ten psts, etc
[14:58:13] <apergos> I wonder how effective it is for them
[14:58:24] <Maryana> apergos: yes, wordpress is definitely a good example. they also surface page view data really intensely.
[14:58:34] <apergos> hmm that's true
[14:58:43] <Isarra> Number of edits, though, does that really mean anything? Meaningful content contribution tends to result in a much lower editcount than typo fixing or countervandalism, so encouraging more edits in such a fashion could detract from content work.
[14:58:43] <StevenW> Right. It's kind of similar to that, though this first test will be telling people when they have reached a milestone, rather than encouraging them to reach an arbitrary edit count milestone.
[14:59:06] <Isarra> On the other hand, most new users seem to not know what the preview button does, so maybe I'm just being silly.
[14:59:08] <BirgitteSB> WereSpielChqrs: Are they really new editors if they do not make a successful edit?
[14:59:30] <StevenW> Agreed Isarra. See my above comment. :)
[14:59:36] <thekaryn> Isarra - do you have any thoughts on other types of post edit feedback you'd like to see?
[14:59:40] <WereSpielChqrs> @Apergos we have http://en.wikipedia.org/wiki/Wikipedia:Service_awards - three systems
[14:59:52] <apergos> looking
[14:59:53] <Isarra> Yes, well, if they can see a pattern...
[15:00:50] <WereSpielChqrs> @BirgitteSB, no they aren't editors but they are attempted editors, and it is an important part of understanding how we are interacting with potential editors
[15:00:53] <StevenW> Ok, so we're at the hour mark. I am okay with sticking around a few mins if anyone wants to talk more.
[15:01:13] <StevenW> But we should let thekaryn go or anyone else who wants to. :)
[15:01:13] <apergos> who has 16 years??
[15:01:18] <Dragonfly6-7> 16 years of what?
[15:01:30] <apergos> 20 Vanguard Editor (or Lord Gom, the Highest Togneme of the Encyclopedia) 132,000 16 years
[15:01:56] <WereSpielChqrs> 16 years is an attempt to futuure proof it - ultimately we will need fifty year awards
[15:02:22] <apergos> I see :-)
[15:02:26] *** Parts: Emw (~emw@wikipedia/Emw) ()
[15:02:44] <Maryana> i should probably get back to family vacation time :) but if anybody has any pressing questions/comments for me in the next minute or so, fire away
[15:02:49] <Maryana> otherwise, catch me on the wikis
[15:02:52] <Isarra> thekaryn: Not that wouldn't require a complete community overhaul.
[15:03:09] <StevenW> I think one of the interesting things in the editing milestone test is that it's basically an automated way of delivering service awards between the #1 and #2 awards.
[15:03:28] <Maryana> yeah, and it's a really critical time, from what i can tell
[15:03:39] <Isarra> If anyone ever tries to deliver me a service award, I will punch their bot in the face.
[15:03:40] <Maryana> right before editors become "wikipedians"
[15:03:44] <BirgitteSB> I wonder if the first edit confirmation will set a false expectation
[15:03:45] <Risker> thanks Maryana and StevenW for accommodating a different group of interested observers
[15:03:52] <WereSpielChqrs> If you look at other volunter organisations who have people who've been with them for decades they often have such systems - doesn't guarantee they work but otherwise is free and harmless
[15:04:11] <Maryana> risker: of course! we try :)
[15:04:13] <BirgitteSB> That they will then not believe that their SECOND edit worked when it did
[15:04:42] *** Parts: Risker (~chatzilla@wikimedia/Risker) ()
[15:04:50] <apergos> bots are elegibile for these awards I guess, it doesn't say anything t the contrary... so be careful who you punch!
[15:04:57] *** Quits: cndiv (~textual@adsl-71-135-43-244.dsl.pltn13.pacbell.net) (Quit: Computer has gone to sleep.)
[15:05:11] <StevenW> It's a good question BirgitteSB. We're thinking that this will be headed off by the fact that the message specifically will refer to their first edit.
[15:05:12] <Maryana> ok, i'm out of here. thanks for coming, everybody, and i hope to catch at least some of you at wikimania next week :)
[15:05:19] <StevenW> Bye maryana
[15:05:21] <apergos> thanks for having the session
[15:05:32] <Maryana> bye!
[15:05:39] *** Quits: Maryana (~justdandy@wikipedia/Accedie) (Quit: bye!)
[15:05:39] <WereSpielChqrs> We exclude bots from the score board though
[15:05:50] <Isarra> I apologise for wandering in and making an arse of myself.
[15:05:57] <apergos> I see you let the bot owner count bot edits towards their score
[15:06:00] <StevenW> You did not such thing.
[15:06:03] <StevenW> no*
[15:06:04] <apergos> sounds like bot enslavemnt to me :-P
[15:06:21] <StevenW> Bot slaves. Making Wikipedia work since 2001.
[15:06:24] <Isarra> Bots are an extension of one's will.
[15:06:29] <apergos> and maybe before :-D
[15:06:42] <Isarra> All automated edits receive the same scrutiny as manual edits.
[15:06:44] <WereSpielChqrs> Not having a bot myself I hadn't noticed that bit
[15:07:07] <Isarra> And get the owner blocked just as quickly, if not more so, than manual edits.
[15:07:29] <StevenW> Before everyone else heads out from the team, just want to double check: does anyone have final questions for us about the experiments work?
[15:07:46] <apergos> but it's a lot easier to rack up several tens of thousands of bot edits in a short time (I say looking knowingly at my bot)
[15:08:03] <WereSpielChqrs> What reality test do you do before running an experiment?
[15:08:04] <Thogo> please keep us informed about the experiments, StevenW :)
[15:08:26] <StevenW> Will do.
[15:08:31] *** Joins: mindspillage (~kat@wikimedia/KatWalsh/x-0001)
[15:08:54] <StevenW> We make sure all our work is documented on Meta, the Wikipedia we're working on, and we announce new test on the Village Pump.
[15:09:22] <StevenW> WereSpielChqrs: not sure what you mean. Reality test?
[15:09:53] <StevenW> What is this thing reality you speak of ;-)
[15:10:00] <WereSpielChqrs> If you keep experimenting on the community without discussing them on Wiki beforehand eventully something will go wrong and people will want to know where they were discussed. But a Village pump discussion should sufice
[15:10:46] <StevenW> Well, that's part of the point of documenting upcoming tests in public
[15:11:07] <StevenW> Though obviously that's not the same as trumpeting an upcoming thing everywhere or in the best places.
[15:11:24] <WereSpielChqrs> I was worried you migt only be doing it on Meta
[15:11:29] <thekaryn> WereSpielChqrs we aim to be as transparent as possible about our experiment plans - it does us no good in our editor retention work if we piss off the community we're trying to help out
[15:11:30] <apergos> do you solicit ideas for future experiments when you announce new tests?
[15:11:35] <StevenW> yes
[15:11:44] <StevenW> https://en.wikipedia.org/wiki/Wikipedia:Editor_engagement_experiments <- also on EN
[15:11:56] <StevenW> Watch that page for sure if you want to keep tabs on what we're doing
[15:12:40] <StevenW> How much impact we think something is going to have definitely informs the amount of communication we do beforehand.
[15:13:13] <StevenW> For instance, if we were to test anything resembling a big new feature like Special:NewPagesFeed or Article Feedback Tool, that requires a lot of discussion beforehand
[15:13:25] <tommorris> ooh, things going on in office?
[15:13:25] <tommorris> ;-)
[15:13:30] <tommorris> hey StevenW
[15:13:33] <StevenW> hi
[15:13:40] <StevenW> But if we run a test say, only on anonymous or completely new editors it's hard to inform them beforehand.
[15:14:29] <StevenW> The other option, especially when it comes to experiments for more experienced people, is trying to run an opt-in test rather than an opt-out one
[15:15:30] <Thogo> will be hard to get a balanced sample of users then
[15:15:40] <Thogo> if you need to control for certain factors,.
[15:15:41] <StevenW> Right, it does present challenges.
[15:15:45] <WereSpielChqrs> This maybe out of scope, but have you considered testing whether templating new articles as quickly as possible is effective at getting newbies to fix their articles or just bites them? I've got a fairly strong suspicion its the latter, but others believe the former, independent research could really make a difference here.
[15:16:01] <StevenW> I think that's a good idea to test.
[15:16:36] <StevenW> We should add that to the backlog.
[15:16:44] <StevenW> A somewhat similar idea is this: https://meta.wikimedia.org/wiki/Research:Improve_your_edit
[15:16:50] <WereSpielChqrs> You can probably measue that from what is already happening
[15:17:23] <StevenW> There are sampling problems in that approach, since you're comparing people who's articles may naturally be better (and thus get no tags) versus those that need more improvements.
[15:18:21] <StevenW> Anyway, I think doing this on a Saturday worked out pretty well. I think we'll try it again for next time, instead of during working hours North American time.
[15:18:57] <WereSpielChqrs> It does rahter cut into the weekend though
[15:19:27] *** Quits: BirgitteSB (~birgittes@24-217-149-193.dhcp.stls.mo.charter.com) (Quit: Left)
[15:19:30] <StevenW> That's okay. I'd probably be editing anyway.
[15:19:39] <apergos> :-D
[15:19:51] <WereSpielChqrs> I like the improve your edit concept, but fear it might encourage even more people to auto revert all unsourced edits
[15:20:14] * StevenW kind of feels that happens anyway.
[15:20:16] <apergos> I suppose the list of reasons would have to be carefully selcted
[15:20:56] <WereSpielChqrs> A large proprtion of unsourced edits re reverted, but not by any means all
[15:21:49] <StevenW> It's a valid concern. Want to note it on talk page WSC? :)
[15:22:18] <StevenW> I should probably head out.
[15:22:24] <WereSpielChqrs> I think it also needs an option for unsourced but plausible edit not involving a living persoon. And if they tick that it doesn't revert but adds a fact tag instead
[15:22:43] *** Parts: Sven_Manguard (~~~@wikipedia/Sven-Manguard) ()
[15:22:58] <StevenW> Thanks for attending everyone.
[15:23:04] *** Parts: gmaxwell (~greg@wikimedia/KatWalsh/x-0001) ()
[15:23:06] <apergos> thanks for having us, it was very interesting
[15:23:26] <Thogo> thank you StevenW and the rest of the team ^^
[15:23:37] *** StevenW changes topic to 'IRC office hours -- https://meta.wikimedia.org/wiki/IRC_office_hours'
[15:23:42] <StevenW> Have a good weekend!
[15:23:53] *** Quits: StevenW (~textual@wikimedia/steven-walling) (Quit: ["Textual IRC Client: www.textualapp.com"])