IRC office hours/Office hours 2011-10-19

zackexley: Hi!  I'm really sorry I'm late. I made the mistake of saying yes to a system update -- like 20 minutes ago

[4:04pm] ppena joined the chat room.

[4:04pm] FT2: "oops"

[4:04pm] jsalsman: welcome and good morning, Zack

[4:04pm] zackexley: Welcome everyone -- today we're going to talk about the 2011 Fundraiser. Who's got questions?

[4:04pm] zackexley was granted voice by ChanServ.

[4:05pm] jsalsman: I have 4 at http://meta.wikimedia.org/wiki/User_talk:Zackexley#Fundraising_office_hour_topics

[4:05pm] zackexley: Which first James?

[4:05pm] jsalsman: (well 5 but the first really isn't fundraising-related so I didn't number it)

[4:05pm] zackexley: there are few in there

[4:05pm] jsalsman: I hope you will test the user submitted banners

[4:05pm] zackexley: OK, let's talk about that

[4:06pm] jsalsman: I understand that you have been trying to optimize the subsequent portions of the fundraising pages, and I agree that is good

[4:06pm] jsalsman: however, the click through rate is the gateway to people seeing those other pages, so it is paramount

[4:06pm] jsalsman: there are still at least 300 user submitted banners which have not been tested, and I understand they could be in less than a week if you so choose

[4:07pm] zackexley: I should give an intro to the banner->landing page fundraising dynamics first for everyone here's benefit

[4:07pm] jsalsman: the variation we have observed in click through rates has been so high that it would be folly to leave all but the least appropriate untested

[4:08pm] jsalsman: it is not only important to optimize the click through rate, but to further optimize message selection for future banners

[4:08pm] zackexley: OK. let me give a quick overview of how the fundraising works. And then I'll talk about the banner testing

[4:08pm] jsalsman: but also* to further optimize

[4:09pm] pgehres joined the chat room.

[4:09pm] Seddon joined the chat room.

[4:09pm] zackexley: During the fundraiser, we run banners on top of all Wikimedia projects all over the world. Generally around 1% of users will click on a banner. Of those around 1% will donate on the landing page. That was the general rate last year -- and that was a HUGE improvement over past years

[4:10pm] StevenW joined the chat room.

[4:10pm] pgehres left the chat room.

[4:10pm] zackexley was granted voice by ChanServ.

[4:11pm] zackexley: In all of our testing in the past -- and we tested scores, or maybe over 100, of different messages, most submitted by volunteers -- the "Please Read: A personal appeal from Wikimedia ____ ___" did FAR FAR better than anything else….with only 2 exceptions

[4:11pm] zackexley: Let me get to those exceptions in a second.

[4:11pm] StevenW was promoted to operator by ChanServ.

[4:11pm] pgehres joined the chat room.

[4:11pm] zackexley: One other thing to know is that adding an image of a person (we tested other images) increased click rates by about 30% very consistently

[4:12pm] • jsalsman notes that implies a very large standard deviation (and variance) with bimodal outliers, supporting the rationale for testing all user submissions

[4:12pm] StevenW left the chat room. (Remote host closed the connection)

[4:12pm] harej joined the chat room.

[4:12pm] wctaiwan joined the chat room.

[4:12pm] StevenW joined the chat room.

[4:12pm] mroth joined the chat room.

[4:12pm] zackexley: James - I'm not arguing against testing other banners. Just hang on there...

[4:12pm] jubo2 joined the chat room.

[4:12pm] jsalsman: okay

[4:13pm] StevenW was promoted to operator by ChanServ.

[4:13pm] Sven_Manguard joined the chat room.

[4:13pm] tommorris joined the chat room.

[4:13pm] Sven_Manguard: Has the meeting started yet?

[4:13pm] Philippe: yes 

[4:14pm] Sven_Manguard: Oops.

[4:14pm] Fluffernutter joined the chat room.

[4:14pm] tommorris: hey all

[4:14pm] zackexley: You're right about the variance between many banners click rates. BUT the "Please read a personal appeal" banner beat all other slogans (except 2 in the very last days of the campaign) by a HUGE and very confident margin.

[4:14pm] Sven_Manguard left the chat room.

[4:14pm] StevenW: I think Zack is answering a question from jsalsman right now, but feel free to PM me anything you want to ask after, or just keep it handy 

[4:15pm] JAYANTA joined the chat room.

[4:15pm] jsalsman: Zack is addressing question #1 on http://meta.wikimedia.org/wiki/User_talk:Zackexley#Fundraising_office_hour_topics

[4:15pm] Platonides joined the chat room.

[4:15pm] zackexley: the two exceptions were in the last days of the fundraiser when 2 time-specific messages worked a tiny bit better: regarding end-of-year taxes and ending the fundraiser

[4:15pm] AzaToth joined the chat room.

[4:15pm] jsalsman: zackexley: it is not clear to me that you are using the words "confident margin" in a statistically accurate sense

[4:16pm] tejinashi joined the chat room.

[4:16pm] StevenW: jsalsman: our analytics engineer is a masters in CS and machine learning, so it is what we mean when we say confidence margin. 

[4:16pm] FT2: Can we hear the rough cut first, then ask refining and double-checking questions.

[4:16pm] jsalsman: in cases where the variance is large and the outliers are bimodal, there is very little confidence that any unexplored portions of the population are not outliers

[4:17pm] jsalsman: StevenW: to whom do you refer?

[4:17pm] azariv joined the chat room.

[4:17pm] jsalsman: if anyone would like to continue this discussion later, please do so at http://meta.wikimedia.org/wiki/Talk:Fundraising_2011#User_made_banners:_outstanding_question_from_archived_thread

[4:18pm] zackexley: James -- please, really, I know what confidence means in a statistical sense. We have spent countless hours, and talked with many statisticians about how we're testing to make sure we're not screwing up. We always invite scrutiny. But I want you to know we're not just throwing around "confidence" without rigor

[4:18pm] Avruch: other than the fundraiser, what does the CCO do?

[4:18pm] jsalsman: but for now, I will need more than an appeal to authority to convince me that Zack was using "confident margin" correctly in the case of the observed bimodal distribution with large variance

[4:19pm] jsalsman: zackexley: if there is a statistically rigorous reason that you have not tested the remaining user submitted banners, why has it not been brought forth at http://meta.wikimedia.org/wiki/Talk:Fundraising_2011#User_made_banners:_outstanding_question_from_archived_thread ?

[4:19pm] Theo10011: Yes, hello.

[4:19pm] Theo10011: Timely reminder StevenW.

[4:19pm] Jamesofur: jsalsman, I'm sorry. But that's a bit offensive since it sounds very much like you're coming in assuming that he or the rest of the office has no clue about statistical significance

[4:20pm] zackexley: I don't think these office hours are the place to go into our statistical methods. But I could be wrong. What do folks want to hear about? Statistical confidence or how the fundraiser works with regard to messaging and testing?

[4:20pm] StevenW: zackexley: want to take Avruch's question?

[4:20pm] jsalsman: my questions stands.  I'm sorry if I offended anyone with my request to state and argument which people claim has already been decided correct but not published

[4:20pm] Fae joined the chat room.

[4:20pm] jsalsman: state an* argument

[4:20pm] Fluffernutter: zackexley, please leave the statistical techniques for another time 

[4:21pm] Avruch: ^

[4:21pm] FT2: maybe james and those involved could engage by email to check his concerns. If we go into them now with limited time, 1/ most people arent here for that, 2/ we lose the communication of the big picture.

[4:21pm] AzaToth: statistics are good, as long you aint try to apply it to reality

[4:21pm] Theo10011: tru dat

[4:21pm] Saibo joined the chat room.

[4:21pm] zackexley: James - Megan is about to give us a link to Ryan Faulkner's explanation of our stats methods from last year. We've improved a bit on them this year too.

[4:21pm] jsalsman: FT2: my second question is, why are people uncomfortable discussing fundraising with me on the email list

[4:22pm] PhancyPhysicist joined the chat room.

[4:22pm] StevenW: Let's leave personal discussions about one person to another time.

[4:22pm] guillom joined the chat room.

[4:22pm] FT2: james - I have no idea, nor any idea if that's so or just individual perception. But either way my above answer stands here.

[4:22pm] jsalsman: I'll read it carefully, Zack, and respond on your talk page.

[4:23pm] jsalsman: please move on since I've offended Jamesofur by assuming that someone has made a mistake about bimodality

[4:23pm] jsalsman: (however, I would like to point out that mathematical facts are not assumptions)

[4:23pm] mhernandez_: here's the link to Ryan's explanation with some example test reports: http://meta.wikimedia.org/wiki/Fundraising_2010/Report/Example_Methodology

[4:24pm] zackexley: The stats stuff has been a really interesting experience. Like AzaToth alluded to, we talked to a ton of stats experts. And they all used big words and all argued for different approaches. We really had to get them to look at our conditions closely and in detail. Over and over different experts would say, "OH! Now I see what you're doing. OK, no, you need to use different methods than what I was originally suggesting"

[4:25pm] jsalsman: mhernandez_: thank you, but that document does not address the observed bimodality of the distribution, or say anything about its variance

[4:25pm] zackexley: Especially when talking with stats people coming from the hard sciences, they generally wanted to apply the wrong set of methods. When they looked closer at our actual problem, they changed their minds and recommended different approaches.

[4:26pm] Theo10011: Can we move on from statistics please?

[4:26pm] Odisha1 left the chat room. (Ping timeout: 245 seconds)

[4:26pm] jsalsman: the simple fact is that you don't know whether you are missing any other outliers, and it would take you 4 days to find out

[4:26pm] jsalsman: yes, please move on

[4:26pm] zackexley: James - we had exactly that kind of conversation with a lot of people. And when they looked closer, they were operating on some of the wrong assumptions about our application, population, etc

[4:26pm] Theo10011: Thank you.

[4:26pm] jsalsman: I agree that those people must have been operating on wrong assumptions

[4:27pm] StevenW: If anyone has other fundraiser questions, I think it's a good time to throw them out there.  Or we can suggest some topics.

[4:27pm] zackexley: OK, but one last consideration is that 4 days will cost us a lot of money -- maybe over a million.

[4:27pm] Philippe: I think Avruch had one 

[4:27pm] Avruch: Did my question get answered?

[4:27pm] Theo10011: How much is the daily average needed to hit the target?

[4:27pm] zackexley: Avruch - the topic is fundraising for this week. Can we stick with that a little more?

[4:28pm] StevenW: Good question Theo

[4:28pm] Avruch: might've missed it while skimming over bimodal marginal confidence variables or whatever

[4:28pm] Avruch: zackexley: ok

[4:28pm] AzaToth: Noob question: what is the target?

[4:29pm] Theo10011: Good one.

[4:29pm] zackexley: The daily average depends on how long we run the fundraiser and what our goal is.  And we don't have any of that totally fixed yet.

[4:29pm] Theo10011: I thought it was $25 Million-ish. I might be wrong.

[4:29pm] zackexley: 25milionish could be right -- but what if in the first days of the fundraiser we realize that's totally unrealistic?

[4:29pm] zackexley: then the goal might be 20

[4:30pm] zackexley: We don't want to set a goal we can't reach

[4:30pm] Theo10011: Ok.

[4:30pm] zackexley: But 24m would be the max

[4:30pm] AzaToth: last year was US$15 right?

[4:30pm] Theo10011: The budget and planning mentioned it closer 25 but it's understandable if it is not achievable.

[4:30pm] Theo10011: 16

[4:31pm] zackexley: We have a ~30m budget to raise. We think 6 can come from foundations and major gifts.  So 24 is the max for the fundraiser

[4:31pm] Theo10011: ohai harej

[4:31pm] harej: hy

[4:31pm] zackexley: So…James with the banner discussion was getting us to something really interesting that I'd like to talk about

[4:31pm] • AzaToth has difficult to understand why you need $10m more this year

[4:31pm] jsalsman: thank you!

[4:32pm] zackexley: Why does "Please read: A personal appeal from Wikimedia  ___" do so well?

[4:32pm] jsalsman: I think one of the reasons I make people uncomfortable is that rigor and accuracy is more important to me than preventing offense

[4:32pm] AzaToth: zackexley: stupidity of the masses

[4:32pm] zackexley: AzaToth: I would disagree -- it's exactly the opposite

[4:33pm] zackexley: it's the coolest thing about the fundraiser

[4:33pm] jsalsman: that's a great question, it's because people trust authorities and they want to hear what their money will be used for from people who appear to know

[4:33pm] zackexley: what we learned last year beyond any doubt was that normal marketing slogans don't work on our users

[4:33pm] zackexley: i think that's great

[4:33pm] AzaToth: zackexley: unless you starts to wonder if it's really a "personal message"

[4:33pm] AzaToth: zackexley: it's probably a cultural thing

[4:34pm] zackexley: But it's not about trusting authorities because some of our best appeals have been from "wikimedia foundation programmer" or "wikipedia editor" or "wikimedia volunteer" -- not such huge authorities in the traditional sense

[4:34pm] Philippe: I think expectations are realistic there, AzaToth.  I don't, for instance, expect that my US Senator wrote the response letter that I got from him.  I do suspect that he approves general broad themes and probably writes part of the language though.

[4:34pm] FT2: I could speculate why it works well. But do we have any formal analysis, feedback, or "very likely" guesses?

[4:34pm] jsalsman: everyone in the developed world who has had any experience with marketing expects non-personal messages to be focus grouped an therefore at least marginally deceptive

[4:34pm] AzaToth: Philippe: then it's not a personal message

[4:35pm] zackexley: AzaToth:  that is also the great thing. Our appeals are REALLY written by the people. And we have seen in research that readers can hear that in the voices

[4:35pm] StevenW: Writing real personal messages is why we hired the storytellers.

[4:35pm] jvandavier joined the chat room.

[4:35pm] jsalsman: how much is the difference between pictures of people with their arms crossed and otherwise?

[4:36pm] AzaToth: zackexley: yea, but it's difficult to know, especially when you assume all "personal messages" are really not

[4:36pm] FT2: Personal appeals do often do well. It personalizes the speaker. An African charity might ask for money to help "Lita who is blind" rather than "blind people" generally, for example. An individual is assumed to have some integrity, we care because they care.

[4:36pm] FT2: my guess.

[4:36pm] Theo10011: I am not sure about some of the banners the Storytellers(tm) came up with.

[4:36pm] jsalsman: I would like to know how a hands-on-hips photo performs compared to cross arms and arms at sides

[4:37pm] Maryana joined the chat room.

[4:37pm] AzaToth: The big question I think is: How far are you willing to go to manipulate the masses to give you money?

[4:37pm] jsalsman: if I thought they would be tested, I would submit a series of poses

[4:37pm] Theo10011: This might be mostly because of my non-american perspective, but the one about Maryana leacing harvard to work at wikipedia, didn't connect on any level.

[4:37pm] harej: Question -- if we want to go with as intimate an experience as possible, shouldn't we be striving for pictures as faces as much as possible, as zoomed-in as possible, as big as possible?

[4:37pm] the_wub joined the chat room.

[4:38pm] ChristineM: wouldn't that just be creepy?

[4:38pm] Fae: harej: Like a scarily big Jimbo face, say?

[4:38pm] Theo10011: No.

[4:38pm] Jamesofur: oh please no

[4:38pm] jsalsman: "A personal appeal from a Wikipedian banned for trying to get people to stop breathing uranium fumes" -- I bet that would be an outlier, not sure on which side of the distribution

[4:38pm] Moonriddengirl: LOL!

[4:38pm] Theo10011: Neva forget! Jimmy eyes.

[4:39pm] zackexley: Let me explain the process by which we're generating these appeals. We're also looking for more people to participate -- so please suggest people to intervview

[4:39pm] ChristineM: Jimmy is always watching.

[4:39pm] Theo10011:        http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B_0710_Maryana_Quote_US    

[4:39pm] Theo10011:        http://meta.wikimedia.org/w/index.php?title=Special:NoticeTemplate/view&template=B_1014_Rob_dollar2_US    

[4:39pm] zackexley: We hired 3 interviewers for the fundraiser (we have been calling them storytellers).  They are interviewing a ton of people from the community.

[4:39pm] zackexley: And also staff

[4:40pm] zackexley: We then -- ideally, working with them -- write an appeal using their words from the interview. We try a few different approaches for each one and test them.

[4:40pm] AzaToth:        http://commons.wikimedia.org/wiki/File:Jimbo-eyes.ogg    

[4:40pm] TheCavalry joined the chat room.

[4:41pm] Jamesofur: Just as a comment Theo10011 (you may know) but both of those banners never ran. Were created but never used in a test (decided against)

[4:41pm] AzaToth: from the 2007 fundraiser

[4:41pm] zackexley: It's fascinating what kinds of approaches work best -- and heartening. The ones that work best are those that call out the best values of the wikimedia movement

[4:41pm] zackexley: but they only work if they have these two other pieces -- right up top in the appeal:

[4:41pm] Theo10011: Jamesofur, there's 5-6 variations each, and I saw most of the tests were in US, so I have no idea to know either way.

[4:42pm] Maryana: Oof, that quote sounds really dumb out of context. Glad you nixed it.

[4:42pm] Jamesofur: yeah, and given how many banners get created definitely hard to know exactly which ones got used without really digging into the logs or test reports

[4:42pm] Odisha1 joined the chat room.

[4:43pm] zackexley: 1) a personal hook. i.e. something to draw in people's attention. That's why maryana worked (even though many people were put off by any mention of harvard, they still read, because they were hearing from a real person). But esp. see Brandon, Jimmy, Alan

[4:43pm] zackexley: 2) something that explains why we need the money

[4:43pm] rfarrand_ joined the chat room.

[4:43pm] FT2: zack

[4:43pm] AzaToth: I think you've used up all "Personal appeal" cards though

[4:43pm] Theo10011: cool Jamesofur.

[4:43pm] FT2: is a real name an important factor?

[4:44pm] FT2: or are people equally ok donating to a user identified by pseudonym?

[4:44pm] zackexley: and when we can roll all those into one or two paragraphs: personal, why we need the money, and mikimedia values, then we have a winning appeal

[4:44pm] Theo10011: I think so, AzaToth.

[4:44pm] Theo10011: Personal appeal is getting over-used.

[4:44pm] zackexley: AzaToth: what do you mean?

[4:44pm] zackexley: Ah

[4:44pm] AzaToth: zackexley: well, people in general have memory

[4:44pm] zackexley: that is a common mistake in this kind of fundraising or campaigning

[4:44pm] zackexley: you and I experience EVERY banner

[4:45pm] zackexley: because we're watching them all

[4:45pm] zackexley: but 99% of the readers -- maybe 99.9 -- barely notice them

[4:45pm] AzaToth: aint the 1% who have donated before most likely to donate again?

[4:45pm] zackexley: the personal appeal is not only the truest to our mission and values and movement -- it's also the most effective for those 99% who are noticing and clicking for the first time

[4:46pm] AzaToth: (and please, no _personal_ email this time)

[4:46pm] zackexley: AzaToth: no, they're not

[4:46pm] zackexley: only about 10% give again when we don't email

[4:46pm] AzaToth: the last one almost made me throw up

[4:46pm] Fae: Do we have anything on the cultural side, like the Indianapolis Children's museum <insert photo of delighted child>

[4:46pm] wctaiwan: What makes people who have used Wikipedia for years donate for the first time?

[4:46pm] zackexley: by emailing we can up that 10% to 20-30%

[4:46pm] zackexley: wctaiwan: They just finally click on that banner, read the appeal and decide to donate

[4:47pm] AzaToth: zackexley: at least don't make it a fake personal email this time

[4:47pm] zackexley: our research shows that most of our readers don't know that we're a non-profit

[4:47pm] zackexley: they think we're a for-profit company that is supported by ads.

[4:47pm] jsalsman: zackexley: please remind us all which 2 slogans beat the personal appeal banners

[4:47pm] zackexley: when they click and read the appeal, they learn for the first time what this movement is all about.

[4:48pm] zackexley: AzaToth:

[4:48pm] AzaToth: zackexley:

[4:48pm] zackexley: AzaToth: why do you say it was a fake?

[4:48pm] zackexley: Jimmy REALLY did write that email

[4:48pm] AzaToth: zackexley: well, it was "From" "Jimmy Wales", and it started with "Hello Carl"

[4:48pm] zackexley: he wrote it on a plane to Russia. I bugged him for months to write it, and he finally did

[4:49pm] AzaToth: zackexley: what I meant is that he didn't write it to me

[4:49pm] TheCavalry: ...but he personally wrote it

[4:49pm] AzaToth: but not to me

[4:49pm] mhernandez_: jsalsman, you can see the year-end banners here http://meta.wikimedia.org/wiki/Fundraising_2010/Report#Banners

[4:49pm] StevenW: But he can't actually write emails to thousands of people one by one....

[4:49pm] AzaToth: StevenW: then it shouldn't be personally addressed to me

[4:49pm] zackexley: that makes it a fake?  well, that's good feedback. Maybe we should take that out. But keep in mind that most of our donors aren't so critical of that kind of practice. They are used to the convention of a "Hello ___," in an email from an org they donated to

[4:49pm] wctaiwan: (does 'personal' mean that the email was written by a person rather than a PR team? or that the email is addressed to a person? To a reader, that is.)

[4:50pm] AzaToth: zackexley: perhaps I'm overly sceptic to letters like that ツ

[4:50pm] FT2: Personalized emails are usually accepted, in this culture anyway. Other cultures - I dont have that experience

[4:50pm] AzaToth: wctaiwan: well, both, but in this case, that is was addressed personally to me

[4:51pm] AzaToth: I'm Swede by the way

[4:51pm] jsalsman: mhernandez_: thanks; the excerpt is: In the last few days of the year we were able to find banner text that performed better than Jimmy's appeal banner. We saw two messages in particular that were performing well: "Only 4 days left in 2010 to make a tax-deductible contribution to keep Wikipedia free. Please help Wikipedia pay its bills in 2011." and "If everyone reading this donated $5 our fundraiser would end today. Please

[4:52pm] zackexley: so - one more thing on why the "personal appeal" works best in the banners. The one thing that most potential donors have in common with each other is that they love the site they're using and would like to know more about it.  That's why an offer of a letter from someone closely connected to the site is appealing.

[4:52pm] AzaToth: jsalsman: we could start with them ツ

[4:52pm] zackexley: and that's why the normal marketing slogans don't work

[4:52pm] FT2: basically the 2 exceptions were both time-limit based. "Tax year is coming so donating before matters, and fundraiser is ending, donate before it ends

[4:52pm] Philippe: and they were time boxed… the critical moment was within 48 hours.

[4:52pm] FT2: so people respond to a sense of deadline or urgency

[4:52pm] zackexley: So-- the "we could end it today" banner probably only works at the end when the claim is actually true

[4:53pm] AzaToth: "[COUNTER] until wikipedia shuts down, please donate to help us avoid that"

[4:53pm] jsalsman: AzaToth: well the thermometer / progress meter kind of performs the function of the first one, so it's a good idea to include that. I would like to see what the second one would be on the first day of a $20 million fundraiser

[4:53pm] zackexley: even if it worked better in the beginning, it wouldn't be good to use it until it was actually a true statement

[4:53pm] FT2: aza - no. If you do that then by the time it's true, its impact has worn off, over familiarity

[4:53pm] FT2: (I suspect)

[4:53pm] jsalsman: "If everyone reading this donated $20 million, the fundraiser would end in the next minute"

[4:53pm] AzaToth: haha

[4:54pm] zackexley: the other banner that worked better was "only 1 day left to make your tax deductible contribution" which only works in countries where tax deduction is available, and only will work in the last few days

[4:54pm] AzaToth: "If someone donated $30 million we wouldn't need to have an fundraiser, but the reality is that you all are needed"

[4:54pm] AzaToth: ±spelling

[4:54pm] zackexley: ALSO - those two banners only seemed to beat "appeal" by a little, and only when used together in an alternating banner

[4:54pm] TheCavalry: Most people in the UK today seemed to be assuming that Wikipedia was actually going to shut down if they didn't donate

[4:55pm] zackexley: AzaToth:

[4:55pm] zackexley: er

[4:55pm] jsalsman: this is why we need a constantly-running, year-round, 0.01% page view banner text measurement queue

[4:55pm] zackexley: AzaToth: we should test that one

[4:55pm] zackexley: we will

[4:55pm] ToAruShiroiNeko: hmm

[4:55pm] AzaToth: zackexley: fix the spelling then please

[4:55pm] FT2: point of information zack - something you said very early on.

[4:55pm] zackexley: will do!

[4:55pm] FT2: Around 1% of users will click on a banner and of those around 1% donate on the landing page.

[4:55pm] FT2: How do those %s compare to the rates we should be able to obtain, based on usual stats and allowing that we might have slightly above-average attractiveness to click or donate?  (due to the nature of the site/readers/appeal)

[4:55pm] ToAruShiroiNeko: I dont like "shuts down" kind of a thing

[4:55pm] AzaToth: FT2: statistics?

[4:56pm] ToAruShiroiNeko: I prefer a simple progress bar towards a goal

[4:56pm] wctaiwan left the chat room. (Quit: wctaiwan)

[4:56pm] harej left the chat room.

[4:56pm] FT2: click-through rates have been analyzed masively, whole iindustries are based on them. We should know if we are roughly in a "decent ballpark" or not. I don't have that knowledge so I thought it's worth asking.

[4:56pm] ToAruShiroiNeko: also please no creepy Jimbo photos 

[4:56pm] jsalsman: zackexley: if there's time for one more, what is the feeling on auctioning search failover link order between Bing and Google?

[4:56pm] rfarrand_ left the chat room. (Quit: Page closed)

[4:57pm] zackexley: The other thing about those two non "appeal" banners is that they might only work after a solid 5 weeks of the "appeal" -- by then, there are enough people who have clicked, read, understand what we're doing but still haven't moved to donate

[4:57pm] zackexley: jsalsman: do you really think that would be a good idea?

[4:57pm] jsalsman: I do!

[4:57pm] jsalsman: I miss search failover, I'm always retying

[4:57pm] jsalsman: retyping*

[4:58pm] zackexley: i think there'd be a strong consensus against becoming dependent on google/bing

[4:58pm] jsalsman: why not get them in a bidding war?

[4:58pm] lyzzy joined the chat room.

[4:58pm] jsalsman: presumably the term would be for a year, and they could bid again in a year

[4:58pm] jsalsman: maybe Yahoo will return to solvency, etc.

[4:58pm] FT2: zack - can we analyze some time, what features of a personal appeal pay off?  For example, do people give because someone wants to spread knowledge applicable to them, to say thankyou, to help developing cultures, etc?  Can we test correlations between "what a personal appeal is targeting" and the result?

[4:58pm] zackexley: So, one last thing about testing all the marketing-style banners…the hard part is that most of them would have to go to a custom appeal that is a logical fit with the banner. We couldn't send them to our appeals that are doing well.

[4:59pm] zackexley: and the appeals are doing most of the work -- not the banners -- in getting people to donate

[4:59pm] ppena left the chat room. (Quit: Leaving.)

[4:59pm] zackexley: so even a high-click rate banner that is not "appeal" (which is very hard to find) won't do well if it's not going to a great appeal letter

[5:00pm] jsalsman: zackexley: you have a generic landing page to test click through rates prior to their isolation as outliers

[5:00pm] ppena joined the chat room.

[5:00pm] StevenW: We should wrap up soon.

[5:00pm] zackexley: FT2: the problem is that all those kinds of things are dependent variables, interacting with so many things. But..YES we know somethings

[5:00pm] jsalsman: the click through rate is independent of the landing page text

[5:01pm] zackexley: quickly: no ads, spread knowledge, keep wikipedia free, WP is a public utility...

[5:01pm] Theo10011: hi ppena

[5:01pm] amuszalski joined the chat room.

[5:01pm] FT2: yeah, what Im wondering is what mix of "targets" makes for a good set of personal appeals (different people are motivated by different targets)

[5:01pm] FT2: so hopefully we get data on that some time.

[5:01pm] jsalsman: the amount donated s not independent of either banner text or landing page text

[5:01pm] zackexley: jsalsman: yes, click rate is independent, but a high click rate is useless if it has to go to bad appeal letter

[5:01pm] FT2: nod

[5:01pm] FT2: thanks

[5:01pm] Philippe: "Stay Curious"

[5:02pm] zackexley: and you can't surprise people with a mis-matched appeal.  Also -- appeal performance IS dependent on banner

[5:02pm] Theo10011: Guys, what happened to community involvement this year?

[5:02pm] Theo10011: banner submission or the like?

[5:02pm] jsalsman: if you don't isolate the high click through rate outliers, you'll never have the opportunity to see how well they perform with generic and custom landing page text

[5:02pm] Moonriddengirl: Theo, for one form of community involvement, the Storytellers have been busily interviewing community members.

[5:02pm] Seddon left the chat room. (Read error: Connection reset by peer)

[5:02pm] zackexley: Theo10011: We are interviewing a ton of community members and asking people through banners to submit stories

[5:03pm] Theo10011: zackexley, Moonriddengirl I meant actually getting the community to make banners and suggest stuff?

[5:03pm] zackexley: what we learned last year (as I've been explaining this whole hour) is that banner slogans are not a valuable place for ANY of us to put a ton of time

[5:03pm] jsalsman: I respectfully disagree, because they will always be the gateway to the landing page. And because four days is not a ton of time

[5:04pm] mhernandez_: here's where you can submit your story: https://wikimediafoundation.org/wiki/Stories/en

[5:04pm] zackexley: OH! One last thing: jsalsman, we WILL test a handful more of community banners from last year, just to be thurough

[5:04pm] zackexley: There are 10 or so in a queue that were chosen by volunteers last year

[5:04pm] zackexley: we'll test those

[5:04pm] jsalsman: thank you. I hope your hand can hold the 27 I submitted based on Sue's goals and overlooked esoterica

[5:05pm] jsalsman: they are linked from your talk page on meta

[5:05pm] jsalsman:        http://meta.wikimedia.org/wiki/Fundraising_2010/Messages/Mission#26_from_Sue.27s_goals_list_and_overlooked_esoterica    

[5:05pm] StevenW: Okay I think Zack has to go now.

[5:05pm] StevenW: Thanks everyone.

[5:05pm] Philippe: Thanks, Zack 

[5:05pm] FT2: thanks zack

[5:05pm] jsalsman: and one more at http://meta.wikimedia.org/wiki/Fundraising_2010/Messages/Mission#Make_the_Wikimedia_Foundation_more_useful bye!

[5:05pm] Moonriddengirl: Thanks, Zack!

[5:05pm] zackexley: thanks everyone!  And submit your story to that link megan just gave

[5:05pm] You left the chat by being disconnected from the server.