Grants:Project/Stephane/Kiwix/Final

This project is funded by a Project Grant

proposal

people

timeline & progress

finances

midpoint report

final report

Report accepted

This report for a Project Grant approved in FY 2016-17 has been reviewed and accepted by the Wikimedia Foundation.

• To read the approved grant submission describing the plan for this project, please visit Grants:Project/Stephane/Kiwix.
• You may still review or add to the discussion about this report on its talk page.
• You are welcome to email projectgrantswikimedia.org at any time if you have questions or concerns about this report.

Welcome to this project's final report! This report shares the outcomes, impact and learnings from the grantee's project.

Part 1: The Project

Summary

Kiwix is an offline solutions allowing its users to access internet content - e.g., Wikipedia- without internet access. Wikimed is the first "custom app" that we developed, focusing on Medicine-related content as curated by the Wikiproject Medicine. Wikimed functions as a standalone app that could be installed (at the time) on any Android device, taking up a fraction of the space necessary for the full Wikipedia (1.2 Gb vs. 65+ Gb). The app' success (more than 200k downloads in two years and 10 languages) shows that there is a strong demand for such content.

In spite of this, there were a series of issues remaining with the app:

• High resolution content (e.g. medical videos) could not be played;
• Display was not adapted to mobile devices;
• iOS versions were not available;
• With no automated system to update the corresponding .zim files, the App's release largely remained an amateur endeavour that had to be done largely by hand.

What we promised (Project Goals)

The overall goal was to increase the use of Wikimed and broaden access to Wikimedia contents as a whole: most of the improvements we proposed pertained to Kiwix core code itself and benefited the full Kiwix ecosystem.

Objective 1: Offer 3 different versions of Wikimed (nopic, standard, extended imageview); expand to iOS versions;

Done: We now offer all three versions as zim files which can be downloaded and read via Kiwix. The "standard" version is the one available on the standalone app, though we ended up publishing a "mini" Wikimed on Android following some interesting research we got from the Foundation. All three files (mini, standard, video) are available for all Kiwix users via the Kiwix library (i.e. along every other content we made available offline).

Objective 2: Support video and audio content. This goes hand-in-hand with MW developers solving ticket 64270 on Phabricator.

Done: That was the one part we did not control as it had to be solved Foundation-side, but we're happy to report that collaboration went super smoothly: thanks to everyone involved (there were many).

Objective 3: industrialize app release.

Done We've rolled out what we decided to call a "zimfarm": under this very rural name is now a very industrious slave process helping us with the generation of fresh zim files in a distributed manner (in layman's terms: we now more or less only need to press a button to get our zims, the slave doing all the background work of connecting to the right servers and making them work together).

What we delivered (Project Impact)

Important: The Wikimedia Foundation is no longer collecting Global Metrics for Project Grants. We are currently updating our pages to remove legacy references, but please ignore any that you encounter until we finish.

What we nailed (Targets)

Planned measure of success (include numeric target, if applicable)	Actual result	Explanation
Tech goals are listed here (29 in total)	all 29 solved.	(•_•) ( •_•)>⌐■-■ (⌐■_■)
100'000 Wikimed installs by EoY 2017	We passed the 270'000 mark \o/	This number does not include downloads of high-resolution zim files, which are harder to track, but does cover all 10 language versions of Wikimed (+mini EN)

The initial idea was to add video support for high-resolution content. However, in order to get there we first had to disentangle the blob of code that made up the Kiwix core: concretely, this meant inp particular building a separate Kiwix library (now zimlib and kiwix-lib). The continuous integration process we had to implement means that we now can publish daily nightlies and clean binary files: it is now easier to test, validate and merge volunteers' pull requests (less bottleneck -> faster work for everyone). Last but not least (at least for end users), multithreading now allows articles to load faster and opens up more improvements (see below "next steps").

Story time!

Old, non-responsive layout.

New Mobile view

We basically delivered what we had promised, and that's the bottom line: more people than expected now benefit from quality medical content, and in their own language. We've got a good relationship going with the Foundation, and we very much appreciate this. The more interesting anecdotes will be listed in the "Learnings" section below, but in the grand scheme of things this grant was a turning point for Kiwix: this money allowed us to do things that had been long overlooked or set aside in terms of code maintenance, and the difference between now and then is as much as that between night and day. So if there's any place to say it, this might as well be here: thanks to the grants committee and thanks to the Foundation staff.

This grant made a difference on how people can access knowledge, and having a robust app means that the effect will be long-lasting: our impact, therefore, should not be measured in bugs solved, improvements made or people reached this year, but should be seen in light of all those who will download Wikimed over the next few years: as one can see below, the people we reach aren't located in the most wired parts of the planet, and in the case of medical content there is a level of certainty that such knowledge means life. With all due respect, we do not think there's another project in the Movement that can honestly pretend it saved actual lives in so many places around the world. We have enough feedback from our users to think that we do.

An interesting side effect of spending some money to clean our code was that it made it more approachable and easier to handle for volunteers. We've started the process of moving from a one-size-fits-all blob of code to a series of neatly organized libraries that people can pick from when they want to work on a particular project (android, iOS, etc.). As a result of this increased ease of (re)use, volunteer activity has notably increased this year.

Numbers

We did not conduct a survey per se, but user comments -overall very positive- are for all to see on the apps' respective pages.

Better than survey are actual distribution numbers and app rankings: here is the breakdown by language/platform/mark at time of writing this report (end of December 2017):

Project	Downloads	Active users	App Store/Play Store mark	Main countries (>10%)
English (Android)	169'394	58'012	4.70	India/Pakistan
Arabic (Android)	36'084	9'239	4.62	Iraq, Yemen, Egypt, Algeria
French (Android)	22'691	11'357	4.68	Algeria, France
Spanish (Android)	7'130	2'557	4.71	Mexico, Cuba
Portuguese (Android)	2'469	386	4.62	Brasil
German (Android)	8'584	3'863	4.73	Germany
Farsi (Android)	7'883	2'733	4.58	Iran, Afghanistan
Orya (Android)	2'240	328	4.75	India
Japanese (Android)	1'245	603	4.89	Japan
Traditional Chinese (Android)	8'344	3'109	4.50	Taiwan, HongKong
English Mini (Android)	7'007	3'012	4.48	India, Pakistan
English (iOS)	1'780	569	n/a	USA, India

There are some variations, obviously, but the main takeaway for us is that Wikimed clearly is in demand in the Global South. Also, the average mark is above both Kiwix and the Wikipedia app (4.5 and 4.4 on the Play Store, respectively). At around 30%, retention is also markedly above market levels (20%), eventhough the app is consistently in the top 1% by file size. Last but certainly not least, Wikimed Mini has had a stellar start: with 7'000 downloads, it achieved in three months what the original Wikimed did in about a year - eventhough it scores lower in users' rating.

If we were to venture an analysis, we'd say that

1. Size does matter - following research shared by the Foundation we decided to set a ceiling at 100Mb for the mini app. This proved to be a winning bet.;
2. Focus does matter - a likely explanation for the consistently better marks is that if a medicine-inclined user were to download the full English Wikipedia (5M articles), only 1% of it would be useful. But with 50k articles solely focused on medicine, 100% of its content becomes suddendly relevant to her;
3. As expected, iOS is a niche/northern market.

Useful links (Project resources)

We broke down the Kiwix github repository into several more focused repos:

• https://github.com/kiwix/kiwix-android Where the android engine is specifically developed;
• https://github.com/kiwix/kiwix-android-custom Where all custom apps (e.g. Wikimed) are stored/improved;
• https://github.com/openzim/mwoffliner mwoffliner is the tool used to generate zim files from Mediawiki.

We did share occasional news on our Facebook and Twitter feeds, but more interesting/rewarding is the #userfeedbackoftheday hashtag, with quite a few specifically mentioning WikiMed. Our hackathon plans and achievements can be seen here (Spring) and here (Wikimania).

Downloads

• 
  Kiwix Android / iOS
• 
  Wikimed Android / iOS
• 
  Wikimed Mini

All the other apps benefited from the work we've done (starting with the mobile view), you can find them listed here

Learning

This was the first direct grant request from Kiwix, and lots of learnings came along the way. Almost nothing that we had planned happened as expected, and we're all the wiser for it.

It all started with registering the structure at the bank, which nearly didn't happen - apparently, compliance officers do check the links given to them, and the fact that Kiwix is being used in North Korea as a counter-propaganda tool did not go down well with them. Explaining that no, we weren't directly involved in subverting a foreign country's government took some time and delayed things by a month, until we finally got a green light. The first learning, therefore, is to keep a low profile (at least when opening a Swiss bank account which, in hindsight, does make sense).

The second learning came in April when we decided to do a hackathon in Lyons, France. The city is rather centrally located considering that most of our volunteers come from Europe: with 12 participants it was indeed our largest to date. Yet this was an unplanned, opportunistic move on our part, which we hadn't budgeted for in the grant request (we also sent two people to a Google-Serve event in Zurich so they could boss Googlers around for a day solving bugs). Hackathons are a great thing, if only because they create a sense of belonging among our volunteers (also, ours are not open bar but invite-only, meaning that people have to show some earlier work before getting freebies: at the end it ensured we got the best of the best, and the general mood during the week-long event was extremely studious). It also proved crucial in advancing certain tasks whose difficulty we'd underestimated: if not for Lyons, then we'd probably be even more late on our schedule than we ended up being. And here is a basic axiom of coding 101: anyone pretending "this is easy; at most a week's worth of work" is probably lying to your face (and to himself). The tech milestones we'd set were all completed, but not anywhere near the order expected - some tasks proved easier, some were a lot harder. Just like legal drinking age in Europe, milestones should therefore be seen as gentle suggestions rather than a list of hard deadlines.

We also had to face the fact that the event did impact our budget quite negatively (we basically had spent the Wikimania money before registrations for it were even open), but we have no regrets: we cut the number of participants invited to Montreal, and hoped for the best (a well-know budgeting technique). Our next grant request therefore includes an "unexpected and unforeseen spending" line that we took from the WikiAfrica team.

Talking of unforeseen events brings us to another learning: don't book AirBnB to save costs while there is a major conference happening. Or at least think twice about it and understand the risks: ours was cancelled the morning before our departure (some previous tenant visibly had left a faucet open or something). In a normal, business relationship, a hotel would try to relocate its angry customers. Not so much in the sharing economy: with such a short notice and no one to turn to (the landlady was genuinely sorry but evidently unhelpful) we basically had to fend for ourselves and take whatever was available - at a 2k extra cost vs. our initial booking . We're happy to report that the Holiday Inn across from the Conference venue was very comfortable - at least to the team members who didn't end up having to commute from outside of town.

Other recommendations

If you have additional recommendations or reflections that don’t fit into the above sections, please list them here.

• Accounting is a real job, and we clearly underestimated the amount of work needed to sort through expenses in several currencies over several months. If anything, we would have planned for higher PM costs.

Next steps and opportunities

The Wikimed Project was a great exercise to think how users interact with our content. A few ideas have popped, which will be rolled out sometime in the future (or be subject to another grant request). Among these:

• Core Wikimed and thematic extensions: we're more than ever looking to reduce file size - the crudest strategy for this was Wikimed mini, whereby we kept only the infobox and lead paragraph and removed everything below. It made the whole thing more approachable / easier to download, but at a cost: there was less information. The idea therefore would be to create an algorithm that would select the "must have" content and make it into a "core Wikimed" of 100 Mb or so (the maximum limit for comfort download, apparently). Then users would be able to download additional packages (e.g. dentistry, ophtalmology, etc.) into the app according to their own needs. We've laid out the groundwork for that one by developing multizim search: we'll see how we can keep building on it over the next few months.
• Reverse search: we think it could be possible to enhance the Kiwix search engine so that a user would type symptoms and then get the articles pertaining to the relevant conditions. One of our volunteers has started working on a proof-of-concept, so we'll see where that leads.

Part 2: The Grant

Finances

Actual spending

Please copy and paste the completed table from your project finances page. Check that you’ve listed the actual expenditures compared with what was originally planned. If there are differences between the planned and actual use of funds, please use the column provided to explain them.

	Approved amount	Actual funds spent	Difference
Software dev.	55,000 CHF	57,849 CHF	+2,000 CHF. Pretty much as expected, eventhough we had one extra month of work billed.
Hackathons	10,000 CHF	23,641 CHF	+13,641 CHF. The Lyons and Google serve event greatly increased our cost, though they had a much lower per participant cost basis.
Project Management	13,000 CHF	5,644 CHF	-7,356 CHF. We limited costs to infrastructure, no salaries/compensations involved (easier for our accounting).
Total	78,000 CHF	87,292 CHF	+9,292 CHF.

Overall ended up with 11.5% cost overrun, which can almost entirely be attributed to the Lyons hackathon and AirBnB failure in Montreal.

Remaining funds

Do you have any unspent funds from the grant?

Please answer yes or no. If yes, list the amount you did not use and explain why.

• no.

Documentation

Did you send documentation of all expenses paid with grant funds to grantsadminwikimedia.org, according to the guidelines here?

Please answer yes or no. If no, include an explanation.

• yes.

Confirmation of project status

Did you comply with the requirements specified by WMF in the grant agreement?

Please answer yes or no.

• yes.

Is your project completed?

Please answer yes or no.

• yes.

Grantee reflection

We’d love to hear any thoughts you have on what this project has meant to you, or how the experience of being a grantee has gone overall. Is there something that surprised you, or that you particularly enjoyed, or that you’ll do differently going forward as a result of the Project Grant experience? Please share it here!

The experience was great on our side, and the Foundation got excellent value for money. If anything had to be done differently then probably managing accounting would be it, as well as setting aside more time and money for unexpected delays and problems.