Grants:Project/Stephane/Kiwix/Timeline

This project is funded by a Project Grant

Timeline for Offline medical

Timeline	Date
Administrative logistics Kiwix-lib Kiwix-tools	End January
Kiwix-android own repo Kiwix-Android compilation against kiwix-lib Mobile layout Better dependencies support in mwoffliner	February
Multizim search fulltext index reduction Integration between suggestions and fulltext search Better specific language support	March
zimdiff/zimpatch merge geoloc search/index merge seek support (for video) kiwix-serve bytrange support Stub of unit-tests for zimlib	April
kiwix-serve with OPDS support Better metada in ZIM Example of OPDS client Full automation of custom apps creation Android improved video support	May
zim-plit tool Scheduling infrastructure for ZIM creation Mwoffliner video suport	June
Android multi-zim support Wikimed Mini (English) for Android Wikimed for iOS	July
Testing, debugging	August to ~~October~~ November

Monthly updates

Please prepare a brief project update each month, in a format of your choice, to share progress and learnings with the community along the way. Submit the link below as you complete each update.

See also milestones for a (slightly) more technically detailed overview.

January

Shared by email (no link) to task members. See below for summary.

Admin:

Awaiting clearance from Bank;

We have shared the issue with our current contractor, he’s okay with delaying his invoicing;

We recruited someone to join from 20 February until about Wikimania : contract remains to be signed (he had another opportunity at Mozilla but that did not pan out. Hopefully none of them changes their mind).

Dev:

Emmanuel has established a task plan that covers all development tasks until completion of the project;
Contacted by Srishti and Quim @WMF for tech support;
We’ve already asked a couple of volunteer devs to start booking time for the August hackathon in Montreal;
We’ve also included Wikimed-related tasks in our Spring Hackathon (Lyon, April);
We’ve touched base with WMAT in case anyone with our volunteers was interested in going to the MediaWiki hackathon in Vienna (May): we haven’t budgeted for it, but if anyone can go and help improve parsoid and/or the video extension, that’s a net gain;
Ongoing work: Kiwix-library. Kiwix tools almost done.

February

Admin: all clear

Banking issues have been cleared and first grant installment received (big up to Janice);
Started asking for quotes for iOS dev;
5 people on our side have asked for travel grants to attend the Vienna hackathon;
We're finalizing our schedule for the Wikimania hackathon: it looks like we will make it last until after Wikimania (tbc).

Dev: A short month. We're having our first delays in delivery, but as we were ahead of schedule in January things are still under control.

Upcoming OCG deprecation on the Foundation's side (https://phabricator.wikimedia.org/T150871) will negatively impact zim generation. It would be nice if zim rendering could be included upstream of the new tool (a decision to be made at Foundation management level, hence my flagging this here);
DONE: Kiwix-tools cross-compilation for Windows/Ubuntu;
DONE: Kiwix-build cross-compilation;
DONE: Kiwix-library upload of artefacts. We've started looking at Android use;
DONE: Kiwix-android split.

March

Admin: our first hackathon will take place next week in Lyon.

Two of our travel grant requests for the Vienna hackathon have been turned down, we still have three in stage two.

Tech: slight delays but moving quicker than expected on other issues (see Tech milestones).

[Foundation] the Audio/Video support patch has been under review for the past two weeks, so when this will be done we can start testing;
[Foundation] we need to know what the status with OCG (exporting books to zim) is. Looks like electron can’t help and we’ll need to work straight with OCG. It would be good to have a discussion with the product owner;
We’ve published a first docker image for the offliner, which in the long run will allow for automated update and uploads of our custom apps (starting with Wikimed);
We’ve discussed with James about the format of the app: we’ll have a core Wikimed with about 5’000 articles (most read/useful/complete) and then people will be able to download additional subjects (dentistry, physiology, etc.) which will then integrate seamlessly into the core (the other choice was to have separate apps for each subject). The current work on multizim search (which is slightly delayed btw) should allow this;
Delays on the implementation of mobile display. It works in theory... but failed our testing :-(

April

Admin: additional costs that were unbudgeted for.

The Spring hackathon was held at the beginning of the month. We haven’t received all expenses yet but considering it’s the biggest one we had and was unbudgeted for in the grant request, it will negatively impact our budget (it was worth it though: see [1] for achievements). Not as many people as planned will come to Wikimania though, so we’ll see how this evens out;
Three people will end up going to the Vienna hackathon, including one new recruit;
Montreal hackathon coming together[2].
Still looking for an iOS developer.

Tech: significant progress on all fronts (see Tech milestones).

[Foundation] The phabricator task for audio/video support has been closed. Thanks;
mwoffliner dependencies management done (a bit hacky, but it works);
We now have an automated android testing solution running on BitBar’s device farm;
We now have a working prototype of the zimfarm (meaning that zim files can be generated and uploaded automatically);
Kiwix-lib base running on macOS. Working on the same for iOS;
Search: fixed stemming and stop words.

Not in the grant request but impactful: mediawiki dependencies have been solved. We should be able to have a better mobile rendering soon enough.
Side benefit of the hackathon: we now have functioning Chrome and Firefox extensions. For a host of reasons this is not as good as having a full replacement for the desktop solution, but for the time being it’s an MVP and a good addition to our portfolio.

May

Admin: paperwork submitted

We filed our midpoint report and asked for a small grant extension to cover up the Lyon hackathon.

Tech: much progress in spite of a traditionnally short month in Europe (several bank holidays lining up nicely)

Mobile layout is out! The last bugs were cleared and the lighter design will roll out before Summer. You can get a sneak peak here (spoiler alert: it looks like the regular mobile view);
We have an early solution to generate the « core » articles for Wikimed (sorted by size, priority, etc.). We’ll start testing in the next few weeks;
Fixed a few bugs in the multizim search.
Now that the phabricator task has been fixed by the Foundation, we’ve started working on the video support.

Unrelated to this grant but mentioned before: the Chrome and Firefox extensions have been approved and released. Also of note : we’re invited for a Google Serve day at the end of June in Zurich. Around 10 Googlers will help us fix Android (and therefore Wikimed-related) bugs.

June

Admin:

Second half of the grant received. Thank you.

Tech

Officially released zim-lib
Parsoid now a local dependence in mwoffliner
Introduced audio/video support: it tends to increase zim file size quite a bit, so we won't use it directly everywhere.

July

Admin: nothing to report.

Tech

We now have a working version of mini-wikimed, a simplified app that only offers the leade and infobox for each article. This allows us to dramatically reduce the size of the app to 400Mb or less;
Last-minute show-stopper while we were about to release Kiwix 2.3 (and the corresponding Wikimed app): problem with the bookmark tool. Release will probably happen early August;
Checked relevance for search results;
Completed planning for libzim development;
Completed sorting algorithm to identify core (=biggest, highest rated, most read) articles.

August

Admin: unexpected booking issue raises costs

Had our booking cancelled the morning of our departure to Wikimania. Found a last minute replacement, which obviously ended up being a lot more expensive -> this will negatively impact our budget, but should nevertheless remain within acceptable limits as we saved on other costs. Still waiting for all expense reports to come in, we'll figure things out then.

Tech: not one but two hackathons' worth of work: pre- and post-conference (Montreal/Potsdam)

Updated all Wikimed apps (with new, simplified layout on the English one as per Abbey's recommendations);
Released WikiMed mini (EN);
Clean cross-compilation of libzim/kiwix lib for iOS/macOS;
iOS planning complete (est. release of iOS app: early October);
Clean video support in libzim/kiwix-lib/kiwix-serve;
Multithreading support added to libzim/kiwixlib;
Revamped mwoffliner;
Last rounds of zimfarm testing before releasing to production.

September

Admin: -

Tech: debugging in progress but heavier than expected. WikiMed iOS beta is out!

The iOS version of Wikimed is out for beta testing. If anyone is interested in having an early peek, drop us a note (or DM me directly);
Libzim file access/buffering revamping is completed. Libzim is also now threaded, ran unit testing successfully.
More bugs and issues than expected with multithreading, byte-range support, and video will however, probably cause delays.

October

Admin: New end date to November (see below)

Cleaned the WikiMed list of articles to remove biographies and help reduce file size.

Tech: More debugging

Started zimfarm deployment;
Started release of zimlib/kiwix-lib/kiwix-tools
Still a stopword bug in the search function. One big OPDS bug fixed;
Writing patches for iOS.

New end date

30 November 2017

Rationale

Integrating OPDS (basically the tool that manages the library) into Kiwix-serve and Kiwix-lib is a bit more time-consuming than expected. We need some more time for implementation and debugging. Stephane (talk) 09:34, 17 October 2017 (UTC)

Approving a new project end date of 30 November 2017. --Marti (WMF) (talk) 18:09, 18 October 2017 (UTC)

Your new final report due date 30 December 2017. -- JTud (WMF), Grants Administrator (talk) 19:20, 19 October 2017 (UTC)

Thank you. Stephane (talk) 08:53, 30 October 2017 (UTC)

9/30/18 end date approved

Per my post on the finances page, the new end date for this grant is September 30, 2018.

Is your final report due but you need more time?