Grants:Project/Maxlath/WikidataJS/Midpoint

This project is funded by a Project Grant

Report accepted

This midpoint report for a Project Grant approved in FY 2017-18 has been reviewed and accepted by the Wikimedia Foundation.

To read the approved grant submission describing the plan for this project, please visit Grants:Project/Maxlath/WikidataJS.
You may still review or add to the discussion about this report on its talk page.
You are welcome to email projectgrantswikimedia.org at any time if you have questions or concerns about this report.

Welcome to this project's midpoint report! This report shares progress and learning from the grantee's first 6 months.

Summary

In a few short sentences or bullet points, give the main highlights of what happened with your project so far.

Following the needs expressed by the community, many developments and bug fixes happened on the different tools. Hereafter, a brief overview of the activity since the grant started in May 2018

Methods and activities

The 4 tools repository have known various levels of activities, mostly following requests expressed in each repositories issues by the community.

wikidata-sdk

More utils functions were added to work with simplified qualifiers and work with claims GUIDs.

wikidata-edit

Working toward feature completion, now getting into more subtle parts of the Wikidata data model, such as statement ranks

wikidata-cli

Features from wikidata-sdk and wikidata-edit are being progressively integrated and used to provide new features, such as the new wd convert, that can be used to convert batches of external ids to Wikidata ids and vice versa. There is a growing demand for this tool to cover parts of the Wikidata data model other tools don't cover, such as setting special snaktypes or rank: those features largely depend on wikidata-edit progresses, and progressively integrated as they become available.

wikidata-filter

The filter seems to have reach a good level of maturity, there were some requests for performance improvement in some edge cases that couldn't be met yet.

Midpoint outcomes

Out of statistics on the number of commits, issues, and pull requests, it is hard to access the exact outcomes of those developments, as good metrics on the use made of those tools are hard to get:

wikidata-edit and wikidata-cli edits on Wikidata now include a signature in the edit summary (starting by #wikidatajs), but unfortunately, we couldn't figure-out a way to retrieve a list of those signed edits
we can recover edits and creation made on Wikidata by a given OAuth consumer such as inventaire.io though
download counts are interesting but, out of CLI tools, should not be considered an accurate measure of the use of a tool:
- one download of wikidata-sdk or wikidata-edit might result in many visitors, users, contributors, benefiting from the tool's possibilities
- automated tools (such as CI tools) downloads are also included

metrics

wikidata-sdk downloads

Chart comment: the reason for the bump in downloads in the first week of May 2018 isn't identified

wikidata-edit downloads

wikidata-cli downloads

wikidata-filter downloads

Finances

Please take some time to update the table in your project finances page. Check that you’ve listed all approved and actual expenditures as instructed. If there are differences between the planned and actual use of funds, please use the column provided there to explain them.

Then, answer the following question here: Have you spent your funds according to plan so far? Please briefly describe any major changes to budget or expenditures that you anticipate for the second half of your project.

Financial plans are going as planned: the largest share of the work time is passed on code developments and support requests
Communication efforts have been mostly social media presence and updates in the Wikidata newsletter, and being present to answer question during WikiCite 2018
Communication in 2019 is likely to further intensify with possible presentation of the tools during Hackathon 2019, and possibly Wikimania 2019

Learning

What are the challenges

What challenges or obstacles have you encountered? What will you do differently going forward? Please list these as short bullet points.

With a growing set of features, some architectural choices become increasingly painful: the is mostly sensible for wikidata-edit, due to the added complexity required by the authentification workflow. A refactor might be needed to increase lower the complexity of the code and thus increase the maintainability of the tool.
Regarding the grant itself, it is hard to keep up with the planned features, as new requirements coming from the community make the priorities evolve. It is also hard to track the time used for one feature or the other, as the time writing the code is just the tip of the iceberg, making reporting efforts feel approximative. The timeline as been used to keep track of developments and try to make this work visible.

What is working well

What have you found works best so far? To help spread successful strategies so that they can be of use to others in the movement, rather than writing lots of text here, we'd like you to share your finding in the form of a link to a learning pattern.

communication through Github as been working quite well: more contributors have been opening issues and pull requests than before
downloads stats seems to be on an upward trend
while hard to precisely measure, edits made with wikidata-edit and CLI are on the rise

Used learning patters

Learning patterns/Git repository for software: tracking code changes using git and a forge such as Github is a must! We should nevertheless consider moving to libre/free software platforms, such as GitLab, which offer close to equivalent features.
Learning patterns/Features requests for a software: the issue tracker is the heart of the communication between the developers and the community

Next steps and opportunities

What are the next steps and opportunities you’ll be focusing on for the second half of your project? Please list these as short bullet points. If you're considering applying for a 6-month renewal of this grant at the end of your project, please also mention this here.

getting ranks features to work from wikidata-edit and cli, getting those edit tools closer to feature parity with the API
considering enhancing wikidata-cli scripting workflow with edit groups, so that a batch of edits can be documented and eventually reverted

Grantee reflection

We’d love to hear any thoughts you have on how the experience of being an grantee has been so far. What is one thing that surprised you, or that you particularly enjoyed from the past 6 months?

Working with community issues is definitely the most gratifying, as it reflects on the real use-cases those tools are used for, and the work put into answering those is directly validated by the request.
This is contrasting with other developments that often seem to pass under the community radar, giving the feeling that the tools potential isn't used at it's maximum. For instance, I was expecting that the combination of features such as wd sparql and wd data to generate partial dumps (example) would have a larger echo.
This kind of feature always make for a great demo though