Grants:IEG/Wikimaps Warper 2.0/Midpoint

This project is funded by an Individual Engagement Grant

Welcome to this project's midpoint report! This report shares progress and learnings from the Individual Engagement Grantee's first 3 months.

Summary

In a few short sentences or bullet points, give the main highlights of what happened with your project so far.

The broader goal of the project is to advance the use of historical maps as a source of information in Wikimedia products. This project focuses on Wikimaps Warper, a tool that aligns the scanned, historical map with geographic coordinates. More specifically, we work to make the code of the tool usable through an API. This will make it easier for front end developers to create better user interfaces or external applications to take advantage of the project.

Wikimaps Warper code refactoring made by Tim Waters
Enhancing the upload process for maps via the Metapipe project by Ari Häyrinen

Methods and activities

How have you setup your project, and what work has been completed so far?

Describe how you've setup your experiment or pilot, sharing your key focuses so far and including links to any background research or past learning that has guided your decisions. List and describe the activities you've undertaken as part of your project to this point.

In this project, we focus on code development for the Warper tool. In the first part of the project, we have invited a group of advisors/contributors to reflect on our plans.

The advisors

We decided to call all our contributors advisors instead of making a division between advisors and volunteers. We invited them all in the "General Assembly" and will continue individually with each of them. The following people are following up the process to varying degrees.

The New York Public Library: Matt Knutzen, Bert Spaan

Have initiated the original MapWarper. Surfacing the needs that appeared over the course of years and aligning with the original Warper are the key areas where we collaborate. We also bring maps and rectification data to the Warper at Wikimedia.

The British Library: Nora McGregor

The British Library has georeferenced a lot of maps with the aid of a different tool. With BL we aim to facilitate the path of maps and control points from that environment to the Warper at Wikimedia, making test batch uploads together.

Wikimedia maps: Yuri Astrakhan

We ask Yuri's advice to keep the development aligned with the Wikimedia maps initiative

The Wikimedia community: Alex

Alex works in the German mapping community in Wikimedia, and he has created a display of historical maps georeferenced with the Wikimaps Warper.

Broader context: Tod Robbins, Shekhar Krishnan, Vahur Puik

We gain a lot of insight from people who have significantly contributed to crowdsourcing environments. Shekhar Krishnan is an original project manager for the MapWarper project, Vahur Puik runs a very popular photo georeferencing site Ajapaik.ee, and Tod Robbins participates in several crowdsourced cultural heritage project across the globe.

Transferring and evaluating the vision

Mockups: Future interface mockups and feature sketches have been produced to give an idea of what we are aiming for with the project. The UI will not be completed during this phase, however.
Survey: Vote for features that you find most urgently needed in AllOurIdeas

Warper development

Code repository: The code is shared in GitHub
Dev log at Commons:Wikimaps: Logged entries of development activities

Developing supporting workflows

Metapipe: The Metapipe project is Ari Häyrinen's contribution to the project. Matapipe is a tool to take a large metadata set, clean and transform it to a suitable format such as a Commons template and upload the files to Commons.
The Map template: The Map template holds all relevant metadata for a map. We enhance it to communicate data between the upload, the Warper and also Wikidata.
Bot/Gadget to mass-insert Map templates to map images in Commons: Since not all map images are equipped with Map templates, we are creating ways to insert it to many images at once.

Testing the environment with pilot content

The British Library: The War Office East Africa Maps
New York Public Library: Transferring and synchronizing content, data and software features.
Linked Data Finland: Linked Data Finland has rectifies The Senate Atlas and another set of Karelian maps in collaboration with the Wikimaps project. The next task is to transfer the control points from their MapWarper instance to the Wikimaps Warper.
Ajapaik: Ajapaik hosts an impressive amount of geolocated images, part of which can be uploaded to Wikimedia Commons. We will look into the workflow of uploading content from an archive and making available the Ajapaik environment for them while being connected to Wikimedia Commons. This happens mainly in another project, and will address it here time allowing.

Learning by doing in workshops

Workshop initiatives for Wikimania and the Swiss Cultural Hackathon: We have prepared a proposal for a historical maps workshop for Wikimania, and would like to combine it with other maps workshop initiatives, the Wikimedia maps and Wikidata proposals.

Related activities

Historical map data in Wikidata: We have drafted how to store map metadata in Wikidata. We invite you to participate in defining the properties!
WikiProject Historical Place in Wikidata: We have also initiated a discussion among historical geodata scholars and Wikidatans about how to store the changing names and boundaries of places.

Midpoint outcomes

What are the results of your project or any experiments you’ve worked on so far?

Please discuss anything you have created or changed (organized, built, grown, etc) as a result of your project to date.

New features developed, partly before the grant period

Oauth authentication - leading to the development of a Devise / Omniauth OAuth strategy for mediawiki. see blog post.
Commons category import - a Category of maps can be imported all at once into the warper.
Import of batch control points - a CSV of control points for many maps can be imported.
Download of control points - control points can be easily downloaded for a map
Bot edit of wiki page to update bounding box / extent and status - when warped, the warper will act as a bot and update the commons wiki page with the results of the bounding box.

Blog posts we have written: Wikimaps Warper 2.0 - Susanna Ånäs; strace and ruby-prof to identify slow rails startup on wikimaps - Tim Waters; Introducing MetaPipe - Ari Häyrinen
Blog posts about Wikimaps: Wikimaps in MapsMania - Tim Waters; Así trabajan los cartógrafos de Wikipedia: "Acaba convirtiéndose en un vicio" - Lucía El Asri
Hangouts: We hosted the "General Assembly" for our advisors. This is the agenda/documentation of the meeting.
Mockups: A more extensive mockup document; Brief presentation for the advisor Hangout, including UI mockups, instructional videos and drafts for future functionality
Instructional videos: From Wikimedia Commons to the Warper; Import a category of maps into the Warper; OpenHistoricalMap editor inside the Warper
Survey: Survey to vote for the most urgent issues, and to propose your own.

Finances

Please take some time to update the table in your project finances page. Check that you’ve listed all approved and actual expenditures as instructed. If there are differences between the planned and actual use of funds, please use the column provided there to explain them.

Then, answer the following question here: Have you spent your funds according to plan so far? Please briefly describe any major changes to budget or expenditures that you anticipate for the second half of your project.

The project costs have covered work by each project participant as outlined in the report. The same will apply to the remaining funds with the exception of 500 euros for administrative costs. The administrative costs have not been specified, but they are intended to cover costs of presenting the project or administering the funds. We are requesting to use the funds for participation in Wikimania, to cover the costs of Albin Larsson and/or Susanna Ånäs with the amount of 500 euros. The remaining costs not covered by this 500 euros will be sought elsewhere.

Learning

The best thing about trying something new is that you learn from it. We want to follow in your footsteps and learn along with you, and we want to know that you are taking enough risks to learn something really interesting! Please use the below sections to describe what is working and what you plan to change for the second half of your project.

What are the challenges

What challenges or obstacles have you encountered? What will you do differently going forward? Please list these as short bullet points.

Keeping in contact with all our advisors is a big task, since we have invited so many. They are all very important to us and the project, and we hope to communicate with everyone to the extent they have time available. I cannot correct it in this project, but for the next 6-month grant I would have plans for communication ready before the grant starts.

What is working well

What have you found works best so far? To help spread successful strategies so that they can be of use to others in the movement, rather than writing lots of text here, we'd like you to share your finding in the form of a link to a learning pattern.

About fostering relationships with memory organizations in Towards a Successful GLAM Partnership by Barbara Fischer (WMDE) and Lilli Iliev (WMDE)
About the use of templates in photographic uploads in Working with cultural institutions to upload archival photographs by Harej.

Next steps and opportunities

What are the next steps and opportunities you’ll be focusing on for the second half of your project? Please list these as short bullet points. If you're considering applying for a 6-month renewal of this IEG at the end of your project, please also mention this here.

Provide a test environment, get feedback
Carry out pilot uploads
Wrap up, documentation, dissemination
Create the workshops for Wikimania
Plan next steps:
- Create the user interface for Wikimaps Warper, and establish a site / hub / address for it and other tools added in the future
- Connect historical maps to Wikidata, store historical geodata in Wikidata
- Create workflows to insert annotated, layered historical maps on Wikipedias
- Support the development of OpenHistoricalMap, an OpenStreetMap for historical geodata

We are interested in grant renewals for one or more of the above mentioned areas.

Grantee reflection

We’d love to hear any thoughts you have on how the experience of being an IEGrantee has been so far. What is one thing that surprised you, or that you particularly enjoyed from the past 3 months?

I am very happy about the fact that institutions and individuals are excited about the development and provide their assistance. I am also very happy about our team, who work separately on different areas, but can together tackle a complex set of challenges. Marti Johnson has been very constructive and helpful as a Grans Officer, so thank you for that! / Susannaanas (talk)