Wikidata For Wikimedia Projects/AddIssue/Automated links to Wikipedia

Automated links to Wikipedia

Description of issue

On Dutch Wiktionary I made a template that automatically links books/movies/writers/publishers/etc om Wikipedia. For example, {{Wikipedia artikel link|schrijver|Max Jacob}} results in Max Jacob. This part relies only on data imported from petscan.

Wikidata is however used to identify publishers/websites by domain. For example, {{Wikipedia artikel|websitedomein|gutenberg.org}} returns w:nl:Project Gutenberg. In this case, that information is stored in wikt:nl:Sjabloon:Wikipedia artikel/nlwiki-websitedomein-all.*

There's no real issue here, though the scraping process is kinda slow. I need sitelinks+P856 for (depending on the wiki) hundreds or thousands of Q-numbers that Petscan gave me. Example API request. Doing this for 50 Q-numbers at once doesn't work, the JSON get truncated sometimes and is extremely slow to process on the client side. So I'm only getting 10 Q-numbers at a time.

All I need are the sitelinks and P856, but I'm getting a ton of other claims too that I simply discard. https://www.wikidata.org/w/api.php?action=help&modules=wbgetentities has no filter options for claims. Luckily I don't have to run this very often.

* I'm aware of the data: namespace. It doesn't perform any better, can't be kept locally and counts the parsed size as the page size further limiting its usefulness.

Affected Wikimedia Project(s)

Dutch Wiktionary

Phabricator Ticket

phab:T243138 from 2020 and phab:T93519 from 2015 seem to cover it.

Submitted By

— Alexis Jazz (talk or ping me) 01:39, 4 April 2024 (UTC)