Toolhub/Progress reports/2022-02-18
Report on activities in the Toolhub project for the week ending 2022-02-18.
A mysterious build system error arrives and is fixed
editOn 2022-02-14 Slavina did a normal make restart in her local development environment and was surprised to find the "web" container failing to build. The error message she saw was "AttributeError: 'PathDistribution' object has no attribute '_normalized_name'". Raymond and Bryan were not able to reproduce the problem in their local environments, but Bryan started trying to find a likely cause. Not long after we saw the same failure in the Jenkins jobs for Toolhub. This helped us confirm that the problem was not local to Slavina's laptop.
Bryan noticed that the base Docker image used to build our web container had been updated the previous day which seemed to be suspicious. Based on a hunch, Bryan asked Slavina to try making an edit to her local .pipeline/blubber.yaml file which is used to generate the Dockerfile instructions for building the container so that it would use the prior base image (python3-buster:0.0.1-20220206). This change did fix Slavina's local environment (yay!), but when Bryan tried to do the same thing in his local environment he found that it caused the problem seen by Slavina and Jenkins to start happening for him too.
The error being fixed for Slavina by pinning to an older base image and starting for Bryan with the same pinning seemed like a clue to examine more closely. This behavior actually also matched Bryan's initial hunch that reusing a cached Docker image "layer" would fix Slavina's problem temporarily. Pinning to an older version that she had a locally cached layer for kept the error from happening not because of a change in the base image, but because of another upstream change in the setuptools library. JJMC89 noticed around the same time that the same "AttributeError: 'PathDistribution' object has no attribute '_normalized_name'" error had been reported in their bug tracker. This error had started happening for folks with the setuptools 60.9.0 release which had happened within the prior 24 hours. This fell nicely between when Slavina had first seen the error and the last time that Jenkins had successfully run tests for Toolhub.
Closer examination of the Blubber generated Dockerfile pointed to the instruction RUN python3 "-m" "easy_install" "pip" && python3 "-m" "pip" "install" "-U" "setuptools" "wheel" "tox" "pip" as the cause for installing the buggy setuptools. Environments where this instruction found a cached output layer to reuse (avoiding the new and buggy setuptools 60.9.0) worked. Environments where the instructions were executed failed.
We ended up doing three things to fix the problem:
- We temporarily pinned our base image to python3-buster:0.0.1-20220206 which both Slavina and Jenkins had locally cached layers for.
- We also patched Blubber to prevent it from ever installing the buggy setuptools 60.9.0 version.
- Once setuptools 60.9.1 was released we switched back to using the python3-buster:latest base image tag.
Thank you to Jeena for her help in rolling out the Blubber change.
Make all vuex actions that make API requests return a promise
editRaymond fixed some technical debt by updating all of our existing vuex actions which make API requests to follow a pattern of returning a Promise object. This is a pattern that we adopted sometime in the past 6 months or so, but had not retroactively applied everywhere until this week. The pattern makes it much easier for Vue components and other callers to "chain" additional business logic to the vuex call.
Always cast suffix passed to getMetaTitle as String
editBryan ran into a known issue with our frontend getMetaTitle() helper related to it's use of the banana-i18n library's bidi instruction. It turns out that banana-i18n assumes that any argument wrapped with ... is a string. In some cases Toolhub is passing integers as this argument. This was previously noticed and worked around with an explicit cast of the integer to a string in ListHistory.vue. When he ran across the same problem in another Vue component, Bryan decided to fix this inside getMetaTitle() instead.
N-gram tokenizers for autocomplete searches
editRaymond's patch adding 2-gram and 3-gram indices for autocomplete searches has been merged. This change will make the "search as you type" lookups that have been added in various places more likely to find matches. Previously searches would not match anything until the user's input matched a word stem. Now matches can happen with as few as two letters typed. This more closely matches the search behavior of Hay's Directory and should be slightly more intuitive for users.
UI to toggle "featured" status of a published list
editSlavina continues to work on missing features for list management in Toolhub. gerrit:762809 has added a button to the detail screen for published lists to allow users with the appropriate rights to toggle the list's "featured" status. Featured lists are shown on the Toolhub landing page.