Tangent: is it worthwhile to establish a consensus for best practices with package pinning and package management for Python projects in the Wikimedia ecosystem? When I last worked on a python project (https://wikitech.wikimedia.org/wiki/Add_Link) I found it confusing that we have so many different tools and approaches for doing these things, and seems like we'd benefit from having a standard, supported way. (Or maybe that already exists and I haven't found it?)
Tangent: is it worthwhile to establish a consensus for best practices with package pinning and package management for Python projects in the Wikimedia ecosystem? When I last worked on a python project (https://wikitech.wikimedia.org/wiki/Add_Link) I found it confusing that we have so many different tools and approaches for doing these things, and seems like we'd benefit from having a standard, supported way. (Or maybe that already exists and I haven't found it?)_______________________________________________KostaOn 5. May 2023, at 13:51, Slavina Stefanova <sstefanova@wikimedia.org> wrote:Poetry is a modern lockfile-based packaging and dependency management tool worth looking into. It also supports exporting dependencies into a requirements.txt file, should you need that (nice if you want to containerize an app without bloating the image with Poetry, for instance).
https://python-poetry.org/--
Slavina Stefanova (she/her)
Software Engineer - Technical Engagement
Wikimedia Foundation_______________________________________________On Fri, May 5, 2023 at 1:38 PM Sebastian Berlin <sebastian.berlin@wikimedia.se> wrote:A word of warning: using `pip freeze` to populate requirements.txt can result in a hard to read (very long) file and other issues: https://medium.com/@tomagee/pip-freeze-requirements-txt-considered-harmful-f0bce66cf895.E-post/E-Mail: sebastian.berlin@wikimedia.seSebastian Berlin
Utvecklare/DeveloperWikimedia Sverige (WMSE)
Telefon/Phone: (+46) 0707 - 92 03 84_______________________________________________On Fri, 5 May 2023 at 13:17, Amir Sarabadani <ladsgroup@gmail.com> wrote:You can also create an empty virtual env, install all requirements and then dopip freeze > requirements.txtThat should take care of pinningAm Fr., 5. Mai 2023 um 13:11 Uhr schrieb Lucas Werkmeister <lucas.werkmeister@wikimedia.de>:For the general case of Python projects, I’d argue that a better solution is to adopt the lockfile pattern (package-lock.json, composer.lock, Cargo.lock, etc.) and pin all dependencies, and only update them when the new versions have been tested and are known to work. pip-tools can help with that, for example (requirements.in specifies “loose” dependencies; pip-compile creates a pinned requirements.txt; pip-sync installs it; pip-compile -U upgrades requirements.txt later; you check both requirements.in and requirements.txt into version control.) But I don’t know if that applies in your integration/config case.Am Do., 4. Mai 2023 um 18:08 Uhr schrieb Antoine Musso <hashar@free.fr>:_______________________________________________Hello,
This is for python projects.
Today, May 4th, urllib3 has released a new major version 2.0.2 which breaks the extremely popular requests library.
The fix is to pin urllib3<2 to prevent the new major version from being installed (example).
https://phabricator.wikimedia.org/T335977
Upstream issue: https://github.com/psf/requests/issues/6432
Antoine "hashar" Musso
Wikimedia Release Engineering
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
--_______________________________________________Lucas Werkmeister (he/er)
Software Engineer
Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin
Phone: +49 (0)30-577 11 62-0
https://wikimedia.de
Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision!
https://spenden.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
--_______________________________________________Amir (he/him)
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/
Wikitech-l mailing list -- wikitech-l@lists.wikimedia.org
To unsubscribe send an email to wikitech-l-leave@lists.wikimedia.org
https://lists.wikimedia.org/postorius/lists/wikitech-l.lists.wikimedia.org/