taskManeage - Tasks: task #16270, Uninstall method for python +...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #16270: Uninstall method for python + python modules

Submitter:  Boud Roukema <boud>
Submitted:  Tue 04 Oct 2022 04:49:17 PM UTC
   
 
Should Start On:  Tue 04 Oct 2022 12:00:00 AM UTC Should be Finished on:  Tue 04 Oct 2022 12:00:00 AM UTC
Category:  Software Priority:  5 - Normal
Status:  None Privacy:  Public
Assigned to:  None Percent Complete:  0%
Open/Closed:  Open Effort:  0.00

Tue 18 Mar 2025 05:00:37 PM UTC, comment #3: 

There are at least two challenges to using "RECORD" to cleanly remove all previously installed files of a given software package:

  • three files in setuptools-75.8.0 have non-standard characters in the filenames (space, and parentheses '(' ')'); one file in scipy-1.15.2 has a non-standard character in the filename (space); other packages might also create files with non-standard characters;


  • if the previous partial install was incomplete, then the file RECORD may not exist, even though some of the old files exist.


The following patch should in principle solve the former of these two problems (a pyhook_before for scipy is not yet included), but the latter is not solved.


diff --git a/reproduce/software/make/python.mk b/reproduce/software/make/python.mk
index 871bad7..8db733b 100644
--- a/reproduce/software/make/python.mk
+++ b/reproduce/software/make/python.mk
@@ -210,6 +210,16 @@ $(ibidir)/python-$(python-version): $(ibidir)/libffi-$(libffi-version)
 # [7] https://ninja-build.org
 # [8] https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html

+#TODO: packagedir will not always be the maneage PACKAGE-$(PACKAGE-version)
+
+# WARNING (2025-03-18): Do not take the risk of adding 'r' (recursive) to
+# the 'rm' command for files listed in RECORD. A few rare cases of files
+# with unusual characters are known (setuptools-75.8.0: 3 filenames with '
+# ','(',')'; scipy-1.15.2: 1 file with ' '), and a python package with more
+# dangerous characters might be written. Conservatively, any files with
+# unusual names will have to be removed with an explicit exception with
+# 'pyhook_before' for building that package. Reporting the unusual filename
+# upstream should be considered too, to be helpful to the wider community.

 pybuild = cd $(ddir); \
        packagedir=$(strip $(2)); \
@@ -218,6 +228,20 @@ pybuild = cd $(ddir); \
        if ! $(1) $(tdir)/$$tarball; then \
          echo; echo "Tar error"; exit 1; \
        fi; \
+       printf "Will check files in $(ilibpymajorversion)/site-packages/$${packagedir}.dist-info/RECORD for removal ...\n"; \
+       if [ -e $(ilibpymajorversion)/site-packages/$${packagedir}.dist-info/RECORD ]; then \
+          for file in $$(cat $(ilibpymajorversion)/site-packages/$${packagedir}.dist-info/RECORD | \
+            awk -F ',' '{print $$1}' | sed -e 's;[^a-zA-Z0-9_./-];UNSAFECHAR;g'); do \
+              if [ -e $(ilibpymajorversion)/site-packages/$${file} ]; then \
+                 rm -fv $(ilibpymajorversion)/site-packages/$${file}; \
+              else \
+                 printf "The filename $(ilibpymajorversion)/site-packages/$${file}"; \
+                 printf " has unsafe characters and should be removed manually.\n"; \
+              fi; \
+            done; \
+       else \
+          printf "Record file $(ilibpymajorversion)/site-packages/$${packagedir}.dist-info/RECORD does not exist.\n"; \
+       fi; \
        cd $$packagedir; \
        if [ "x$(strip $(3))" != x ]; then \
          sed -e 's|@LIBDIR[@]|'"$(ildir)"'|' \
@@ -958,6 +982,14 @@ $(ipydir)/setuptools-$(setuptools-version): \
                      $(ipydir)/python-installer-$(python-installer-version)
        tarball=setuptools-$(setuptools-version).tar.lz
        $(call import-source, $(setuptools-url), $(setuptools-checksum))
+#      Remove old files with unsafe names manually
+#      setuptools-75.8.0 has three unsafe filenames
+       pyhook_before() {
+              printf "Setuptools-75.8.0 or similar: manually remove three badly named old files:\n"; \
+              rm -fv "$(ilibpymajorversion)/site-packages/setuptools/script (dev).tmpl"; \
+              rm -fv "$(ilibpymajorversion)/site-packages/setuptools/_vendor/jaraco/text/Lorem ipsum.txt"; \
+              rm -fv "$(ilibpymajorversion)/site-packages/setuptools/command/launcher manifest.xml"
+       }
        $(call pybuild, tar -xf, setuptools-$(setuptools-version), , \
                        Setuptools $(setuptools-version), GPEP517)


Boud Roukema <boud>
Group Member
Mon 17 Mar 2025 07:25:29 PM UTC, comment #2: 

There's a wider uninstall task at https://savannah.nongnu.org/task/?16287 .

Boud Roukema <boud>
Group Member
Thu 27 Feb 2025 01:36:51 AM UTC, comment #1: 

With the new gpep517/installer build system in task #16625, files such as

.build/software/installed/lib/python3.13/site-packages/astropy-7.0.1.dist-info/RECORD

appear to follow the PEP 517 [1] metadata requirement, providing a list of (apparently) all installed files (with their paths relative to the install directory) together with their sha256sums, except for the RECORD file itself.

This list of files should enable the use of 'rm -fv' or 'rm -f' without the '-r' recursive option, which is the big risk with the 'rm' command.

Although uninstalls are not a Maneage priority, most of the python installer backends appear to refuse overwriting at least some particular old files in the install target directory. The problem as described on 04 Oct 2022 remains a practical problem for anyone wishing to re-install one of the python packages or add a new one and fixing mistakes. It appears that a clean and safe solution is available - probably not much more complicated than:


rm -fv $$(cat /path/package.dist-info/RECORD | awk -F ',' '{print $$1}')


within a makefile rule.


[1] Python Enhancement Proposal 517 https://peps.python.org/pep-0517

Boud Roukema <boud>
Group Member
Tue 04 Oct 2022 04:49:17 PM UTC, original submission:  

When debugging or testing improvements to the python build system (python.mk) in Maneage, it would be good to have a clean method for uninstalling all python libraries and binaries and python module libraries and binaries.

My best recent attempt was

rm -fv .local/version-info/python/; rm -fv .local/bin/python; rm -fr .local/lib/python*; rm -fr .build/.cache/Python-Eggs/; rm -fv .local/version-info/proglib/python-3.10.6; rm -fv .local/bin/cy*; rm -fv .local/version-info/proglib/cy*; rm -fvr .local/include/python3.10* ; rm -fv .local/bin/f2py*

but on doing './project configure --existing-conf' I still got fatal errors of the style

FileExistsError: File already exists: ...


The only practical method I found of uninstalling python, in order to get a clean test of an update to python, was to remove the whole '.build/' tree.  (In principle, removing '.build/software/' should be enough.)

(In Debian, aptitude purge PACKAGE removes all traces of the package, including any configure files installed with PACKAGE and subsequently modified by the sysadmin. An alternative is aptitude remove PACKAGE , which does not remove configuration files.)

Boud Roukema <boud>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Carbon-Copy List
  • -email is unavailable- added by boud (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2025-03-17 boud Dependencies- task #16287 is dependent
    2025-03-17 boud Dependencies- task #16625 is dependent

    Back to the top

    Powered by Savane 3.15-e6e5.
    Corresponding source code