bugmake - Bugs: bug #44853, gmake: execvp: bash: The parameter...

 
 

bug #44853: gmake: execvp: bash: The parameter or environment lists are too long.

Submitter:  Daniel Richard G. <iskunk>
Submitted:  Thu 16 Apr 2015 05:37:23 AM UTC
Votes: 1
 
Severity:  3 - Normal Item Group:  Bug
Status:  None Privacy:  Public
Assigned to:  None Open/Closed:  Open
Component Version:  4.1 Operating System:  POSIX-Based
Fixed Release:  None Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 27 Jun 2021 11:28:40 AM UTC, comment #9: 

related/duplicate: #45763

I would like to add that (to my surprise) even the workaround offered by the file function (and specifically documented for this purpose in section 8.6 of the manual, with example recipes) may not work:


    program: $(OBJECTS)
             $(file >$@.in,$^)
             $(CMD) $(CMDFLAGS) @$@.in
             @rm $@.in


It seems like the reason for me was that the makefile contained an export directive (for all variables), which causes OBJECTS getting appended to the envp of the execve call of the recipe. Since E2BIG is caused by either the argv or the envp argument, the result is the same (just the error message is confusing).

So as long as make has no general solution for E2BIG, I recommend at least documenting that .EXPORT_ALL_VARIABLES or export will break the file function workaround.

Robert Sachunsky <bertsky>
Tue 26 May 2015 03:32:23 PM UTC, comment #8: 

SHELL=perl:

    .SHELLSOURCE = require "$(.SHELLSCRIPT)";

SHELL=ruby:

    .SHELLSOURCE = require "$(.SHELLSCRIPT)"

If SHELL doesn't have an include/source directive, then there's a challenge---but no worse than the E2BIG.

One issue with using a construct that is parallel to .SHELLFLAGS rather than additive is that if you add e.g. -e to .SHELLFLAGS, then the parallel form won't necessarily pick it up.

Daniel Richard G. <iskunk>
Tue 26 May 2015 12:06:01 PM UTC, comment #7: 

No automated conversion of the command is going to work: you point out csh (which shouldn't be used with make actually) but what if SHELL is Perl?  Or Ruby?  Or...

If you're going to introduce a new variable anyway, it seems like it would be more straightforward to introduce an alternative to SHELLFLAGS that is used when make decides to create a batch file, rather than a variable containing a "source" operation.

Paul D. Smith <psmith>
Group administrator
Tue 26 May 2015 04:45:37 AM UTC, comment #6: 

Well, it would be possible to transform

    $(SHELL) -c "very long command"

into

    [write "very long command" to /tmp/long.sh]
    $(SHELL) -c ". /tmp/long.sh"

so mucking around with .SHELLFLAGS is unnecessary.

The syntax for sourcing a script may vary (e.g. csh would use "source" rather than "."), but a new variable could be provided to control that, like

    .SHELLSOURCE = . $(.SHELLSCRIPT)
    .SHELLSOURCE = source $(.SHELLSCRIPT)

Would this address your concerns?

Daniel Richard G. <iskunk>
Sat 23 May 2015 03:34:11 PM UTC, comment #5: 

This is not so easy.  Make has exactly one way to invoke an interpreter today: essentially '$(SHELL) $(.SHELLFLAGS) <recipeline>"  The setting of .SHELLFLAGS is geared towards this configuration.

Many/most interpreters require a different set of flags to process a file than they do to process a script via the command line, and there is no facility in make today that would allow one to provide those alternative flags.  It's not enough to just assume you can remove the "-c" option and call it good.  Any solution to this problem would need to provide a solution to this aspect as well.

Paul D. Smith <psmith>
Group administrator
Fri 22 May 2015 09:05:12 PM UTC, comment #4: 

I have filed the bug report against Libtool:

    http://debbugs.gnu.org/20632

Libtool can handle overlong object lists capably, but the generated Make rule in its current form will not allow it.

Daniel Richard G. <iskunk>
Fri 22 May 2015 07:42:05 PM UTC, comment #3: 

I'm afraid this is a project using Automake-generated makefiles, and it has to support non-GNU Make programs. If I had full control over the rules, I could certainly go a different route.

(This issue will, in fact, likely require a companion bug to be filed in Libtool. As I found using this hack, even if Make doesn't choke executing that very long command, the interpreter running the libtool script will.)

Daniel Richard G. <iskunk>
Fri 22 May 2015 06:47:23 PM UTC, comment #2: 

Given that you already have GNU make 4.1, any chance you can use the new $(file ...) function (http://www.gnu.org/software/make/manual/make.html#File-Function) to do the same thing in your own recipe? It could be a path of much less resistance.

David Boyce <boyski>
Fri 22 May 2015 03:56:36 PM UTC, comment #1: 

I've confirmed that the problem is a too-large argument to "$SHELL -c". The execvp() call is failing with E2BIG.

I've put together a workaround hack for this issue. A patch against git master is attached (which also applies to 4.1).

This adds a handler for E2BIG in exec_command(). The error occurs when executing

    argv[0] == $SHELL
    argv[1] == "-c"
    argv[2] == "way too much shell code"

so it writes argv[2] to a temporary file, and then calls exec_command() recursively to invoke

    argv[0] == $SHELL
    argv[1] == "/tmp/gmake-e2big.XXXXXX"

This is only a quick-and-dirty hack, however, because (1) the temp file is not cleaned up, as exec_command() does not return, and (2) the code only works in the specific case of a "$SHELL -c '...'" invocation rather than generally.

A real fix may require changes outside of exec_command(), in the places where the exec argument vector is constructed to begin with.

(file #34079)

Daniel Richard G. <iskunk>
Thu 16 Apr 2015 05:37:23 AM UTC, original submission:  

I am attempting to build a project on AIX 4.3.

The makefile is specifying a library target that has a large number of objects (1000+). GNU Make prints the (very long) command that it is about to execute, but then, it returns

    gmake: execvp: bash: The parameter or environment lists are too long.
    Makefile:16554: recipe for target 'libfoo.la' failed
    gmake: * [libfoo.la] Error 127

This is with GNU Make 4.1, and GNU Bash 4.3.30.

I suspect that GNU Make is passing the recipe command as a direct argument to "bash -c", which then exceeds the system's limits on execvp(). What is probably needed is to place the command in a temporary file, and then pass that file to the shell for execution.

Daniel Richard G. <iskunk>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #34079:  make-e2big-hack.patch added by iskunk (963B - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by bertsky (Posted a comment)
  • -email is unavailable- added by bertsky (Voted in favor of this item)
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by boyski (Posted a comment)
  • -email is unavailable- added by iskunk (Submitted the item)
  •  

    There is 1 vote so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-06-27 bertsky Carbon-Copy- Added bertsky
    2015-05-22 iskunk Attached File- Added make-e2big-hack.patch, #34079

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code