bugGNU Octave - Bugs: bug #52414, llvm 4.0.1 fails on ./configure...

 
 

bug #52414: llvm 4.0.1 fails on ./configure --enable-jit

Submitter:  Michael Godfrey <godfrey>
Submitted:  Wed 15 Nov 2017 05:18:54 PM UTC
   
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Build Failure
Status:  Fixed Assigned to:  None
Originator Name:  Godfrey Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 16 Jan 2019 07:22:12 PM UTC, comment #18: 

I opened bug #55492 for the crash during "make check".

The JIT compiler seems to work. With your test, I see:

>> jit_enable (false)
>> tic;  x = 0;  for i=1:1e6,  x = x+1;  end, toc, x
Elapsed time is 0.940764 seconds.
x =  1000000
>> tic;  x = 0;  for i=1:1e6,  x = x+1;  end, toc, x
Elapsed time is 0.954818 seconds.
x =  1000000
>> jit_enable (true)
>> tic;  x = 0;  for i=1:1e6,  x = x+1;  end, toc, x
Elapsed time is 0.070678 seconds.
x =  1000000
>> tic;  x = 0;  for i=1:1e6,  x = x+1;  end, toc, x
Elapsed time is 0.0122459 seconds.
x =  1000000


Markus Mützel <mmuetzel>
Group administrator
Tue 15 Jan 2019 09:18:59 PM UTC, comment #17: 

@Markus: yes, I get the same LLVM ERROR with LLVM 4.0.  (But this is no longer a configuration problem ;-)

I haven't started to investigate this yet, but I think that this is a new problem, not related the one that you mention.  Can you open a dedicated bug report for it please ?

You should be able to accelerate some simple loops, however.  Try this for example:

> jit_enable (false)
> tic;  x = 0;  for i=1:1e6,  x = x+1;  end, toc, x
> jit_enable (true)
> tic;  x = 0;  for i=1:1e6,  x = x+1;  end, toc, x


Julien Bect <jbect>
Tue 15 Jan 2019 07:31:01 PM UTC, comment #16: 

On Ubuntu 18.10, I successfully configured with:

LLVM_CONFIG=`which llvm-config-4.0` ../configure CPPFLAGS=-I/usr/include/hdf5/serial LDFLAGS="-L/usr/lib/$(dpkg-architecture -qDEB_HOST_MULTIARCH)/hdf5/serial" --enable-jit


But "make check" crashes with:

  jit.tst ........................................................LLVM ERROR: Program used extern function 'jit_paren_subsref_matrix_scalar2' which could not be resolved!


Is this bug #55469 or something different?

Markus Mützel <mmuetzel>
Group administrator
Tue 15 Jan 2019 12:09:59 PM UTC, comment #15: 

@Dimitri : Thank you for your report.

You are trying to build with LLVM 3.9, so this bug report (which is about configure + LLVM 4.0) is not the correct place to report a problem.

Moreover, it seems that the problem is actually about building in Fedora with a given version of LLVM when several are installed (in your case, building with LLVM 3.9 when LLVM 7 is installed too).  I suggest to open a different bug report (and, if possible, to try first on a system with LLVM 3.9 only.)


@admin: As I said in my comment #9, as far as I can tell, the original issue (configure fails with LLVM 4.0) is solved, so this bug report can be closed.

Further problems with Octave + LLVM 4.0 (or 3.9, or 3.8) would be better discussed elsewhere.

Julien Bect <jbect>
Mon 14 Jan 2019 07:46:52 PM UTC, comment #14: 

Perhaps llvm-7 got pulled in by some other dependencies/libraries.

The other infor from the stack that I omitted last time:



Process 4329 (lt-octave-gui) crashed in llvm::SmallPtrSetImplBase::FindBucketFor(void const*) const()


Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 14 Jan 2019 07:15:41 PM UTC, comment #13: 

+verrbatim+
../configure LLVM_CONFIG="/usr/lib64/llvm3.9/bin/llvm-config" --enable-jit
-verbatim-

compiles, but does not run, trying to plot anything results in segfault. Here is the top of the trace (what does llvm-7 do there?):



systemd-coredump[8597]: Process 8451 (lt-octave-gui) of user 1001 dumped core.

                                                  Stack trace of thread 8451:
                                                  #0  0x00007f1eab67e1a2 _ZNK4llvm19SmallPtrSetImplBase13FindBucketForEPKv (libLLVM-3.9.so)
                                                  #1  0x00007f1eab67e3c2 _ZN4llvm19SmallPtrSetImplBase14insert_imp_bigEPKv (libLLVM-3.9.so)
                                                  #2  0x00007f1eab64dedd _ZN4llvm2cl14OptionCategory16registerCategoryEv (libLLVM-3.9.so)
                                                  #3  0x00007f1e6d17d58e n/a (libLLVM-7.so)
                                                  #4  0x00007f1eb1ff5e0a call_init.part.0 (ld-linux-x86-64.so.2)
                                                  #5  0x00007f1eb1ff5f0a _dl_init (ld-linux-x86-64.so.2)
                                                  #6  0x00007f1eb1ff9eff dl_open_worker (ld-linux-x86-64.so.2)

....


(llvm-7 is the default llvm on this system.)

Dmitri.
--



Dmitri A. Sergatskov <dasergatskov>
Mon 14 Jan 2019 03:42:21 PM UTC, comment #12: 

@Julien

Thanks for trying to help. But, Fedora 29 does
not appear to offer configuration as you suggested.
The only config entries are:
/usr/bin/llvm-config@
/usr/bin/llvm-config-64*
and config@ points through /etc/alternatives/llvm-config
back to /usr/bin/llvm-config-64

And, using /usr/bin/llvm-config results in:
configure: WARNING: Missing LLVM file TargetData.h.  JIT compiler is disabled.

So, best to let this go for now.

Thanks anyhow.

Michael Godfrey <godfrey>
Group Member
Mon 14 Jan 2019 02:15:14 PM UTC, comment #11: 

Hi Michael,

Your question is slightly off-topic IMO, since this ticket is about LLVM 3.9...

Trying to anwser anyway :

1) Have you set the environment variable LLVM_CONFIG before building?  If not, you are probably trying to build with LLVM 7.0, which as you know is not supported.

2) In the list of LLVM versions that you indicate, only LLVM 3.9 is currently supposed to work.  Therefore you should set LLVM_CONFIG to point to the llvm-config utility of LLVM 3.9.  In Debian this is:

export LLVM_CONFIG=`which llvm-config-3.9`

If Octave fails to configure or build with this, please open a dedicated bug report about LLVM 3.9.

3) About supporting LLVM 5, 6 and 7 as well: sure, it would be better, but I need help if you want things to move faster...  Since you happen to have LLVM 5 installed, perhaps can you try to configure+build with it, and then tell us about it in a dedicated bug report?

Please always provide your config.log when reporting about configure issues.

@++
Julien

Julien Bect <jbect>
Mon 14 Jan 2019 01:01:43 PM UTC, comment #10: 

On Fedora 29 I get for ./configure --enable-jit:


  Build Octave Qt GUI:                  yes (version: 5)
  JIT compiler for loops:               no
  Build Java interface:                 yes
  Build static libraries:               no
  Build shared libraries:               yes
  Dynamic Linking API:                  dlopen
  Include support for GNU readline:     yes
  64-bit array dims and indexing:       yes
  64-bit BLAS array dims and indexing:  no
  OpenMP SMP multithreading:            yes
  Truncate intermediate FP results:     yes
  Build cross tools:                    no
  Build docs:                           yes

configure: WARNING: Missing LLVM file TargetData.h.  JIT compiler is disabled.


Probably due to the fact that available llvm versions are:

llvm       (which is really llvm 7.0.0-2.fc29
llvm3.9
llvm5.0
llvm6.0

What would be needed to allow 5, 6, or 7?

Michael Godfrey <godfrey>
Group Member
Mon 14 Jan 2019 09:07:20 AM UTC, comment #9: 

The patches referred to in comment #6 have now been applied to stable, which means that Octave 5.0 (to be released soon) can be built with LLVM 4.0.

There some unit tests that pass with LLVM 3.8 and crash Octave with LLVM 4.0, but they will be reported in separate bug reports.

I don't know if this report must be closed or not: strictly speaking, stable and default now build with LLVM 4.0, but "make check" fails...

Julien Bect <jbect>
Mon 07 Jan 2019 06:09:10 AM UTC, comment #8: 

Yes, as I said, you can build with LLVM 3.9 if you apply my patches on default (dev).   They can also easily be rebased to stable (5.0).

Julien Bect <jbect>
Mon 07 Jan 2019 12:24:54 AM UTC, comment #7: 

Julian,

You said: Currently, with both 5.0 and dev, LLVM 3.8 is the only version that you can use.

Fedora 29 does not offer 3.8. 3.9, 5.0, and 6.0 are the choices.
Is it possible that 3.9 will work? Any other suggestions?

Michael Godfrey <godfrey>
Group Member
Sun 06 Jan 2019 05:45:29 PM UTC, comment #6: 

Currently, with both 5.0 and dev, LLVM 3.8 is the only version that you can use.

With the patches from https://savannah.gnu.org/patch/?9741 applied, you can also build 3.7, 3.9 and 4.0, but the unit tests crash Octave in 3.7 and 4.0 (not 3.9).  I haven't investigated why.

Julien Bect <jbect>
Sun 06 Jan 2019 12:05:41 PM UTC, comment #5: 

export LLVM_CONFIG=`which llvm-config-3.9`

THis is useful. For Octave 5.0 or dev which versions
of llvm are currently supported ? Is any one "preferred"?

Thanks

Michael Godfrey <godfrey>
Group Member
Fri 04 Jan 2019 01:49:32 PM UTC, comment #4: 

Hi Michael.  What is your point ?

As far as Octave is concerned, in principle you can have several versions of LLVM installed as long as you provide the correct llvm-config in the LLVM_CONFIG vaiable when you build.

For instance, in Debian 9 (stretch) I can have LLVM 3.8, 3.9 and 4.0 installed.  In order to build againt LLVM 3.9 I just do this:

export LLVM_CONFIG=`which llvm-config-3.9`



Julien Bect <jbect>
Fri 04 Jan 2019 12:42:37 PM UTC, comment #3: 

It appears that https://llvm.org/ is the home page
for the LLVM project. This shows a release list:

7.0.1: Dec 2018
7.0.0: Sep 2018
6.0.1: July 2018
5.0.2: May 2018
6.0.0: Mar 2018
5.0.1: Dec 2017
5.0.0: Sep 2017
4.0.1: Jul 2017
4.0.0: Mar 2017
3.9.1: Dec 2016
3.9.0: Sep 2016
3.8.1: July 2016
3.8.0: Mar 2016

However, on Fedora 29 I have installed:
[godfrey@pbdsl4 octave]$ rpm -qa|grep llvm
621:llvm6.0-libs-6.0.1-7.fc29.x86_64
675:llvm-libs-7.0.0-2.fc29.x86_64
1231:llvm-doc-7.0.0-2.fc29.noarch
1682:llvm3.9-libs-3.9.1-14.fc29.x86_64
2474:llvm-devel-7.0.0-2.fc29.x86_64
3049:llvm-7.0.0-2.fc29.x86_64

This is just accidental.

dnf list all llvm* shows a long list of available packages.
The list starts with 3.9.1-14 and ends with 7.0.0-2.
The 7.0.0-2 version is just llvm.*

Previous versions are, for example, llvm3.9.x86_64

Clearly (???) any combination of llvm and llvmx.y can be
installed together.

However, the list goes from 3.9 to 5.0. Fedora appears to have
dropped 4.0.x

How all this is supposed to work seems as unclear as before.

And, different linuxes likely handle all this differently?

Michael Godfrey <godfrey>
Group Member
Thu 03 Jan 2019 09:25:01 AM UTC, comment #2: 

Patches that make it possible to build with LLVM 3.9 and 4.0 are now available on the patch tracker: https://savannah.gnu.org/patch/?9741.

Julien Bect <jbect>
Wed 15 Nov 2017 11:44:46 PM UTC, comment #1: 

I looked at: https://llvm.org/devmtg/2017-10/

They are on a six month major release schedule:

5.0.0: Sep 2017
4.0.1: Jul 2017
4.0.0: Mar 2017
3.9.1: Dec 2016
3.9.0: Sep 2016

And, they seem to make major compatibility changes
in each release as they did before. Is there some way
that we can keep this under control? I did not find
any obvious advice on their web page.

Fedora seems to just accept their "current" release with
some months delay.

Michael Godfrey <godfrey>
Group Member
Wed 15 Nov 2017 05:18:54 PM UTC, original submission:  

In Fedora 26 and 27 the current llvm is: llvm-4.0.1
and with JWT's latest push (HG ID f74678605da9)
./configure --enable-jit gives:

configure: using -isystem for LLVM headers
checking for LLVMBuildAdd in -lLLVM-4.0.1... yes
checking llvm/Support/TargetSelect.h usability... yes
checking llvm/Support/TargetSelect.h presence... yes
checking for llvm/Support/TargetSelect.h... yes
checking llvm/IR/Function.h usability... no
checking llvm/IR/Function.h presence... no
checking for llvm/IR/Function.h... no
checking llvm/Function.h usability... no
checking llvm/Function.h presence... no
checking for llvm/Function.h... no
checking llvm/Support/IRBuilder.h usability... no
checking llvm/Support/IRBuilder.h presence... no
checking for llvm/Support/IRBuilder.h... no
checking llvm/IR/IRBuilder.h usability... no
checking llvm/IR/IRBuilder.h presence... no
checking for llvm/IR/IRBuilder.h... no
checking llvm/IRBuilder.h usability... no
checking llvm/IRBuilder.h presence... no
checking for llvm/IRBuilder.h... no
checking llvm/Target/TargetData.h usability... no
checking llvm/Target/TargetData.h presence... no
checking for llvm/Target/TargetData.h... no
checking llvm/IR/DataLayout.h usability... no
checking llvm/IR/DataLayout.h presence... no
checking for llvm/IR/DataLayout.h... no
checking llvm/DataLayout.h usability... no
checking llvm/DataLayout.h presence... no
checking for llvm/DataLayout.h... no
checking llvm/IR/Verifier.h usability... no
checking llvm/IR/Verifier.h presence... no
checking for llvm/IR/Verifier.h... no
checking check llvm::Function::addAttribute arg type is llvm::Attributes... no
checking check LLVM::Function::addFnAttr arg type is llvm::Attributes... no
checking check LLVM::CallInst::addAttribute arg type is llvm::Attributes... no
checking check LLVM::raw_fd_ostream arg type is llvm::sys:fs... no
checking check for LLVM::legacy::PassManager... no
configure: WARNING: Missing LLVM file TargetData.h.  JIT compiler is disabled.

It appears that the llvm folks are still making version
incompatible changes...


Michael Godfrey <godfrey>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by jbect (Posted a comment)
  • -email is unavailable- added by doronbehar
  • -email is unavailable- added by godfrey (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-01-24 mtmiller StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2019-01-15 mmuetzel StatusNone Ready For Test
    2018-11-06 doronbehar Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-cf05.
    Corresponding source code