maindmidecode - Support: sr #109697, Segmentation fault on Ubuntu 18.04...

 
 

sr #109697: Segmentation fault on Ubuntu 18.04 LTS aarch64

Submitter:  None
Submitted:  Thu 06 Jun 2019 11:06:28 PM UTC
   
 
Category:  None Priority:  5 - Normal
Severity:  3 - Normal Status:  Done
Privacy:  Public Assigned to:  khali
Originator Email:  -email is unavailable- Open/Closed:  Closed
Operating System:  GNU/Linux
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 01 Apr 2020 08:11:28 AM UTC, comment #10: 

Fixed by:

commit e12ec26e19e02281d3e7258c3aabb88a5cf5ec1d
Author: Jean Delvare
Date:   Mon Aug 26 14:20:15 2019 +0200

    dmidecode: Only scan /dev/mem for entry point on x86

Jean Delvare <khali>
Group administrator
Thu 21 Nov 2019 08:05:11 AM UTC, comment #9: 

Thanks for testing and reporting Stephen, very appreciated.

Jean Delvare <khali>
Group administrator
Tue 19 Nov 2019 09:00:24 PM UTC, comment #8: 

I have tested the patch on the same NVIDIA Jetson AGX Xavier hardware that James Glenn is using. The patch works.

Specifically: I noticed that the patch is checked in to git. Building master in git yields a dmidecode that refuses to operate saying the following, as expected:
    # dmidecode 3.2
    # No SMBIOS nor DMI entry point found, sorry.
However, if I revert the patch in question, then the SError is reproduced, as expected.

Some notes:

This platform does not support UEFI or DMI. It's expected that kernel options to support those features are not enabled.

The problem is an SError (system error; reported by a bus/HW-module when an invalid physical address is accessed), not a segmentation fault (which is raised by MMU-related HW, when an invalid virtual address is accessed).

Stephen Warren <swarren>
Mon 23 Sep 2019 07:40:38 AM UTC, comment #7: 

Ping.

Jean Delvare <khali>
Group administrator
Fri 02 Aug 2019 09:25:09 AM UTC, comment #6: 

Can you please test the patch and report? I'd like to commit the fix and close this bug if the fix works for you.

Jean Delvare <khali>
Group administrator
Wed 26 Jun 2019 10:43:28 AM UTC, comment #5: 

Dmidecode has 3 ways to get the DMI data: from sysfs (both the entry point and the actual table), from EFI (for the entry point, and then the actual table if read from /dev/mem), or from /dev/mem (both the entry point and the actual table). The comment in the source code indicates that the last method only works on x86, however, there is currently nothing preventing it from being attempted on other architectures because it was expected that either method #1 or method #2 would have succeeded before we reached that point.

However looking at the strace which leads to your system rebooting, I can see that both methods #1 and #2 have failed. This suggests that your kernel offers no DMI and no EFI support, which seems wrong. I'm pretty sure that you want both enabled on an arm64 kernel.

That being said, dmidecode should not make this assumption, and scanning /dev/mem for the entry point should be explicitly restricted to x86. I'll provide a patch doing that for you to test. This will only prevent the reboot though. dmidecode won't be able to provide any information as long as both DMI and EFI are disabled in the kernel.

Jean Delvare <khali>
Group administrator
Tue 25 Jun 2019 06:36:07 PM UTC, comment #4: 


> Please apply commit 82497fa02d60757c2cfa645cf89a79abb1435273 ("dmidecode: Don't use memcpy on /dev/mem on arm64") and see if it happens again:


Done. No change in symptoms. Still reboots.

> Is it a virtual machine?


No. This is an Nvidia Jetson AGX Xavier Developer Kit.

> Was the kernel built without CONFIG_DMI?


Looks like no. At least I do not see that referenced in the running kernel config.

I'm going to try to rebuild with CONFIG_DMI set, but have not had time. Pardon my delay.

I found we have a few internal bugs open because of dmidecode causing this type of system to reboot.


apt-get install salt-minion


Even simply installing some packages causes problems. Would be nice to not crash when installing software.

James Glenn <jglenn>
Tue 11 Jun 2019 06:58:23 AM UTC, comment #3: 

OK, this is probably a duplicate of bug #55026.

Please apply commit 82497fa02d60757c2cfa645cf89a79abb1435273 ("dmidecode: Don't use memcpy on /dev/mem on arm64") and see if it happens again:

http://git.savannah.gnu.org/cgit/dmidecode.git/commit/?id=82497fa02d60757c2cfa645cf89a79abb1435273

That being said, you should normally not reach that code path to start with.


openat(AT_FDCWD, "/sys/firmware/dmi/tables/smbios_entry_point", O_RDONLY) = -1 ENOENT (No such file or directory)


means that the kernel did not provide the DMI table to user-space. Kernel 4.9 should definitely provide it if it exists.

Is it a virtual machine?

Was the kernel built without CONFIG_DMI?

Is there anything in the kernel boot log related to DMI or SMBIOS?

Jean Delvare <khali>
Group administrator
Mon 10 Jun 2019 07:13:15 PM UTC, comment #2: 

Apologies, I am unable to reproduce a Segmentation fault. Now I just get a reboot. I have an strace below. System reboots and leaves no logs. Please let me if I can provide any other information.

> Which version of dmidecode is this?


HEAD is now at a448baa Set version to 3.2 from https://git.savannah.nongnu.org/git/dmidecode.git

> Does it have any patch applied?


No.

> What compiler version and compilation flags were used to build it?



root@jetson-0423318029481:~# gcc --version
gcc (Ubuntu/Linaro 7.3.0-27ubuntu1~18.04) 7.3.0
root@jetson-0423318029481:/usr/local/src/dmidecode# make
gcc -W -Wall -Wshadow -Wstrict-prototypes -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wmissing-prototypes -Winline -Wundef -D_FILE_OFFSET_BITS=64 -O2 -c dmidecode.c -o dmidecode.o
gcc -W -Wall -Wshadow -Wstrict-prototypes -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wmissing-prototypes -Winline -Wundef -D_FILE_OFFSET_BITS=64 -O2 -c dmiopt.c -o dmiopt.o
gcc -W -Wall -Wshadow -Wstrict-prototypes -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wmissing-prototypes -Winline -Wundef -D_FILE_OFFSET_BITS=64 -O2 -c dmioem.c -o dmioem.o
gcc -W -Wall -Wshadow -Wstrict-prototypes -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wmissing-prototypes -Winline -Wundef -D_FILE_OFFSET_BITS=64 -O2 -c util.c -o util.o
gcc  dmidecode.o dmiopt.o dmioem.o util.o -o dmidecode


> Does "dmidecode -V" crash too?


No.


root@jetson-0423318029481:~# /usr/local/sbin/dmidecode -V
3.2


root@jetson-0423318029481:~# strace /usr/local/sbin/dmidecode
execve("/usr/local/sbin/dmidecode", ["/usr/local/sbin/dmidecode"], 0x7ff325a350 /* 15 vars */) = 0
brk(NULL)                               = 0x55a5c6a000
faccessat(AT_FDCWD, "/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
faccessat(AT_FDCWD, "/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=133276, ...}) = 0
mmap(NULL, 133276, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f7e8b1000
close(3)                                = 0
faccessat(AT_FDCWD, "/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/aarch64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0 \10\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1341080, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7e8fa000
mmap(NULL, 1409880, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f7e758000
mprotect(0x7f7e898000, 61440, PROT_NONE) = 0
mmap(0x7f7e8a7000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13f000) = 0x7f7e8a7000
mmap(0x7f7e8ad000, 13144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f7e8ad000
close(3)                                = 0
mprotect(0x7f7e8a7000, 16384, PROT_READ) = 0
mprotect(0x5577c5e000, 12288, PROT_READ) = 0
mprotect(0x7f7e8fe000, 4096, PROT_READ) = 0
munmap(0x7f7e8b1000, 133276)            = 0
fstat(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 1), ...}) = 0
brk(NULL)                               = 0x55a5c6a000
brk(0x55a5c8b000)                       = 0x55a5c8b000
write(1, "# dmidecode 3.2\n", 16# dmidecode 3.2
)       = 16
openat(AT_FDCWD, "/sys/firmware/dmi/tables/smbios_entry_point", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/sys/firmware/efi/systab", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/proc/efi/systab", O_RDONLY) = -1 ENOENT (No such file or directory)
write(1, "Scanning /dev/mem for entry poin"..., 35Scanning /dev/mem for entry point.
) = 35
openat(AT_FDCWD, "/dev/mem", O_RDONLY)  = 3
fstat(3, {st_mode=S_IFCHR|0640, st_rdev=makedev(1, 1), ...}) = 0
mmap(NULL, 65536, PROT_READ, MAP_SHARED, 3, 0xf0000

# System reboots here




James Glenn <jglenn>
Fri 07 Jun 2019 07:00:30 AM UTC, comment #1: 

Which version of dmidecode is this?

Does it have any patch applied?

What compiler version and compilation flags were used to build it?

Does "dmidecode -V" crash too?

Please enable core dumping (ulimit -c unlimited), run gdb on the core, and provide a backtrace of the segmentation fault.

Jean Delvare <khali>
Group administrator
Thu 06 Jun 2019 11:06:28 PM UTC, original submission:  

Please let me know if I can provide any other info.

root@galen:~# dmidecode &> dmidecode.log
Segmentation fault
[  189.421741] CPU2: SError detected, daif=1c0, spsr=0x80c000c5, mpidr=80000100, esr=be000000
[  189.421751] CPU3: SError detected, daif=140, spsr=0x20000000, mpidr=80000101, esr=be000000
[  189.421765] CPU1: SError detected, daif=1c0, spsr=0x80c000c5, mpidr=80000001, esr=be000000
[  189.421874] CPU:0, Error:CBB-NOC@0x2300000,irq=485
[  189.421877] ************************************
[  189.421879] * For more Internal Decode Help
[  189.421884] *     http://nv/cbberr
[  189.421884] * NVIDIA userID is required to access
[  189.421889] ************************************
[  189.421890] CPU:0, Error:CBB-NOC
[  189.421892] Error Logger : 1
[  189.421901] ErrLog0 : 0x800f0100
[  189.421907] ************************************
[  189.421911]   Transaction Type : RD  - Read, Incrementing
[  189.421914] * For more Internal Decode Help
[  189.421920]   Error Code : DEC
[  189.421923] *     http://nv/cbberr
[  189.421927]   Error Source : Initiator NIU
[  189.421930] * NVIDIA userID is required to access
[  189.421935]   Error Description : Address decode error
[  189.421938] ************************************
[  189.421941]   Packet header Lock : 0
[  189.421945] CPU:3, Error:CBB-NOC
[  189.421949]   Packet header Len1 : 15
[  189.421983] Error Logger : 1
[  189.421986]   NOC protocol version : version >= 2.7
[  189.421993] ErrLog1 : 0x320026
[  189.421997] ErrLog0 : 0x800f0100
[  189.422000] ErrLog2 : 0x0
[  189.422005]   Transaction Type : RD  - Read, Incrementing
[  189.422008]   RouteId : 0x320026
[  189.422012]   Error Code : DEC
[  189.422015]   InitFlow : ccroc_p2ps/I/ccroc_p2ps
[  189.422019]   Error Source : Initiator NIU
[  189.422023]   Targflow : axis_satellite_grout/T/axis_satellite_grout
[  189.422026]   Error Description : Address decode error
[  189.422029]   TargSubRange : 0
[  189.422032]   Packet header Lock : 0
[  189.422035]   SeqId : 0
[  189.422038]   Packet header Len1 : 15
[  189.422041] ErrLog3 : 0xf0000
[  189.422044]   NOC protocol version : version >= 2.7
[  189.422048] ErrLog4 : 0x0
[  189.422050] ErrLog1 : 0x320026
[  189.422052] ErrLog2 : 0x0
[  189.422057]   RouteId : 0x320026
[  189.422060]   InitFlow : ccroc_p2ps/I/ccroc_p2ps
[  189.422062]   Targflow : axis_satellite_grout/T/axis_satellite_grout
[  189.422063]   TargSubRange : 0
[  189.422064]   SeqId : 0
[  189.422066] ErrLog3 : 0xf0000
[  189.422067] ErrLog4 : 0x0
[  189.422134]   Address : 0xf0000 (unknown device)
[  189.422135] ErrLog5 : 0x989f850
[  189.422137]   Non-Modify : 0x1
[  189.422138]   AXI ID : 0x13
[  189.422140]   Master ID : CCPLEX
[  189.422141]   Security Group(GRPSEC): 0x7e
[  189.422143]   Cache : 0x0 -- Non-cacheable/Non-Bufferable)
[  189.422147]   Protection : 0x2 -- Unprivileged, Non-Secure, Data Access
[  189.422152]   Address : 0xf0000 (unknown device)
[  189.422155]   FALCONSEC : 0x0
[  189.422158] ErrLog5 : 0x989f850
[  189.422161]   Virtual Queuing Channel(VQC): 0x0
[  189.422163]   Non-Modify : 0x1
[  189.422166]   AXI ID : 0x13
[  189.422173]   Master ID : CCPLEX
[  189.422177] ************************************
[  189.422179]   Security Group(GRPSEC): 0x7e
[  189.422184] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[  189.422188]   Cache : 0x0 -- Non-cacheable/Non-Bufferable)
[  189.422192]   Protection : 0x2 -- Unprivileged, Non-Secure, Data Access
[  189.422194]   FALCONSEC : 0x0
[  189.422195]   Virtual Queuing Channel(VQC): 0x0
[  189.422198] ************************************
[  189.422210] CPU0: SError detected, daif=140, spsr=0x60000000, mpidr=80000000, esr=be000000
[  189.422244] ************************************
[  189.422247] RAS Error in SCF:IOB:
[  189.422249] Status = 0xf4009604
[  189.422251] IERR = CBB Interface Error: 0x96
[  189.422254] SERR = Assertion Failure: 0x4
[  189.422255] Uncorrectable (this is fatal)
[  189.422271] MISC0 = 0x40
[  189.422272] MISC1 = 0x66466444e3
[  189.422280] ADDR = 0x80000000000f0000
[  189.422292] ************************************
[  189.422300] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[  189.422347] ************************************
[  189.422348] RAS Error in L2:
[  189.422351] Status = 0xf400640d
[  189.422353] IERR = SCF to L2 Decode Error Read: 0x64
[  189.422358] SERR = Illegal address (software fault): 0xd
[  189.422360] Uncorrectable (this is fatal)
[  189.422369] MISC0 = 0x100000000100000
[  189.422370] MISC1 = 0x40240000001
[  189.422376] ADDR = 0x80000000000f0000
[  189.422384] ************************************
[  189.422418] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[  189.422586] Bad mode in Error handler detected on CPU3, code 0xbe000000 -- SError
[  189.422595] Internal error: Oops - bad mode: 0 [#1] PREEMPT SMP
[  189.422663] Modules linked in: bnep fuse nvs_bmi160 nvs nvgpu bluedroid_pm ip_tables x_tables
[  189.422686] CPU: 3 PID: 8834 Comm: dmidecode Not tainted 4.9.140-tegra #1
[  189.422687] Hardware name: jetson-xavier (DT)
[  189.422691] task: ffffffc37c4f5400 task.stack: ffffffc34c7c0000
[  189.422694] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[  189.422698] PC is at 0x7f831ec280
[  189.422701] LR is at 0x55895e8aa8
[  189.422704] pc : [<0000007f831ec280>] lr : [<00000055895e8aa8>] pstate: 20000000
[  189.422705] sp : 0000007fc2a277d0
[  189.422722] x29: 0000007fc2a277d0 x28: 0000000000000000
[  189.422730] x27: 0000000000010000 x26: 0000007f832d1000
[  189.422738] x25: 0000000000000000 x24: 00000000000f0000
[  189.422750] x23: 0000005589601000
[  189.422751] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[  189.422759] x22: 0000000000010000
[  189.422767] x21: 000000558d3174a0 x20: 0000000000000003
[  189.422775] x19: 00000055895eb042 x18: 0000007f832bca70
[  189.422782] x17: 0000007f831ec190 x16: 0000005589601e50
[  189.422790] x15: 0000000000000000 x14: 0000000000000000
[  189.422798] x13: 0000000000000000 x12: 0000000000000000
[  189.422830] x11: 0000000000000000 x10: 0000000000000400
[  189.422846] x9 : 0000007f8330a2a0 x8 : 00000000000000de
[  189.422847] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[  189.422858] x7 : 0000000000000000 x6 : 0000000000000000
[  189.422867] x5 : 000000558d3274a0 x4 : 0000007f832e1000
[  189.422874] x3 : 000000558d3174a0 x2 : 0000000000010000
[  189.422883] x1 : 0000007f832d1000 x0 : 000000558d3174a0

[  189.422891] Process dmidecode (pid: 8834, stack limit = 0xffffffc34c7c0000)
[  189.422900] ---[ end trace 02dc7e6b87ec70f9 ]---
[  189.425280] CPU1: SError detected, daif=1c0, spsr=0x80c000c5, mpidr=80000001, esr=be000000
[  189.425282] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[  189.425319] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[  189.425367] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[  189.425547] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[  189.425583] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[  189.425629] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[  189.996002] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[  190.004635] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[  190.013750] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
[  190.021776] CPU2: SError detected, daif=1c0, spsr=0x80c000c5, mpidr=80000100, esr=be000000
[  190.029986] ras_ccplex_serr_callback: Scanning CCPLEX Error Records for Uncorrectable Errors
[  190.038838] ras_corecluster_serr_callback:Scanning CoreCluster Error Records for Uncorrectable Errors
[  190.048116] ras_core_serr_callback: Scanning Core Error Records for Uncorrectable Errors
root@galen:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.2 LTS
Release: 18.04
Codename: bionic
root@galen:~# uname -a
Linux galen 4.9.140-tegra #1 SMP PREEMPT Wed Mar 13 00:30:11 PDT 2019 aarch64 aarch64 aarch64 GNU/Linux

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #47135:  dmidecode-scan-mem-x86-only.patch added by khali (2KiB - text/x-patch - [PATCH] dmidecode: Only scan /dev/mem for entry point on x86)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by swarren (Posted a comment)
  • -email is unavailable- added by jglenn (Posted a comment)
  • -email is unavailable- added by khali (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-04-01 khali StatusNeed Info Done
        Open/ClosedOpen Closed
    2019-06-26 khali Attached File- Added dmidecode-scan-mem-x86-only.patch, #47135
    2019-06-07 khali StatusNone Need Info
        Assigned toNone khali

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code