Regression
Build / HIP-5.6 / projectroot.test.ArborX_Test_DetailsDistributedTreeImpl (from CTest)
Failing for the past 1 build
(Since Failed
)
Stacktrace
[6c009d7e2d54:18848] *** Process received signal *** [6c009d7e2d54:18848] Signal: Segmentation fault (11) [6c009d7e2d54:18848] Signal code: Address not mapped (1) [6c009d7e2d54:18848] Failing at address: 0x18 [6c009d7e2d54:18848] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7ff30f4f0420] [6c009d7e2d54:18848] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7ff30dd4835d] [6c009d7e2d54:18848] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7ff30dd4c47d] [6c009d7e2d54:18848] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7ff30dd57926] [6c009d7e2d54:18848] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7ff30dc8fc90] [6c009d7e2d54:18848] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7ff30dc9ab5e] [6c009d7e2d54:18848] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [6c009d7e2d54:18848] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [6c009d7e2d54:18848] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [6c009d7e2d54:18848] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [6c009d7e2d54:18848] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7ff30d3b7083] [6c009d7e2d54:18848] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [6c009d7e2d54:18848] *** End of error message *** [6c009d7e2d54:18854] *** Process received signal *** [6c009d7e2d54:18854] Signal: Segmentation fault (11) [6c009d7e2d54:18854] Signal code: Address not mapped (1) [6c009d7e2d54:18854] Failing at address: 0x18 [6c009d7e2d54:18854] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f7294449420] [6c009d7e2d54:18854] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f7292ca135d] [6c009d7e2d54:18854] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f7292ca547d] [6c009d7e2d54:18854] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f7292cb0926] [6c009d7e2d54:18854] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f7292be8c90] [6c009d7e2d54:18854] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f7292bf3b5e] [6c009d7e2d54:18854] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [6c009d7e2d54:18854] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [6c009d7e2d54:18854] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [6c009d7e2d54:18854] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [6c009d7e2d54:18854] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f7292310083] [6c009d7e2d54:18854] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [6c009d7e2d54:18854] *** End of error message *** [6c009d7e2d54:18859] *** Process received signal *** [6c009d7e2d54:18859] Signal: Segmentation fault (11) [6c009d7e2d54:18859] Signal code: Address not mapped (1) [6c009d7e2d54:18859] Failing at address: 0x18 [6c009d7e2d54:18859] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f3799ef3420] [6c009d7e2d54:18859] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f379874b35d] [6c009d7e2d54:18859] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f379874f47d] [6c009d7e2d54:18859] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f379875a926] [6c009d7e2d54:18859] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f3798692c90] [6c009d7e2d54:18859] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f379869db5e] [6c009d7e2d54:18859] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [6c009d7e2d54:18859] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [6c009d7e2d54:18859] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [6c009d7e2d54:18859] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [6c009d7e2d54:18859] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f3797dba083] [6c009d7e2d54:18859] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [6c009d7e2d54:18859] *** End of error message *** [6c009d7e2d54:18849] *** Process received signal *** [6c009d7e2d54:18849] Signal: Segmentation fault (11) [6c009d7e2d54:18849] Signal code: Address not mapped (1) [6c009d7e2d54:18849] Failing at address: 0x18 [6c009d7e2d54:18849] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f088f0db420] [6c009d7e2d54:18849] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f088d93335d] [6c009d7e2d54:18849] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f088d93747d] [6c009d7e2d54:18849] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f088d942926] [6c009d7e2d54:18849] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f088d87ac90] [6c009d7e2d54:18849] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f088d885b5e] [6c009d7e2d54:18849] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [6c009d7e2d54:18849] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [6c009d7e2d54:18849] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [6c009d7e2d54:18849] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [6c009d7e2d54:18849] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f088cfa2083] [6c009d7e2d54:18849] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [6c009d7e2d54:18849] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 0 with PID 0 on node 6c009d7e2d54 exited on signal 11 (Segmentation fault). --------------------------------------------------------------------------
Standard Output
[6c009d7e2d54:18848] *** Process received signal *** [6c009d7e2d54:18848] Signal: Segmentation fault (11) [6c009d7e2d54:18848] Signal code: Address not mapped (1) [6c009d7e2d54:18848] Failing at address: 0x18 [6c009d7e2d54:18848] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7ff30f4f0420] [6c009d7e2d54:18848] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7ff30dd4835d] [6c009d7e2d54:18848] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7ff30dd4c47d] [6c009d7e2d54:18848] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7ff30dd57926] [6c009d7e2d54:18848] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7ff30dc8fc90] [6c009d7e2d54:18848] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7ff30dc9ab5e] [6c009d7e2d54:18848] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [6c009d7e2d54:18848] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [6c009d7e2d54:18848] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [6c009d7e2d54:18848] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [6c009d7e2d54:18848] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7ff30d3b7083] [6c009d7e2d54:18848] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [6c009d7e2d54:18848] *** End of error message *** [6c009d7e2d54:18854] *** Process received signal *** [6c009d7e2d54:18854] Signal: Segmentation fault (11) [6c009d7e2d54:18854] Signal code: Address not mapped (1) [6c009d7e2d54:18854] Failing at address: 0x18 [6c009d7e2d54:18854] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f7294449420] [6c009d7e2d54:18854] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f7292ca135d] [6c009d7e2d54:18854] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f7292ca547d] [6c009d7e2d54:18854] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f7292cb0926] [6c009d7e2d54:18854] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f7292be8c90] [6c009d7e2d54:18854] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f7292bf3b5e] [6c009d7e2d54:18854] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [6c009d7e2d54:18854] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [6c009d7e2d54:18854] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [6c009d7e2d54:18854] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [6c009d7e2d54:18854] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f7292310083] [6c009d7e2d54:18854] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [6c009d7e2d54:18854] *** End of error message *** [6c009d7e2d54:18859] *** Process received signal *** [6c009d7e2d54:18859] Signal: Segmentation fault (11) [6c009d7e2d54:18859] Signal code: Address not mapped (1) [6c009d7e2d54:18859] Failing at address: 0x18 [6c009d7e2d54:18859] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f3799ef3420] [6c009d7e2d54:18859] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f379874b35d] [6c009d7e2d54:18859] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f379874f47d] [6c009d7e2d54:18859] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f379875a926] [6c009d7e2d54:18859] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f3798692c90] [6c009d7e2d54:18859] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f379869db5e] [6c009d7e2d54:18859] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [6c009d7e2d54:18859] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [6c009d7e2d54:18859] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [6c009d7e2d54:18859] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [6c009d7e2d54:18859] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f3797dba083] [6c009d7e2d54:18859] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [6c009d7e2d54:18859] *** End of error message *** [6c009d7e2d54:18849] *** Process received signal *** [6c009d7e2d54:18849] Signal: Segmentation fault (11) [6c009d7e2d54:18849] Signal code: Address not mapped (1) [6c009d7e2d54:18849] Failing at address: 0x18 [6c009d7e2d54:18849] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f088f0db420] [6c009d7e2d54:18849] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f088d93335d] [6c009d7e2d54:18849] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f088d93747d] [6c009d7e2d54:18849] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f088d942926] [6c009d7e2d54:18849] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f088d87ac90] [6c009d7e2d54:18849] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f088d885b5e] [6c009d7e2d54:18849] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [6c009d7e2d54:18849] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [6c009d7e2d54:18849] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [6c009d7e2d54:18849] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [6c009d7e2d54:18849] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f088cfa2083] [6c009d7e2d54:18849] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [6c009d7e2d54:18849] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 0 with PID 0 on node 6c009d7e2d54 exited on signal 11 (Segmentation fault). --------------------------------------------------------------------------