Regression
Build / HIP-5.6 / projectroot.test.ArborX_Test_DetailsDistributedTreeImpl (from CTest)
Failing for the past 1 build
(Since Failed
)
Stacktrace
[54febd312b64:18352] *** Process received signal *** [54febd312b64:18352] Signal: Segmentation fault (11) [54febd312b64:18352] Signal code: Address not mapped (1) [54febd312b64:18352] Failing at address: 0x18 [54febd312b64:18352] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f151435e420] [54febd312b64:18352] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f1512bb635d] [54febd312b64:18352] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f1512bba47d] [54febd312b64:18352] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f1512bc5926] [54febd312b64:18352] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f1512afdc90] [54febd312b64:18352] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f1512b08b5e] [54febd312b64:18352] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [54febd312b64:18352] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [54febd312b64:18352] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [54febd312b64:18352] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [54febd312b64:18352] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f1512225083] [54febd312b64:18352] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [54febd312b64:18352] *** End of error message *** [54febd312b64:18347] *** Process received signal *** [54febd312b64:18347] Signal: Segmentation fault (11) [54febd312b64:18347] Signal code: Address not mapped (1) [54febd312b64:18347] Failing at address: 0x18 [54febd312b64:18347] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f984d77b420] [54febd312b64:18347] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f984bfd335d] [54febd312b64:18347] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f984bfd747d] [54febd312b64:18347] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f984bfe2926] [54febd312b64:18347] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f984bf1ac90] [54febd312b64:18347] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f984bf25b5e] [54febd312b64:18347] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [54febd312b64:18347] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [54febd312b64:18347] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [54febd312b64:18347] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [54febd312b64:18347] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f984b642083] [54febd312b64:18347] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [54febd312b64:18347] *** End of error message *** [54febd312b64:18342] *** Process received signal *** [54febd312b64:18342] Signal: Segmentation fault (11) [54febd312b64:18342] Signal code: Address not mapped (1) [54febd312b64:18342] Failing at address: 0x18 [54febd312b64:18342] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7efea7fc1420] [54febd312b64:18342] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7efea681935d] [54febd312b64:18342] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7efea681d47d] [54febd312b64:18342] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7efea6828926] [54febd312b64:18342] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7efea6760c90] [54febd312b64:18342] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7efea676bb5e] [54febd312b64:18342] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [54febd312b64:18342] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [54febd312b64:18342] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [54febd312b64:18342] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [54febd312b64:18342] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7efea5e88083] [54febd312b64:18342] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [54febd312b64:18342] *** End of error message *** [54febd312b64:18339] *** Process received signal *** [54febd312b64:18339] Signal: Segmentation fault (11) [54febd312b64:18339] Signal code: Address not mapped (1) [54febd312b64:18339] Failing at address: 0x18 [54febd312b64:18339] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fd92e4e0420] [54febd312b64:18339] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fd92cd3835d] [54febd312b64:18339] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fd92cd3c47d] [54febd312b64:18339] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fd92cd47926] [54febd312b64:18339] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fd92cc7fc90] [54febd312b64:18339] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fd92cc8ab5e] [54febd312b64:18339] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [54febd312b64:18339] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [54febd312b64:18339] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [54febd312b64:18339] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [54febd312b64:18339] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fd92c3a7083] [54febd312b64:18339] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [54febd312b64:18339] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 3 with PID 0 on node 54febd312b64 exited on signal 11 (Segmentation fault). --------------------------------------------------------------------------
Standard Output
[54febd312b64:18352] *** Process received signal *** [54febd312b64:18352] Signal: Segmentation fault (11) [54febd312b64:18352] Signal code: Address not mapped (1) [54febd312b64:18352] Failing at address: 0x18 [54febd312b64:18352] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f151435e420] [54febd312b64:18352] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f1512bb635d] [54febd312b64:18352] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f1512bba47d] [54febd312b64:18352] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f1512bc5926] [54febd312b64:18352] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f1512afdc90] [54febd312b64:18352] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f1512b08b5e] [54febd312b64:18352] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [54febd312b64:18352] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [54febd312b64:18352] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [54febd312b64:18352] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [54febd312b64:18352] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f1512225083] [54febd312b64:18352] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [54febd312b64:18352] *** End of error message *** [54febd312b64:18347] *** Process received signal *** [54febd312b64:18347] Signal: Segmentation fault (11) [54febd312b64:18347] Signal code: Address not mapped (1) [54febd312b64:18347] Failing at address: 0x18 [54febd312b64:18347] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f984d77b420] [54febd312b64:18347] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f984bfd335d] [54febd312b64:18347] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f984bfd747d] [54febd312b64:18347] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f984bfe2926] [54febd312b64:18347] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f984bf1ac90] [54febd312b64:18347] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f984bf25b5e] [54febd312b64:18347] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [54febd312b64:18347] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [54febd312b64:18347] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [54febd312b64:18347] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [54febd312b64:18347] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f984b642083] [54febd312b64:18347] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [54febd312b64:18347] *** End of error message *** [54febd312b64:18342] *** Process received signal *** [54febd312b64:18342] Signal: Segmentation fault (11) [54febd312b64:18342] Signal code: Address not mapped (1) [54febd312b64:18342] Failing at address: 0x18 [54febd312b64:18342] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7efea7fc1420] [54febd312b64:18342] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7efea681935d] [54febd312b64:18342] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7efea681d47d] [54febd312b64:18342] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7efea6828926] [54febd312b64:18342] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7efea6760c90] [54febd312b64:18342] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7efea676bb5e] [54febd312b64:18342] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [54febd312b64:18342] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [54febd312b64:18342] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [54febd312b64:18342] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [54febd312b64:18342] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7efea5e88083] [54febd312b64:18342] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [54febd312b64:18342] *** End of error message *** [54febd312b64:18339] *** Process received signal *** [54febd312b64:18339] Signal: Segmentation fault (11) [54febd312b64:18339] Signal code: Address not mapped (1) [54febd312b64:18339] Failing at address: 0x18 [54febd312b64:18339] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fd92e4e0420] [54febd312b64:18339] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fd92cd3835d] [54febd312b64:18339] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fd92cd3c47d] [54febd312b64:18339] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fd92cd47926] [54febd312b64:18339] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fd92cc7fc90] [54febd312b64:18339] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fd92cc8ab5e] [54febd312b64:18339] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [54febd312b64:18339] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [54febd312b64:18339] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [54febd312b64:18339] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [54febd312b64:18339] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fd92c3a7083] [54febd312b64:18339] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [54febd312b64:18339] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 3 with PID 0 on node 54febd312b64 exited on signal 11 (Segmentation fault). --------------------------------------------------------------------------