Regression
Build / HIP-5.6 / projectroot.test.ArborX_Test_DetailsDistributedTreeImpl (from CTest)
Failing for the past 1 build
(Since Failed
)
Stacktrace
[e621f6f40f56:18501] *** Process received signal *** [e621f6f40f56:18501] Signal: Segmentation fault (11) [e621f6f40f56:18501] Signal code: Address not mapped (1) [e621f6f40f56:18501] Failing at address: 0x18 [e621f6f40f56:18501] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f612a40a420] [e621f6f40f56:18501] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f6128c6235d] [e621f6f40f56:18501] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f6128c6647d] [e621f6f40f56:18501] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f6128c71926] [e621f6f40f56:18501] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f6128ba9c90] [e621f6f40f56:18501] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f6128bb4b5e] [e621f6f40f56:18501] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [e621f6f40f56:18501] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [e621f6f40f56:18501] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [e621f6f40f56:18501] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [e621f6f40f56:18501] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f61282d1083] [e621f6f40f56:18501] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [e621f6f40f56:18501] *** End of error message *** [e621f6f40f56:18506] *** Process received signal *** [e621f6f40f56:18506] Signal: Segmentation fault (11) [e621f6f40f56:18506] Signal code: Address not mapped (1) [e621f6f40f56:18506] Failing at address: 0x18 [e621f6f40f56:18506] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fb4ebc5d420] [e621f6f40f56:18506] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fb4ea4b535d] [e621f6f40f56:18506] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fb4ea4b947d] [e621f6f40f56:18506] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fb4ea4c4926] [e621f6f40f56:18506] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fb4ea3fcc90] [e621f6f40f56:18506] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fb4ea407b5e] [e621f6f40f56:18506] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [e621f6f40f56:18506] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [e621f6f40f56:18506] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [e621f6f40f56:18506] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [e621f6f40f56:18506] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fb4e9b24083] [e621f6f40f56:18506] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [e621f6f40f56:18506] *** End of error message *** [e621f6f40f56:18496] *** Process received signal *** [e621f6f40f56:18496] Signal: Segmentation fault (11) [e621f6f40f56:18496] Signal code: Address not mapped (1) [e621f6f40f56:18496] Failing at address: 0x18 [e621f6f40f56:18496] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fa2956c7420] [e621f6f40f56:18496] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fa293f1f35d] [e621f6f40f56:18496] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fa293f2347d] [e621f6f40f56:18496] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fa293f2e926] [e621f6f40f56:18496] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fa293e66c90] [e621f6f40f56:18496] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fa293e71b5e] [e621f6f40f56:18496] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [e621f6f40f56:18496] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [e621f6f40f56:18496] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [e621f6f40f56:18496] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [e621f6f40f56:18496] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fa29358e083] [e621f6f40f56:18496] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [e621f6f40f56:18496] *** End of error message *** [e621f6f40f56:18495] *** Process received signal *** [e621f6f40f56:18495] Signal: Segmentation fault (11) [e621f6f40f56:18495] Signal code: Address not mapped (1) [e621f6f40f56:18495] Failing at address: 0x18 [e621f6f40f56:18495] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f7e4e109420] [e621f6f40f56:18495] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f7e4c96135d] [e621f6f40f56:18495] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f7e4c96547d] [e621f6f40f56:18495] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f7e4c970926] [e621f6f40f56:18495] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f7e4c8a8c90] [e621f6f40f56:18495] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f7e4c8b3b5e] [e621f6f40f56:18495] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [e621f6f40f56:18495] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [e621f6f40f56:18495] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [e621f6f40f56:18495] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [e621f6f40f56:18495] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f7e4bfd0083] [e621f6f40f56:18495] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [e621f6f40f56:18495] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 2 with PID 0 on node e621f6f40f56 exited on signal 11 (Segmentation fault). --------------------------------------------------------------------------
Standard Output
[e621f6f40f56:18501] *** Process received signal *** [e621f6f40f56:18501] Signal: Segmentation fault (11) [e621f6f40f56:18501] Signal code: Address not mapped (1) [e621f6f40f56:18501] Failing at address: 0x18 [e621f6f40f56:18501] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f612a40a420] [e621f6f40f56:18501] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f6128c6235d] [e621f6f40f56:18501] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f6128c6647d] [e621f6f40f56:18501] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f6128c71926] [e621f6f40f56:18501] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f6128ba9c90] [e621f6f40f56:18501] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f6128bb4b5e] [e621f6f40f56:18501] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [e621f6f40f56:18501] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [e621f6f40f56:18501] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [e621f6f40f56:18501] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [e621f6f40f56:18501] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f61282d1083] [e621f6f40f56:18501] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [e621f6f40f56:18501] *** End of error message *** [e621f6f40f56:18506] *** Process received signal *** [e621f6f40f56:18506] Signal: Segmentation fault (11) [e621f6f40f56:18506] Signal code: Address not mapped (1) [e621f6f40f56:18506] Failing at address: 0x18 [e621f6f40f56:18506] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fb4ebc5d420] [e621f6f40f56:18506] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fb4ea4b535d] [e621f6f40f56:18506] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fb4ea4b947d] [e621f6f40f56:18506] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fb4ea4c4926] [e621f6f40f56:18506] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fb4ea3fcc90] [e621f6f40f56:18506] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fb4ea407b5e] [e621f6f40f56:18506] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [e621f6f40f56:18506] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [e621f6f40f56:18506] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [e621f6f40f56:18506] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [e621f6f40f56:18506] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fb4e9b24083] [e621f6f40f56:18506] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [e621f6f40f56:18506] *** End of error message *** [e621f6f40f56:18496] *** Process received signal *** [e621f6f40f56:18496] Signal: Segmentation fault (11) [e621f6f40f56:18496] Signal code: Address not mapped (1) [e621f6f40f56:18496] Failing at address: 0x18 [e621f6f40f56:18496] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fa2956c7420] [e621f6f40f56:18496] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fa293f1f35d] [e621f6f40f56:18496] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fa293f2347d] [e621f6f40f56:18496] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fa293f2e926] [e621f6f40f56:18496] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fa293e66c90] [e621f6f40f56:18496] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fa293e71b5e] [e621f6f40f56:18496] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [e621f6f40f56:18496] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [e621f6f40f56:18496] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [e621f6f40f56:18496] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [e621f6f40f56:18496] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fa29358e083] [e621f6f40f56:18496] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [e621f6f40f56:18496] *** End of error message *** [e621f6f40f56:18495] *** Process received signal *** [e621f6f40f56:18495] Signal: Segmentation fault (11) [e621f6f40f56:18495] Signal code: Address not mapped (1) [e621f6f40f56:18495] Failing at address: 0x18 [e621f6f40f56:18495] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f7e4e109420] [e621f6f40f56:18495] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f7e4c96135d] [e621f6f40f56:18495] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f7e4c96547d] [e621f6f40f56:18495] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f7e4c970926] [e621f6f40f56:18495] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f7e4c8a8c90] [e621f6f40f56:18495] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f7e4c8b3b5e] [e621f6f40f56:18495] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [e621f6f40f56:18495] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [e621f6f40f56:18495] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [e621f6f40f56:18495] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [e621f6f40f56:18495] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f7e4bfd0083] [e621f6f40f56:18495] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [e621f6f40f56:18495] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 2 with PID 0 on node e621f6f40f56 exited on signal 11 (Segmentation fault). --------------------------------------------------------------------------