Skip to content

Regression

Build / HIP-5.6 / projectroot.benchmarks.distributed_tree_driver.ArborX_Benchmark_DistributedTree (from CTest)

Failing for the past 1 build (Since Failed #108 )
Took 35 sec.

Stacktrace

ArborX version: 1.7 (dev)
ArborX hash   : a403dad8
Kokkos version: 4.2.0
[6a99dc0fa1df:28104] *** Process received signal ***
[6a99dc0fa1df:28104] Signal: Segmentation fault (11)
[6a99dc0fa1df:28104] Signal code: Address not mapped (1)
[6a99dc0fa1df:28104] Failing at address: 0x18
[6a99dc0fa1df:28104] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f1abb47d420]
[6a99dc0fa1df:28104] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f1ab9cd535d]
[6a99dc0fa1df:28104] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f1ab9cd947d]
[6a99dc0fa1df:28104] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f1ab9ce4926]
[6a99dc0fa1df:28104] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f1ab9c1cc90]
[6a99dc0fa1df:28104] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f1ab9c27b5e]
[6a99dc0fa1df:28104] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0x1006b4b]
[6a99dc0fa1df:28104] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc52c]
[6a99dc0fa1df:28104] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc458]
[6a99dc0fa1df:28104] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3fec]
[6a99dc0fa1df:28104] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f1ab9344083]
[6a99dc0fa1df:28104] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3bee]
[6a99dc0fa1df:28104] *** End of error message ***
[6a99dc0fa1df:28115] *** Process received signal ***
[6a99dc0fa1df:28115] Signal: Segmentation fault (11)
[6a99dc0fa1df:28115] Signal code: Address not mapped (1)
[6a99dc0fa1df:28115] Failing at address: 0x18
[6a99dc0fa1df:28115] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f67b8bc9420]
[6a99dc0fa1df:28115] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f67b742135d]
[6a99dc0fa1df:28115] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f67b742547d]
[6a99dc0fa1df:28115] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f67b7430926]
[6a99dc0fa1df:28115] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f67b7368c90]
[6a99dc0fa1df:28115] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f67b7373b5e]
[6a99dc0fa1df:28115] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0x1006b4b]
[6a99dc0fa1df:28115] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc52c]
[6a99dc0fa1df:28115] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc458]
[6a99dc0fa1df:28115] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3fec]
[6a99dc0fa1df:28115] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f67b6a90083]
[6a99dc0fa1df:28115] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3bee]
[6a99dc0fa1df:28115] *** End of error message ***
[6a99dc0fa1df:28110] *** Process received signal ***
[6a99dc0fa1df:28110] Signal: Segmentation fault (11)
[6a99dc0fa1df:28110] Signal code: Address not mapped (1)
[6a99dc0fa1df:28110] Failing at address: 0x18
[6a99dc0fa1df:28110] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fd5d5fb4420]
[6a99dc0fa1df:28110] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fd5d480c35d]
[6a99dc0fa1df:28110] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fd5d481047d]
[6a99dc0fa1df:28110] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fd5d481b926]
[6a99dc0fa1df:28110] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fd5d4753c90]
[6a99dc0fa1df:28110] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fd5d475eb5e]
[6a99dc0fa1df:28110] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0x1006b4b]
[6a99dc0fa1df:28110] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc52c]
[6a99dc0fa1df:28110] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc458]
[6a99dc0fa1df:28110] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3fec]
[6a99dc0fa1df:28110] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fd5d3e7b083]
[6a99dc0fa1df:28110] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3bee]
[6a99dc0fa1df:28110] *** End of error message ***
[6a99dc0fa1df:28105] *** Process received signal ***
[6a99dc0fa1df:28105] Signal: Segmentation fault (11)
[6a99dc0fa1df:28105] Signal code: Address not mapped (1)
[6a99dc0fa1df:28105] Failing at address: 0x18
[6a99dc0fa1df:28105] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fbb9dcf5420]
[6a99dc0fa1df:28105] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fbb9c54d35d]
[6a99dc0fa1df:28105] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fbb9c55147d]
[6a99dc0fa1df:28105] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fbb9c55c926]
[6a99dc0fa1df:28105] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fbb9c494c90]
[6a99dc0fa1df:28105] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fbb9c49fb5e]
[6a99dc0fa1df:28105] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0x1006b4b]
[6a99dc0fa1df:28105] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc52c]
[6a99dc0fa1df:28105] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc458]
[6a99dc0fa1df:28105] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3fec]
[6a99dc0fa1df:28105] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fbb9bbbc083]
[6a99dc0fa1df:28105] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3bee]
[6a99dc0fa1df:28105] *** End of error message ***
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpiexec noticed that process rank 3 with PID 0 on node 6a99dc0fa1df exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------

Standard Output

ArborX version: 1.7 (dev)
ArborX hash   : a403dad8
Kokkos version: 4.2.0
[6a99dc0fa1df:28104] *** Process received signal ***
[6a99dc0fa1df:28104] Signal: Segmentation fault (11)
[6a99dc0fa1df:28104] Signal code: Address not mapped (1)
[6a99dc0fa1df:28104] Failing at address: 0x18
[6a99dc0fa1df:28104]
[6a99dc0fa1df:28104 [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f1abb47d420]
[6a99dc0fa1df:28104] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f1ab9cd535d]
[6a99dc0fa1df:28104] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f1ab9cd947d]
[6a99dc0fa1df:28104] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f1ab9ce4926]
[6a99dc0fa1df:28104] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f1ab9c1cc90]
[6a99dc0fa1df:28104] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f1ab9c27b5e]
[6a99dc0fa1df:28104] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0x1006b4b]
[6a99dc0fa1df:28104] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc52c]
[6a99dc0fa1df:28104] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc458]
[6a99dc0fa1df:28104] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3fec]
[6a99dc0fa1df:28104] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f1ab9344083]
[6a99dc0fa1df:28104] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3bee]
[6a99dc0fa1df:28104] *** End of error message ***
[6a99dc0fa1df:28115] *** Process received signal ***
[6a99dc0fa1df:28115] Signal: Segmentation fault (11)
[6a99dc0fa1df:28115] Signal code: Address not mapped (1)
[6a99dc0fa1df:28115] Failing at address: 0x18
[6a99dc0fa1df:28115] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f67b8bc9420]
[6a99dc0fa1df:28115] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f67b742135d]
[6a99dc0fa1df:28115] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f67b742547d]
[6a99dc0fa1df:28115] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f67b7430926]
[6a99dc0fa1df:28115] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f67b7368c90]
[6a99dc0fa1df:28115] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f67b7373b5e]
[6a99dc0fa1df:28115] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0x1006b4b]
[6a99dc0fa1df:28115] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc52c]
[6a99dc0fa1df:28115] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc458]
[6a99dc0fa1df:28115] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3fec]
[6a99dc0fa1df:28115] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f67b6a90083]
[6a99dc0fa1df:28115] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3bee]
[6a99dc0fa1df:28115] *** End of error message ***
[6a99dc0fa1df:28110] *** Process received signal ***
[6a99dc0fa1df:28110] Signal: Segmentation fault (11)
[6a99dc0fa1df:28110] Signal code: Address not mapped (1)
[6a99dc0fa1df:28110] Failing at address: 0x18
[6a99dc0fa1df:28110] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fd5d5fb4420]
[6a99dc0fa1df:28110] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fd5d480c35d]
[6a99dc0fa1df:28110] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fd5d481047d]
[6a99dc0fa1df:28110] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fd5d481b926]
[6a99dc0fa1df:28110] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fd5d4753c90]
[6a99dc0fa1df:28110] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fd5d475eb5e]
[6a99dc0fa1df:28110] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0x1006b4b]
[6a99dc0fa1df:28110] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc52c]
[6a99dc0fa1df:28110] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc458]
[6a99dc0fa1df:28110] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3fec]
[6a99dc0fa1df:28110] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fd5d3e7b083]
[6a99dc0fa1df:28110] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3bee]
[6a99dc0fa1df:28110] *** End of error message ***
[6a99dc0fa1df:28105] *** Process received signal ***
[6a99dc0fa1df:28105] Signal: Segmentation fault (11)
[6a99dc0fa1df:28105] Signal code: Address not mapped (1)
[6a99dc0fa1df:28105] Failing at address: 0x18
[6a99dc0fa1df:28105] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fbb9dcf5420]
[6a99dc0fa1df:28105] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fbb9c54d35d]
[6a99dc0fa1df:28105] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fbb9c55147d]
[6a99dc0fa1df:28105] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fbb9c55c926]
[6a99dc0fa1df:28105] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fbb9c494c90]
[6a99dc0fa1df:28105] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fbb9c49fb5e]
[6a99dc0fa1df:28105] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0x1006b4b]
[6a99dc0fa1df:28105] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc52c]
[6a99dc0fa1df:28105] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xfdc458]
[6a99dc0fa1df:28105] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3fec]
[6a99dc0fa1df:28105] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fbb9bbbc083]
[6a99dc0fa1df:28105] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xcb3bee]
[6a99dc0fa1df:28105] *** End of error message ***
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpiexec noticed that process rank 3 with PID 0 on node 6a99dc0fa1df exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------