Skip to content

Regression

Build / HIP-5.6 / projectroot.benchmarks.distributed_tree_driver.ArborX_Benchmark_DistributedTree (from CTest)

Failing for the past 1 build (Since Failed #57 )
Took 12 sec.

Stacktrace

ArborX version: 1.6 (dev)
ArborX hash   : b4f6a161
Kokkos version: 4.2.0
[9cc6956df2dc:19137] *** Process received signal ***
[9cc6956df2dc:19137] Signal: Segmentation fault (11)
[9cc6956df2dc:19137] Signal code: Address not mapped (1)
[9cc6956df2dc:19137] Failing at address: 0x18
[9cc6956df2dc:19137] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f5637464420]
[9cc6956df2dc:19137] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f5635cbc35d]
[9cc6956df2dc:19137] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f5635cc047d]
[9cc6956df2dc:19137] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f5635ccb926]
[9cc6956df2dc:19137] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f5635c03c90]
[9cc6956df2dc:19137] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f5635c0eb5e]
[9cc6956df2dc:19137] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e92b]
[9cc6956df2dc:19137] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe2430c]
[9cc6956df2dc:19137] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe24238]
[9cc6956df2dc:19137] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb305ac]
[9cc6956df2dc:19137] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f563532b083]
[9cc6956df2dc:19137] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb301ae]
[9cc6956df2dc:19137] *** End of error message ***
[9cc6956df2dc:19138] *** Process received signal ***
[9cc6956df2dc:19138] Signal: Segmentation fault (11)
[9cc6956df2dc:19138] Signal code: Address not mapped (1)
[9cc6956df2dc:19138] Failing at address: 0x18
[9cc6956df2dc:19138] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fa4b59a0420]
[9cc6956df2dc:19138] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fa4b41f835d]
[9cc6956df2dc:19138] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fa4b41fc47d]
[9cc6956df2dc:19138] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fa4b4207926]
[9cc6956df2dc:19138] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fa4b413fc90]
[9cc6956df2dc:19138] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fa4b414ab5e]
[9cc6956df2dc:19138] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e92b]
[9cc6956df2dc:19138] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe2430c]
[9cc6956df2dc:19138] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe24238]
[9cc6956df2dc:19138] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb305ac]
[9cc6956df2dc:19138] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fa4b3867083]
[9cc6956df2dc:19138] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb301ae]
[9cc6956df2dc:19138] *** End of error message ***
[9cc6956df2dc:19148] *** Process received signal ***
[9cc6956df2dc:19148] Signal: Segmentation fault (11)
[9cc6956df2dc:19148] Signal code: Address not mapped (1)
[9cc6956df2dc:19148] Failing at address: 0x18
[9cc6956df2dc:19148] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fbffc58e420]
[9cc6956df2dc:19148] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fbffade635d]
[9cc6956df2dc:19148] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fbffadea47d]
[9cc6956df2dc:19148] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fbffadf5926]
[9cc6956df2dc:19148] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fbffad2dc90]
[9cc6956df2dc:19148] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fbffad38b5e]
[9cc6956df2dc:19148] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e92b]
[9cc6956df2dc:19148] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe2430c]
[9cc6956df2dc:19148] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe24238]
[9cc6956df2dc:19148] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb305ac]
[9cc6956df2dc:19148] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fbffa455083]
[9cc6956df2dc:19148] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb301ae]
[9cc6956df2dc:19148] *** End of error message ***
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
[9cc6956df2dc:19143] *** Process received signal ***
[9cc6956df2dc:19143] Signal: Segmentation fault (11)
[9cc6956df2dc:19143] Signal code: Address not mapped (1)
[9cc6956df2dc:19143] Failing at address: 0x18
[9cc6956df2dc:19143] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f91059fd420]
[9cc6956df2dc:19143] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f910425535d]
[9cc6956df2dc:19143] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f910425947d]
[9cc6956df2dc:19143] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f9104264926]
[9cc6956df2dc:19143] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f910419cc90]
[9cc6956df2dc:19143] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f91041a7b5e]
[9cc6956df2dc:19143] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e92b]
[9cc6956df2dc:19143] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe2430c]
[9cc6956df2dc:19143] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe24238]
[9cc6956df2dc:19143] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb305ac]
[9cc6956df2dc:19143] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f91038c4083]
[9cc6956df2dc:19143] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb301ae]
[9cc6956df2dc:19143] *** End of error message ***
--------------------------------------------------------------------------
mpiexec noticed that process rank 3 with PID 0 on node 9cc6956df2dc exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------

Standard Output

ArborX version: 1.6 (dev)
ArborX hash   : b4f6a161
Kokkos version: 4.2.0
[9cc6956df2dc:19137] *** Process received signal ***
[9cc6956df2dc:19137] Signal: Segmentation fault (11)
[9cc6956df2dc:19137] Signal code: Address not mapped (1)
[9cc6956df2dc:19137] Failing at address: 0x18
[9cc6956df2dc:19137] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f5637464420]
[9cc6956df2dc:19137] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f5635cbc35d]
[9cc6956df2dc:19137] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f5635cc047d]
[9cc6956df2dc:19137] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f5635ccb926]
[9cc6956df2dc:19137] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f5635c03c90]
[9cc6956df2dc:19137] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f5635c0eb5e]
[9cc6956df2dc:19137] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e92b]
[9cc6956df2dc:19137] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe2430c]
[9cc6956df2dc:19137] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe24238]
[9cc6956df2dc:19137] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb305ac]
[9cc6956df2dc:19137] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f563532b083]
[9cc6956df2dc:19137] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb301ae]
[9cc6956df2dc:19137] *** End of error message ***
[9cc6956df2dc:19138] *** Process received signal ***
[9cc6956df2dc:19138] Signal: Segmentation fault (11)
[9cc6956df2dc:19138] Signal code: Address not mapped (1)
[9cc6956df2dc:19138] Failing at address: 0x18
[9cc6956df2dc:19138] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fa4b59a0420]
[9cc6956df2dc:19138] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fa4b41f835d]
[9cc6956df2dc:19138] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fa4b41fc47d]
[9cc6956df2dc:19138] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fa4b4207926]
[9cc6956df2dc:19138] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fa4b413fc90]
[9cc6956df2dc:19138] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fa4b414ab5e]
[9cc6956df2dc:19138] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e92b]
[9cc6956df2dc:19138] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe2430c]
[9cc6956df2dc:19138] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe24238]
[9cc6956df2dc:19138] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb305ac]
[9cc6956df2dc:19138] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fa4b3867083]
[9cc6956df2dc:19138] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb301ae]
[9cc6956df2dc:19138] *** End of error message ***
[9cc6956df2dc:19148] *** Process received signal ***
[9cc6956df2dc:19148] Signal: Segmentation fault (11)
[9cc6956df2dc:19148] Signal code: Address not mapped (1)
[9cc6956df2dc:19148] Failing at address: 0x18
[9cc6956df2dc:19148] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fbffc58e420]
[9cc6956df2dc:19148] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fbffade635d]
[9cc6956df2dc:19148] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fbffadea47d]
[9cc6956df2dc:19148] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fbffadf5926]
[9cc6956df2dc:19148] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fbffad2dc90]
[9cc6956df2dc:19148] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fbffad38b5e]
[9cc6956df2dc:19148] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e92b]
[9cc6956df2dc:19148] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe2430c]
[9cc6956df2dc:19148] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe24238]
[9cc6956df2dc:19148] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb305ac]
[9cc6956df2dc:19148] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fbffa455083]
[9cc6956df2dc:19148] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb301ae]
[9cc6956df2dc:19148] *** End of error message ***
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
[9cc6956df2dc:19143] *** Process received signal ***
[9cc6956df2dc:19143] Signal: Segmentation fault (11)
[9cc6956df2dc:19143] Signal code: Address not mapped (1)
[9cc6956df2dc:19143] Failing at address: 0x18
[9cc6956df2dc:19143] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f91059fd420]
[9cc6956df2dc:19143] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f910425535d]
[9cc6956df2dc:19143] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f910425947d]
[9cc6956df2dc:19143] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f9104264926]
[9cc6956df2dc:19143] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f910419cc90]
[9cc6956df2dc:19143] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f91041a7b5e]
[9cc6956df2dc:19143] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e92b]
[9cc6956df2dc:19143] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe2430c]
[9cc6956df2dc:19143] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe24238]
[9cc6956df2dc:19143] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb305ac]
[9cc6956df2dc:19143] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f91038c4083]
[9cc6956df2dc:19143] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb301ae]
[9cc6956df2dc:19143] *** End of error message ***
--------------------------------------------------------------------------
mpiexec noticed that process rank 3 with PID 0 on node 9cc6956df2dc exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------