Skip to content

Regression

Build / HIP-5.6 / projectroot.benchmarks.distributed_tree_driver.ArborX_Benchmark_DistributedTree (from CTest)

Failing for the past 1 build (Since Failed #53 )
Took 13 sec.

Stacktrace

ArborX version: 1.6 (dev)
ArborX hash   : 5bfe666c
Kokkos version: 4.2.0
[8ec9c67b5d7f:18635] *** Process received signal ***
[8ec9c67b5d7f:18635] Signal: Segmentation fault (11)
[8ec9c67b5d7f:18635] Signal code: Address not mapped (1)
[8ec9c67b5d7f:18635] Failing at address: 0x18
[8ec9c67b5d7f:18635] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f3b64851420]
[8ec9c67b5d7f:18635] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f3b630a935d]
[8ec9c67b5d7f:18635] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f3b630ad47d]
[8ec9c67b5d7f:18635] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f3b630b8926]
[8ec9c67b5d7f:18635] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f3b62ff0c90]
[8ec9c67b5d7f:18635] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f3b62ffbb5e]
[8ec9c67b5d7f:18635] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb]
[8ec9c67b5d7f:18635] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc]
[8ec9c67b5d7f:18635] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8]
[8ec9c67b5d7f:18635] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc]
[8ec9c67b5d7f:18635] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f3b62718083]
[8ec9c67b5d7f:18635] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe]
[8ec9c67b5d7f:18635] *** End of error message ***
[8ec9c67b5d7f:18627] *** Process received signal ***
[8ec9c67b5d7f:18627] Signal: Segmentation fault (11)
[8ec9c67b5d7f:18627] Signal code: Address not mapped (1)
[8ec9c67b5d7f:18627] Failing at address: 0x18
[8ec9c67b5d7f:18627] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f78d2a20420]
[8ec9c67b5d7f:18627] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f78d127835d]
[8ec9c67b5d7f:18627] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f78d127c47d]
[8ec9c67b5d7f:18627] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f78d1287926]
[8ec9c67b5d7f:18627] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f78d11bfc90]
[8ec9c67b5d7f:18627] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f78d11cab5e]
[8ec9c67b5d7f:18627] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb]
[8ec9c67b5d7f:18627] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc]
[8ec9c67b5d7f:18627] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8]
[8ec9c67b5d7f:18627] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc]
[8ec9c67b5d7f:18627] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f78d08e7083]
[8ec9c67b5d7f:18627] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe]
[8ec9c67b5d7f:18627] *** End of error message ***
[8ec9c67b5d7f:18630] *** Process received signal ***
[8ec9c67b5d7f:18630] Signal: Segmentation fault (11)
[8ec9c67b5d7f:18630] Signal code: Address not mapped (1)
[8ec9c67b5d7f:18630] Failing at address: 0x18
[8ec9c67b5d7f:18630] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fd24c759420]
[8ec9c67b5d7f:18630] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fd24afb135d]
[8ec9c67b5d7f:18630] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fd24afb547d]
[8ec9c67b5d7f:18630] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fd24afc0926]
[8ec9c67b5d7f:18630] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fd24aef8c90]
[8ec9c67b5d7f:18630] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fd24af03b5e]
[8ec9c67b5d7f:18630] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb]
[8ec9c67b5d7f:18630] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc]
[8ec9c67b5d7f:18630] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8]
[8ec9c67b5d7f:18630] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc]
[8ec9c67b5d7f:18630] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fd24a620083]
[8ec9c67b5d7f:18630] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe]
[8ec9c67b5d7f:18630] *** End of error message ***
[8ec9c67b5d7f:18638] *** Process received signal ***
[8ec9c67b5d7f:18638] Signal: Segmentation fault (11)
[8ec9c67b5d7f:18638] Signal code: Address not mapped (1)
[8ec9c67b5d7f:18638] Failing at address: 0x18
[8ec9c67b5d7f:18638] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fe7b6b1e420]
[8ec9c67b5d7f:18638] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fe7b537635d]
[8ec9c67b5d7f:18638] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fe7b537a47d]
[8ec9c67b5d7f:18638] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fe7b5385926]
[8ec9c67b5d7f:18638] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fe7b52bdc90]
[8ec9c67b5d7f:18638] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fe7b52c8b5e]
[8ec9c67b5d7f:18638] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb]
[8ec9c67b5d7f:18638] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc]
[8ec9c67b5d7f:18638] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8]
[8ec9c67b5d7f:18638] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc]
[8ec9c67b5d7f:18638] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fe7b49e5083]
[8ec9c67b5d7f:18638] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe]
[8ec9c67b5d7f:18638] *** End of error message ***
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpiexec noticed that process rank 2 with PID 0 on node 8ec9c67b5d7f exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------

Standard Output

ArborX version: 1.6 (dev)
ArborX hash   : 5bfe666c
Kokkos version: 4.2.0
[8ec9c67b5d7f:18635] *** Process received signal ***
[8ec9c67b5d7f:18635] Signal: Segmentation fault (11)
[8ec9c67b5d7f:18635] Signal code: Address not mapped (1)
[8ec9c67b5d7f:18635] Failing at address: 0x18
[8ec9c67b5d7f:18635] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f3b64851420]
[8ec9c67b5d7f:18635] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f3b630a935d]
[8ec9c67b5d7f:18635] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f3b630ad47d]
[8ec9c67b5d7f:18635] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f3b630b8926]
[8ec9c67b5d7f:18635] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f3b62ff0c90]
[8ec9c67b5d7f:18635] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f3b62ffbb5e]
[8ec9c67b5d7f:18635] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb]
[8ec9c67b5d7f:18635] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc]
[8ec9c67b5d7f:18635] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8]
[8ec9c67b5d7f:18635] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc]
[8ec9c67b5d7f:18635] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f3b62718083]
[8ec9c67b5d7f:18635] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe]
[8ec9c67b5d7f:18635] *** End of error message ***
[8ec9c67b5d7f:18627] *** Process received signal ***
[8ec9c67b5d7f:18627] Signal: Segmentation fault (11)
[8ec9c67b5d7f:18627] Signal code: Address not mapped (1)
[8ec9c67b5d7f:18627] Failing at address: 0x18
[8ec9c67b5d7f:18627] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f78d2a20420]
[8ec9c67b5d7f:18627] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f78d127835d]
[8ec9c67b5d7f:18627] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f78d127c47d]
[8ec9c67b5d7f:18627] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f78d1287926]
[8ec9c67b5d7f:18627] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f78d11bfc90]
[8ec9c67b5d7f:18627] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f78d11cab5e]
[8ec9c67b5d7f:18627] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb]
[8ec9c67b5d7f:18627] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc]
[8ec9c67b5d7f:18627] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8]
[8ec9c67b5d7f:18627] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc]
[8ec9c67b5d7f:18627] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f78d08e7083]
[8ec9c67b5d7f:18627] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe]
[8ec9c67b5d7f:18627] *** End of error message ***
[8ec9c67b5d7f:18630] *** Process received signal ***
[8ec9c67b5d7f:18630] Signal: Segmentation fault (11)
[8ec9c67b5d7f:18630] Signal code: Address not mapped (1)
[8ec9c67b5d7f:18630] Failing at address: 0x18
[8ec9c67b5d7f:18630] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fd24c759420]
[8ec9c67b5d7f:18630] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fd24afb135d]
[8ec9c67b5d7f:18630] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fd24afb547d]
[8ec9c67b5d7f:18630] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fd24afc0926]
[8ec9c67b5d7f:18630] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fd24aef8c90]
[8ec9c67b5d7f:18630] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fd24af03b5e]
[8ec9c67b5d7f:18630] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb]
[8ec9c67b5d7f:18630] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc]
[8ec9c67b5d7f:18630] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8]
[8ec9c67b5d7f:18630] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc]
[8ec9c67b5d7f:18630] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fd24a620083]
[8ec9c67b5d7f:18630] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe]
[8ec9c67b5d7f:18630] *** End of error message ***
[8ec9c67b5d7f:18638] *** Process received signal ***
[8ec9c67b5d7f:18638] Signal: Segmentation fault (11)
[8ec9c67b5d7f:18638] Signal code: Address not mapped (1)
[8ec9c67b5d7f:18638] Failing at address: 0x18
[8ec9c67b5d7f:18638] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fe7b6b1e420]
[8ec9c67b5d7f:18638] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fe7b537635d]
[8ec9c67b5d7f:18638] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fe7b537a47d]
[8ec9c67b5d7f:18638] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fe7b5385926]
[8ec9c67b5d7f:18638] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fe7b52bdc90]
[8ec9c67b5d7f:18638] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fe7b52c8b5e]
[8ec9c67b5d7f:18638] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb]
[8ec9c67b5d7f:18638] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc]
[8ec9c67b5d7f:18638] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8]
[8ec9c67b5d7f:18638] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc]
[8ec9c67b5d7f:18638] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fe7b49e5083]
[8ec9c67b5d7f:18638] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe]
[8ec9c67b5d7f:18638] *** End of error message ***
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpiexec noticed that process rank 2 with PID 0 on node 8ec9c67b5d7f exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------