Regression
Build / HIP-5.6 / projectroot.benchmarks.distributed_tree_driver.ArborX_Benchmark_DistributedTree (from CTest)
Failing for the past 1 build
(Since Failed
)
Stacktrace
ArborX version: 1.6 (dev) ArborX hash : ef959524 Kokkos version: 4.2.0 [e140bc562587:18745] *** Process received signal *** [e140bc562587:18745] Signal: Segmentation fault (11) [e140bc562587:18745] Signal code: Address not mapped (1) [e140bc562587:18745] Failing at address: 0x18 [e140bc562587:18745] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f2058269420] [e140bc562587:18745] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f2056ac135d] [e140bc562587:18745] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f2056ac547d] [e140bc562587:18745] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f2056ad0926] [e140bc562587:18745] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f2056a08c90] [e140bc562587:18745] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f2056a13b5e] [e140bc562587:18745] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb] [e140bc562587:18745] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc] [e140bc562587:18745] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8] [e140bc562587:18745] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [e140bc562587:18745] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f2056130083] [e140bc562587:18745] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [e140bc562587:18745] *** End of error message *** [e140bc562587:18748] *** Process received signal *** [e140bc562587:18748] Signal: Segmentation fault (11) [e140bc562587:18748] Signal code: Address not mapped (1) [e140bc562587:18748] Failing at address: 0x18 [e140bc562587:18748] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fb29ad06420] [e140bc562587:18748] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fb29955e35d] [e140bc562587:18748] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fb29956247d] [e140bc562587:18748] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fb29956d926] [e140bc562587:18748] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fb2994a5c90] [e140bc562587:18748] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fb2994b0b5e] [e140bc562587:18748] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb] [e140bc562587:18748] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc] [e140bc562587:18748] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8] [e140bc562587:18748] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [e140bc562587:18748] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fb298bcd083] [e140bc562587:18748] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [e140bc562587:18748] *** End of error message *** [e140bc562587:18753] *** Process received signal *** [e140bc562587:18753] Signal: Segmentation fault (11) [e140bc562587:18753] Signal code: Address not mapped (1) [e140bc562587:18753] Failing at address: 0x18 [e140bc562587:18753] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f5030c42420] [e140bc562587:18753] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f502f49a35d] [e140bc562587:18753] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f502f49e47d] [e140bc562587:18753] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f502f4a9926] [e140bc562587:18753] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f502f3e1c90] [e140bc562587:18753] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f502f3ecb5e] [e140bc562587:18753] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb] [e140bc562587:18753] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc] [e140bc562587:18753] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8] [e140bc562587:18753] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [e140bc562587:18753] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f502eb09083] [e140bc562587:18753] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [e140bc562587:18753] *** End of error message *** [e140bc562587:18756] *** Process received signal *** [e140bc562587:18756] Signal: Segmentation fault (11) [e140bc562587:18756] Signal code: Address not mapped (1) [e140bc562587:18756] Failing at address: 0x18 [e140bc562587:18756] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f8e29ad6420] [e140bc562587:18756] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f8e2832e35d] [e140bc562587:18756] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f8e2833247d] [e140bc562587:18756] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f8e2833d926] [e140bc562587:18756] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f8e28275c90] [e140bc562587:18756] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f8e28280b5e] [e140bc562587:18756] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb] [e140bc562587:18756] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc] [e140bc562587:18756] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8] [e140bc562587:18756] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [e140bc562587:18756] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f8e2799d083] [e140bc562587:18756] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [e140bc562587:18756] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 0 with PID 0 on node e140bc562587 exited on signal 11 (Segmentation fault). --------------------------------------------------------------------------
Standard Output
ArborX version: 1.6 (dev) ArborX hash : ef959524 Kokkos version: 4.2.0 [e140bc562587:18745] *** Process received signal *** [e140bc562587:18745] Signal: Segmentation fault (11) [e140bc562587:18745] Signal code: Address not mapped (1) [e140bc562587:18745] Failing at address: 0x18 [e140bc562587:18745] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f2058269420] [e140bc562587:18745] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f2056ac135d] [e140bc562587:18745] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f2056ac547d] [e140bc562587:18745] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f2056ad0926] [e140bc562587:18745] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f2056a08c90] [e140bc562587:18745] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f2056a13b5e] [e140bc562587:18745] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb] [e140bc562587:18745] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc] [e140bc562587:18745] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8] [e140bc562587:18745] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [e140bc562587:18745] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f2056130083] [e140bc562587:18745] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [e140bc562587:18745] *** End of error message *** [e140bc562587:18748] *** Process received signal *** [e140bc562587:18748] Signal: Segmentation fault (11) [e140bc562587:18748] Signal code: Address not mapped (1) [e140bc562587:18748] Failing at address: 0x18 [e140bc562587:18748] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fb29ad06420] [e140bc562587:18748] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fb29955e35d] [e140bc562587:18748] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fb29956247d] [e140bc562587:18748] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fb29956d926] [e140bc562587:18748] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fb2994a5c90] [e140bc562587:18748] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fb2994b0b5e] [e140bc562587:18748] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb] [e140bc562587:18748] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc] [e140bc562587:18748] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8] [e140bc562587:18748] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [e140bc562587:18748] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fb298bcd083] [e140bc562587:18748] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [e140bc562587:18748] *** End of error message *** [e140bc562587:18753] *** Process received signal *** [e140bc562587:18753] Signal: Segmentation fault (11) [e140bc562587:18753] Signal code: Address not mapped (1) [e140bc562587:18753] Failing at address: 0x18 [e140bc562587:18753] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f5030c42420] [e140bc562587:18753] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f502f49a35d] [e140bc562587:18753] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f502f49e47d] [e140bc562587:18753] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f502f4a9926] [e140bc562587:18753] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f502f3e1c90] [e140bc562587:18753] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f502f3ecb5e] [e140bc562587:18753] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb] [e140bc562587:18753] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc] [e140bc562587:18753] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8] [e140bc562587:18753] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [e140bc562587:18753] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f502eb09083] [e140bc562587:18753] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [e140bc562587:18753] *** End of error message *** [e140bc562587:18756] *** Process received signal *** [e140bc562587:18756] Signal: Segmentation fault (11) [e140bc562587:18756] Signal code: Address not mapped (1) [e140bc562587:18756] Failing at address: 0x18 [e140bc562587:18756] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f8e29ad6420] [e140bc562587:18756] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f8e2832e35d] [e140bc562587:18756] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f8e2833247d] [e140bc562587:18756] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f8e2833d926] [e140bc562587:18756] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f8e28275c90] [e140bc562587:18756] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f8e28280b5e] [e140bc562587:18756] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1eb] [e140bc562587:18756] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bcc] [e140bc562587:18756] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23af8] [e140bc562587:18756] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [e140bc562587:18756] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f8e2799d083] [e140bc562587:18756] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [e140bc562587:18756] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 0 with PID 0 on node e140bc562587 exited on signal 11 (Segmentation fault). --------------------------------------------------------------------------