Regression
Build / HIP-5.6 / projectroot.benchmarks.distributed_tree_driver.ArborX_Benchmark_DistributedTree (from CTest)
Failing for the past 1 build
(Since Failed
)
Stacktrace
ArborX version: 1.6 (dev) ArborX hash : ea784f69 Kokkos version: 4.2.0 [54febd312b64:18616] *** Process received signal *** [54febd312b64:18616] Signal: Segmentation fault (11) [54febd312b64:18616] Signal code: Address not mapped (1) [54febd312b64:18616] Failing at address: 0x18 [54febd312b64:18616] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f7636af6420] [54febd312b64:18616] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f763534e35d] [54febd312b64:18616] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f763535247d] [54febd312b64:18616] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f763535d926] [54febd312b64:18616] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f7635295c90] [54febd312b64:18616] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f76352a0b5e] [54febd312b64:18616] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1cb] [54febd312b64:18616] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bac] [54febd312b64:18616] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23ad8] [54febd312b64:18616] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [54febd312b64:18616] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f76349bd083] [54febd312b64:18616] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [54febd312b64:18616] *** End of error message *** [54febd312b64:18613] *** Process received signal *** [54febd312b64:18613] Signal: Segmentation fault (11) [54febd312b64:18613] Signal code: Address not mapped (1) [54febd312b64:18613] Failing at address: 0x18 [54febd312b64:18613] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7ff305177420] [54febd312b64:18613] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7ff3039cf35d] [54febd312b64:18613] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7ff3039d347d] [54febd312b64:18613] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7ff3039de926] [54febd312b64:18613] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7ff303916c90] [54febd312b64:18613] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7ff303921b5e] [54febd312b64:18613] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1cb] [54febd312b64:18613] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bac] [54febd312b64:18613] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23ad8] [54febd312b64:18613] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [54febd312b64:18613] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7ff30303e083] [54febd312b64:18613] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [54febd312b64:18613] *** End of error message *** [54febd312b64:18605] *** Process received signal *** [54febd312b64:18605] Signal: Segmentation fault (11) [54febd312b64:18605] Signal code: Address not mapped (1) [54febd312b64:18605] Failing at address: 0x18 [54febd312b64:18605] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f4f79f0d420] [54febd312b64:18605] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f4f7876535d] [54febd312b64:18605] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f4f7876947d] [54febd312b64:18605] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f4f78774926] [54febd312b64:18605] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f4f786acc90] [54febd312b64:18605] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f4f786b7b5e] [54febd312b64:18605] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1cb] [54febd312b64:18605] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bac] [54febd312b64:18605] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23ad8] [54febd312b64:18605] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [54febd312b64:18605] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f4f77dd4083] [54febd312b64:18605] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [54febd312b64:18605] *** End of error message *** [54febd312b64:18608] *** Process received signal *** [54febd312b64:18608] Signal: Segmentation fault (11) [54febd312b64:18608] Signal code: Address not mapped (1) [54febd312b64:18608] Failing at address: 0x18 [54febd312b64:18608] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f447ed4f420] [54febd312b64:18608] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f447d5a735d] [54febd312b64:18608] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f447d5ab47d] [54febd312b64:18608] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f447d5b6926] [54febd312b64:18608] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f447d4eec90] [54febd312b64:18608] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f447d4f9b5e] [54febd312b64:18608] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1cb] [54febd312b64:18608] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bac] [54febd312b64:18608] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23ad8] [54febd312b64:18608] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [54febd312b64:18608] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f447cc16083] [54febd312b64:18608] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [54febd312b64:18608] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 3 with PID 0 on node 54febd312b64 exited on signal 11 (Segmentation fault). --------------------------------------------------------------------------
Standard Output
ArborX version: 1.6 (dev) ArborX hash : ea784f69 Kokkos version: 4.2.0 [54febd312b64:18616] *** Process received signal *** [54febd312b64:18616] Signal: Segmentation fault (11) [54febd312b64:18616] Signal code: Address not mapped (1) [54febd312b64:18616] Failing at address: 0x18 [54febd312b64:18616] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f7636af6420] [54febd312b64:18616] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f763534e35d] [54febd312b64:18616] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f763535247d] [54febd312b64:18616] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f763535d926] [54febd312b64:18616] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f7635295c90] [54febd312b64:18616] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f76352a0b5e] [54febd312b64:18616] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1cb] [54febd312b64:18616] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bac] [54febd312b64:18616] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23ad8] [54febd312b64:18616] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [54febd312b64:18616] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f76349bd083] [54febd312b64:18616] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [54febd312b64:18616] *** End of error message *** [54febd312b64:18613] *** Process received signal *** [54febd312b64:18613] Signal: Segmentation fault (11) [54febd312b64:18613] Signal code: Address not mapped (1) [54febd312b64:18613] Failing at address: 0x18 [54febd312b64:18613] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7ff305177420] [54febd312b64:18613] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7ff3039cf35d] [54febd312b64:18613] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7ff3039d347d] [54febd312b64:18613] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7ff3039de926] [54febd312b64:18613] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7ff303916c90] [54febd312b64:18613] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7ff303921b5e] [54febd312b64:18613] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1cb] [54febd312b64:18613] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bac] [54febd312b64:18613] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23ad8] [54febd312b64:18613] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [54febd312b64:18613] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7ff30303e083] [54febd312b64:18613] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [54febd312b64:18613] *** End of error message *** [54febd312b64:18605] *** Process received signal *** [54febd312b64:18605] Signal: Segmentation fault (11) [54febd312b64:18605] Signal code: Address not mapped (1) [54febd312b64:18605] Failing at address: 0x18 [54febd312b64:18605] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f4f79f0d420] [54febd312b64:18605] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f4f7876535d] [54febd312b64:18605] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f4f7876947d] [54febd312b64:18605] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f4f78774926] [54febd312b64:18605] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f4f786acc90] [54febd312b64:18605] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f4f786b7b5e] [54febd312b64:18605] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1cb] [54febd312b64:18605] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bac] [54febd312b64:18605] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23ad8] [54febd312b64:18605] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [54febd312b64:18605] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f4f77dd4083] [54febd312b64:18605] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [54febd312b64:18605] *** End of error message *** [54febd312b64:18608] *** Process received signal *** [54febd312b64:18608] Signal: Segmentation fault (11) [54febd312b64:18608] Signal code: Address not mapped (1) [54febd312b64:18608] Failing at address: 0x18 [54febd312b64:18608] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f447ed4f420] [54febd312b64:18608] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f447d5a735d] [54febd312b64:18608] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f447d5ab47d] [54febd312b64:18608] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f447d5b6926] [54febd312b64:18608] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f447d4eec90] [54febd312b64:18608] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f447d4f9b5e] [54febd312b64:18608] [ 6] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe4e1cb] [54febd312b64:18608] [ 7] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23bac] [54febd312b64:18608] [ 8] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xe23ad8] [54febd312b64:18608] [ 9] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb303fc] [54febd312b64:18608] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f447cc16083] [54febd312b64:18608] [11] /var/jenkins/workspace/ArborX_master/build/benchmarks/distributed_tree_driver/ArborX_Benchmark_DistributedTree.exe[0xb2fffe] [54febd312b64:18608] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 3 with PID 0 on node 54febd312b64 exited on signal 11 (Segmentation fault). --------------------------------------------------------------------------