![[Jenkins]](/jenkins-ci/static/2d70a61a/images/svgs/logo.svg){kind=logo}
Failed
Build / HIP-5.6 / projectroot.examples.distributed_tree.ArborX_Example_DistributedTree_KNN (from CTest)
Failing for the past 1 build
(Since Failed
)
Stacktrace
terminate called after throwing an instance of 'std::runtime_error' terminate called after throwing an instance of 'std::runtime_error' what(): Kokkos failed to allocate memory for label "ArborX::DistributedTree::query::nearest::phaseI::distances". Allocation using MemorySpace named "HIP" failed with the following error: Allocation of size 1.718e+10 G failed, likely due to insufficient memory. (The allocation mechanism was hipMalloc(). The HIP allocation returned the error code "hipErrorOutOfMemory".) what(): Kokkos failed to allocate memory for label "ArborX::DistributedTree::query::nearest::phaseI::distances". Allocation using MemorySpace named "HIP" failed with the following error: Allocation of size 1.718e+10 G failed, likely due to insufficient memory. (The allocation mechanism was hipMalloc(). The HIP allocation returned the error code "hipErrorOutOfMemory".) [f4be2b48863f:28311] *** Process received signal *** [f4be2b48863f:28313] *** Process received signal *** [f4be2b48863f:28313] Signal: Aborted (6) [f4be2b48863f:28313] Signal code: (-6) [f4be2b48863f:28311] Signal: Aborted (6) [f4be2b48863f:28311] Signal code: (-6) [f4be2b48863f:28313] [ 0] [f4be2b48863f:28311] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f6c47f02420] [f4be2b48863f:28311] [ 1] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f8cf8f8a420] [f4be2b48863f:28313] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7f8cf6e7000b] [f4be2b48863f:28313] [ 2] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7f6c45de800b] [f4be2b48863f:28311] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7f6c45dc7859] /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7f8cf6e4f859] [f4be2b48863f:28313] [ 3] [f4be2b48863f:28311] [ 3] /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e8d1)[0x7f8cf72278d1] [f4be2b48863f:28313] [ 4] /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e8d1)[0x7f6c4619f8d1] [f4be2b48863f:28311] [ 4] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa37c)[0x7f6c461ab37c] [f4be2b48863f:28311] [ 5] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa37c)[0x7f8cf723337c] [f4be2b48863f:28313] [ 5] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3e7)[0x7f6c461ab3e7] [f4be2b48863f:28311] [ 6] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3e7)[0x7f8cf72333e7] [f4be2b48863f:28313] [ 6] Memory access fault by GPU node-33 (Agent handle: 0x2dd9a50) on address 0x7ef2d015f000. Reason: Unknown. /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa699)[0x7f6c461ab699] [f4be2b48863f:28311] [ 7] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa59cd1] [f4be2b48863f:28311] [ 8] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa5dd7f] [f4be2b48863f:28311] [ 9] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa6a8d0] [f4be2b48863f:28311] [10] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa6a621] [f4be2b48863f:28311] [11] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x928c14] [f4be2b48863f:28311] [12] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa699)[0x7f8cf7233699] [f4be2b48863f:28313] [ 7] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa59cd1] [f4be2b48863f:28313] [ 8] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa5dd7f] [f4be2b48863f:28313] [ 9] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa6a8d0] [f4be2b48863f:28313] [10] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa6a621] [f4be2b48863f:28313] [11] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x928c14] [f4be2b48863f:28313] [12] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x928994] [f4be2b48863f:28313] [13] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x928994] [f4be2b48863f:28311] [13] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x989d45] [f4be2b48863f:28311] [14] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x989538] [f4be2b48863f:28311] [15] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x984222] [f4be2b48863f:28311] [16] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x989d45] [f4be2b48863f:28313] [14] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x989538] [f4be2b48863f:28313] [15] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x984222] [f4be2b48863f:28313] [16] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x983cce] [f4be2b48863f:28313] [17] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x9839e4] [f4be2b48863f:28313] [18] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x963754] [f4be2b48863f:28313] [19] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x929b97] [f4be2b48863f:28313] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x983cce] [f4be2b48863f:28311] [17] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x9839e4] [f4be2b48863f:28311] [18] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x963754] [f4be2b48863f:28311] [19] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x929b97] [f4be2b48863f:28311] [20] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x9266c1] [f4be2b48863f:28311] [21] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x9249b3] [f4be2b48863f:28311] [22] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x8fc5a4] [f4be2b48863f:28311] [f4be2b48863f:28312] *** Process received signal *** [20] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x9266c1] [f4be2b48863f:28313] [21] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x9249b3] [f4be2b48863f:28313] [22] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x8fc5a4] [f4be2b48863f:28313] [23] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x8f2cac] [f4be2b48863f:28313] [24] [f4be2b48863f:28312] Signal: Aborted (6) [f4be2b48863f:28312] Signal code: (-6) [23] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x8f2cac] [f4be2b48863f:28311] [24] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f8cf6e51083] [f4be2b48863f:28313] [25] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x8f28de] [f4be2b48863f:28313] *** End of error message *** /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f6c45dc9083] [f4be2b48863f:28311] [25] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x8f28de] [f4be2b48863f:28311] *** End of error message *** [f4be2b48863f:28312] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7efd10457420] [f4be2b48863f:28312] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7efd0e33d00b] [f4be2b48863f:28312] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7efd0e31c859] [f4be2b48863f:28312] [ 3] /opt/rocm-5.6.0/lib/libhsa-runtime64.so.1(+0x243ef)[0x7efd04f573ef] [f4be2b48863f:28312] [ 4] /opt/rocm-5.6.0/lib/libhsa-runtime64.so.1(+0x6955c)[0x7efd04f9c55c] [f4be2b48863f:28312] [ 5] /opt/rocm-5.6.0/lib/libhsa-runtime64.so.1(+0x2840b)[0x7efd04f5b40b] [f4be2b48863f:28312] [ 6] /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609)[0x7efd1044b609] [f4be2b48863f:28312] [ 7] /lib/x86_64-linux-gnu/libc.so.6(clone+0x43)[0x7efd0e419133] [f4be2b48863f:28312] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 0 with PID 0 on node f4be2b48863f exited on signal 6 (Aborted). --------------------------------------------------------------------------
Standard Output
terminate called after throwing an instance of 'std::runtime_error' terminate called after throwing an instance of 'std::runtime_error' what(): Kokkos failed to allocate memory for label "ArborX::DistributedTree::query::nearest::phaseI::distances". Allocation using MemorySpace named "HIP" failed with the following error: Allocation of size 1.718e+10 G failed, likely due to insufficient memory. (The allocation mechanism was hipMalloc(). The HIP allocation returned the error code "hipErrorOutOfMemory".) what(): Kokkos failed to allocate memory for label "ArborX::DistributedTree::query::nearest::phaseI::distances". Allocation using MemorySpace named "HIP" failed with the following error: Allocation of size 1.718e+10 G failed, likely due to insufficient memory. (The allocation mechanism was hipMalloc(). The HIP allocation returned the error code "hipErrorOutOfMemory".) [f4be2b48863f:28311] *** Process received signal *** [f4be2b48863f:28313] *** Process received signal *** [f4be2b48863f:28313] Signal: Aborted (6) [f4be2b48863f:28313] Signal code: (-6) [f4be2b48863f:28311] Signal: Aborted (6) [f4be2b48863f:28311] Signal code: (-6) [f4be2b48863f:28313] [ 0] [f4be2b48863f:28311] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f6c47f02420] [f4be2b48863f:28311] [ 1] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f8cf8f8a420] [f4be2b48863f:28313] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7f8cf6e7000b] [f4be2b48863f:28313] [ 2] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7f6c45de800b] [f4be2b48863f:28311] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7f6c45dc7859] /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7f8cf6e4f859] [f4be2b48863f:28313] [ 3] [f4be2b48863f:28311] [ 3] /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e8d1)[0x7f8cf72278d1] [f4be2b48863f:28313] [ 4] /lib/x86_64-linux-gnu/libstdc++.so.6(+0x9e8d1)[0x7f6c4619f8d1] [f4be2b48863f:28311] [ 4] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa37c)[0x7f6c461ab37c] [f4be2b48863f:28311] [ 5] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa37c)[0x7f8cf723337c] [f4be2b48863f:28313] [ 5] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3e7)[0x7f6c461ab3e7] [f4be2b48863f:28311] [ 6] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa3e7)[0x7f8cf72333e7] [f4be2b48863f:28313] [ 6] Memory access fault by GPU node-33 (Agent handle: 0x2dd9a50) on address 0x7ef2d015f000. Reason: Unknown. /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa699)[0x7f6c461ab699] [f4be2b48863f:28311] [ 7] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa59cd1] [f4be2b48863f:28311] [ 8] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa5dd7f] [f4be2b48863f:28311] [ 9] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa6a8d0] [f4be2b48863f:28311] [10] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa6a621] [f4be2b48863f:28311] [11] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x928c14] [f4be2b48863f:28311] [12] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xaa699)[0x7f8cf7233699] [f4be2b48863f:28313] [ 7] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa59cd1] [f4be2b48863f:28313] [ 8] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa5dd7f] [f4be2b48863f:28313] [ 9] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa6a8d0] [f4be2b48863f:28313] [10] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0xa6a621] [f4be2b48863f:28313] [11] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x928c14] [f4be2b48863f:28313] [12] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x928994] [f4be2b48863f:28313] [13] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x928994] [f4be2b48863f:28311] [13] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x989d45] [f4be2b48863f:28311] [14] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x989538] [f4be2b48863f:28311] [15] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x984222] [f4be2b48863f:28311] [16] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x989d45] [f4be2b48863f:28313] [14] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x989538] [f4be2b48863f:28313] [15] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x984222] [f4be2b48863f:28313] [16] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x983cce] [f4be2b48863f:28313] [17] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x9839e4] [f4be2b48863f:28313] [18] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x963754] [f4be2b48863f:28313] [19] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x929b97] [f4be2b48863f:28313] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x983cce] [f4be2b48863f:28311] [17] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x9839e4] [f4be2b48863f:28311] [18] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x963754] [f4be2b48863f:28311] [19] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x929b97] [f4be2b48863f:28311] [20] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x9266c1] [f4be2b48863f:28311] [21] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x9249b3] [f4be2b48863f:28311] [22] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x8fc5a4] [f4be2b48863f:28311] [f4be2b48863f:28312] *** Process received signal *** [20] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x9266c1] [f4be2b48863f:28313] [21] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x9249b3] [f4be2b48863f:28313] [22] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x8fc5a4] [f4be2b48863f:28313] [23] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x8f2cac] [f4be2b48863f:28313] [24] [f4be2b48863f:28312] Signal: Aborted (6) [f4be2b48863f:28312] Signal code: (-6) [23] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x8f2cac] [f4be2b48863f:28311] [24] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f8cf6e51083] [f4be2b48863f:28313] [25] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x8f28de] [f4be2b48863f:28313] *** End of error message *** /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f6c45dc9083] [f4be2b48863f:28311] [25] /var/jenkins/workspace/ArborX_PR-1073/build/examples/distributed_tree/ArborX_Example_DistributedTree_KNN.exe[0x8f28de] [f4be2b48863f:28311] *** End of error message *** [f4be2b48863f:28312] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7efd10457420] [f4be2b48863f:28312] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7efd0e33d00b] [f4be2b48863f:28312] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7efd0e31c859] [f4be2b48863f:28312] [ 3] /opt/rocm-5.6.0/lib/libhsa-runtime64.so.1(+0x243ef)[0x7efd04f573ef] [f4be2b48863f:28312] [ 4] /opt/rocm-5.6.0/lib/libhsa-runtime64.so.1(+0x6955c)[0x7efd04f9c55c] [f4be2b48863f:28312] [ 5] /opt/rocm-5.6.0/lib/libhsa-runtime64.so.1(+0x2840b)[0x7efd04f5b40b] [f4be2b48863f:28312] [ 6] /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609)[0x7efd1044b609] [f4be2b48863f:28312] [ 7] /lib/x86_64-linux-gnu/libc.so.6(clone+0x43)[0x7efd0e419133] [f4be2b48863f:28312] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 0 with PID 0 on node f4be2b48863f exited on signal 6 (Aborted). --------------------------------------------------------------------------