Regression
Build / HIP-5.6 / projectroot.test.ArborX_Test_DetailsDistributedTreeImpl (from CTest)
Failing for the past 1 build
(Since Not built
)
Stacktrace
[fef06214d65b:18792] *** Process received signal *** [fef06214d65b:18792] Signal: Segmentation fault (11) [fef06214d65b:18792] Signal code: Address not mapped (1) [fef06214d65b:18792] Failing at address: 0x18 [fef06214d65b:18792] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fe8ce154420] [fef06214d65b:18792] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fe8cc9ac35d] [fef06214d65b:18792] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fe8cc9b047d] [fef06214d65b:18792] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fe8cc9bb926] [fef06214d65b:18792] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fe8cc8f3c90] [fef06214d65b:18792] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fe8cc8feb5e] [fef06214d65b:18792] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [fef06214d65b:18792] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [fef06214d65b:18792] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [fef06214d65b:18792] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [fef06214d65b:18792] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fe8cc01b083] [fef06214d65b:18792] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [fef06214d65b:18792] *** End of error message *** [fef06214d65b:18787] *** Process received signal *** [fef06214d65b:18787] Signal: Segmentation fault (11) [fef06214d65b:18787] Signal code: Address not mapped (1) [fef06214d65b:18787] Failing at address: 0x18 [fef06214d65b:18787] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f4845d2e420] [fef06214d65b:18787] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f484458635d] [fef06214d65b:18787] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f484458a47d] [fef06214d65b:18787] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f4844595926] [fef06214d65b:18787] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f48444cdc90] [fef06214d65b:18787] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f48444d8b5e] [fef06214d65b:18787] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [fef06214d65b:18787] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [fef06214d65b:18787] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [fef06214d65b:18787] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [fef06214d65b:18787] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f4843bf5083] [fef06214d65b:18787] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [fef06214d65b:18787] *** End of error message *** [fef06214d65b:18784] *** Process received signal *** [fef06214d65b:18784] Signal: Segmentation fault (11) [fef06214d65b:18784] Signal code: Address not mapped (1) [fef06214d65b:18784] Failing at address: 0x18 [fef06214d65b:18784] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f640d628420] [fef06214d65b:18784] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f640be8035d] [fef06214d65b:18784] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f640be8447d] [fef06214d65b:18784] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f640be8f926] [fef06214d65b:18784] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f640bdc7c90] [fef06214d65b:18784] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f640bdd2b5e] [fef06214d65b:18784] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [fef06214d65b:18784] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [fef06214d65b:18784] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [fef06214d65b:18784] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [fef06214d65b:18784] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f640b4ef083] [fef06214d65b:18784] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [fef06214d65b:18784] *** End of error message *** [fef06214d65b:18795] *** Process received signal *** [fef06214d65b:18795] Signal: Segmentation fault (11) [fef06214d65b:18795] Signal code: Address not mapped (1) [fef06214d65b:18795] Failing at address: 0x18 [fef06214d65b:18795] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fd9b098f420] [fef06214d65b:18795] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fd9af1e735d] [fef06214d65b:18795] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fd9af1eb47d] [fef06214d65b:18795] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fd9af1f6926] [fef06214d65b:18795] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fd9af12ec90] [fef06214d65b:18795] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fd9af139b5e] [fef06214d65b:18795] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [fef06214d65b:18795] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [fef06214d65b:18795] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [fef06214d65b:18795] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [fef06214d65b:18795] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fd9ae856083] [fef06214d65b:18795] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [fef06214d65b:18795] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 2 with PID 0 on node fef06214d65b exited on signal 11 (Segmentation fault). --------------------------------------------------------------------------
Standard Output
[fef06214d65b:18792] *** Process received signal *** [fef06214d65b:18792] Signal: Segmentation fault (11) [fef06214d65b:18792] Signal code: Address not mapped (1) [fef06214d65b:18792] Failing at address: 0x18 [fef06214d65b:18792] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fe8ce154420] [fef06214d65b:18792] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fe8cc9ac35d] [fef06214d65b:18792] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fe8cc9b047d] [fef06214d65b:18792] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fe8cc9bb926] [fef06214d65b:18792] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fe8cc8f3c90] [fef06214d65b:18792] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fe8cc8feb5e] [fef06214d65b:18792] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [fef06214d65b:18792] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [fef06214d65b:18792] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [fef06214d65b:18792] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [fef06214d65b:18792] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fe8cc01b083] [fef06214d65b:18792] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [fef06214d65b:18792] *** End of error message *** [fef06214d65b:18787] *** Process received signal *** [fef06214d65b:18787] Signal: Segmentation fault (11) [fef06214d65b:18787] Signal code: Address not mapped (1) [fef06214d65b:18787] Failing at address: 0x18 [fef06214d65b:18787] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f4845d2e420] [fef06214d65b:18787] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f484458635d] [fef06214d65b:18787] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f484458a47d] [fef06214d65b:18787] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f4844595926] [fef06214d65b:18787] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f48444cdc90] [fef06214d65b:18787] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f48444d8b5e] [fef06214d65b:18787] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [fef06214d65b:18787] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [fef06214d65b:18787] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [fef06214d65b:18787] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [fef06214d65b:18787] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f4843bf5083] [fef06214d65b:18787] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [fef06214d65b:18787] *** End of error message *** [fef06214d65b:18784] *** Process received signal *** [fef06214d65b:18784] Signal: Segmentation fault (11) [fef06214d65b:18784] Signal code: Address not mapped (1) [fef06214d65b:18784] Failing at address: 0x18 [fef06214d65b:18784] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f640d628420] [fef06214d65b:18784] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7f640be8035d] [fef06214d65b:18784] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7f640be8447d] [fef06214d65b:18784] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7f640be8f926] [fef06214d65b:18784] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7f640bdc7c90] [fef06214d65b:18784] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7f640bdd2b5e] [fef06214d65b:18784] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [fef06214d65b:18784] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [fef06214d65b:18784] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [fef06214d65b:18784] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [fef06214d65b:18784] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f640b4ef083] [fef06214d65b:18784] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [fef06214d65b:18784] *** End of error message *** [fef06214d65b:18795] *** Process received signal *** [fef06214d65b:18795] Signal: Segmentation fault (11) [fef06214d65b:18795] Signal code: Address not mapped (1) [fef06214d65b:18795] Failing at address: 0x18 [fef06214d65b:18795] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7fd9b098f420] [fef06214d65b:18795] [ 1] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33235d)[0x7fd9af1e735d] [fef06214d65b:18795] [ 2] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x33647d)[0x7fd9af1eb47d] [fef06214d65b:18795] [ 3] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x341926)[0x7fd9af1f6926] [fef06214d65b:18795] [ 4] /opt/rocm-5.6.0/lib/libamdhip64.so.5(+0x279c90)[0x7fd9af12ec90] [fef06214d65b:18795] [ 5] /opt/rocm-5.6.0/lib/libamdhip64.so.5(hipStreamCreate+0x14e)[0x7fd9af139b5e] [fef06214d65b:18795] [ 6] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x467f2b] [fef06214d65b:18795] [ 7] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cbbc] [fef06214d65b:18795] [ 8] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x43cae8] [fef06214d65b:18795] [ 9] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x4370e2] [fef06214d65b:18795] [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7fd9ae856083] [fef06214d65b:18795] [11] /var/jenkins/workspace/ArborX_master/build/test/ArborX_Test_DetailsDistributedTreeImpl.exe[0x38966e] [fef06214d65b:18795] *** End of error message *** -------------------------------------------------------------------------- Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. -------------------------------------------------------------------------- -------------------------------------------------------------------------- mpiexec noticed that process rank 2 with PID 0 on node fef06214d65b exited on signal 11 (Segmentation fault). --------------------------------------------------------------------------