Simple NVSHMEM example based on NVSHMEM docs

Compiling

cmake -B build -S .
cmake --build build

Run example on JSC-Cluster

First customize the batch script nvshmem_shift. In particular account and partition and, if necessary, nodes, tasks and gres. Make sure to choose a partition with GPUs.

module load CMake GCC CUDA OpenMPI NCCL NVSHMEM MPI-settings/CUDA
export NVSHMEM_BOOTSTRAP=MPI
sbatch nvshmem_shift

A possible output could be the following:

PE 0 received message 7
   hostname: jrc0215
   NVSHMEM_TEAM_WORLD  my team pe 0 of 8
   NVSHMEM_TEAM_SHARED my team pe 0 of 4
   NVSHMEMX_TEAM_NODE  my team pe 0 of 4
PE 1 received message 0
   hostname: jrc0215
   NVSHMEM_TEAM_WORLD  my team pe 1 of 8
   NVSHMEM_TEAM_SHARED my team pe 1 of 4
   NVSHMEMX_TEAM_NODE  my team pe 1 of 4
PE 2 received message 1
   hostname: jrc0215
   NVSHMEM_TEAM_WORLD  my team pe 2 of 8
   NVSHMEM_TEAM_SHARED my team pe 2 of 4
   NVSHMEMX_TEAM_NODE  my team pe 2 of 4
PE 3 received message 2
   hostname: jrc0215
   NVSHMEM_TEAM_WORLD  my team pe 3 of 8
   NVSHMEM_TEAM_SHARED my team pe 3 of 4
   NVSHMEMX_TEAM_NODE  my team pe 3 of 4
PE 4 received message 3
   hostname: jrc0354
   NVSHMEM_TEAM_WORLD  my team pe 4 of 8
   NVSHMEM_TEAM_SHARED my team pe 0 of 4
   NVSHMEMX_TEAM_NODE  my team pe 0 of 4
PE 5 received message 4
   hostname: jrc0354
   NVSHMEM_TEAM_WORLD  my team pe 5 of 8
   NVSHMEM_TEAM_SHARED my team pe 1 of 4
   NVSHMEMX_TEAM_NODE  my team pe 1 of 4
PE 6 received message 5
   hostname: jrc0354
   NVSHMEM_TEAM_WORLD  my team pe 6 of 8
   NVSHMEM_TEAM_SHARED my team pe 2 of 4
   NVSHMEMX_TEAM_NODE  my team pe 2 of 4
PE 7 received message 6
   hostname: jrc0354
   NVSHMEM_TEAM_WORLD  my team pe 7 of 8
   NVSHMEM_TEAM_SHARED my team pe 3 of 4
   NVSHMEMX_TEAM_NODE  my team pe 3 of 4

This example was generated on 2 nodes with 4 GPUs each.