HPCE/gitea-pages
HPCE/gitea-pages
0
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Activity

MAPDL: Intel vs IBM MPI

Browse Source 
CFX: Intel vs IBM MPI
This commit is contained in:
caubet_m 2021-02-15 12:24:10 +01:00
parent 9c5d9b50d8
commit 19c7f9bb79
2 changed files with 42 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
pages/merlin6/05 Software Support/ansys-cfx.md
View File

@ -89,8 +89,29 @@ export ANSYSLI_SERVERS=2325@$LICENSE_SERVER
export HOSTLIST=$(scontrol show hostname | tr '\n' ',' | sed 's/,$//g') export HOSTLIST=$(scontrol show hostname | tr '\n' ',' | sed 's/,$//g')
JOURNAL_FILE=/data/user/caubet_m/CFX/myjournal.in JOURNAL_FILE=myjournal.in
cfx5solve -batch -def "$JOURNAL_FILE" -par-dist "$HOSTLIST" -part $SLURM_NTASKS -start-method 'IBM MPI Distributed Parallel'
# INTELMPI=no for IBM MPI
# INTELMPI=yes for INTEL MPI
INTELMPI=no
if [ "$INTELMPI" == "yes" ]
then
export I_MPI_DEBUG=4
export I_MPI_PIN_CELL=core
# Simple example: cfx5solve -batch -def "$JOURNAL_FILE" -par-dist "$HOSTLIST" \
# -part $SLURM_NTASKS \
# -start-method 'Intel MPI Distributed Parallel'
cfx5solve -batch -part-large -double -verbose -def "$JOURNAL_FILE" -par-dist "$HOSTLIST" \
-part $SLURM_NTASKS -par-local -start-method 'Intel MPI Distributed Parallel'
else
# Simple example: cfx5solve -batch -def "$JOURNAL_FILE" -par-dist "$HOSTLIST" \
# -part $SLURM_NTASKS \
# -start-method 'IBM MPI Distributed Parallel'
cfx5solve -batch -part-large -double -verbose -def "$JOURNAL_FILE" -par-dist "$HOSTLIST" \
-part $SLURM_NTASKS -par-local -start-method 'IBM MPI Distributed Parallel'
fi
``` ```
In the above example, one can increase the number of *nodes* and/or *ntasks* if needed and combine it In the above example, one can increase the number of *nodes* and/or *ntasks* if needed and combine it

23
pages/merlin6/05 Software Support/ansys-mapdl.md
View File

@ -128,11 +128,26 @@ export ANSYSLMD_LICENSE_FILE=1055@$LICENSE_SERVER
export ANSYSLI_SERVERS=2325@$LICENSE_SERVER export ANSYSLI_SERVERS=2325@$LICENSE_SERVER
# [Optional:END] # [Optional:END]
# When using -mpi=intelmpi, KMP Affinity must be disabled SOLVER_FILE=input.dat
export KMP_AFFINITY=disabled
SOLVER_FILE=/data/user/caubet_m/MAPDL/mysolver.in # INTELMPI=no for IBM MPI
mapdl -b -dis -np ${SLURM_NTASKS} -i "$SOLVER_FILE" # INTELMPI=yes for INTEL MPI
INTELMPI=no
if [ "$INTELMPI" == "yes" ]
then
# When using -mpi=intelmpi, KMP Affinity must be disabled
export KMP_AFFINITY=disabled
# INTELMPI is not aware about distribution of tasks.
# - We need to define tasks distribution.
HOSTLIST=$(srun hostname | sort | uniq -c | awk '{print $2 ":" $1}' | tr '\n' ':' | sed 's/:$/\n/g')
mapdl -b -dis -mpi intelmpi -machines $HOSTLIST -np ${SLURM_NTASKS} -i "$SOLVER_FILE"
else
# IBMMPI (default) will be aware of the distribution of tasks.
# - In principle, no need to force tasks distribution
mapdl -b -dis -mpi ibmmpi -np ${SLURM_NTASKS} -i "$SOLVER_FILE"
fi
``` ```
In the above example, one can increase the number of *nodes* and/or *ntasks* if needed and combine it In the above example, one can increase the number of *nodes* and/or *ntasks* if needed and combine it