/*! @page pag_run Running PMSCO
\section sec_run Running PMSCO

To run PMSCO you need the PMSCO code and its dependencies (cf. @ref pag_install),
a customized code module that contains the project-specific code,
and one or several files containing the scan parameters and experimental data.
Please check the <code>projects</code> folder for examples of project modules.

The run-time arguments can either be passed on the command line
(@ref pag_command - the older and less flexible way)
or in a JSON-formatted run-file
(@ref pag_runfile - the recommended new and flexible way).
For beginners, it's also possible to hard-code all project parameters in the custom project module.


\subsection sec_run_single Single Process

Run PMSCO from the command prompt:

@code{.sh}
cd work-dir
python pmsco-dir -r run-file
@endcode

where <code>work-dir</code> is the destination directory for output files,
<code>pmsco-dir</code> is the directory containing the <code>__main__.py</code> file,
<code>run-file</code> is a json-formatted configuration file that defines run-time parameters.
The format and content of the run-file is described in a separate section.

In this form, PMSCO is run in one process which handles all calculations sequentially.

Example command line for a single EDAC calculation of the two-atom project:
@code{.sh}
cd work/twoatom
python ../../pmsco -r twoatom-hemi.json
@endcode

This command line executes the main pmsco module <code>pmsco.py</code>.
The information which project to load is contained in the <code>twoatom-hemi.json</code> file,
along with all common and specific project arguments.

This example can be run for testing.
All necessary parameters and data files are included in the code repository.


\subsection sec_run_parallel Parallel Processes

PMSCO handles parallelization automatically and transparently.
To start PMSCO in a parallel environment in the login shell,
just prefix the command with <code>mpiexec -np N</code>,
where N is the number of processes.
One process will assume the role of the master, and the remaining will assume the role of slaves.
The slave processes will run the scattering calculations, while the master coordinates the tasks,
and optimizes the model parameters (depending on the operation mode).

For optimum performance, the number of processes should not exceed the number of available processors.
To start an optimization job with multiple processes on an quad-core workstation with hyperthreading:
@code{.sh}
cd work/my_project
mpiexec -np 8 --use-hwthread-cpus python pmsco-dir -r run-file
@endcode

The `--use-hwthread` option may be necessary on certain hyperthreading architectures.


\subsection sec_run_hpc High-Performance Cluster

PMSCO is ready to run with resource managers on cluster machines.
Code for submitting jobs to the slurm queue of the Ra cluster at PSI is included in the pmsco.schedule module
(see also the PEARL wiki pages in the PSI intranet).
The job parameters are entered in a separate section of the run file, cf. @pag_runfile for details.
Other machines can be supported by sub-classing pmsco.schedule.JobSchedule or pmsco.schedule.SlurmSchedule.

If a schedule section is present and enabled in the run file,
the following command will submit a job to the cluster machine
rather than starting a calculation directly:

@code{.sh}
cd ~/pmsco
python pmsco -r run-file.json
@endcode

The command will copy the pmsco and project source trees as well as the run file and job script to a job directory
under the output directory specified in the project section of the run file.
The full path of the job directory is _output-dir/job-name.
The directory must be empty or not existing when you run the above command.

Be careful to specify correct project file paths.
The output and data directories should be specified as absolute paths.

The scheduling command will also load the project and scan files.
Many parameter errors can, thus, be caught and fixed before the job is submitted to the queue.
The run file also offers an option to stop just before submitting the job
so that you can inspect the job files and submit the job manually.

Be sure to consider the resource allocation policy of the cluster
before you decide on the number of processes.
Requesting less resources will prolong the run time but might increase the scheduling priority.
*/
