/*! @page pag_runfile Run File
\section sec_runfile Run File

This section describes the format of a run-file.
Run-files are a flexible way of passing arguments to a PMSCO process.
The benefits are:

- contain all essential parameters to repeat a calculation - no need to remember or record the command line
- avoid cluttering up the command line or frequent changes of source code
- can be versioned or stored separately from the code, maintain a single file or multiple files - up to the user
- any property and sub-property of the project object can be assigned in a generic way - even custom properties that are unknown to PMSCO
- no necessity for the project code to parse the command line
- schema validation can help to find syntax errors while editing


\subsection sec_runfile_how How It Works

Run-files are text files in machine and human readable [JSON](https://en.wikipedia.org/wiki/JSON) format.
In PMSCO, run-files contain dictionaries of parameters to be passed to the project object.
For the calculations, internally, the project object is the main container of calculation parameters, model objects and input data.

Upon launching PMSCO, a generic parser reads the run-file,
constructs the project object from the specified custom project class
and assigns the attributes defined in the run-file.
Run-files are a sort of script that assign data to the project.
The parser does not expect specific data types or classes.
It merely copies data items to the project attributes of the same name.
The validation and interpretation of the data is up to the project object.

The parser handles the following situations:

- Strings, numbers as well as dictionaries and lists of simple objects are assigned directly to project attributes.
  If the project class declares a setter method for the attribute, the setter is called.
  Else, the existing attribute is overwritten.
  Setters can execute custom code to validate the data value.
- If specified in the run-file, the parser creates objects from classes in the namespace of the project module
  and recursively assigns their properties.

\note There are no implicit checks of correctness of the assigned data objects!
The author of the run-file must make sure that the run-file is compatible with the project class,
else the calculation process might fail.

There are three ways to check assigned attributes before the calculations are started.
All have to be implemented explicitly by the project maintainer:

1. The run-file can be validated against a JSON schema before launching PMSCO (see below).
   Schema validation may catch some obvious mistakes
   but is not complete in the sense that it cannot guarantee error-free execution of the project code.
2. The classes used with run-files define property setters.
   The setters can raise an exception or post an error in the log.
   (The latter won't stop the calculation process.)
3. The project class implements a validation method to check and fix important or error-prone attributes.
   It can write warnings and errors to the log, or raise an exception if the process should be aborted.


\subsection sec_runfile_general General File Format

Run-files must adhere to the [JSON](https://en.wikipedia.org/wiki/JSON) format.
Specifically, a JSON file can declare dictionaries, lists and simple objects
such as strings, numbers and `null`.
The syntax of these basic elements is similar to Python source code (there are some differences, though).

At the top level, a PMSCO run-file contains a dictionary with up to two items:

1. The _project_ item is the most important, it is described in the following under @ref sec_runfile_project.
2. The _schedule_ item is an optional section for passing the parameters to a job queue of a computing cluster.
   See @ref sec_runfile_schedule .


\subsection sec_runfile_schema Schema

The structure of a JSON file can be described in a _schema_ file that can be used to check the syntax and structure programmatically.
The `schema/runfile.schema.json` file of the PMSCO distribution describes the structure of a run-file as well as common properties of the project.
The schema is, however, rather basic and does not cover all parameters, conditional cases or custom project properties.

A run-file can be easily validated against the schema while editing in the PyCharm IDE.
Alternatively, the jsonschema validator from the Python distribution can be used on the command line.


\subsection sec_runfile_project Project Specification

The following minimum run-file from the twoatom project demonstrates how to specify the project:

~~~~~~{.py}
{
  "project": {
    "__module__": "twoatom",
    "__class__": "TwoatomProject",
    "mode": "single",
    "job_name": "twoatom0001"
  }
}
~~~~~~

Here, the `project` keyword denotes the dictionary that is used to construct the project object.

Within the project dictionary, the `__module__` key selects the Python module file that contains the project code,
and `__class__` refers to the name of the actual project class.
Further dictionary items correspond to attributes of the project class.

The module name is the same as would be used in a Python import statement.
It must be findable on the Python path.
Alternatively, a file path may be specified.
PMSCO ensures that the directory containing the `pmsco` and `projects` sub-directories is on the Python path.
The class name must be in the namespace of the loaded module.

As PMSCO starts, it imports the specified module,
constructs an object of the specified project class,
and assigns any further items to project attributes.
In the example above, it creates an object of type `TwoatomProject` from the `twoatom` module
and assigns `single` to the `mode` property and `twoatom0001` to the `job_name` property.

Any attributes not specified in the run-file remain at their default values
that were set by the `__init__` constructor of the project class.

Note that parameter names must start with an alphabetic character, else they are ignored
(useful for comments as JSON does not have a syntax for comments).
Also note that PMSCO does not spell-check parameter names.
The parameter values are just written to the corresponding object attribute.
If a name is misspelled, the value will be written to the wrong attribute.

PMSCO carries out only some most important checks on the given parameter values.
Incorrect values may lead to improper operation or exceptions later in the calculations.
The project class can explicitly check and fix important or error-prone attributes, or report errors.

The following sub-sections describe the most common properties of the project class.


\subsubsection sec_runfile_common Common Arguments

The following table lists some important parameters controlling the calculations.
They are declared in the pmsco.projects.Project class.

| Key | Values | Description |
| --- | --- | --- |
| mode | `single` (default), `grid`, `swarm`, `genetic`, `table`, `test`, `validate` | Operation mode. `validate` can be used to check the syntax of the run-file, the process exits before starting calculations. |
| directories | dictionary | This dictionary lists common file paths used in the project. It contains keys such as `home`, `project`, `output` (see documentation of Project class in pmsco.project). |
| output_dir | path | Shortcut for directories["output"] |
| data_dir | path | Shortcut for directories["data"] |
| job_name | string, must be a valid and unique file name (see note below) | Base name for all produced output files. It is recommended to set a unique name for each calculation run. Do not include a path. The path can be set in _output_dir_. |
| cluster_generator | dictionary | Class name and attributes of the cluster generator. See below. |
| atomic_scattering_factory | string<br>Default: InternalAtomicCalculator from pmsco.calculators.calculator | Class name of the atomic scattering calculator. This name must be in the namespace of the project module. |
| multiple_scattering_factory | string<br>Default: EdacCalculator from  pmsco.calculators.edac | Class name of the multiple scattering calculator. This name must be in the namespace of the project module. |
| model_space | dictionary | See @ref sec_runfile_space below. |
| domains | list of dictionaries | See @ref sec_runfile_domains below. |
| scans | list of dictionaries | See @ref sec_runfile_scans below. |
| optimizer_params | dictionary | See @ref sec_runfile_optimizer below. |

\note The *job name* parameter appears most visibly as the prefix of output file names.
It is also registered in the `jobs` table of the results database (if used),
and it is used to identify the job with a job scheduling system.
For these reasons, it is important that the job name be unique within the respective subsystem.
Specifically, you need to *provide a new job name each time you start pmsco*, otherwise the job may fail.
It may be more natural to specify the job name on the command line using the `-o` argument
rather than changing the run file every time.
Unfortunately, PMSCO cannot auto-generate, auto-increment or verify the job name.

File names specified in a runfile can include an explicit path or a placeholder.
Placeholders have the format `${key}` where `key` must be one of the keys of the `directories` dictionary.
The placeholder will then be replaced by the corresponding value before the calculation starts
(as a part of the pmsco.project.Project.validate method).
The `directories` dictionary can be filled by the project class or in the runfile.
In addition, a number of keys are defined by PMSCO and can be used as placeholders in other directories and file paths.

| Key | Type | Description |
| --- | --- | --- |
| data | absolute | Directory with experimental data. Must be set by user if needed. |
| home | absolute | Home directory of the current user |
| pmsco | absolute | Directory that contains the loaded pmsco.py module. Note: This may be in a site packages directory. |
| output | absolute |  Output directory. Must be set by the user. |
| project | absolute |  Directory where the project module is located. |
| project_name | relative | Name of the project. By default, the name of the project class. |
| job_name | relative | Name of the calculation job. |
| mode | relative | Calculation mode |
| report | absolute |  Report directory. Defaults to `${output}/report`. |
| run | absolute |  Directory where the runfile is located (if used). |
| temp | absolute |  Directory for temporary files. Currently not used. |
| work | absolute |  Current working directory |

Placeholders of absolute paths must be used at the beginning of a path.
Relative paths can be used at any position in a file path.
Some of the keys may have empty values if PMSCO was loaded in a non-standard way.
For verification of the path resolution, all directories are printed to the log file at WARNING level (default).

The following table lists some common control parameters and metadata
that affect the behaviour of the program but do not affect the calculation results.
The job metadata is used to identify and describe a job in the results database if requested.

| Key | Values | Description |
| --- | --- | --- |
| db_file | new or existing file path or `:memory:` | SQLite3 database file to receive the optimization results. If the database exists, results are inserted under the given job name. If it doesn't exist, a new file is created. If the attribute is `:memory:`, an in-memory database is used internally and flushed at the end of processing. |
| job_tags | dictionary of strings | User-specified job tags in key-value format (metadata). |
| description | string | Description of the calculation job (metadata) |
| time_limit | decimal number<br>Default: 24. | Wall time limit in hours. The optimizers try to finish before the limit. This cannot be guaranteed, however. |
| keep_files | list of file categories | Output file categories to keep after the calculation. Multiple values can be specified and must be separated by spaces. By default, cluster and model (simulated data) of a limited number of best models are kept. See @ref sec_runfile_files below. |
| keep_best | integer number<br>Default: 10 | number of best models for which result files should be kept. |
| keep_levels | integer number<br>Default: 1 | numeric task level down to which files are kept. 1 = scan, 2 = domain, 3 = emitter. |
| log_level | DEBUG, INFO, WARNING, ERROR, CRITICAL | Minimum level of messages that should be added to the log. Empty string turns off logging. |
| log_file | file system path<br>Default: job_name + ".log". | Name of the main log file. Under MPI, the rank of the process is inserted before the extension. The log name is created in the working directory.  |


\subsubsection sec_runfile_space Model Space

The `model_space` parameter is a dictionary of model parameters.
The key is the name of the parameter as used by the cluster and input-formatting code,
the value is a dictionary holding the `start`, `min`, `max`, `step` values to be used by the optimizer.
Instead of `min` and `max` you may declare the `width`, which will center the space on the start value.

All parameter values can be declared as numbers or as simple Python expressions in double quotes.
Expressions are evaluated by the Python `eval` function.
All functions in the namespace of the project module.
Note that you have to import the `math` or `numpy` modules in your project module
if you want to use their functions.

~~~~~~{.py}
{
  "project": {
    // ...
    "model_space": {
      "dAB": {
        "start": 2.109,
        "min": "2.109 - 0.1",
        "max": "2.109 + 0.1",
        "step": 0.05
      },
      "pAB": {
        "start": "4 * 3.56 / math.sqrt(3.0)",
        "width": 4.0,
        "step": 0.5
      },
      // ...
    }
  }
}
~~~~~~

Alternatively, the `model_space` can be declared as a `ModelSpace` object.
However, this shall not be described in detail here.


\subsubsection sec_runfile_domains Domains

Domains is a list of dictionaries.
Each dictionary holds keys describing the domain to the cluster and input-formatting code.
The meaning of these keys is up to the project.

An example:

~~~~~~{.py}
{
  "project": {
    // ...
    "domains": [
      {"surface": "Te", "doping": null, "zrot": 0.0},
      {"surface": "Te", "doping": null, "zrot": 60.0}
    ],
  }
}
~~~~~~


\subsection sec_runfile_scans Experimental Scan Files

The pmsco.scan.Scan objects used in the calculation cannot be instantiated from the run-file directly.
Instead, the scans object of the run-file is a list of scan creators/loaders which specify how to create a Scan object.
The pmsco.scan module defines four scan creators: `ScanLoader`, `ScanCreator`, `HoloScanCreator` and `ScanKey`.
The following code block shows examples:

~~~~~~{.py}
{
  "project": {
    // ...
    "scans": [
      {
        "__class__": "ScanCreator",
        "filename": "twoatom_energy_alpha.etpai",
        "emitter": "N",
        "initial_state": "1s",
        "positions": {
          "e": "np.arange(10, 400, 5)",
          "t": "0",
          "p": "0",
          "a": "np.linspace(-30, 30, 31)"
        }
      },
      {
        "__class__": "ScanLoader",
        "filename": "${project}/twoatom_hemi_250e.etpi",
        "emitter": "N",
        "initial_state": "1s",
        "is_modf": false
      },
      {
        "__class__": "HoloScanCreator",
        "filename": "${project}/twoatom_scan3.etpi",
        "emitter": "N",
        "initial_state": "1s",
        "generator": "pmsco.data.holo_grid",
        "generator_args": {
          "theta_start": 90,
          "theta_step": 1,
          "theta_range": 90,
          "phi_start": 0,
          "phi_range": 360,
          "phi_refinement": 1
        },
        "other_positions": {"e": 250, "a": 0}
      },
      {
        "__class__": "HoloScanCreator",
        "filename": "${project}/twoatom_scan4.etpi",
        "emitter": "N",
        "initial_state": "1s",
        "other_positions": {"e": 250, "a": 0}
      }
    ]
  }
}
~~~~~~

The class name must be specified as it would be called in the custom project module.
For the example shown above, the following import statements are necessary in the `pmsco.projects.twoatom.py` module.
(Other forms of the import statement can be used accordingly.)

~~~~~~{.py}
import numpy as np
import pmsco.data
from pmsco.scan import ScanKey, ScanLoader, ScanCreator, HoloScanCreator
~~~~~~

The *ScanCreator* object creates a scan using Numpy array constructors in `positions`.
In the example above, a two-dimensional rectangular energy-alpha scan grid is created.
The values of the positions axes are passed to Python's `eval` function
and must return a one-dimensional Numpy `ndarray`.

The `emitter` and `initial_state` keys define the probed core level.

The *HoloScanCreator* object creates a /holo scan/, i.e., an angle scan of the theta and phi axes.
The distribution of the grid points is defined by a separate generator function.
Usually, the default pmsco.data.holo_grid function is used
which generates the well-known Osterwalder holo scan
with constant point density in solid angle and equidistant polar steps.

The `generator` and `generator_args` properties have default values.
The two example holo scans above are equivalent,
as the first one above just uses default values explicitly.
If you want to specify a generator function explicitly,
you must import it into the namespace of your project.
E.g. for `pmsco.data.holo_grid` you have to `import pmsco.data`.

The *ScanLoader* object loads a data file, specified under `filename`.
The filename can include a placeholder which is replaced by the corresponding item from Project.directories.
Note that some of the directories (including `project`) are pre-set by PMSCO.
It is recommended to add a `data` key under `directories` in the run-file
if the data files are outside of the PMSCO directory tree.
The `is_modf` key indicates whether the file contains a modulation function (`true`) or intensity (`false`).
In the latter case, the modulation function is calculated after loading.

The *ScanKey* is the shortest scan specification in the run-file.
It should not be used in new projects as it uses hard-coded data links in program code.

ScanKey is a shortcut to a complete scan dictionary in the project object.
The `scan_dict` must be set up in the `__init__` method of the project class.
The `key` item specifies which key of `scan_dict` should be used to create the Scan object.
Each item of `scan_dict` holds a dictionary
that holds the attributes for either a `ScanCreator`, `HoloScanCreator` or a `ScanLoader`.
If it contains a `positions` (`other_positions`) key, it represents a `ScanCreator` (`HoloScanCreator`), else a `ScanLoader`.


\subsection sec_runfile_optimizer Optimizer Parameters

The `optimizer_params` is a dictionary holding one or more of the following items.

| Key | Values | Description |
| --- | --- | --- |
| pop_size | integer<br>The default value is the greater of 4 or the number of parallel calculation processes. | Population size (number of particles) in swarm and genetic optimization mode. |
| seed_file | file system path | Name of the population seed file. Population data of previous optimizations can be used to seed a new optimization. The file must have the same structure as the .pop or .dat files. See @ref pmsco.optimizers.population.Population.seed_from_file. |
| seed_limit | integer | Number of seed models to import. |
| recalc_seed | true or false | If true, the seed models are calculated. Otherwise, the R-factor from the seed file is used as result. Use true if the seed file contains no or outdated R-factors. |
| table_source | file system path | Name of the model table file in table scan mode. |


\subsubsection sec_runfile_files File Categories

The following category names can be used with the `keep_files` option.
Multiple names can be specified as a list.

| Category | Description | Default Action |
| --- | --- | --- |
| all | shortcut to include all categories | |
| input |      raw input files for calculator, including cluster and phase files in custom format | delete |
| output |     raw output files from calculator | delete |
| atomic |     atomic scattering and emission files in portable format | delete |
| cluster |    cluster files in portable XYZ format for report | keep |
| debug |      debug files |  delete |
| model |       output files in ETPAI format: complete simulation  (a_-1_-1_-1_-1) | keep |
| scan |       output files in ETPAI format: scan (a_b_-1_-1_-1) |  keep |
| domain |     output files in ETPAI format: domain (a_b_c_-1_-1) |  delete |
| emitter |    output files in ETPAI format: emitter (a_b_c_d_-1) |  delete |
| region |     output files in ETPAI format: region (a_b_c_d_e) |  delete |
| report|      final report of results | keep always |
| population |  final state of particle population | keep |
| rfac |        files related to models which give bad r-factors, see warning below | delete |

\note
The `report` category is always kept and cannot be turned off.
The `model` category is always kept in single calculation mode.

\warning
If you want to specify `rfac` with the `keep_files` option,
you have to add the file categories that you want to keep, e.g.,
`"keep_files": ["rfac", "cluster", "model", "scan", "population"]`
(to return the default categories for all calculated models).
Do not specify `rfac` alone as this will effectively not return any file.


\subsection sec_runfile_reports Reports

Run-time graphical reports are configured in the `reports` section.
The section is organized as a list of dictionaries.
Each dictionary sets up a specific report.
For example:

~~~~~~{.py}
{
  "project": {
    // ...
    "reports": [
      {
        "__class__": "ConvergencePlot",
        "filename_format": "${base}.convergence",
        "title_format": "my_calc"
      },
      {
        "__class__": "SwarmPlot",
        "filename_format": "${base}-${param0}-${param1}-${gen}.swarmplot",
        "title_format": "my_calc ${param0}-${param1} gen ${gen}",
        "params": [["A", "B"], ["C", "D"]]
      }
    ]
  }
}
~~~~~~

The class name must be specified as it would be called in the custom project module.
For the example above, the import section of the project must include:

~~~~~~{.py}
from pmsco.reports.convergence import ConvergencePlot
from pmsco.reports.swarm import SwarmPlot
~~~~~~

For details on reports and their configuration, see @ref sec_reports.


\subsection sec_runfile_schedule Job Scheduling

To submit a job to a resource manager such as Slurm, add a `schedule` section to the run file
(section ordering is not important):

~~~~~~{.py}
{
  "schedule": {
    "__module__": "pmsco.schedule",
    "__class__": "PsiRaSchedule",
    "nodes": 1,
    "tasks_per_node": 24,
    "walltime": "2:00",
    "manual_run": true,
    "enabled": true
  },
  "project": {
    "__module__": "projects.twoatom.twoatom",
    "__class__": "TwoatomProject",
    "mode": "single",
    "output_file": "${home}/pmsco/twoatom0001",
    ...
  }
}
~~~~~~

In the same way as for the project, the `__module__` and `__class__` keys select the class that handles the job submission.
In this example, it is pmsco.schedule.PsiRaSchedule which is tied to the Ra cluster at PSI.
For other machines, you can sub-class one of the classes in the pmsco.schedule module and include it in your project package.
The derived job submission class must prepare the code, run file and job script, and submit the job to the queue.
It should copy the code to the calculation directory to avoid version conflicts if the user continues to edit the code.
Compilation of the code can be done before submission or as a part of the job script.

@note It is difficult to check the run file and code against errors that may abort job execution.
New code and run files should be tested with a modified, fast-running calculation.

The parameters of pmsco.schedule.PsiRaSchedule are as follows.
Information about the computing nodes and partitions can be printed by the `sinfo -Nel` and `sinfo --long` commands.

| Key | Values | Description |
| --- | --- | --- |
| nodes | integer: 1..2 | Number of compute nodes (main boards on Ra). The maximum number available for PEARL is 2. |
| tasks_per_node | integer: 1..24, 32 | Number of tasks (CPU cores on Ra) per node. Jobs with less than 24 tasks are assigned to the shared partition. |
| wall_time | string: [days-]hours[:minutes[:seconds]] <br> dict: with any combination of days, hours, minutes, seconds | Maximum run time (wall time) of the job. |
| manual | bool | Manual submission (true) or automatic submission (false). Manual submission allows you to inspect the job files before submission. |
| enabled | bool | Enable scheduling (true). Otherwise, the calculation is started directly (false). |

@note The calculation job may run in a different working directory than the current one.
It is important to specify absolute data and output directories in the run file (project/directories section).
Placeholders like `${home}` can be used to make run files portable, cf. @ref sec_run_dirs.
*/
