/*! @page pag_project Setting up a new project
\section sec_project Setting Up a New Project

This topic guides you through the setup of a new project.
Be sure to check out the examples in the projects folder
and the code documentation as well.

The basic steps are:

1. Create a new folder under `projects`.
2. In the new folder, create a Python module for the project (subsequently called _the project module_).
3. In the project module, define a cluster generator class which derives from pmsco.cluster.ClusterGenerator.
4. In the project module, define a project class which derives from pmsco.project.Project.
5. In the same folder as the project module, create a JSON run-file.

\subsection sec_project_module Project Module

A skeleton of the project module file (with some common imports) may look like this:

~~~~~~{.py}
import logging
import math
import numpy as np
import periodictable as pt
from pathlib import Path

import pmsco.cluster
import pmsco.data
import pmsco.dispatch
import pmsco.elements.bindingenergy
import pmsco.project

logger = logging.getLogger(__name__)


class MyClusterGenerator(pmsco.cluster.ClusterGenerator):
    def create_cluster(self, model, index):
        clu = pmsco.cluster.Cluster()
        # ...
        return clu

    def count_emitters(self, model, index):
        # ...
        return 1


class MyProject(pmsco.project.Project):
    def __init__(self):
        super().__init__()
        # ...
        self.cluster_generator = MyClusterGenerator(self)

    def create_model_space():
        spa = pmsco.project.ModelSpace()
        # ...
        return spa

    def create_params(self, model, index):
        par = pmsco.project.CalculatorParams()
        # ...
        return par
~~~~~~

The main purpose of the `MyProject` class is to bundle the project-specific calculation parameters and code.
The purpose of the `MyClusterGenerator` class is to produce atomic clusters as a function of a number of model parameters.
For the project to be useful, some of the methods in the skeleton above need to be implemented.
The individual methods are discussed in the following.
Further descriptions can be found in the documentation of the code.

\subsection sec_project_cluster Cluster Generator

The cluster generator is a project-specific Python object that produces a cluster, i.e., a list of atomic coordinates,
based on a small number of model parameters whenever PMSCO requires it.
The most important member of a cluster generator is its `create_cluster` method.
At least this method must be implemented for a functional cluster generator.

A generic `count_emitters` method is implemented in the base class.
It needs to be overridden if you want to use parallel calculation of multiple emitters.

\subsubsection sec_project_cluster_create Cluster Definition

The `create_cluster` method takes the model parameters (a dictionary)
and the task index (a pmsco.dispatch.CalcID, cf. @ref pag_concepts_tasks) as arguments.
Given these arguments, it must create and fill a pmsco.cluster.Cluster object.
See pmsco.cluster.ClusterGenerator.create_cluster for details on the method contract.

As an example, have a look at the following simplified excerpt from the twoatom demo project.

~~~~~~{.py}
    def create_cluster(self, model, index):
        # access model parameters
        # dAB - distance between atoms in Angstroms
        # th - polar angle in degrees
        # ph - azimuthal angle in degrees
        r = model['dAB']
        th = math.radians(model['th'])
        ph = math.radians(model['ph'])

        # prepare a cluster object
        clu = pmsco.cluster.Cluster()
        # the comment line is optional but can be useful
        clu.comment = "{0} {1}".format(self.__class__, index)
        # set the maximum radius of the cluster (outliers will be ignored)
        clu.set_rmax(r * 2.0)

        # calculate atomic vectors
        dx = r * math.sin(th) * math.cos(ph)
        dy = r * math.sin(th) * math.sin(ph)
        dz = r * math.cos(th)
        a_top = np.array((0.0, 0.0, 0.0))
        a_bot = np.array((-dx, -dy, -dz))

        # add an oxygen atom at a_top position and mark it as emitter
        clu.add_atom('O', a_top, 1)
        # add a copper atom at a_bot position
        clu.add_atom('Cu', a_bot, 0)

        # pass the created cluster to the calculator
        return clu
~~~~~~

In this example, two atoms are added to the cluster.
The pmsco.cluster.Cluster class provides several methods to simplify the task,
such as adding layers or bulk regions, rotation, translation, trim, emitter selection, etc.
Please refer to the documentation of its code for details.
It may also be instructive to have a look at the demo projects.

The main purposes of the cluster object are to store an array of atoms and to read/write cluster files in a variety of formats.
For each atom, the following properties are stored:

- sequential atom index (1-based, maintained by cluster code)
- atom type (chemical element number)
- chemical element symbol from periodic table
- x coordinate of the atom position
- t coordinate of the atom position
- z coordinate of the atom position
- emitter flag (0 = scatterer, 1 = emitter, default 0)
- charge/ionicity (units of elementary charge, default 0)
- scatterer class (default 0)

All of these properties except the scatterer class can be set by the add methods of the cluster.
The scatterer class is used internally by the atomic scattering factor calculators.
Whether the charge/ionicity is used, depends on the particular calculators, EDAC does not use it, for instance.

Note: You do not need to take care how many emitters a calculator allows,
or whether the emitter needs to be at the origin or the first place of the array.
These technical aspects are handled by PMSCO code transparently.

\subsubsection sec_project_cluster_domains Domains

Domains refer to regions of inequivalent structure in the probing region.
This may include regions of different orientation, different lattice constant, or even different structure.
The cluster methods can read the selected domain from the `index.domain` argument.
This is an index into the pmsco.project.Project.domains list where each item is a dictionary
that holds additional, invariable structural parameters.

A common case are rotational domains.
In this case, the list of domains may look like `[{"zrot": 0.0}, {"zrot": 60.0}]`, for example,
and the `create_cluster` method would include additional code to rotate the cluster:

~~~~~~{.py}
    def create_cluster(self, model, index):
        # filling atoms here
        # ...

        dom = self.domains[index.domain]
        try:
            z_rot = dom['zrot']
        except KeyError:
            z_rot = 0.0
        if z_rot:
            clu.rotate_z(z_rot)

        # selecting emitters
        # ...

        return clu
~~~~~~

Depending on the complexity of the system, it may, however, be necessary to write a specific sub-routine for each domain.

The pmsco.project.Project class includes generic code to add intensities of domains incoherently (cf. pmsco.project.Project.combine_domains).
If the model space contains parameters 'wdom0', 'wdom1', etc.,
these parameters are interpreted at weights of domain 0, 1, etc.
One domain must have a fixed weight to avoid correlated parameters.
Typically, 'wdom0' is left undefined and defaults to 1.

\subsubsection sec_project_cluster_emitters Emitter Configurations

If your project has a large cluster and/or many emitters, have a look at @ref pag_concepts_emitter.
In this case, you should override the `count_emitters` method and return the number of emitter configurations.
In the simplest case, this is the number of inequivalent emitters, and the implementation would be:

~~~~~~{.py}
    def count_emitters(self, model, index):
        index = index._replace(emit=-1)
        clu = self.create_cluster(model, index)
        return clu.get_emitter_count()
~~~~~~

Next, modify the `create_cluster` method to check the emitter index (`index.emit`).
If it is -1, the method must return the full cluster with all inequivalent emitters marked.
If it is positive, only the corresponding emitter must be marked.
The code could be similar to this example:

~~~~~~{.py}
    def create_cluster(self, model, index):
        # filling atoms here
        # ...

        # select all possible emitters (atoms of a specific element) in a cylindrical volume
        # idx_emit is an array of atom numbers (0-based atom index)
        idx_emit = clu.find_index_cylinder(origin, r_xy, r_z, self.project.scans[index.scan].emitter)
        # if a specific emitter should be marked, restrict the array index.
        if index.emit >= 0:
            idx_emit = idx_emit[index.emit]
        # mark the selected emitters
        # if index.emit was < 0, all emitters are marked
        clu.data['e'][idx_emit] = 1

        return clu
~~~~~~

Now, the individual emitter configurations will be calculated in separate tasks
which can be run in parallel in a multi-process environment.
Note that the processing time of EDAC scales linearly with the number of emitters.
Thus, parallel execution is beneficial.

Advanced programmers may exploit more of the flexibility of emitter configurations, cf. @ref pag_concepts_emitter.

\subsection sec_project_project Project Class

Most commonly, a project class overrides the `__init__`, `create_model_space` and `create_params` methods.
Most other inherited methods can be overridden optionally,
for instance `validate`, `setup`, `calc_modulation`, `rfactor`,
as well as the combine methods `combine_rfactors`, `combine_domains`, `combine_emitters`, etc.
Int his introduction, we focus on the most basic three methods.

\subsubsection sec_project_project_init Initialization and Defaults

In the `__init__` method, you define and initialize (with default values) additional project properties.
You may also redefine properties of the base class.
The following code is just an example to give you some ideas.

~~~~~~{.py}
class MyProject(pmsco.project.Project):
    def __init__(self):
        # call the inherited method first
        super().__init__()
        # re-define an inherited property
        self.directories["data"] = Path("/home/pmsco/data")
        # define a scan dictionary
        self.scan_dict = {}
        # fill the scan dictionary
        self.build_scan_dict()
        # create the cluster generator
        self.cluster_generator = MyClusterGenerator(self)
        # declare the list of domains (at least one is required)
        self.domains = [{"zrot": 0.}]

    def build_scan_dict(self):
        self.scan_dict["empty"] = {"filename": "{pmsco}/projects/common/empty-hemiscan.etpi",
                                   "emitter": "Si", "initial_state": "2p3/2"}
        self.scan_dict["Si2p"]  = {"filename": "{data}/xpd-Si2p.etpis",
                                   "emitter": "Si", "initial_state": "2p3/2"}
~~~~~~

The scan dictionary can come in handy if you want to select scans by a shortcut on the command line or in a run file.

Note that most of the properties can be assigned from a run file.
This happens after the `__init__` method.
The values set by `__init__` serve as default values.

\subsubsection sec_project_project_space Model Space

The model space defines the keys and value ranges of the model parameters.
There are three ways to declare the model space in order of priority:

1. Declare the model space in the run-file.
2. Assign a ModelSpace to the self.model_space property directly in the `__init__` method.
3. Implement the `create_model_space` method.

We begin the third way:

~~~~~~{.py}
# under class MyProject(pmsco.project.Project):
    def create_model_space(self):
        # create an empty model space
        spa = pmsco.project.ModelSpace()

        # add parameters
        spa.add_param('dAB',    2.10,  2.00,   2.25,  0.05)
        spa.add_param('th',    15.00,  0.00,  30.00,  1.00)
        spa.add_param('ph',    90.00)
        spa.add_param('V0',    21.96, 15.00,  25.00,  1.00)
        spa.add_param('Zsurf',  1.50)
        spa.add_param('wdom1',  0.5,   0.10,  10.00,  0.10)

        # return the model space
        return spa
~~~~~~

This code declares six model parameters: `dAB`, `th`, `ph`, `V0`, `Zsurf` and `wdom1`.
Three of them are structural parameters (used by the cluster generator above),
two are used by the `create_params` method (see below),
and `wdom1` is used in pmsco.project.Project.combine_domains while summing up contributions from different domains.

The values in the arguments list correspond to the start value (initial guess),
the lower and upper boundaries of the value range,
and the step size for optimizers that require it.
If just one value is given, like for `ph` and `Zsurf`, the parameter is held constant during the optimization.

The equivalent declaration in the run-file would look like (parameters after `th` omitted):

~~~~~~{.py}
{
  "project": {
    // ...
    "model_space": {
      "dAB": {
        "start": 2.109,
        "min": 2.0,
        "max": 2.25,
        "step": 0.05
      },
      "th": {
        "start": 15.0,
        "min": 0.0,
        "max": 30.0,
        "step": 1.0
      },
      // ...
    }
  }
}
~~~~~~

\subsubsection sec_project_project_params Calculation Parameters

Non-structural parameters that are needed for the input files of the calculators are passed
in a pmsco.project.CalculatorParams object.
This object should be created and filled in the `create_params` method of the project class.

The following example is from the twoatoms demo project:

~~~~~~{.py}
# under class MyProject(pmsco.project.Project):
    def create_params(self, model, index):
        params = pmsco.project.CalculatorParams()

        # meta data
        params.title = "two-atom demo"
        params.comment = "{0} {1}".format(self.__class__, index)

        # initial state and binding energy
        initial_state = self.scans[index.scan].initial_state
        params.initial_state = initial_state
        emitter = self.scans[index.scan].emitter
        params.binding_energy = pt.elements.symbol(emitter).binding_energy[initial_state]

        # experimental setup
        params.polarization = "H"
        params.polar_incidence_angle = 60.0
        params.azimuthal_incidence_angle = 0.0
        params.experiment_temperature = 300.0

        # material parameters
        params.z_surface = model['Zsurf']
        params.work_function = 4.5
        params.inner_potential = model['V0']
        params.debye_temperature = 356.0

        # multiple-scattering parameters (EDAC)
        params.emitters = []
        params.lmax = 15
        params.dmax = 5.0
        params.orders = [25]

        return params
~~~~~~

Most of the code is generic and can be copied to other projects.
Only the experimental and material parameters need to be adjusted.
Other properties can be changed as needed, see the documentation of pmsco.project.CalculatorParams for details.

\subsection sec_project_args Passing Runtime Parameters

Runtime parameters can be passed in one of three ways:

1. hard-coded in the project module,
2. on the command line, or
3. in a JSON run-file.

In the first way, all parameters are hard-coded in the `create_project` function of the project module.
This is the simplest way for a quick start to a small project.
However, as the project code grows, it's easy to loose track of revisions.
In programming it is usually best practice to separate code and data.

The command line is another option for passing parameters to a process.
It requires extra code for parsing the command line and is not very flexible.
It is difficult to pass complex data types.
Using the command line is no longer recommended and may become deprecated in a future version.

The recommended way of passing parameters is via run-files.
Run-files allow for complete separation of code and data in a generic and flexible way.
For example, run-files can be stored along with the results.
However, the semantics of the run-file may look intimidating at first.

\subsubsection sec_project_args_runfile Setting Up a Run-File

The usage and format of run-files is described in detail under @ref pag_runfile.

\subsubsection sec_project_args_code Hard-Coded Arguments

Hard-coded parameters are usually set in a `create_module` function of the project module.
At the end of the module, this function can easily be found.
The function has two purposes: to create the project object and to set parameters.
The parameters can be any attributes of the project class and its ancestors.
See the parent pmsco.project.Project class for a list of common attributes.

The `create_project` function may look like in the following example.
It must return a project object, i.e. an object instance of a class that inherits from pmsco.project.Project.

~~~~~~{.py}
def create_project():
    project = MyProject()

    project.optimizer_params["pop_size"] = 20

    project_dir = Path(__file__).parent
    scan_file = Path(project_dir, "hbnni_e156_int.etpi")
    project.add_scan(filename=scan_file, emitter="N", initial_state="1s")

    project.add_domain({"zrot": 0.0})
    project.add_domain({"zrot": 60.0})

    return project
~~~~~~

To have PMSCO call this function,
pass the file path of the containing module as the first command line argument of PMSCO, cf. @ref pag_command.
PMSCO calls this function in absence of a run-file.


\subsubsection sec_project_args_cmd Command Line

Since it is not recommended to pass calculation parameters on the command line,
this mechanism is not described in detail here.
It is, however, still available.
If you really need to use it,
have a look at the code of the pmsco.pmsco.main function
and how it calls the `create_project`, `parse_project_args` and `set_project_args` of the project module.

*/
