From 42b80245b34bdb828ca29ebb4c153ac9fec96e39 Mon Sep 17 00:00:00 2001 From: Andrew Johnson Date: Thu, 17 Jun 2004 18:22:06 +0000 Subject: [PATCH] Significantly revised, added stuff about default values. --- msi.html | 579 +++++++++++++++++++++++++++++++++---------------------- 1 file changed, 350 insertions(+), 229 deletions(-) diff --git a/msi.html b/msi.html index 86ec39018..639553968 100644 --- a/msi.html +++ b/msi.html @@ -6,215 +6,400 @@ +

msi: Macro Substitution and Include Tool

-Marty Kraimer
-Argonne National Laboratory - Advanced Photon Source
-Original:  April 26, 1999 + +
+

Version 1.4, June 2004
+Marty Kraimer (Advanced Photon Source, Argonne National Laboratory)

+

Introduction

-msi is a general purpose macro substitution/include tool. It accepts as input -an ascii template file. It looks for lines containing two reserved command -names: include and substitute. It also looks for and performs substitutions -on macros of the form $(var) and ${var}. It uses the macLib, an epics base -library created by William Lupton, to perform the substitutions. + +

msi is a general purpose macro substitution/include tool. It accepts as +input an ascii template file. It looks for lines containing two reserved +command names: include and substitute. It also looks for and performs +substitutions on macros of the form $(var) and ${var}. It uses macLib, an EPICS +Base library created by William Lupton, to perform the substitutions.

msi also allows substitutions to be specified via a separate substitution file. This substitution file allows the same format as the substitution files -accepted by dbLoadTemplate or subtool.
- 

+accepted by dbLoadTemplate or subtool.

Command Syntax:

+ 
msi -V -ofile -Idir -Msub -Ssubfile template
-NOTE: All parameters are optional and a space is optional between  -o, --I, -M, and -S  and the  associated value. Output is written to -stdout unless the -o option is given. -

Where:

+

All parameters are optional. The -o, -I, -M, and -S switches may be +separated from their associated value string by spaces if desired. Output will +be written to stdout unless the -o option is given.

-

-V

- --o file - --I dir -
-
    msi -I "/home/phoebus/MRK/examples:." -I".." - template
+
-V
+
If this parameter is specified then any undefined macro discovered in + the template file which does not have an associated default value is + considered an error. An error message is generated and when msi terminates + it will do so with an exit status of 2.
+ +
-o file
+
Output will be written to the specifed file rather than to the standard + output.
+ +
-I dir
+
This parameter, which may be repeated, specifies a search path for + include commands. For example: + +
+ 
msi -I "/home/phoebus/MRK/examples:." -I".." template
+
+ + specifies that all named files should be searched for in the following + locations in the order given: + +
    +
  1. /home/phoebus/MRK/examples
    2. +
  2. . (the current directory)
    3. +
  3. .. (the parent of the current directory)
    4. +
+
+ +
-M substitutions
+
This parameter specifies macro values for the template instance. + Multiple macro values can be specified in one substitution parameter, or in + multiple -M parameters. For example: + +
+ 
msi -M "a=aval,b=bval" -Mc=cval template
+
+ + specifies that in the template file each occurrence of: + +
+
$(a) or ${a} is replaced by aval
+
$(b) or ${b} is replaced by bval
+
$(c) or ${c} is replaced by cval
+
+
+ +
-S subfile
+
The substitution file. See below for format.
+ +
template
+
The input file. If no file is specified then input is taken from + stdin, i.e. msi can be used as a filter. See below for a description of + commands that can be embedded in the template file.
-
specifies the search path:
-
    /home/phoebus/MRK/examples:.:..
-
--M substitutions - -
-
    msi -M "a=aval,b=bval" -M"c=cval" - template
-
- - --S subfile - -template - -NOTE: It is not possible to display usage by just typing msi since -executing the command with no arguments i s a valid command. To show usage -specify an illegal switch, e.g. - + +

It is not possible to display usage by just typing msi since +executing the command with no arguments is a valid command. To show usage +specify an illegal switch, e.g.

+ +
msi -help
+
+ +

Exit Status

+ +
+
0
Success. +
1
Can't open/create file, or other I/O error. +
2
Undefined macros encountered with the -V option specified. +

Template File Format

-This file contains the text to be read and written to the output after macro + +

This file contains the text to be read and written to the output after macro substitution is performed. If no file is given then input is read from stdin. -In addition the file can have lines containing include and substitute -commands. The format of these commands are: -

    include "file"
-
    substitute "var=value,var=value,..."
-For example let the command be: -
    msi template
-and file includeFile contain: -
    first name is ${first}
-    family name is ${family}
-and template is
-  -
    substitute "first=Marty,family=Kraimer"
-    include "includeFile"
-    substitute "first=Irma,family=Kraimer"
-    include "includeFile"
-then the following is written to the output. -
    first name is Marty
-    family name is Kraimer
-    first name is Irma
-    family name is Kraimer
+Variable instances to be substituted by macro values are expressed in the +template using the syntax $(name) or +${name}. If msi has been built with EPICS Base version +3.14.7 or later, the template can also provide default values to be used when a +macro has not been given a value, using the syntax +$(name=default) or +${name=default}.

+ +

For example, using the command

+ +
+
msi -M name=Marty template
+
+ +

where the file template contains

+ +
+
My name is $(name)
+My age is $(age=none of your business)
+
+ +

results in this output:

+ +
+
My name is Marty
+My age is none of your business
+
+ +

Macro variables and their default values can be expressed in terms of other +macros if necessary, to almost any level of complexity. Recursive definitions +will generate warning messages on stderr and result in undefined output.

+ +

The template file is read and processed one line at a time, where the +maximum length of a line before and/or after macro expansion is 1023 characters +— longer input or output lines will cause msi to fail. Within the context +of a single line, macro expansion does not occur when the variable instance +appears inside a single-quoted string, or where the dollar sign $ is +preceded by a back-slash character \, but as with the standard Unix +shells, variables inside double quoted strings are expanded properly.

+ +

However neither back-slash characters nor quotes or either variety are +removed when generating the output file, so depending on what is being output +the single quote behaviour may not be useful and may even be a hinderance. It +cannot be disabled in the current version of msi.

+ +

Template file commands

+ +

In addition to the regular text and variable instances described above, the +template file may also contain commands which allow the insertion of other +template files and the ability to set macro values inside the template file +itself. These commands are:

+ +
+
include "file"
+substitute "var=value,var=value,..."
+
+ +

Lines containing commands must be in one of these forms:

+ + + +

White space is allowed before and after the command verb, and after the +quoted string. If embedded quotes are needed, the backslash character +\ can be used as an escape character. For example

+ +
+
substitute "a=\"val\""
+
+ +

specifies that (unless a is subsequently redefined) wherever a +$(a) macro appears in the template below this point, the text +"val" (including the double quote characters) will appear in the +output instead.

+ +

If a line does match either syntax above it is just passed to macLib for +processing without any notification. Thus the input line:

+ +
+
include "myfile" #include file
+
+ +

would just be passed to macLib, i.e. it would not be considered an +include command.

+ +

As an example of these commands, let the Unix command be:

+ +
+
msi template
+
+ +

and file includeFile contain:

+ +
+
first name is ${first}
+family name is ${family}
+
+ +

and template is

+ +
+
substitute "first=Marty,family=Kraimer"
+include "includeFile"
+substitute "first=Irma,family=Kraimer"
+include "includeFile"
+
+ +

then the following is written to the output.

+ +
+
first name is Marty
+family name is Kraimer
+first name is Irma
+family name is Kraimer
+
+ +

Note that on an IOC when using the dbLoadTemplate command, the +template file does not support the substitute command, although +include is supported.

Substitution File Format

-The optional substitution file has three formats: regular, pattern, and -dbTemplate format. Lets discuss  each separately
-  -

regular format

+

The optional substitution file has three formats: regular, pattern, and +dbTemplate format. We will discuss each separately.

+ +

Regular format

- 
{var1=value1,var2=value2,...}
-{var1=value1,var2=value2,...}
-...
+
{var1=set1_val1, var2=set1_val2, ...}
+{var2=set2_val2, var1=set2_val1, ...}
+{var1=set3_val1, var2=set3_val2, ...}
+{var2=set4_val2, var1=set4_val1, ...}
-After reading each set of replacements within braces,   the -template file is read and macro substitution performed.
-  -

pattern  format:

+

The template file is output with macro substitutions performed once for each +set of braces containing macro replacement values.

+ +

Pattern format

- 
pattern {var1,var2,...}
-{value1,value2,...}
-{value1,value2,...}
-pattern {var1,var2,...}
-{value1,value2,...}
-{value1,value2,...}
+
pattern {var1, var2, ...}
+{set1_val1, set1_val2, ...}
+{set2_val1, set2_val2, ...}
+pattern {var2, var1, ...}
+{set3_val2, set3_val1, ...}
+{set4_val2, set4_val2, ...}
-This is the same as the regular format: -
- 
{var1=value1,var2=value2}
-...
-
+

This produces the same result as the regular format example above.

dbTemplate Format

+

This format is an extension of the format accepted by the EPICS IOC command +dbLoadTemplate, and allows templates to be expanded on the host rather +by using dbLoadTemplate at IOC boot time.

+
- 
file template {
-    pattern format or regular format
-}
- 
file template {
-    pattern format or regular format
+
file template {
+    pattern format or regular format
+}
+file "${WHERE}/template2" {
+    pattern format or regular format
 }
-For the template format, the command line template argument is optional. If -it specified it is used, otherwise the file template is used. This format is -an extension of the format accepted by dbLoadTemplate. It allows templates to -be  expanded on the host rather via dbLoadTemplate. The file name may -appear inside double quotation marks; these are only required if the name -contains any environment variable macros of the form ${ENV_VAR} which will be -expanded before the file is opened. Environment variable macro expansion -is available when msi is linked with EPICS base R3.14.3 or newer. + +

For the dbTemplate format, the template filename does not have to be given +on the command line, and is usually specified in the substitutions file +instead. If a template filename is given on the command line it will override +the filenames listed in the substitutions files.

+ +

Syntax for all formats

+ +

A comment line may appear anywhere in a substitution file, and will be +ignored. A comment line is any line beginning with the character #, +which must be the very first character on the line.

+ +

For definitions within braces given in any of the file formats, a separator +must be given between items. A separator is either a comma, or one or more of +the standard white space characters (space, formfeed, newline, carriage return, +tab or vertical tab).

+ +

Each item within braces can be an alphanumeric token, or a double-quoted +string. A back-slash character \ can be used to escape a quote +character needed inside a quoted string. These three sets of substitutions are +all equivalent:

+ +
+
{a=aa b=bb c="\"cc\""}
+{b="bb",a=aa,c="\"cc\""}
+{
+    c="\"cc\""
+    b=bb
+    a="aa"
+}
+
+ +

Within a substitutions file, the file name may appear inside double +quotation marks; these are only required if the name contains environment +variable macros of the form ${ENV_VAR} or $(ENV_VAR), which will be expanded +before the file is opened. Environment variable macro expansion is only +available when msi has been built using EPICS base R3.14.3 or newer.

Regular substitution example

-Let the command be: -
    msi -S substitute template
-template is -
    first name is ${first}
-    family name is ${family}
-substitute is -
    {first=Marty,family=Kraimer}
-    {first=Irma,family=Kraimer}
-The following is written to the output. -
    first name is Marty
-    family name is Kraimer
-    first name is Irma
-    family name is Kraimer
+ +

Let the command be:

+ +
+
msi -S substitute template
+
+ +

The file template contains

+ +
+
first name is ${first}
+family name is ${family}
+
+ +

and the file substitute is

+ +
+
{first=Marty,family=Kraimer}
+{first=Irma,family=Kraimer}
+
+ +

The following is the output produced:

+ +
+
first name is Marty
+family name is Kraimer
+first name is Irma
+family name is Kraimer
+

Pattern substitution example

-Let the command be: -
    msi -S pattern template
-pattern is -
    pattern {first,last}
-    {Marty,Kraimer}
-    {Irma,Kraimer}
-template is the same as in the previous example. -

The following is written to the output.

-
    first name is Marty
-    family name is Kraimer
-    first name is Irma
-    family name is Kraimer
+

Let the command be:

+ +
+
msi -S pattern template
+
+ +

The file pattern contains

+ +
+
pattern {first,last}
+{Marty,Kraimer}
+{Irma,Kraimer}
+
+ +

and template is the same as the previous example:

+ +
+
first name is ${first}
+family name is ${family}
+
+ +

This is the output:

+ +
+
first name is Marty
+family name is Kraimer
+first name is Irma
+family name is Kraimer
+

dbTemplate example

Let the command be
- 
msi -S xxx.substitutions
+
msi -S xxx.substitutions
+ xxx.substitutions is
- 
file template {
-    pattern {first,last}
-    {Marty,Kraimer}
-    {Irma,Kraimer}
-    pattern {last,first}
-    {Smith,Bill}
-    {Smith,Mary}
+
file template {
+pattern {first,last}
+{Marty,Kraimer}
+{Irma,Kraimer}
+pattern {last,first}
+{Smith,Bill}
+{Smith,Mary}
 }
 file template {
-    {first=Marty,last=Kraimer}
-    {first=Irma,last=Kraimer}
+{first=Marty,last=Kraimer}
+{first=Irma,last=Kraimer}
 }
template is the same as in the previous example.. @@ -222,7 +407,7 @@ file template {

The following is written to the output

- 
first name is Marty
+
first name is Marty
 family name is Kraimer
 first name is Irma
 family name is Kraimer
@@ -236,77 +421,13 @@ first name is Irma
 family name is Kraimer
-

Some Details

-Building msi - -
-
Line length limits
-
- -macLib - -template file syntax - -
        include "<file name>"
-
        or
-
        substitute "<substitutions>"
- -
        substitute "a=\"val\""
- -
        a=val
- -
        include "myfile" #include file
- -substitution  file syntax - -
    {a=aa b=bb c="\"cc\""}
-and -
    {a=aa,b=bb,c="\"cc\""}
-and -
    {
-        a="aa"
-        b="bb",
-        c="\"cc\""
-    }
-are all equivalent. - -

- -

Exit Status

-

0 - Success.

-

1 - Can't open/create file, or other I/O error.

-

2 - -V option specified and one or more undefined macros were encountered.