msi: Macro Substitution and Include Tool

Marty Kraimer
Argonne National Laboratory - Advanced Photon Source
Original: April 26, 1999

Introduction

msi is a general purpose macro substitution/include tool. It accepts as input an ascii template file. It looks for lines containing two reserved command names: include and substitute. It also looks for and performs substitutions on macros of the form $(var) and ${var}. It uses the macLib, an epics base library created by William Lupton, to perform the substitutions.

msi also allows substitutions to be specified via a separate substitution file. This substitution file allows the same format as the substitution files accepted by dbLoadTemplate or subtool.

Command Syntax:

msi -V -Idir -Msub -Ssubfile template

NOTE: All parameters are optional and a space is optional between -I, -M, and -S and the associated value. Output is written to stdout.

Where:

-V

If this parameter is specified then an undefined macro is considered an error. An error message is generated and the line containing the macro is not written to stdout. -I dir

This parameter, which may be repeated, specifies a search path for include commands. For example:

msi -I "/home/phoebus/MRK/examples:." -I".." template

specifies the search path:

/home/phoebus/MRK/examples:.:..

-M substitutions

This parameter, which may be repeated, specifies marco substitutions. For example:

msi -M "a=aval,b=bval" -M"c=cval" template

specifies that in the template file each occurrence of:

$(a) or ${a} is replaced by aval

$(b) or ${b} is replaced by bval

$(c) or ${c} is replaced by cval

-S subfile

The substitution file. See below for format. template

The input file. If no file is specified then input is taken from stdin, i.e. msi can be used as a filter. See below for a description of commands that can be imbedded in the template file. NOTE: It is not possible to display usage by just typing msi since executing the command with no arguments i s a valid command. To show usage specify an illegal switch, e.g.

msi -help

Template File Format

This file contains the text to be read and written to stdout after macro substitution is performed. If no file is given then input is read from stdin. In addition the file can have lines containing include and substitute commands. The format of these commands are:

    include "file"

    substitute "var=value,var=value,..."

For example let the command be:

    msi template

and file includeFile contain:

    first name is ${first}
    family name is ${family}

and template is

    substitute "first=Marty,family=Kraimer"
    include "includeFile"
    substitute "first=Irma,family=Kraimer"
    include "includeFile"

then the following is written to stdout.

    first name is Marty
    family name is Kraimer
    first name is Irma
    family name is Kraimer

Substitution File Format

The optional substitution file has three formats: regular, pattern, and dbTemplate format. Lets discuss each separately

regular format

{var1=value1,var2=value2,...}
{var1=value1,var2=value2,...}
...

After reading each set of replacements within braces, the template file is read and macro substitution performed.

pattern format:

pattern {var1,var2,...}
{value1,value2,...}
{value1,value2,...}
pattern {var1,var2,...}
{value1,value2,...}
{value1,value2,...}

This is the same as the regular format:

{var1=value1,var2=value2}
...

dbTemplate Format

file template {
    pattern format or regular format
}

file template {
    pattern format or regular format
}

For thedbTemplate format, the command line template argument is optional. If it specified it is used, otherwise the file template is used. This format is an extension of the format accepted by dbLoadTemplate. It allows templates to be expanded on the host rather via dbLoadTemplate.

Regular substitution example

Let the command be:

    msi -S substitute template

template is

    first name is ${first}
    family name is ${family}

substitute is

    {first=Marty,family=Kraimer}
    {first=Irma,family=Kraimer}

The following is written to stdout.

    first name is Marty
    family name is Kraimer
    first name is Irma
    family name is Kraimer

Pattern substitution example

Let the command be:

    msi -S pattern template

pattern is

    pattern {first,last}
    {Marty,Kraimer}
    {Irma,Kraimer}

template is the same as in the previous example.

The following is written to stdout.

    first name is Marty
    family name is Kraimer
    first name is Irma
    family name is Kraimer

dbTemplate example

Let the command be

msi -S xxx.substitutions

xxx.substitutions is

file template {
    pattern {first,last}
    {Marty,Kraimer}
    {Irma,Kraimer}
    pattern {last,first}
    {Smith,Bill}
    {Smith,Mary}
}
file template {
    {first=Marty,last=Kraimer}
    {first=Irma,last=Kraimer}
}

template is the same as in the previous example..

The following is written to stdout

first name is Marty
family name is Kraimer
first name is Irma
family name is Kraimer
first name is Bill
last name is Smith
first name is Mary
last name is Smith
first name is Marty
family name is Kraimer
first name is Irma
family name is Kraimer

Some Details

Building msi

msi is built as a normal extensions product. It should be built with base version 3.13.0beta11 or later.

Line length limits

All buffers containing input or output lines are set for a maximum line length of 1024. Longer input or output lines will cause msi to fail. macLib

Currently the only macLib documentation is in macLibNOTES and macLibREADME which are in base/src/libCom. template file syntax

except for lines containing include or substitute commands each line is just passed to macLib. Lines containing these commands MUST be of the form:

        include "<file name>"

or

        substitute "<substitutions>"

White space is allowed before and after the command and after the quoted string. If imbedded quotes are needed the \ character can be used as an escape character. For example:

        substitute "a=\"val\""

specifies the substitution

        a=val

If a line does have the above syntax it is just passed to macLib for processing without any notification. Thus the input line:

        include "myfile" #include file

would just be passed to macLib, i.e. it would NOT be considered an include command. substitution file syntax

For items within braces separators may be given between items. A separator is either white space or a comma. White space is any of the following: space, formfeed, newline, carriage return, tab, vertical tab.

Each item within braces can be an alphanumeric token or a quoted string. The characters \" can be used for imbedded quotes. The rules for non quoted strings are actually more relaxed but any item that is not an alphanumeric string should be given as a quoted string.

Thus

    {a=aa b=bb c="\"cc\""}

    {a=aa,b=bb,c="\"cc\""}

    {
        a="aa"
        b="bb",
        c="\"cc\""
    }