Re: found a bug in 3.13.9 dbLoadTemplate - after all that time

[Andrew Johnson <anj@aps.anl.gov>]

kuner wrote:
> Remark: The substittutions $HOPR and $LOPR are used in both files, but 
> in the .substitution files it is forgotten for the second file, 
> test1.template.
>
> After the run of make dbloadTemplat will create:
>
> record(calc,"X:rdbk") {
>    field(DESC,"calculation")
>    field(SCAN,".5 second")
>    field(CALC,"A")
>    field(INPA,"X:rd .PP.NMS")
>    field(HOPR,"9")
>    field(LOPR,"1")
> }
> record(ai,"Y:$(SNAME):rd") {
>    field(DESC,"position readback")
>    field(HOPR,"9")
>    field(LOPR,"1")
> }

I am unable to replicate your results using EPICS R3.13.9, R3.13.10 or 
R3.14.6.  These are my source files:

::::::::::::::
test.substitutions
::::::::::::::
file test.template {
 { NAME="X", HOPR="9", LOPR="1"}
}
file test1.template {
 { NAME="Y" }
}
::::::::::::::
test.template
::::::::::::::
record(calc,"$(NAME):rdbk") {
       field(DESC,"calculation")
       field(SCAN,".5 second")
       field(CALC,"A")
       field(INPA,"$(NAME):rd .PP.NMS")
       field(HOPR,"$(HOPR)")
       field(LOPR,"$(LOPR)")
}
::::::::::::::
test1.template
::::::::::::::
 record(ai,"$(NAME):rd") {
     field(DESC,"position readback")
     field(HOPR,"$(HOPR)")
     field(LOPR,"$(LOPR)")
 }

These are the messages I get when expanding that substitutions file on the 
host using dbLoadTemplate:

> Inflating database from test.substitutions
> <string>: HOPR referenced but undefined
> macExpandString failed
> <string>: LOPR referenced but undefined
> macExpandString failed
> Installing database ../../../db/test.db
> rm test.t.db.raw

Unfortunately it does install the test.db file, when it should have 
aborted the make because of the errors - this is not a bug in 
dbLoadTemplates, it's a problem with the EPICS build system.  However, it 
did generate warning messages, and the file it actually installed looks 
like this:

> record(calc,"X:rdbk") {
>                 field(DESC, "calculation")
>                 field(SCAN, ".5 second")
>                 field(CALC, "A")
>                 field(INPA, "X:rd .PP.NMS")
>                 field(HOPR, "9")
>                 field(LOPR, "1")}
> record(ai,"Y:rd") {
>                 field(DESC, "position readback")
>                 field(HOPR, "$(HOPR)")
>                 field(LOPR, "$(LOPR)")}

i.e. it did not keep the old values for $(HOPR) and $(LOPR), it just 
didn't replace the macros.

If I use dbLoadRecords on the IOC instead, I get these error messages:

> iocexample> dbLoadTemplate "test.substitutions"
> IocLogClient: EPICS environment variable "EPICS_IOC_LOG_INET" undefined
> iocLogClient: logging disabled
> macExpandString failed
> Cannot set field HOPR to $(HOPR)
> macExpandString failed
> Cannot set field LOPR to $(LOPR)
> filename="../macCore.c" line number=709
> <string>: HOPR referenced but undefined
> Y:rd.HOPR Has unexpanded macro
> filename="../macCore.c" line number=709
> <string>: LOPR referenced but undefined
> Y:rd.LOPR Has unexpanded macro

The database after loading looks like this:

> iocexample> dbDumpRecord pdbbase, 0, 0
> record(ai,"Y:rd") {
>         field(DESC,"position readback")
>         field(DTYP,"Soft Channel")
> }
> record(calc,"X:rdbk") {
>         field(DESC,"calculation")
>         field(SCAN,".5 second")
>         field(CALC,"A")
>         field(INPA,"X:rd PP NMS")
>         field(HOPR,"9")
>         field(LOPR,"1")
> }
> value = 0 = 0x0

i.e. no value is given to Y:rd.HOPR and Y:rd.LOPR, they remain at their 
default values.

On R3.14.6, the IOC expansion is even more verbose with error messages:

> dbLoadTemplate("test.substitutions")
> Error   Last token ")"
> input line:      field(HOPR,"$(HOPR)")
>    in: path "."  file test1.template line 3
>
> Error   Last token ")"
> input line:      field(LOPR,"$(LOPR)")
>    in: path "."  file test1.template line 4
>
> db_parse returned -1
> filename="../../../src/libCom/macLib/macCore.c" line number=734
> <string>: HOPR referenced but undefined
> filename="../dbLexRoutines.c" line number=313
> macExpandString failed for file test1.template
> Y:rd.HOPR Has unexpanded macro
> filename="../dbLexRoutines.c" line number=942
> Bad Field value
> filename="../../../src/libCom/macLib/macCore.c" line number=734
> <string>: LOPR referenced but undefined
> filename="../dbLexRoutines.c" line number=313
> macExpandString failed for file test1.template
> Y:rd.LOPR Has unexpanded macro
> filename="../dbLexRoutines.c" line number=942
> Bad Field value 

The only way we can replicate your results here is by using msi to do the 
macro expansion on the host, which gives:

> phoebus% msi -S test.substitutions
> record(calc,"X:rdbk") {
>        field(DESC,"calculation")
>        field(SCAN,".5 second")
>        field(CALC,"A")
>        field(INPA,"X:rd .PP.NMS")
>        field(HOPR,"9")
>        field(LOPR,"1")
> }
>  record(ai,"Y:rd") {
>      field(DESC,"position readback")
>      field(HOPR,"9")
>      field(LOPR,"1")
> }

We will therefor be releasing a new version of MSI which behaves like the 
old dbLoadTemplate program and leaves the unexpanded $(HOPR) and $(LOPR) 
variables in the output.

Note that the rules for how templates are expanded are found in the IOC 
application's <top>/config/RULES.Db file.  We assume that your RULES.Db 
has been modified to use msi instead of dbLoadTemplate.

We are also leaning towards implementing Andreas Luedeke's suggestion of 
being able to put default values for particular macros inside the template 
file.  If you wish to persue your idea of including defaults in the 
substitutions file pattern line, you'll have to write your own tool to do 
that (or modify one of the existing EPICS programs).

- Andrew
--
Dear God, I didn't think orange went with purple until I saw
the sunset you made last night.  That was really cool. - Caro