Re: EPICS Base R3.14.9-RC1 Released

[Andrew Johnson <anj@aps.anl.gov>]

Benjamin Franksen wrote:
> From the release notes:
>
> ---------------------------------------------
> dbLoadTemplate
>
> The parser for the substitutions file accepted by dbLoadTemplate() has been 
> revised, and is now stricter than it used to be â unmatched characters that 
> were accepted and discarded without warning before will now generate 
> errors. The quote characters at the two ends of a quoted string value must 
> now match, although either single or double quotes can be used. Escaped 
> characters inside a quoted string should pass safely through to the 
> underlying dbLoadRecord() command. The characters permitted in a bareword 
> parameter (i.e. a filename or value that is not inside quotes) have been 
> reduced from the overly wide set allowed before; the permitted characters 
> now comprise: a-z A-Z 0-9 _ - + : . / \ [ ] < > ;
> ---------------------------------------------
>
> I am a bit astonished about this. This whole change, but especially the last 
> one, introduces a major incompatibility. I would have expected at least a 
> note to tech-talk beforehand, inviting requests for comments from concerned 
> parties. I am astonished about this because in many other respects 
> compatibility on the database level has been a top-priority for EPICS 
> (sometimes pursued almost to the point of stiffling progress).

I don't think this change is as large as you make it.  Section 6.20 of 
the R3.14.8 Application Developers' Guide requires that any filenames 
containing macros be given in a quoted string, which is a standard 
behaviour in EPICS DB and DBD files.

To allow escaped characters in quoted strings for record field values I 
had to make some changes to the definition of a quoted string, and when 
I looked at all the lex and yacc templates in Base I found some horrible 
coding in dbLoadTemplate - for instance, what self-respecting programmer 
would regard this as a valid string?
	"quoted string'
MSI won't accept that, but dbLoadTemplate() did.  There were a few more 
horrors like that hidden in the R3.14.8 dbLoadTemplate_lex.l which I 
just couldn't allow to survive any longer, hence the cleanup.

> I would be interested to hear of the rationale i.e. what is the great 
> improvement, considering the possible disruption such an incompatible 
> change may cause, and also considering that 3.14.9 has been advertised as 
> a 'mostly bug-fix' release?

The syntax allowed by the dbLoadTemplate() command was actually looser 
than that permitted by MSI, and in places it still is.  For example, MSI 
only permits quoted strings to use a double quote character, whereas 
dbLoadTemplate() did and still does permit you to use single quotes (but 
now they do have to match at the two ends of the string).

> One further issue that I would be interested in is: will the corresponding 
> tool on the host side (MSI) be adapted according to the new rules, so that 
> inflating substitutions on host and target side will remain equivalent? (Or 
> have they ever been after dbLoadTemplate was no longer available on the 
> host side?)

The tightening up of the syntax I have done has not affected MSI at all, 
and I don't intend to make any changes to that tool; the two have 
probably never been completely identical in the syntax they will accept 
since they are independently written code, but there is and always will 
be a core syntax that both understand, which is the syntax described in 
the Application Developers' Guide.

If this change prevents an operational site from upgrading to R3.14.9 I 
am willing to write a perl script that fixes any files that are no 
longer accepted by the dbLoadTemplate() command.  However I do expect 
such changes to be minor and probably quicker to fix by hand in most 
cases - quoting some filenames is likely to be the most common change.

- Andrew
--
The right to be heard does not automatically include
the right to be taken seriously. -- Hubert H. Humphrey