Re: RFC: Creating st.cmd from snippets

[Ralph Lange <ralph.lange@gmx.de>]

Hi Andrew,

On 22/09/2015 22:27, Andrew Johnson wrote:

On 09/22/2015 04:28 AM, Ralph Lange wrote:

https://github.com/epics-base/epics-base/wiki/Simple-mechanism-to-create-files-from-ordered-snippets-(blocks)

My original thought was that the output file would only contain a list
of iocshLoad commands for all the snippet files that it found, not that
it would read in and spit out the contents of those files. This might
result in a much simpler Perl script, but it would then have to handle
the case where the IOC's directory tree looks different to the build
host. Your solution has different areas of complexity and simplicity but
overall is probably equivalent.

We are currently using the include-may-snippets approach, and I consider it a nightmare.
Looking at an error while the IOC is booting (or at the log of it), you never know which of the ~20 snippets contains the failing line. You are forced (on a target system with a basic editor) to look at the st.cmd for the names and their order, and then open one snippet after another until you find the failing line.
Before editing a snippet, you have to remember (or find in the documentation) which of the snippets are created and overwritten, and which are the ones intended for user editing. There is always a good chance that you can't add a line where you need to, because that place is inside a series of created-file includes.
With our users, this creates the impression that booting an IOC is an incredibly complex, error-prone, and hard-to-debug operation. Which is just wrong.

Blowing up the st.cmd on the host allows to have a single st.cmd file installed, no dependencies on other places with snippets. Looking at that file you know what the IOC is doing, making the boot process very obvious.

A few comments & questions on your document:

1. Why the design D03 for no dots in snippet filenames? I would suggest
a requirement that filenames all have the same specific extension such
as .cmd or .ish. GNUmake pattern rules are hard to get right without a
unique extension to hang them onto.

Good point. I added specs for a requirement and a design that allows this.
I wasn't thinking about implementation yet, but you are certainly right.

2. The pattern rule that runs the script will need a wildcard dependency
for each snippet directory, but then just removing a snippet file would
be insufficient to trigger a rebuild. Not sure this is going to be easy
to get right even when snippets have an extension.

True. Will see how that works. (Or not.)
Final ugly fallback is running it every time.

3. R21 probably needs a bit more detail, e.g. where you set macro
values, and how expansion should handle environment variable macros in
the snippets.

To be honest, at this point I was adding this as a general placeholder.
Coming from fixed files, we don't need macros for this step, but I was sure people would be asking for them.
My first take would be a basic "key=value,key=value" string on the command line (what I found with a number of EPICS tools and apps), and do plain text replacement in the snippet, leaving any '$...' variables untouched. (See next paragraph.)

BTW there is an EPICS::macLib Perl module for when you get
to implementation, don't go writing your own.

I don't want to add any conflicts with standard EPICS macros that are present in st.cmd parts.
The user should be able to split an existing st.cmd in snippets and use the mechanism without changing the snippet contents. (I should probably add that as a requirement.)
So in this case I don't know if using that module helps a lot.

4. D92 -- is the idea to add rules to base/configure/RULES.ioc or put
them in the iocBoot/ioc*/Makefile itself?

Again: Haven't thought much about implementation yet.
RULES file preferred, in any case.
Aim are generic rules, user API maybe like:

CONFIG_FROM_SNIPPETS += st.cmd
stcmd_DIRS = /opt/ioc/st.cmd.d:$(TOP)/iocBoot/common/st.cmd.d:../st.cmd.d
stcmd_PATTERN = *.cmd

Thanks for your help!
~Ralph