Re: archiver tools macro support

[Bruce Hill <bhill@slac.stanford.edu>]

Hi Robert,
We have a similar situation in our epics development, as we
also wanted to generate archiver lists that used macro substitution
to build customized archive lists for each ioc instance.

As Jack suggested in another reply, there's a script in the archiver
extension that translates a simple text format to the xml.
That way instead of your templates being XML which needs to
be edited carefully to avoid syntax errors, they can contain
lines that look like this:

$(CUBICLE_NAME):FAN    2
to sample the FAN PV once every 2 seconds
or
$(CUBICLE_NAME):DOOR    30    monitor
for a PV which you don't expect to change often but want
to record every change.

The other thing we learned quickly is that it's difficult for a central
archiver script to collect archive files for multiple IOC's and do the
right macro substitution for each.   It's much better for the ioc
developer to use a substitution file and an archive template file
and let the build system expand the macros.

To do this, I created a RULES.archive file modeled after RULES.db
that uses msi to do the macro substitution.   See attached.
It contains build rules that use msi to automatically build foo.arch
from foo.tpl-arch and foo.sub-arch.

The easiest way to deploy it is to copy the attached RULES.archive
to your ioc's $(TOP)/configure directory.

First, create your template in cubicleApp/Db like this:
In cubicleApp/Db/Makefile:
Add the following line at the end of the Makefile
include $(TOP)/configure/RULES.archive
In the middle section of the Makefile, add:
ARCHIVE  += cubicle.tpl-arch

Then add a file cubicleApp/Db/cubicle.tpl-arch with contents similar to this:
# Cubicle archiver template
$(CUBICLE_NAME):FAN    2
$(CUBICLE_NAME):DOOR    30    monitor

Then, follow this pattern for each cubicle ioc.
In iocBoot/ioc-cubicle1/Makefile:
Add the following line at the end of the Makefile
include $(TOP)/configure/RULES.archive
  
In the middle section of the Makefile, add:
ARCHIVE   +=  ioc-cubicle1.arch

Then add file iocBoot/ioc-cubicle1/ioc-cubicle1.sub-arch with contents similar to this:
# Cubicle archiver substitution file
file cubicle.tpl-arch
{
     {   CUBICLE_NAME   =  "Cube1"   }
}

Then run make from $(TOP).
If all goes well, $(TOP)/archive should contain cubicle.tpl-arch and
ioc-cubicle1.arch with all macros expanded.

HTH,
- Bruce

P.S.  We did the same thing for autosave files with RULES.req,
foo.sub-req and foo.tpl-req.     That may not be as needed now since
I recall reading that the latest autosave module supports autosave
configuration via the DB files.

On 5/9/2013 10:17 AM, Tim Mooney wrote:
> Hi Robert,
>
> I don't know all the ways an archiver config file has to be useable,
> but you might be able to use msi
> (http://www.aps.anl.gov/epics/extensions/msi/index.php) to generate
> an application-specific config file whenever one is needed.
>
> Tim
>
> ----- Original Message -----
>> From: "Capobianco Roberto" <roberto.capobianco@igi.cnr.it> To:
>> tech-talk@aps.anl.gov Sent: Thursday, May 9, 2013 10:51:06 AM
>> Subject: archiver tools macro support
>>
>> Hi all,
>>
>> I have created, as template, a simple application to check some
>> signals in a cubicle (doors, fans, power, temperature etc.) Using
>> macro substitution, I have created several identical applications,
>> one for each cubicle to control. Now I want to enable persistance
>> and archiving on some PVs (I would like to do this in the template
>> not by hand in the applications). And this is the problem:
>> persistance supports macro substitution but what about archiver
>> config tools and/or file ?
>>
>>
>> Just to clarify what I mean, these are examples record from the
>> template:
>>
>> record (bi,"$(CUBICLE_NAME):FAN") { field(DTYP, "s7nodave")
>> field(INP, "@CUBICLEM F1.2 bool") field(PINI, "YES") field(SCAN,
>> "I/O Intr") field(ZNAM, "Fan OFF") field(ONAM, "Fan ON") }
>>
>> record (ao,"$(CUBICLE_NAME):WARNING_THRS_W") { field(DTYP,
>> "s7nodave") field(OUT, "@CUBICLEM DB1.DBD14 int32") field(LINR,
>> "LINEAR") field(EGUF, "214748364.7") field(EGUL, "-214748364.8")
>> field(EGU, "°C") field(HOPR, "35") field(LOPR, "20") }
>>
>> For the persistance I have something like this
>>
>> $(CUBICLE_NAME):WARNING_THRS_W.VAL in my _IOC_.req file and
>> create_monitor_set("_IOC_.req", 5, "CUBICLE_NAME=$(CUBICLE_NAME)")
>> in my st.cmd
>>
>> Can I create something like this for the _IOC_.xml archiver config
>> file ?
>>
>> <group > <name >$(CUBICLE_NAME)</name>
>>
>> <channel > <name >$(CUBICLE_NAME):FAN</name> <monitor ></monitor>
>> </channel> </group>
>>
>> Or I have to look at other solutions ?
>>
>> Thanks. Robert
>>
>>
>

####################################################
# RULES for building and installing archive files

# Display how this makefile was called
#$(info RULES.archive: '$(MAKECMDGOALS)')

########################################################
# NOTE: The rules for generating archive files from
# %.substitutions files shouldn't be needed any more.
# If you think you need them, please see Bruce Hill, bhill,
# so we can discuss.

####################################################
# Where to find and install files for archive

vpath %.archive       $(USR_VPATH) $(GENERIC_SRC_DIRS) $(dir $(ARCHIVE))
vpath %.sub-arch	  $(USR_VPATH) $(GENERIC_SRC_DIRS) $(dir $(ARCHIVE)) $(COMMON_DIR)
#vpath %.substitutions $(USR_VPATH) $(GENERIC_SRC_DIRS) $(dir $(ARCHIVE)) $(COMMON_DIR)
vpath %.template      $(USR_VPATH) $(GENERIC_SRC_DIRS) $(dir $(ARCHIVE)) $(COMMON_DIR)
vpath %.tpl-arch      $(USR_VPATH) $(GENERIC_SRC_DIRS) $(dir $(ARCHIVE)) $(COMMON_DIR)

INSTALL_ARCHIVE = $(INSTALL_LOCATION)/archive

####################################################
# archive Flags (-I path)

INSTALL_ARCHIVEFLAGS = -I $(INSTALL_ARCHIVE)
RELEASE_ARCHIVEFLAGS = $(patsubst %/dbd,%/archive, $(RELEASE_DBDFLAGS))
ARCHIVEFLAGS  = $($*_ARCHIVEFLAGS) $(USR_ARCHIVEFLAGS) -I. -I.. $(GENERIC_SRC_INCLUDES) $(INSTALL_ARCHIVEFLAGS) $(RELEASE_ARCHIVEFLAGS)
ARCHIVEFLAGS += -I$(COMMON_DIR)

####################################################
# Build targets

INSTALL_ARCHIVES += $(addprefix $(INSTALL_ARCHIVE)/,$(notdir $(ARCHIVE)))

build:	$(INSTALL_ARCHIVES)

rebuild:        clean install

buildInstall:	$(INSTALL_ARCHIVES)

realclean: clean

clean: archive_clean

archive_clean:
	@$(RM) $(COMMONS)
	@$(RM) $(TARGETS)
	@echo "Cleaning"

####################################################
# Build and install Rules

#$(COMMON_DIR)/%.archive: %.substitutions
#	@echo "Inflating archive file from $<"
#	@$(RM) $@
#	$(MSI) $(ARCHIVEFLAGS) -S$< > msi.tmp
#	$(MV) msi.tmp $@

$(COMMON_DIR)/%.archive: %.sub-arch
	@echo "Inflating archive file from $<"
	@$(RM) $@
	$(MSI) $(ARCHIVEFLAGS) -S$< > msi.tmp
	$(MV) msi.tmp $@

INSTALL = $(PERL) $(TOOLS)/installEpics.pl

$(INSTALL_ARCHIVE)/%: $(COMMON_DIR)/%
	@echo "Installing archive file $@"
	@$(INSTALL) -d -m 644 $< $(@D)

$(INSTALL_ARCHIVE)/%.archive:	%.archive  
	@echo "Installing archive file $@"
	@$(INSTALL) -d -m 644 $< $(@D)

#.PRECIOUS: $(COMMON_DIR)/%.archive

#$(INSTALL_ARCHIVE)/%.substitutions: %.substitutions
#	@echo "Installing archive substitutions file $@"
#	@$(INSTALL) -d -m 644 $< $(@D)

$(INSTALL_ARCHIVE)/%.sub-arch: %.sub-arch
	@echo "Installing archive substitutions file $@"
	@$(INSTALL) -d -m 644 $< $(@D)

#.PRECIOUS: $(COMMON_DIR)/%.substitutions

$(INSTALL_ARCHIVE)/%.template: %.template
	@echo "Installing archive template file $@"
	@$(INSTALL) -d -m 644 $< $(@D)

$(INSTALL_ARCHIVE)/%.tpl-arch: %.tpl-arch
	@echo "Installing archive template file $@"
	@$(INSTALL) -d -m 644 $< $(@D)

#.PRECIOUS: $(COMMON_DIR)/%.template