Re: User Created File Types: FILE TYPE += ABC [EPICS R3-14-12]

[Bruce Hill <bhill@slac.stanford.edu>]

I like Nick's approach of having all build directories support the
same targets.   Maybe we just need to figure out what kind of
build targets envPaths and st.cmd are and where to install them.

envPaths and st.cmd will typically be unique to each IOC, so
if you install them to bin/$(T_A), you'd need unique filenames
for each.    Our approach for this is to use the ioc instance name
as part of the filename.    For example, instead of st.cmd, we
could use $(IOC).cmd

So where should envPaths and $(IOC).cmd be installed?
I'm tempted to propose
$(INSTALL_LOCATION)/iocBoot/$(IOC)

But I think this would work best if the "source" files were found in
$(TOP)/iocSrc/$(IOC)
That way the distclean targets could remove iocBoot.

To keep compatibility with existing installations, it would probably
be better to keep iocBoot as a source directory, and install to
a new path such as
$(INSTALL_LOCATION)/iocStartup/$(IOC)

That way if you don't have a need to install your startup files
to a different install location, you can continue to just use iocBoot/$(IOC)
for your ioc specific files.

Regards,
- Bruce


On 11/4/2012 11:41 PM, nick.rees@diamond.ac.uk wrote:
> Hi,
>
> At Diamond we do something different again. We basically don't include RULES.ioc  - all leaf build directories include the same set of rules. The boot script is just a SCRIPT target, and gets installed in bin/$(T_A).
>
> The difference is that we have a rule that runs a script through macro substitution to generate an IOC boot script. The default macros that are substituted are the same as the environment variables normally kept in envPaths/cdCommands, e.g. EPICS_BASE=$(EPICS_BASE), INSTALL=$(INSTALL_LOCATION), ARCH=$(T_A) etc. We have modified convertRelease.pl to output these variables in a form suitable for use as msi command line options. With a bit of fiddling you can even have one script work for multiple target architectures.
>
> Cheers
>
> Nick Rees
> Principal Software Engineer           Phone: +44 (0)1235-778430
> Diamond Light Source                  Fax:   +44 (0)1235-446713
>
> ________________________________________
> From: tech-talk-bounces@aps.anl.gov [tech-talk-bounces@aps.anl.gov] on behalf of Bruce Hill [bhill@slac.stanford.edu]
> Sent: 03 November 2012 06:11
> To: Andrew Johnson
> Cc: tech-talk@aps.anl.gov
> Subject: Re: User Created File Types:  FILE TYPE += ABC [EPICS R3-14-12]
>
> We've done something like this in the photon controls group at SLAC.
>
> We wanted to be able to build db files in the iocBoot/ioc-instance-name
> directories.    In addition, we added RULES.archive and RULES.autosave
> so we could create custom db, archive and autosave files
> for each ioc instance.    That way we can create new ioc instances
> where all the ioc specific files are located in the new ioc instance directory.
>
> To do this, we moved everything in base's RULES.ioc after the line
> that includes RULES_DIRS into a new file, RULES.ioc-targets and
> replaced it with a single line:
> include $(CONFIG)/RULES.ioc-targets
>
> This allows traditional ioc's to build as before.
>
> To build db files from the iocBoot/ioc-instance-name directory,
> we change the last line in it's Makefile from:
> include $(TOP)/configure/RULES.ioc
> to:
> include $(TOP)/configure/RULES
> include $(TOP)/configure/RULES.ioc-targets
>
> Now when you run make in the ioc instance directory, your
> Makefile can have DB += lines or other build targets like
> the app build directories, and those targets get built in
> O.Common or O.$(ARCH) as appropriate, and installed as
> they would be from the app build directories.
>
> As to installing the st.cmd or envPaths files, we haven't done
> that exactly, but we did create a RULES.copy file that
> supports installing a set of files to a single directory.
> We've mostly used it to install protocol files.
>
> It's used by adding this line to the end of your Makefile:
> include $(TOP)/configure/RULES.copy
>
> Then, above that in the area for build targets, you set
> COPY_SUBDIR = name_of_destination_dir
> COPY_FILES += name_of_file1
> COPY_FILES += name_of_file2
> ...
>
> The files are installed to $(INSTALL_LOCATION)/$(COPY_SUBDIR).
>
> I've attached copies of these RULES files for
> anyone who's interested.
>
> While you can use them by putting a copy of each in your base/configure
> directory and each IOC's $(TOP)/configure directory, the RULES files we
> put in our $(TOP)/configure typically look like this:
> % cat configure/RULES.ioc-targets
> # RULES.ioc-targets
> include $(EPICS_BASE)/configure/RULES.ioc-targets
>
> Regards,
> - Bruce
>
> P.S.   I just realized in re-reading this that there may be a flaw in our approach,
> in that the envPaths file is generated in O.$(EPICS_HOST_ARCH).
> That hasn't mattered here, as we've only used this for soft IOC's running
> on the same platform as our build, but more work may be needed to
> allow using this for cross compilation.
>
> P.P.S.  I also tried setting INSTALL_LOCATION w/ RULES.copy and
> RULES.ioc-targets to see if I could install st.cmd and envPaths.
> Some things worked, others didn't, so again
> more work may be needed to make it general purpose.
> Be careful what you set COPY_SUBDIR to as the clean target will run
> "rm -rf $(INSTALL_LOCATION)/$(COPY_SUBDIR)" where
> INSTALL_LOCATION defaults to TOP if you don't set it.
> Thus, setting COPY_SUBDIR to "." can delete your ioc directories
> out from under you if you don't set INSTALL_LOCATION. :0(
>
>
> On 11/02/2012 11:38 AM, Andrew Johnson wrote:
> > Hi Guys,
> >
> > The fact that both Ernest and Ralph want to be able to install files in an
> > IOC-specific directory tells me that the build system needs to be able to do
> > something like this, but it's not clear what the requirements are or where the
> > files should go.  The install path ioc/<name>  is an obvious target location,
> > but whatever we come up with will need to support the current way of building
> > IOCs as well.
> >
> > Janet has taken a look at Ernest's original request and says it's not going to
> > work without making build system changes, so I think we/you should agree on
> > what we want to achieve before we fiddle with the current system too much.
> >
> > IIRC Gemini has been using a build system that installs st.cmd files among
> > others for quite a long time, and I hope someone from there will pipe up with
> > what your requirements are or how your current application builds work so we
> > can consider them as well.
> >
> > In the mean-time, it is possible for any application to override the build
> > rules in Base (it actually overrides the setting of the CONFIG variable that
> > gnumake uses to find the files) without having to make any changes to Base
> > itself or without having to have a private copy of those rules.  You just set
> > the RULES variable in your RELEASE file to point to a top area that contains a
> > configure directory with your local version of the files CONFIG, RULES,
> > RULES_DIRS, RULES_TOP and RULES.ioc.
> >
> > Hoping for some collaboration here, although I know you're all busy.
> >
> > - Andrew
> >
> > On 2012-11-01 Williams Jr., Ernest L. wrote:
> > > ________________________________________
> > > From: tech-talk-bounces@aps.anl.gov [tech-talk-bounces@aps.anl.gov] On
> > >    Behalf Of Ralph Lange [Ralph.Lange@gmx.de] Sent: Thursday, November 01,
> > >    2012 8:12 AM
> > > To: EPICS Tech Talk
> > > Subject: Re: User Created File Types:  FILE TYPE += ABC [EPICS R3-14-12]
> > >
> > > Hi Ernest,
> > >
> > > the main issue behind this:
> > > In the traditional EPICS build system, IOC startup files (actually
> > >    everything under iocBoot) are considered source files. Other than for
> > >    libraries, databases etc., these sources are not compiled in any way, and
> > >    they do not create any products that are installed. Without an
> > >    installation step, $(INSTALL_LOCATION) does not apply. ====
> > > ELW
> > > ====
> > > =================================================================
> > > I think this is actually wrong and should be addressed.
> > > They are indeed products built in the IOC area:
> > > (1) envPaths or for vxWorks "cdCommands"
> > > (2) The st.cmd is simply an ASCII text file; like a very simple database
> > >    needing no MACRO expansion. In other words the st.cmd needs to be simply
> > >    installed.
> > >
> > > Bottom line: Anything that is loaded by an IOC should be an installable set
> > >    of libraries, files, etc...
> > >
> > > Not having this of course, breaks symmetry. :(
> > >
> > > Although, I know it has always been this way.  My INSTALLATION_TOP has
> > >    always been right along side my APP. Now, I really want to push the
> > >    products to a different location.
> > > ===================================================================
> > >
> > >
> > >
> > >
> > >
> > > For our HZB/BESSY developments, we are creating the startup files
> > >    programmatically in an ...App directory, so we changed our rules to treat
> > >    .../iocBoot as an installation directory (much like lib, bin, ...), and
> > >    install the created startup scripts there. In this setting, there is an
> > >    installation step for these files, and $(INSTALL_LOCATION) works as
> > >    expected. ======
> > > ELW
> > > ======
> > > ===============================================================
> > > I recommend that this become part of the EPICS main distribution for the
> > >    reasons I stated above. This would everyone creating their own solution.
> > > In this case, for backward compatability can we leave "iocBoot" in tact and
> > >    create such as "ioc" ?
> > >    ================================================================
> > >
> > >
> > > Trying to select the relevant configuration parts....
> > >
> > >
> > > Our CONFIG contains:
> > >
> > > INSTALL_BOOT := $(INSTALL_LOCATION)/iocBoot
> > > UNINSTALL_DIRS += $(INSTALL_BOOT)
> > >
> > >
> > > The ...App Makefile uses this by setting:
> > >
> > > IOCS = $( /* database call to get ioc names */ )
> > > STCMD_INSTALLS = $(IOCS:%=$(INSTALL_BOOT)/ioc%/st.cmd)
> > >
> > > build: $(STCMD_INSTALLS)
> > >
> > > $(STCMD_INSTALLS): $(INSTALL_BOOT)/ioc%/st.cmd: $(COMMON_DIR)/st.cmd.%
> > >           @echo "Installing $@"
> > >           @$(CP) $<  st.cmd
> > >           @$(INSTALL) -d -m 644 st.cmd $(@D)
> > >           @$(RM) st.cmd
> > >
> > > $(COMMON_DIR)/st.cmd.%: ../%.st.cmd
> > >           @$(CP) $<  $@
> > >
> > >
> > > Hope this shows the general direction...
> > > =================================================
> > > =====
> > > ELW
> > > =====
> > > ==================================================
> > > Thanks, this is the direction we would like to go in.
> > > Just be nice to get this into EPICS BASE as well.
> > > Everything loaded on the IOC is a product. :)
> > > ==================================================
> > >
> > >
> > > Thanks again.
> > >
> > > Cheers,
> > > Ernest
> > >
> > >
> > >
> > >
> > >
> > > Cheers,
> > > ~Ralph
> > >
> > >
> > > On 30.10.2012 16:51, Ernest L. Williams Jr. wrote:
> > > Hi Janet,
> > >
> > > How is it going?
> > >
> > > For IOC Application deployment; I would like to take advantage of the
> > >    $(INSTALL LOCATION). The only problem is that IOC Startup files do not get
> > >    installed along with other products such as databases, libraries, etc...
> > >
> > > So, I tried to do something like the following:
> > >
> > > In my  $IOC_APP_TOP/iocBoot/<iocName>
> > >
> > > Create CONFIG_STARTUP:
> > > ============ CONFIG_STARTUP ========================
> > > FILE_TYPE +=  STARTUP
> > > INSTALL_EDL = $(INSTALL_LOCATION)/ioc/vioc-b34-cd39
> > > =====================================================
> > >
> > > Add to Makefile:
> > >
> > > ========== Makefile =====================================
> > > TOP = ../..
> > > include $(TOP)/configure/CONFIG
> > > # =====================================================
> > > # Config for installing IOC startup related files:
> > > # =====================================================
> > > CFG +=  CONFIG_STARTUP
> > >
> > > ARCH = linuxRT-x86
> > > TARGETS = envPaths
> > > INSTALL_STARTUP += envPaths rtPrioritySetup.cmd st.cmd
> > > include $(TOP)/configure/RULES
> > > include $(TOP)/configure/RULES.ioc
> > > # =====================================================
> > >
> > >
> > > The above does not work.
> > >
> > > [ernesto@cdlx01 vioc-b34-cd39]$ make
> > > /afs/slac/g/lcls/epics/base/base-R3-14-12/configure/RULES_DIRS:72: ***
> > >    target file `clean' has both : and :: entries.  Stop.
> > >
> > >
> > > Can you recommend a way to achieve this in the context of the EPICS
> > >    Make/Build system?
> > >
> > > Cheers,
> > > Ernest
> > - Andrew
> --
> Bruce Hill
> Member Technical Staff
> SLAC National Accelerator Laboratory
>
>
> --
> This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
> Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
> Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
> Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom