Experimental Physics and
Industrial Control System

The problem withe the existing runtests target is that it depends on files and has a recipe.

With that setup, there is no way to add a different recipe for different files.

Unless you make it a double colon rule, which basically treats it like a completely separate thing. (The Make doc says: "Double-colon rules are somewhat obscure and not often very useful; they provide a mechanism for cases in which the method used to update a target differs depending on which prerequisite files caused the update." - which exactly describes that case.

The problem with your suggested pattern is that when the ...dependencies... don't exist, the intermediate phony target gets executed unconditionally, probably failing as the variables for prerequisites are empty.

Also, any rules for the generation of the report files should be pattern rules that only apply to the expected targets. It is very hard to work around something like the existing implicit "%.xml: %.tap" rule when - like in the Google Test case - the test executable creates both xml and tap output directly.

On Mon, 20 Apr 2020 at 19:40, Johnson, Andrew N. <anj at anl.gov> wrote:

Hi Ralph,

You committed this to the 3.15 branch last week:

commit cbf917e8332d620b6a2f60ea9822d1f84467ef28

Author: Ralph Lange <ralph.lange at gmx.de>

Date:   Thu Apr 16 12:04:16 2020 +0200

    Improve automated testing rules to allow other test frameworks

    

    - make runtests a double-colon rule, so that other test frameworks

      can add their own recipes independently

    - only define runtests:: $TESTSCRIPTS rule when there are TESTSCRIPTS

      (to avoid having it run every time when no TESTSCRIPTS are defined)

    - $(strip $TAPFILES) inside ifneq to fix trouble when TAPFILES=' '

I would like to undo the double-colon change, it is never actually necessary and double-colon rules can cause problems, although I’m not exactly sure what specific issues caused Janet to very deliberately change the clean rule back from a double-colon to a single-colon rule again (it was single-colon in earlier versions).

You can add additional recipes for the runtests target without them clashing with the existing one by adding intermediate targets, e.g.:

runtests: run-gtests

run-gtests: ...dependencies...

...commands...

With that modification I don’t think your other changes are required either, although I could be wrong. If you want I could add a similar intermediary for the tap-tests.

BTW take a look at the new pre-make and test-results recipes I’m considering adding to RULES_TOP in https://github.com/epics-base/epics-base/compare/3.15...anjohnson:appveyor?expand=1. pre-make may be a bad name, it gets run by RULES_DIRS before any recursion into a subdirectory; I needed it to clean out the new test summary file which the new test-results recipe displays and uses to mark test failures as an error. The exact behavior of the testFailures.pl script could be changed, but the idea is to collect all the test failures and have the test-results target print them out at the end of the build rather than bury them higher up. I realized these were needed after my last commit to the 3.15 branch which stopped the test-results target from ever failing; I wanted it to display all the results before test-results stopped, and for some reason ‘make -k’ isn’t doing that properly on the 7.0 builds.

Comments and suggestions on that welcome. For some reason Appveyor was failing most of my builds of this over the weekend but they’re passing now, I don’t think the two commits I made should have changed anything though.

- Andrew

-- 

Complexity comes for free, simplicity you have to work for.