Re: [EXTERNAL] Re: Db/Makefile infinite loop

[Han Lee via Tech-talk <tech-talk at aps.anl.gov>]

Just additional information, which may help future tech-talkers.

The original code works well with Makefile 3.81 with Base 7.0.7. It doesn't work with Makefile 4.xx with Base 7.0.7

Han

On Wed, Oct 19, 2022 at 10:19 AM Han Lee <jeonglee at lbl.gov> wrote:

Hi Matt,

Please check the comment from Ralph. He is correct, and we should move the 'generated" code into O.Common instead of SRC.

Best,

Han

On Wed, Oct 19, 2022 at 10:00 AM Pearson, Matthew <pearsonmr at ornl.gov> wrote:

Hi Han,

 

Thanks, that’s fixed it (on all the base versions). It’s a nicer solution than what I originally came up with.

 

I’ll paste it below so it’s recorded in tech-talk:

 

diff --git a/loopTestApp/Db/Makefile b/loopTestApp/Db/Makefile

index befc80a..7f60c13 100644

--- a/loopTestApp/Db/Makefile

+++ b/loopTestApp/Db/Makefile

@@ -17,10 +17,11 @@ include $(TOP)/configure/RULES

#----------------------------------------

#  ADD RULES AFTER THIS LINE

 

-example.template: example.generated

+../O.Common/example.db: ../example.generated

 

-example.generated:

-       echo "# some more database logic" > example.generated

+../example.generated:

+       @$(RM) $@

+       echo "# some more database logic" > $@

 

 

Cheers,

Matt

 

 

From: Han Lee <jeonglee at lbl.gov>
Sent: Wednesday, October 19, 2022 12:02 PM
To: Pearson, Matthew <pearsonmr at ornl.gov>
Cc: tech-talk at aps.anl.gov
Subject: [EXTERNAL] Re: Db/Makefile infinite loop

 

I made a few changes and did pull requests. Please check your repository and let me know.

 

Best,

Han

 

On Wed, Oct 19, 2022 at 8:11 AM Pearson, Matthew via Tech-talk <tech-talk at aps.anl.gov> wrote:

Hi,

 

Sure, I’ve put it here, with a description of what I did:

https://github.com/mp49/epics_make_loop_test

 

This is just a simple module I created that reproduces the problem.

 

For this test I used fresh downloads of base 3.14.12.8, 3.15.9 and 7.0.7.

 

If you clone that, then edit the RELEASE file to point to your base versions, and you should see the problem happen (with 3.15.9+).

 

Cheers,

Matt

 

From: Simon Rose <Simon.Rose at ess.eu>
Sent: Wednesday, October 19, 2022 8:33 AM
To: Pearson, Matthew <pearsonmr at ornl.gov>; tech-talk at aps.anl.gov
Subject: [EXTERNAL] Re: Db/Makefile infinite loop

 

Hello Matthew –

 

I have tried to run your code and I do not get the infinite loop, but I find instead the expected output.  Can you please push the exact failing “epics module” online somewhere in order to test it out?

 

Cheers,

 

Simon

 

From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of "Pearson, Matthew via Tech-talk" <tech-talk at aps.anl.gov>
Reply to: "Pearson, Matthew" <pearsonmr at ornl.gov>
Date: Tuesday, 18 October 2022 at 20:28
To: "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
Subject: Db/Makefile infinite loop

 

Hi,

 

One of our support modules that built fine with base 3.14 is failing with base 7.0.7 (and base 3.15.9). This seems to be because the Db/Makefile has additional rules that define targets that already exist, in order to trigger a script to generate database templates before MSI is executed.

 

For example, if we have a database template file called “example.template”, and we want to build an “example.db”, and “example.template” is:

 

# Some database logic

include "example.generated"

 

And “example.generated” is only built at build time via a script. The script needs to run before MSI is run.

 

I’ve reproduced the problem with this simplified Db/Makefile:

 

TOP=../..

include $(TOP)/configure/CONFIG

 

DB += example.db

 

include $(TOP)/configure/RULES

 

example.template: example.generated

 

example.generated:

              echo "some text" > example.generated

 

 

So, the “example.template” target is defined, presumably as a trick to ensure that the script is run before MSI tries to build “example.db”. But “example.template” already exists in the Db directory.

 

With based 3.14 I get this output:

 

perl /home/controls/common/base/base-3.14.12.8/bin/linux-x86_64/makeMakefile.pl O.linux-x86_64 ../../..

mkdir O.Common

make -C O.linux-x86_64 -f ../Makefile TOP=../../.. \

    T_A=linux-x86_64 install

make[1]: Entering directory '/home/controls/inst/st99/st99-example/main/st99-exampleApp/Db/O.linux-x86_64'

echo "some text" > example.generated

#cp ../example.template .

Inflating database from example.template

msi    -I. -I.. -I../O.Common -I../../../db -I/home/controls/common/base/base-3.14.12.8/db example.template > example.tmp

mv example.tmp ../O.Common/example.db

Installing created db file ../../../db/example.db

make[1]: Leaving directory '/home/controls/inst/st99/st99-example/main/st99-exampleApp/Db/O.linux-x86_64'

 

which works fine.

 

However, with base 7.0.7 and 3.15.9 I get:

 

perl -CSD /home/controls/common/base/base-3.15.9/bin/linux-x86_64/makeMakefile.pl O.linux-x86_64 ../../..

mkdir O.Common

make -C O.linux-x86_64 -f ../Makefile TOP=../../.. \

    T_A=linux-x86_64 install

make[1]: Entering directory '/home/controls/inst/st99/st99-example/main/st99-exampleApp/Db/O.linux-x86_64'

echo "some text" > example.generated

#cp ../example.template .

/home/controls/common/base/base-3.15.9/bin/linux-x86_64/msi -D    -I. -I.. -I../O.Common -I../../../db -I/home/controls/common/base/base-3.15.9/db -o ../O.Common/example.db example.template > example.db.d

/home/controls/common/base/base-3.15.9/bin/linux-x86_64/msi -D    -I. -I.. -I../O.Common -I../../../db -I/home/controls/common/base/base-3.15.9/db -o ../O.Common/example.db example.template > example.db.d

/home/controls/common/base/base-3.15.9/bin/linux-x86_64/msi -D    -I. -I.. -I../O.Common -I../../../db -I/home/controls/common/base/base-3.15.9/db -o ../O.Common/example.db example.template > example.db.d

 

and it’s an infinite loop.

 

I believe this is because we have a Makefile target with the same name as the database template file, and the timestamp on that file never changes (because the Makefile rule is not touching the template file, only generating a different file). So this logic may be suspect from the start. Is there a better way to do this? Specially, how to run a script in the Db/Makefile before MSI without resorting to the above trick?

 

I have a workaround, which works for all base versions, which is to make a new copy of the template file so that Make is happy:

 

example.generated:

              echo "some text" > example.generated

              cp ../example.template .

 

 

However, why did it work for 3.14 and not 3.15+? Is it because the db.d format changed and it now references “example.template” whereas before it didn’t? For base 3.14 the O.linux-x86_64/example.db.d file is:

 

# DO NOT EDIT: This file created by mkmf.pl,v 1.5 2002/03/25 21:33:24 jba Exp $

 

../O.Common/example.db : example.generated

 

Whereas the same file using base 3.15 is:

 

../O.Common/example.db: ../example.template \

./example.generated

 

I assume that db.d file is used as a Makefile rule, and since “example.template” is now referenced the build system will repeatedly try to execute the “example.template” rule?

 

Cheers,

Matt

 

 

 

STS Integrated Control System Lead Engineer

Spallation Neutron Source

Oak Ridge National Lab

 

--

Jeong Han Lee, Dr.rer.nat

Staff Scientist and Engineer

Lawrence Berkeley National Laboratory

1 Cyclotron Road Mailstop 46R0161

Berkeley, CA 94720, United States

Tel :+1-510-486-6163

Cell:+1-510-384-3868

--

Jeong Han Lee, Dr.rer.nat

Staff Scientist and Engineer

Lawrence Berkeley National Laboratory

1 Cyclotron Road Mailstop 46R0161

Berkeley, CA 94720, United States

Tel :+1-510-486-6163

Cell:+1-510-384-3868

--

Jeong Han Lee, Dr.rer.nat

Staff Scientist and Engineer

Lawrence Berkeley National Laboratory

1 Cyclotron Road Mailstop 46R0161

Berkeley, CA 94720, United States

Tel :+1-510-486-6163

Cell:+1-510-384-3868