Experimental Physics and
Industrial Control System

"Konrad, Martin" <[email protected]> · Thu, 3 May 2018 18:19:41 +0000

Hi Anton, Ralph,
> I get a bit jealous every time I see "bottom-to-top" setups with
> CIs, push triggering, automatic testing and such - that is until I
> imagine something goes wrong and, say, a malformed input just drags a
> trace through the entire setup.
Sure it does. But the same is true if you build and deploy manually. We
are trying to prevent issues by:

- Maintaining a "master" branch which gets build into an "unstable"
Debian repo and a "release" branch which gets build into a "release"
Debian repo. Releasing means merging to the release branch.
- We have set up a test network that mimics our production network. We
deploy from the "unstable" repo into this test environment and from the
"release" one into the production network. Nothing goes out to
production without test and approval.
- We are trying to have one piece of each device on the test network. In
other cases we are simulating hardware. The fact that FRIB is heavily
using VMs and only a small number of physical IOCs helps to keep the
cost for such a test environment low.

> Even if that doesn't break production right away, cleaning things up
> may be troublesome.
If we broke something we can roll back. This can be done either by

a) reverting the bad commit in Git and waiting for the new packages to
be build and distributed or
b) installing an older version of the affected package(s)

Both approaches work well for us.

> What different in the "FRIB version numbers" is that upload comes 
> before distribution - doesn't it mean that a recent upload for, say, 
> jessie will sort as newer when compared to an older upload for 
> stretch? And the system will consider it to be newer if it sees 
> both.
We attach the timestamp when our CI server builds the _source_ package.
If the same source package gets build for multiple distributions the
version number of the resulting packages is the same except for the
distribution name. The package build for stretch ("debian09") will sort
as newer as the one for jessie ("debian08"). This ensures we upgrade to
the stretch package after dist-upgrading a machine.

Regarding the Git hash in the version number: I like this feature as it
allows me to directly correlate packages (including "unstable" builds)
to a revision in Git. I agree that this is less important for production
builds if they get tagged anyway. But to be honest I trust the Git hash
more than a tag that could get modified later. We still maintain normal
version numbers (e.g. "4.33") for readability but we never saw a reason
to remove the additional stuff (timestamp, build number and Git hash) -
it can easily be ignored if not needed.

-Martin

-- 
Martin Konrad
High Performance Controls Team Leader
Facility for Rare Isotope Beams
Michigan State University
640 South Shaw Lane
East Lansing, MI 48824-1321, USA
Tel. 517-908-7253
Email: [email protected]

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System