Experimental Physics and
Industrial Control System

Mark Rivers <[email protected]> · Mon, 5 Feb 2018 23:32:21 +0000

Hi Lewis,

> Right, and with the way the synApps modules are currently developed,
> I agree with you.  However, I would bet that the reason synApps does
> not get updated regularly is that it's a lot of work to do it.  Part of
> the reason it's a lot of work is all the breaking change.  If all the
> modules in synApps adhered to semantic versioning and did not introduce
> breaking changes left and right, making a new synApps release would be
> significantly easier.

I really disagree the synApps has lots of breaking changes.  I will make the following assertion:

If you use the latest tagged version of every synApps module (including asyn, seq, ipac, areaDetector) it virtually always works.  That is essentially how I have been running our beamlines for a decade.

In my opinion a new version of synApps should simply be a script:  It does a git clone and git checkout of the last tagged version of every module as of that date. If that's what it did it could be updated frequently.  And users would have a git repository that they could trivially update by going into calc, for example, and grab the latest bug fix by checking out master, or a specific release by checking out R3-7.  It does not make sense to me to be using tar files.

Can you give me an example where using the latest tagged release of each synApps module would have led to problems?

Mark

> -----Original Message-----
> From: J. Lewis Muir [mailto:[email protected]]
> Sent: Monday, February 05, 2018 5:19 PM
> To: Mark Rivers <[email protected]>
> Cc: Jörn Dreyer <[email protected]>; [email protected]
> Subject: Re: AreaDetector repository inconsistent
> 
> On 02/05, Mark Rivers wrote:
> > Hi Lewis,
> >
> > > I don't understand why you want to separate out into multiple
> > > repositories things that everyone will just turn around and combine
> > > again.  Why not just leave them together?
> >
> > Here are a number of reasons:
> >
> > - Clear separation.  When we release a new version of ADPilatus people
> > who don't use ADPilatus but do use ADPointGrey know immediately that
> > this change does not affect them.  If the Pilatus and Point Grey code
> > were both in a large areaDetector repository then it would not be
> > obvious.  Currently Point Grey users only need to watch for releases
> > of ADCore and ADPointGrey.  If it were combined they would need to
> > watch for every release of areaDetector.  Since there are now 40
> > detector repositories in areDetector, to first order they would need
> > to read the release notes 20x more often.
> >
> > - Easier collaboration.  With separate repositories we can assign
> > permissions differently for each one.  I could give you write access
> > to ADADSC since you wrote it, but not give you write permission to
> > ADCore.
> >
> > - Lower barriers for new detectors.  Because each detector is in a
> > separate repository the barrier to accepting a new detector is lower.
> > They are well decoupled from ADCore. If ADxxx it does not work with
> > some change to ADCore it does not impact the release of ADCore, and
> > ADxxx will still work for earlier releases of ADCore.  We don't have
> > to wait for that developer to fix it.
> 
> Hi, Mark!
> 
> Thanks for sharing those reasons; they all make sense to me.
> 
> > Consider the analogy with synApps.  That is about 25 modules which are
> > in separate repositories (motor, calc, etc.) and need to be assembled
> > into a tree like areaDetector.  Are you suggesting that they should
> > also be in a single repository?
> 
> Yes, I would love that! :-)
> 
> We obviously differ here, and I do understand your reasons given above,
> but I suspect a more fundamental reason we differ here is because of
> how the modules included in synApps (and synApps itself) are developed
> compared to how, IMO, they could be developed.  What I see with synApps
> is all (well, most of) the modules I want.  And just like areaDetector,
> I might not use them all, but it's convenient to just get them all and
> build them all (or configure the build to include or exclude some).
> synApps is great in that it has provided a way to get all those modules
> at once and working correctly together.  That's a huge plus.  However,
> synApps releases happen so infrequently that I essentially have to make
> my own internal synApps releases which takes a nontrivial amount of
> time.  I would much rather have a synApps (and the modules it includes)
> that adheres to semantic versioning <https://semver.org/>, does not
> bump its major version every release, and releases often so that I can
> get (backward-compatible) features and bug fixes without having to read
> every entry in every module's changelog (if it has one).  I don't want
> to have to spend time dealing with breaking changes (i.e., changes that
> are not backward compatible) for every release.
> 
> Maybe a good analogy to this would be if I wanted to run RHEL 7 on all
> my machines for five years, but instead I had to upgrade to the next
> major release (e.g., RHEL 8, 9, 10, etc.) every three months.  I really
> value the API stability of a RHEL release and don't want to have to
> spend time changing and upgrading my software stack to work with each
> new RHEL release every three months.
> 
> > The synApps repositories get updated regularly, but the parent synApps
> > release has only happened twice in the last 5 years (5-7 and 5-8).  I
> > think that is an indication of how unbundling improves the frequency
> > of releases.
> 
> Right, and with the way the synApps modules are currently developed,
> I agree with you.  However, I would bet that the reason synApps does
> not get updated regularly is that it's a lot of work to do it.  Part of
> the reason it's a lot of work is all the breaking change.  If all the
> modules in synApps adhered to semantic versioning and did not introduce
> breaking changes left and right, making a new synApps release would be
> significantly easier.
> 
> Obviously, though, making backward-compatibility a priority would have
> to be something that the community really wants.  While I and maybe some
> others (others, are you out there?) might wish for it, if the community
> doesn't want it or the developers don't want it, then it's not going
> to happen.  Also, I understand that there can easily be very different
> philosophies and concerns/needs here (e.g., one synchrotron beamline
> adds or changes hardware all the time vs. another that builds out a
> configuration for one type of experiment and doesn't change the hardware
> much).
> 
> So, I understand your three reasons given in support of a
> multiple-repository approach, and they all make sense.  Thanks again for
> that!  I do think that the first one about clear separation and reading
> the release notes would be much less of an issue if the development
> approach was different.  Again, to draw an analogy with RHEL, a new
> minor release could include kernel drivers to support new hardware, and
> I don't mind because it doesn't break backward-compatibility.  I just
> install the latest updates for RHEL 7 (if I'm running 7), and I don't
> need to read release notes.
> 
> Regards,
> 
> Lewis

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System