Re: NSLS-II Debian Repository in 2018

["J. Lewis Muir" <jlmuir@imca-cat.org>]

On 02/13, Anton Derbenev wrote:
> Hello Ralph, Lewis, and Michael,
> 
> thanks for your feedback and thoughts. As Michael noted, the
> question of tools is not the only consideration here. For all the
> packaging/publishing enterprise, I from the very beginning have two
> main concerns:
>
> 1) Real applicability. While it would be great to have everything
> packaged for everything, the question of optimal tools and mechanism
> implementations may be not even viable until a purpose is found. If
> our entire infrastructure runs Debian, testing grounds for the
> packaged software are limited. Building a thing is one thing, making
> sure that it doesn't choke with a segmentation fault during runtime is
> another thing.

Hi, Anton!

I'm not suggesting you take on the burden of testing on all platforms.
I think a good model is to consider yourself a packager only; it's
the upstream developer of a piece of software who is responsible for
ensuring it works correctly on any supported platforms (i.e., platforms
the developer is willing to support).  After that, it would be up to
whoever wants the packages to work well on a particular platform to
ensure they work.

> I personally do not see it any practical to have a collection of
> software which is published just for the sake of it. This can be done
> for fun and consistency of course, but there comes the item #2.

Certainly I wouldn't do it for no reason, but you're limiting yourself
by choosing a packaging system that supports Debian-based platforms
only.  With pkgsrc, you could focus on Debian-based platforms only, but
at least the packaging system would not effectively rule out support for
other platforms.

> 2) Real maintainability. It occurred to me very quickly that
> packaging is not a one-time project - the fact which is sometimes
> overlooked. Keeping it properly viable, updated, and tested
> is a dedicated effort; reacting quickly to upstream changes
> and user requests defines how useful the packaging is to
> users. Multi-distribution and multi-platform packaging requires
> multi-expertise, sometimes application-specific expertise. Unless it
> is a project with proper resources assigned, I doubt it would be very
> practical for anyone to commit to such an over-encompassing effort
> which is quite volatile in its time requirements.

I totally agree that it's not a one-time thing; it takes a lot of effort
to initially create the packages, and then it takes effort to maintain
them.

As far as the cost of multi-distribution and multi-platform packaging
goes, that would be one of the main reasons for an open source packaging
effort: by working on a system that can be extended to meet the needs of
others on different platforms, you increase your chances of getting more
contributions from others.

To me, making a packaging effort for Debian-based platforms only would
be like creating a piece of software for RHEL only.  I could do that,
but normally I don't want to limit my potential user base because that
makes any potential developer community smaller than it otherwise might
be.  By trying to support multiple platforms, I increase the chance of
a larger developer community which essentially increases the amount of
(hopefully good) contributions.

So, right now you're saying it's a lot of work to do all by yourself,
and I hear you, but I'm saying that if you would use a packaging system
that would serve more than just Debian-based platforms (e.g., pkgsrc), I
think you'd get more contributions than you're getting now.

> On top of that, there is requirements split. On one hand, our
> accelerator environment wants stability with no practical reason
> or sometimes even no practical possibility to put new updates in
> place. On the other hand, our beamline environment often wants most
> recent developments which were not published in epicsdeb even, and
> probably will not be published for some while. The packaging is hence
> too fast for the first case and too slow for the second case; that
> is of course a consequence of the amount of work required to make it
> all "right". The practical outcome that we do what we can with things
> which we have.

I understand that, and I agree that part of it comes down to how many
hours of effort you can devote to it.  In pkgsrc, there are stable
quarterly release branches, and there is the current branch.  For your
accelerator environment, you would be free to stay on a quarterly
release branch for as long as you want.  Security updates are not
released for more than two quarters back, I think, but in a trusted
environment, you could ignore that.  (Or I think Joyent maintains LTS
(long-term support) branches that you could use.)  Then obviously for
your beamline environment, you could follow the current branch.  (Or if
the current branch is too new, you could follow the latest quarterly
release branch.)  You could install them under two different prefixes if
you wanted so that they could even coexist (e.g., /opt/pkg-2017Q4 and
/opt/pkg-current).  You could of course also keep multiple quarterly
release branches around too, all coexisting nicely.

>  Maybe in the future? :)
>
> However I am not sure how the future would help with the fact that a
> successful solution needs constant upkeep...

If you switched to pkgsrc, I think you'd get more users and then more
contributors.

Regards,

Lewis