Experimental Physics and
Industrial Control System

"J. Lewis Muir" <[email protected]> · Wed, 16 Nov 2016 15:14:12 -0600

On 11/16, Kasemir, Kay wrote:
> Hi:
> 
> We had very similar discussions for CS-Studio.
> 
> Originally, we included the CS-Studio dependencies in our sources,
> i.e. we had a copy of the JAR files for accessing oracle, mysql, JMS,
> Channel Access.  That made for very easy and repeatable builds:  Unzip
> a snapshot of the CS-Studio sources (which include 3rd party JAR
> files), import into Eclipse or invoke the equivalent of "make", done.
>
> But this obviously results in a growing source repository.  First we
> moved all the 3rd party JARs into their own source repository.  Then
> we switched to using Maven as a build system, and removed many of the
> 3rd party binaries from our source repos.  Maven downloads the 3rd
> party libraries from somewhere on the internet.  There are days where
> it can't find them, or it picks a newer version because we didn't
> take enough care to specify the exact version that we need.  While
> our current approach "works" on most days, saves some space in the
> repository and is buzzword compliant, it does take longer to build.
> Newcomers tend to need about a week to get to their first successful
> build setup.  At least for me, the earlier approach of having the
> plain JAR files together with the sources was easier to understand and
> 100% repeatable.

Hi, Kay!

IMHO, ideally, JARs would be distributed as part of the OS or through
the OS' package manager and maintained just like the shared libraries.
You then either provide a package for your application for the OS'
package manager that pulls in all the packages needed to satisfy the
JAR dependencies, or you tell users what command to run for their OS to
install all the needed JARs (e.g., run "sudo apt-get install PKG1 PKG2
PKG3 ..."). But this is definitely not reality; there are some JARs
packaged this way, but the vast majority on the Maven central repository
are not.

Yes, you do have to be careful to fully specify your dependencies.
Ending up with different versions of dependencies over time is
absolutely unworkable.  I'm not a Maven expert, but I would expect it
to be possible to fully specify all dependency versions so that a build
can be 100% reproducible.  A newcomer taking a week to get their first
successful build sounds nuts.  After doing some Google searches, I
believe 100% reproducible builds are possible with Maven, but it seems
like a pain to get there.  The best I found was this Stack Overflow
question:

  http://stackoverflow.com/q/27016865

The answers suggest that it is possible, but it sounds like you have to
list all the dependencies (including transitive dependencies) explicitly
in a POM or something and then use a Maven plug-in to ensure you didn't
make a mistake.  Anyway, you probably know more about Maven than I do.

I must say, though, that I like Bundler:

  http://bundler.io/

It's for managing Ruby gems in the Ruby world and is really nice
compared to Maven for JARs in the Java world.  With Bundler, you can
get 100% reproducible builds, but it seems *way* easier.  You just
specify your dependencies in a Gemfile, and then Bundler maintains
a Gemfile.lock file with all the specific dependency versions used
(essentially pinning them).  You keep Gemfile *and* Gemfile.lock (e.g.,
under version control), and as long as you don't change Gemfile and as
long as you have Gemfile.lock, you will always get exactly the same
versions of all your dependencies.  See the Bundler rationale page for
how it works:

  http://bundler.io/rationale.html

I started out thinking there must be some Bundler equivalent in the Java
world, but I didn't immediately find it.  The closest I found was a
plug-in for Gradle called Gradle Dependency Lock Plugin:

  https://github.com/nebula-plugins/gradle-dependency-lock-plugin

That would require switching to Gradle, but who knows, maybe it'd be
worth it considering the pain you might save.

Regards,

Lewis

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System