Re: Help needed with E3

[Anders Lindh Olsson via Tech-talk <tech-talk at aps.anl.gov>]

Happy to help.

> Do you have already a time frame for the next generation of e3? Which conda repository are you going to use?
> Taking into account the licensing issues with the conda repo that came up last year, I personally have my doubts how long conda-forge still will be usable without restrictions.

We have been running a version of it for 4-5 years already but with limited scope (neutron instrument "R&D") and a limited user group. We hope to make this our primary architecture in maybe 6-12 months, but it is somewhat hingent on our development as well as the ESS project plan progress. We do not use the anaconda/default channels, and currently we only use conda-forge to the degree that we copy package versions from it to a channel of our own, but we may make use of conda-forge to some degree, as a lower priority channel. How to deal with the channels is part of what we are working on, so I can unfortunately not tell you much more about how it will be at this point in time.

If you did want to also make use of that - new - generation of e3 (if that remains its name), the main thing you would need is probably one or more channels of your own. We have ours on a JFrog artifactory (https://urldefense.us/v3/__https://jfrog.com/help/r/jfrog-artifactory-documentation/conda-repositories__;!!G_uCfscf7eWS!drmEigggKCxJYPaxEZKQB9G3bumXdDI1cRaj2JlK_Fybm_sjeQs8jruQas48y_sKAYtUEIhthzvTp79jC_85D6R3UA-DV2Q$ ). You can read a bit about the current implementation - there is at least a high-level description and some training chapters - at https://urldefense.us/v3/__https://e3.pages.esss.lu.se__;!!G_uCfscf7eWS!drmEigggKCxJYPaxEZKQB9G3bumXdDI1cRaj2JlK_Fybm_sjeQs8jruQas48y_sKAYtUEIhthzvTp79jC_85D6R3TE9z9jE$ . 


Cheers
A


On 2025-03-26, 15:13, "Jörn Dreyer" <j.dreyer at hzdr.de <mailto:j.dreyer at hzdr.de>> wrote:


Hi Anders,


thanks for the information. 


Am Mittwoch, 26. März 2025, 13:36:52 Mitteleuropäische Normalzeit schrieb Anders Lindh Olsson via Tech-talk:
> Hi Jörn,
> 
> On the specification format and the "e3" package's utilities, they are - indeed - currently hardcoded to interact with our gitlab instance and select groups within that instance. It would, however, probably not be all too difficult to separate out these configurations to a separate file, or to expose optional arguments for overriding them. If you do not have your own gitlab instance, and/or if you want to use both our wrappers and your own, it might be better to allow further customisation through the specifications; for example, by allowing the user to specify the path to the wrapper (instead of doing a name-based lookup). Something like
> 
> adaravis:
> source: some/repo.git
> versions:
> - some/reference
> 
> We welcome any contribution as long as it does not break our usage. We would have offered to help, but we are currently focusing on the next generation of e3, using conda, and we are also in the midst of accelerator commissioning.
Do you have already a time frame for the next generation of e3? Which conda repository are you going to use?
Taking into account the licensing issues with the conda repo that came up last year, I personally have my doubts how long conda-forge still will be usable without restrictions.


> 
> I am not sure I understand your question about extracting the time information of the git commit that is part of the module version information, but if you are asking about the reference format, these are tags created by the e3-release utility in the same e3 package. We use this utility to mark and schedule deployments to our shared build servers (together with some additional infrastructure).
That was the missing information I needed to get. That is not documented in the accessible web pages so I was wondering how these versions cam about. I tagged and commited my wrappers to our local gitlab instance. So basically my tags could also be as simple as just 1.0.0.


> 
> When it comes to kernel modules, we handle these with ansible nowadays, and I believe they are built into our so-called "ESS Linux" (Yocto) images. Maybe one of my colleagues can offer further support here.
Ok, after detecting the discrepancy, I assumed something like that already, just wanted confirmation.


Many thanks,


Jörn


> 
> 
> HTH
> Anders
> 
> 
> On 2025-03-26, 11:51, "Tech-talk on behalf of Jörn Dreyer via Tech-talk" <tech-talk-bounces at aps.anl.gov <mailto:tech-talk-bounces at aps.anl.gov> <mailto:tech-talk-bounces at aps.anl.gov <mailto:tech-talk-bounces at aps.anl.gov>> on behalf of tech-talk at aps.anl.gov <mailto:tech-talk at aps.anl.gov> <mailto:tech-talk at aps.anl.gov <mailto:tech-talk at aps.anl.gov>>> wrote:
> 
> 
> Hello folks,
> 
> 
> after experimenting with SUMO I also tried the E3 framework from ESS and found it very usefull.
> It installed without problems on my OpenSuSE Tumbleweed system and compiled the basic modules that I need for my installation without problems.
> I also managed to write wrappers for my modules within a few day's but stumbled across some problems for which I do not have a solution yet.
> 
> 
> One of my modules needs a small kernel module like the mrfioc2 module. When calling "make dkms_add" it failed because the variable SUDO that
> is used in the RULES_DKMS_L file is not defined. Defining this externally made the installation possible but the build with "make dkms_build" failed
> because that is not called with the $SUDO prefix. 
> As I do not have a CentOS installation available, I can not check how the default configuration is there. So I can not tell if this failure comes from
> my specific Linux distribution or it is off more general origin. 
> 
> 
> The second topic I would like to get some help with is the specifications files. As the documentation for this file is ESS internal only
> I could not find a solution how to extract the correct time information of the git commit that is part of the module version information.
> 
> 
> This directly leads to the third problem of how to integrate local modules into the automatic build process of "e3 build".
> The default is using gitlab.ess.lu.se only to search for the wrappers. But looking at the source code of e3 it looks as there could be a third parameter
> to the module version defining the repository from where to take the module.
> Did anybody use E3 outside of ESS with local managed modules and wrappers or knows how to configure it to use a specific repository?
> 
> 
> If I manage to get that to work, I would recomend the tool to be used for our upcoming accelerator control system which will be based on EPICs.
> 
> 
> Kind regards
> 
> 
> Jörn Dreyer
> ---
> Dr. Jörn Dreyer
> HIBEF DAQ + Controls
> Institut für Strahlenpysik
> 
> 
> Helmholtz-Zentrum Dresden - Rossendorf e.V. (HZDR)
> Bautzner Landstr. 400 | 01328 Dresden | Germany
> https://urldefense.us/v3/__http://www.hzdr.de__;!!G_uCfscf7eWS!cTB_fZHiEPC2GSK6iS3ljEBtugk-cl_MOBJwog5vyM3lJ6BGwGkn4y3w-yvLK8QIYkAdzyvZcKRksJnuT5BxPZU$ <https://urldefense.us/v3/__http://www.hzdr.de__;!!G_uCfscf7eWS!cTB_fZHiEPC2GSK6iS3ljEBtugk-cl_MOBJwog5vyM3lJ6BGwGkn4y3w-yvLK8QIYkAdzyvZcKRksJnuT5BxPZU$> <https://urldefense.us/v3/__http://www.hzdr.de__;!!G_uCfscf7eWS!cTB_fZHiEPC2GSK6iS3ljEBtugk-cl_MOBJwog5vyM3lJ6BGwGkn4y3w-yvLK8QIYkAdzyvZcKRksJnuT5BxPZU$ <https://urldefense.us/v3/__http://www.hzdr.de__;!!G_uCfscf7eWS!cTB_fZHiEPC2GSK6iS3ljEBtugk-cl_MOBJwog5vyM3lJ6BGwGkn4y3w-yvLK8QIYkAdzyvZcKRksJnuT5BxPZU$>> 
> Vorstand: Prof. Dr. Sebastian M. Schmidt, Dr. Diana Stiller
> Vereinsregister: VR 1693 beim Amtsgericht Dresden 
> 
> 
> 
> 
> 
> 
> 
> 




-- 
Dr. Jörn Dreyer
HIBEF DAQ + Controls
Institut für Strahlenpysik


Helmholtz-Zentrum Dresden - Rossendorf e.V. (HZDR)
Bautzner Landstr. 400 | 01328 Dresden | Germany
https://urldefense.us/v3/__http://www.hzdr.de__;!!G_uCfscf7eWS!drmEigggKCxJYPaxEZKQB9G3bumXdDI1cRaj2JlK_Fybm_sjeQs8jruQas48y_sKAYtUEIhthzvTp79jC_85D6R3xxhXEwU$  <https://urldefense.us/v3/__http://www.hzdr.de__;!!G_uCfscf7eWS!drmEigggKCxJYPaxEZKQB9G3bumXdDI1cRaj2JlK_Fybm_sjeQs8jruQas48y_sKAYtUEIhthzvTp79jC_85D6R3xxhXEwU$ > 
Vorstand: Prof. Dr. Sebastian M. Schmidt, Dr. Diana Stiller
Vereinsregister: VR 1693 beim Amtsgericht Dresden