Experimental Physics and
Industrial Control System

Freddie Akeroyd - STFC UKRI via Tech-talk <tech-talk at aps.anl.gov> · Tue, 7 Sep 2021 14:24:47 +0000

Hi Tom,

It doesn't really need genie python, that is our local python bundle, it only needs the genie_cachannel_wrapper that just puts an interface onto the CaChannel package. A colleague at CLF told us "I got it to run a single, simple test under linux with only minor changes. I used caproto to re-implement just enough of CaChannelWrapper and UnableToConnectToPVException to get my test to work." We plan to remove the genie python dependency, in the meanwhile I'll see if I can get his changes for you. We have not tested with PVA ourselves this way, but as all channel access is via the wrapper class if you added a PVA binding the framework should likely all work

Regards,

Freddie

> -----Original Message-----
> From: Stainer Tom <Tom.Stainer at sckcen.be>
> Sent: 07 September 2021 15:14
> To: Akeroyd, Freddie (STFC,RAL,ISIS) <freddie.akeroyd at stfc.ac.uk>; tech-
> talk at aps.anl.gov
> Subject: RE: Unit testing IOCs and databases
> 
> Hi Freddie,
> 
> Thanks for the links. I was aware of LeWIS but haven't tried it out yet.
> 
> With regard to the **EPICS-IOC_Test_Framework** repo, it seems to require
> genie_python, which (forgive my ignorance) I am not aware of what this is.
>     ```
>     from genie_python.genie_cachannel_wrapper import CaChannelWrapper,
> UnableToConnectToPVException
>     ```
> 
> How does one go about getting genie_python? And does this also support PVA
> in addition to CA?
> 
> Kind regards,
> Tom
> 
> -----Original Message-----
> From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of Freddie
> Akeroyd - STFC UKRI via Tech-talk
> Sent: Tuesday, September 7, 2021 3:09 PM
> To: tech-talk at aps.anl.gov
> Subject: RE: Unit testing IOCs and databases
> 
> Hi Tom,
> 
> For testing IOCs we run a set of tests from Python that start the IOC and then
> write and read various PVs over channel access to confirm behaviour. In most
> cases we need to have the hardware emulated, for this we use the LeWIS
> framework   https://lewis.readthedocs.io/en/latest/  and we have also
> developed a set of Python routines to make writing and running the tests
> easier https://github.com/ISISComputingGroup/EPICS-IOC_Test_Framework
> 
> Regards,
> 
> Freddie
> 
> > -----Original Message-----
> > From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of Stainer
> > Tom via Tech-talk
> > Sent: 07 September 2021 12:36
> > To: Torsten Bögershausen <Torsten.Bogershausen at ess.eu>; tech-
> > talk at aps.anl.gov
> > Subject: RE: Unit testing IOCs and databases
> >
> > Hi Torsten,
> >
> > Thanks for the response, very helpful Torsten.
> >
> > I also replied inline!
> >
> > >> Essentially this boils down to 4 main questions:
> >
> > >> - Given a db file what is the standard way to test logic and
> > >> records inside it
> > in an automated fashion?
> > > Not what I am aware of - the "standard way". Each record has it's
> > > own
> > specific behaviour....
> > Fair enough. I was curious to know how people using EPICS write tests
> > for logic they put in a record. For example, if you have a db file
> > which implements a CALC, even if very simple (e.g. A+2B-C), but how do
> > you ensure that this does what you think it does and doesn't change
> > later? Coupled with the fact you may have 100+ PVs in a single db file
> > that all depend on each other. I guess this is again at the integration test
> level....
> >
> > >> - Using the V4 smart database, what is the standard way to mock the
> > protocol i.e. PVA, to do more unit testing rather than integration testing?
> > >> - How do people mock hardware? Is this done by using the
> > >> asynPortDriver
> > interface?
> > > I am doing tests using both simulated and real hardware: Motion
> > > controllers
> > and motors.
> > > Some tests need the simulator to simulate failures in HW...
> > > In any sense, there is a common motor driver, a model 3 driver.
> > > The simulator does the same things as the real HW, so we use the
> > > same
> > driver in both cases.
> > Good to know.
> >
> > >> - Has anyone tried using other libraries, such as google test,
> > >> Catch2,
> > doctest,… to do unit testing in EPICS? If so, how can I set this up
> > with the EPICS build system?
> > > I am using pytest.
> > Great. I am using pytest for IOCs written in python, but a lot of the
> > code is in
> > C++ also.
> >
> > > And the whole test-suite is rather a integration test, not a Unit test.
> > Yes indeed, we have many integration tests but we sometimes run into
> > issues with broadcasts on the CI runner. Additionally, they can be
> > quite slow, I was hoping to find ways to look at unit testing through
> > use of mocks. In some cases however, we end up with too many mocked
> > interfaces that at some point you question if this actually tests
> > anything real in the end. Integration tests are more useful in these cases.
> >
> > > If you, or anybody else, is interested, I can explain & show stuff
> > > in a video
> > meeting.
> >
> > Yes, I would find that very helpful!
> >
> > Kind regards,
> > Tom
> >
> > -----Original Message-----
> > From: Torsten Bögershausen <Torsten.Bogershausen at ess.eu>
> > Sent: Tuesday, September 7, 2021 1:07 PM
> > To: Stainer Tom <Tom.Stainer at sckcen.be>; tech-talk at aps.anl.gov
> > Subject: Re: Unit testing IOCs and databases
> >
> > Hej Tom,
> > some answers inline
> >
> > From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Stainer
> > Tom via Tech-talk <tech-talk at aps.anl.gov>
> > Reply-To: Stainer Tom <Tom.Stainer at sckcen.be>
> > Date: Tuesday, 7 September 2021 at 11:13
> > To: "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
> > Subject: Unit testing IOCs and databases
> >
> > Hello EPICS community,
> >
> > I have a question, that is quite broad in its nature (I know these
> > don’t go down well here but thought I’d try anyway), regarding testing in
> EPICS.
> >
> > Specifically I am interested in how people using EPICS generally test
> > their applications and the logic in IOCs, at a unit testing level.
> > I am not talking about testing epics-base, of course I trust that your
> > tests provide good coverage ☺. What I am interested in is how can I
> > test logic that is in an IOC (db file or via smart database).
> >
> > Essentially this boils down to 4 main questions:
> >
> > - Given a db file what is the standard way to test logic and records
> > inside it in an automated fashion?
> > Not what I am aware of - the "standard way". Each record has it's own
> > specific behaviour....
> >
> > - Using the V4 smart database, what is the standard way to mock the
> > protocol i.e. PVA, to do more unit testing rather than integration testing?
> > - How do people mock hardware? Is this done by using the
> > asynPortDriver interface?
> > I am doing tests using both simulated and real hardware: Motion
> > controllers and motors.
> > Some tests need the simulator to simulate failures in HW...
> > In any sense, there is a common motor driver, a model 3 driver.
> > The simulator does the same things as the real HW, so we use the same
> > driver in both cases.
> >
> > - Has anyone tried using other libraries, such as google test, Catch2,
> > doctest,… to do unit testing in EPICS? If so, how can I set this up
> > with the EPICS build system?
> > I am using pytest.
> > And the whole test-suite is rather a integration test, not a Unit test.
> > If you, or anybody else, is interested, I can explain & show stuff in
> > a video meeting.
> >
> > Kind regards,
> > Tom
> >
> >
> 
> 
> This email and any attachments are intended solely for the use of the named
> recipients. If you are not the intended recipient you must not use, disclose,
> copy or distribute this email or any of its attachments and should notify the
> sender immediately and delete this email from your system. UK Research and
> Innovation (UKRI) has taken every reasonable precaution to minimise risk of
> this email or any attachments containing viruses or malware but the recipient
> should carry out its own virus and malware checks before opening the
> attachments. UKRI does not accept any liability for any losses or damages
> which the recipient may sustain due to presence of any viruses.

This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System