Experimental Physics and
Industrial Control System

Stainer Tom via Tech-talk <tech-talk at aps.anl.gov> · Tue, 7 Sep 2021 14:14:28 +0000

Hi Freddie,

Thanks for the links. I was aware of LeWIS but haven't tried it out yet. 

With regard to the **EPICS-IOC_Test_Framework** repo, it seems to require genie_python, which (forgive my ignorance) I am not aware of what this is. 
    ```
    from genie_python.genie_cachannel_wrapper import CaChannelWrapper, UnableToConnectToPVException
    ```

How does one go about getting genie_python? And does this also support PVA in addition to CA?

Kind regards,
Tom

-----Original Message-----
From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of Freddie Akeroyd - STFC UKRI via Tech-talk
Sent: Tuesday, September 7, 2021 3:09 PM
To: tech-talk at aps.anl.gov
Subject: RE: Unit testing IOCs and databases

Hi Tom,

For testing IOCs we run a set of tests from Python that start the IOC and then write and read various PVs over channel access to confirm behaviour. In most cases we need to have the hardware emulated, for this we use the LeWIS framework   https://lewis.readthedocs.io/en/latest/  and we have also developed a set of Python routines to make writing and running the tests easier https://github.com/ISISComputingGroup/EPICS-IOC_Test_Framework 

Regards,

Freddie

> -----Original Message-----
> From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of Stainer Tom
> via Tech-talk
> Sent: 07 September 2021 12:36
> To: Torsten Bögershausen <Torsten.Bogershausen at ess.eu>; tech-
> talk at aps.anl.gov
> Subject: RE: Unit testing IOCs and databases
> 
> Hi Torsten,
> 
> Thanks for the response, very helpful Torsten.
> 
> I also replied inline!
> 
> >> Essentially this boils down to 4 main questions:
> 
> >> - Given a db file what is the standard way to test logic and records inside it
> in an automated fashion?
> > Not what I am aware of - the "standard way". Each record has it's own
> specific behaviour....
> Fair enough. I was curious to know how people using EPICS write tests for logic
> they put in a record. For example, if you have a db file which implements a
> CALC, even if very simple (e.g. A+2B-C), but how do you ensure that this does
> what you think it does and doesn't change later? Coupled with the fact you
> may have 100+ PVs in a single db file that all depend on each other. I guess this
> is again at the integration test level....
> 
> >> - Using the V4 smart database, what is the standard way to mock the
> protocol i.e. PVA, to do more unit testing rather than integration testing?
> >> - How do people mock hardware? Is this done by using the asynPortDriver
> interface?
> > I am doing tests using both simulated and real hardware: Motion controllers
> and motors.
> > Some tests need the simulator to simulate failures in HW...
> > In any sense, there is a common motor driver, a model 3 driver.
> > The simulator does the same things as the real HW, so we use the same
> driver in both cases.
> Good to know.
> 
> >> - Has anyone tried using other libraries, such as google test, Catch2,
> doctest,… to do unit testing in EPICS? If so, how can I set this up with the EPICS
> build system?
> > I am using pytest.
> Great. I am using pytest for IOCs written in python, but a lot of the code is in
> C++ also.
> 
> > And the whole test-suite is rather a integration test, not a Unit test.
> Yes indeed, we have many integration tests but we sometimes run into issues
> with broadcasts on the CI runner. Additionally, they can be quite slow, I was
> hoping to find ways to look at unit testing through use of mocks. In some cases
> however, we end up with too many mocked interfaces that at some point you
> question if this actually tests anything real in the end. Integration tests are
> more useful in these cases.
> 
> > If you, or anybody else, is interested, I can explain & show stuff in a video
> meeting.
> 
> Yes, I would find that very helpful!
> 
> Kind regards,
> Tom
> 
> -----Original Message-----
> From: Torsten Bögershausen <Torsten.Bogershausen at ess.eu>
> Sent: Tuesday, September 7, 2021 1:07 PM
> To: Stainer Tom <Tom.Stainer at sckcen.be>; tech-talk at aps.anl.gov
> Subject: Re: Unit testing IOCs and databases
> 
> Hej Tom,
> some answers inline
> 
> From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Stainer Tom via
> Tech-talk <tech-talk at aps.anl.gov>
> Reply-To: Stainer Tom <Tom.Stainer at sckcen.be>
> Date: Tuesday, 7 September 2021 at 11:13
> To: "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
> Subject: Unit testing IOCs and databases
> 
> Hello EPICS community,
> 
> I have a question, that is quite broad in its nature (I know these don’t go down
> well here but thought I’d try anyway), regarding testing in EPICS.
> 
> Specifically I am interested in how people using EPICS generally test their
> applications and the logic in IOCs, at a unit testing level.
> I am not talking about testing epics-base, of course I trust that your tests
> provide good coverage ☺. What I am interested in is how can I test logic that is
> in an IOC (db file or via smart database).
> 
> Essentially this boils down to 4 main questions:
> 
> - Given a db file what is the standard way to test logic and records inside it in
> an automated fashion?
> Not what I am aware of - the "standard way". Each record has it's own specific
> behaviour....
> 
> - Using the V4 smart database, what is the standard way to mock the protocol
> i.e. PVA, to do more unit testing rather than integration testing?
> - How do people mock hardware? Is this done by using the asynPortDriver
> interface?
> I am doing tests using both simulated and real hardware: Motion controllers
> and motors.
> Some tests need the simulator to simulate failures in HW...
> In any sense, there is a common motor driver, a model 3 driver.
> The simulator does the same things as the real HW, so we use the same driver
> in both cases.
> 
> - Has anyone tried using other libraries, such as google test, Catch2, doctest,…
> to do unit testing in EPICS? If so, how can I set this up with the EPICS build
> system?
> I am using pytest.
> And the whole test-suite is rather a integration test, not a Unit test.
> If you, or anybody else, is interested, I can explain & show stuff in a video
> meeting.
> 
> Kind regards,
> Tom
> 
> 

This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System