RE: xspress3

[<nick.rees@diamond.ac.uk>]

Hi Matt (and Matt),

 

First I want to start by apologising for missing/not being aware of the pull request from Matt Newville. I have just looked at it and it seems we are going in basically the same direction (eliminating a lot of the crap from the data files), except Matt has made quite a few changes to an example IOC which is auto-generated so if we need to keep this we will have to move it somewhere else or else it will be trodden on by the next time we auto-generate the code.

 

The auto-generation has been essential up until now because otherwise there is been no practical way to easily write IOC’s for Xspress3’s with differing numbers of channels. I hope to change this.

 

So, in answer to Matt Moore, the driver still needs work, but we want to fix it. Most of the changes are on GitHub, either in Matt Newvilles area (https://github.com/newville/xspress3) or Adam Bark’s (https://github.com/AdamBark/xspress3). We need to work on a merge of all these.

 

Cheers,

 

Nick Rees

Principal Software Engineer           Phone: +44 (0)1235-778430

Diamond Light Source                  Fax:   +44 (0)1235-446713

 

From: tech-talk-bounces@aps.anl.gov [mailto:tech-talk-bounces@aps.anl.gov] On Behalf Of Matt Newville
Sent: 18 July 2015 00:06
To: Matthew D. Moore
Cc: tech-talk@aps.anl.gov
Subject: Re: xspress3

 

Hi Matt,

 

On Fri, Jul 17, 2015 at 4:42 PM, Matthew D. Moore <mdmoore@aps.anl.gov> wrote:

I am looking for a quick survey of people with a xspress3 from Quantum Detectors.

Do you you have any feedback on the driver?

Have you made any changes to the EPICS driver that aren't GiHub? If so, are you willing to put it up on GitHub?

 

I have a fork of Nick Rees' github repo that I use for our Xspress3.  I changed many of the (weirdly, for a machine from a vendor) DLS-specific configuration, so that it would build well on the system provided by Quantum.  

I also added trivial loading of "simple_mca.db" from the Mca record so that "normal" (IMHO) MCA-style ROI variable names could be used, to act more like any other multi-element MCA.  There isn't support code to have these ROIs actually work to update counts and so on.   I use client code to copy these ROI placeholders to the live Xspress3 ROIs.  Using a "normal" Mca record would certainly be something to consider.

I trimmed many of the useless attributes from being saved to HDF5 files (for example, ROI definitions were being saved at each pixel -- not a lot of extra data, but certainly unnecessary). 

I worked on saving data as Float32 instead of Float64.  Float64 is simply not necessary for this data.   In fact, like the saving of the ROI limits at each pixel, it sort of demonstrates a fundamental lack of understanding of what the detector does.

I made a Pull Request against Nick's repository in January.  It has not been merged, and Nick's repo has not been updated since. I had taken this to mean that no one else was working on this code.

 

Meanwhile, Nick wrote:

> We are doing some work at Diamond - starting by simplifying the driver so that it has far
 > fewer parameters in the NDArray, and making it simpler to scale the system by number of
> channels. This should make it more performant and able to scale to higher speeds and/or > more channels.

I don't see these changes.  Is this code available?

> I would be interested if anyone actually uses the ROI's that are stored in the data file.

You mean the HDF5 data files?  Then, no.  The ROIs are not necessary here.  If one has the full array and knows the ROI definitions, these are as easy to extract as they are to read from a separate dataset. 

One thing we've found to be very important for performance is to use compression on the HDF5 files.   We typically acquire data for 10 to 100 ms per spectra.  Since the max count rate is ~3MHz and there are 4096 bins, the typical count is around 10 to 100 in each bin.   Even with the data stored as Int32 or Float32 (and especially as Float64), the data are highly compressible, and disk i/o dwarfs compression / decompression time.   We typically use Zlib compression level 1.

> These add a lot of attributes and have a lot of overhead - both manipulating the
> NDArray every frame, and also when writing the file at the end - because it writes
> all the parameters after the scan finishes.

 

I think saving the SCA and DTC data is a good idea, but the ROI data is not necessary.   I trimmed my copy of XSP3.xml from ~250 to ~75 lines (for a 4 element array).

The main complaint with the Xspress3 is the lack of dead time information.   This seems to not be an Epics issue, but an issue from the vendor.  

 

--Matt Newville <newville at cars.uchicago.edu> 630-252-0431

 

-- 

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom