Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017  2018  2019  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017  2018  2019 
<== Date ==> <== Thread ==>

Subject: 32-bit TIFFs and PIL
From: Mark Rivers <rivers@cars.uchicago.edu>
To: Timothy Madden <tmadden@aps.anl.gov>
Cc: "dvine@aps.anl.gov" <dvine@aps.anl.gov>, Ross Harder <rharder@aps.anl.gov>, "tech-talk@aps.anl.gov" <tech-talk@aps.anl.gov>
Date: Tue, 20 Mar 2012 13:03:00 +0000
Hi Tim (and tech-talkers),

I know you've done quite a bit with TIFF files, so I've got a question for you.

The areaDetector NDFileTIFF plugin can write 8-bit, 16-bit and 32-bit TIFF files.  All of them can be read fine by ImageJ and IDL.  However, the 32-bit TIFFs cannot be read by the Python Imaging Library (PIL).  But 32-bit TIFF files written by the Dectris Pilatus driver can be read by PIL.

Do you have any idea what needs to be done to make a 32-bit TIFF readable by PIL?

Thanks,
Mark

________________________________________
From: matt.newville@gmail.com [matt.newville@gmail.com] on behalf of Matt Newville [newville@cars.uchicago.edu]
Sent: Monday, March 19, 2012 10:34 PM
To: Mark Rivers
Cc: Ross Harder; dvine@aps.anl.gov
Subject: Re: tiffs

Hi Mark,

Oh, right, I was using tiffinfo on my Mac, after moving the files over
(so that the display was faster).   tiffinfo -v says 3.9.4  I
installed libtiff-tools on corvette, so tiffinfo (3.9.5) now works
there too.

---Matt

On Mon, Mar 19, 2012 at 10:19 PM, Mark Rivers <rivers@cars.uchicago.edu> wrote:
> Hi Matt,
>
> Thanks for that information.
>
> What version of tiffinfo are you running?  Your output format is slightly different from what I get on idea and millenia.  I'd like to run tiffinfo on a Pilatus file and on the areaDetector file and see how they differ.  Then maybe I can change areaDetector's writer slightly to produce files that PIL can read.
>
> Mark
>
> ________________________________________
> From: matt.newville@gmail.com [matt.newville@gmail.com] on behalf of Matt Newville [newville@cars.uchicago.edu]
> Sent: Monday, March 19, 2012 9:54 PM
> To: Mark Rivers
> Cc: Ross Harder; dvine@aps.anl.gov
> Subject: Re: tiffs
>
> Hi Mark, Ross, David,
>
> On Mon, Mar 19, 2012 at 5:46 PM, Mark Rivers <rivers@cars.uchicago.edu> wrote:
>> Ross,
>>
>> I just talked to Matt Newville.  He thought that maybe this was a problem with an older version of PIL.  However, I just tried it on our new Linux system (corvette) and I can read 8-bit and 16-bit TIFFs fine in PIL, but 32-bit does not work.
>>
>> I used the simDetector to create 8-bit, 16-bit and 32-bit integer TIFF files.
>>
>> I then read them in IDL.  It worked fine.
>>
>> IDL> t = read_tiff('8bit_int_1.tiff')
>> IDL> help, t
>> T               BYTE      = Array[1024, 1024]
>> IDL> t = read_tiff('16bit_int_1.tiff')
>> IDL> help, t
>> T               UINT      = Array[1024, 1024]
>> IDL> t = read_tiff('32bit_int_1.tiff')
>> IDL> help, t
>> T               ULONG     = Array[1024, 1024]
>>
>> I then tried to read them in PIL.  8-bit and 16-bit worked, but not 32-bit.
>>
>> corvette:~/scratch>python
>> Python 2.7.1 (r271:86832, Apr 12 2011, 16:15:16)
>> [GCC 4.6.0 20110331 (Red Hat 4.6.0-2)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> Imported history from: '/home/epics/.pyhist'
>>>>> from PIL import Image
>>>>> im = Image.open("8bit_int_1.tiff")
>>>>> im.show()
>>>>> im = Image.open("16bit_int_1.tiff")
>>>>> im.show()
>>>>> im = Image.open("32bit_int_1.tiff")
>> Traceback (most recent call last):
>>  File "<stdin>", line 1, in <module>
>>  File "/usr/lib64/python2.7/site-packages/PIL/Image.py", line 1980, in open
>>    raise IOError("cannot identify image file")
>> IOError: cannot identify image file
>>
>> Matt, is the version of PIL I am running on corvette with Python 2.7.1 a recent version?  Do you understand why I can't read 32-bit TIFFs?
>>
>> Mark
>>
>>
>> -----Original Message-----
>> From: Mark Rivers
>> Sent: Monday, March 19, 2012 2:59 PM
>> To: 'Ross Harder'; 'dvine@aps.anl.gov'
>> Subject: RE: tiffs
>>
>> Hi Ross,
>>
>> I just ran the Linux utility "tiffinfo" on your file:
>>
>> idea:~>tiffinfo /corvette/home/epics/scratch/im.tiff
>> TIFF Directory at offset 0x3fe00c (4186124)
>>  Image Width: 1023 Image Length: 1023
>>  Bits/Sample: 32
>>  Compression Scheme: None
>>  Photometric Interpretation: min-is-black
>>  Samples/Pixel: 1
>>  Rows/Strip: 1023
>>  Planar Configuration: single image plane
>>  Make: Unknown
>>  Model: Unknown
>>
>> It also says that the image directory is near the end of the file.  But it is clearly a valid TIFF file, so why doesn't PIL like it?
>>
>> Mark
>>
>>
>> -----Original Message-----
>> From: Mark Rivers
>> Sent: Monday, March 19, 2012 2:55 PM
>> To: 'Ross Harder'; dvine@aps.anl.gov
>> Subject: RE: tiffs
>>
>> Hi Ross,
>>
>> It is not the "first image" that is near the end of the file, it is the "first image directory".  It looks like the image directory is being placed after the image.
>>
>> If you look at the NDFileTIFF plugin, you will see that it is very simple.  It uses libtiff.  For a 32-bit mono image these are the only calls to libtiff in the plugin, and they are done in this order:
>>
>>    TIFFOpen(fileName, "w")
>>    TIFFSetField(this->output, TIFFTAG_BITSPERSAMPLE, bitsPerSample);
>>    TIFFSetField(this->output, TIFFTAG_SAMPLESPERPIXEL, samplesPerPixel);
>>    TIFFSetField(this->output, TIFFTAG_PHOTOMETRIC, photoMetric);
>>    TIFFSetField(this->output, TIFFTAG_PLANARCONFIG, planarConfig);
>>    TIFFSetField(this->output, TIFFTAG_IMAGEWIDTH, sizeX);
>>    TIFFSetField(this->output, TIFFTAG_IMAGELENGTH, sizeY);
>>    TIFFSetField(this->output, TIFFTAG_ROWSPERSTRIP, rowsPerStrip);
>>    TIFFSetField(this->output, TIFFTAG_MAKE, ManufacturerString);
>>    TIFFSetField(this->output, TIFFTAG_MODEL, ModelString);
>>
>>    stripSize = TIFFStripSize(this->output);
>>    TIFFGetField(this->output, TIFFTAG_IMAGELENGTH, &sizeY);
>>
>>    nwrite = TIFFWriteEncodedStrip(this->output, 0, pArray->pData, stripSize);
>>    TIFFClose(this->output);
>>
>> So if the directory is being put at the end of the file, it must be because that's how libtiff does it.
>>
>> Mark
>>
>>
>> -----Original Message-----
>> From: ross.harder@gmail.com [mailto:ross.harder@gmail.com] On Behalf Of Ross Harder
>> Sent: Monday, March 19, 2012 2:40 PM
>> To: Mark Rivers; dvine@aps.anl.gov
>> Subject: tiffs
>>
>> Hi Mark,
>> I've placed a tiff from a recent scan on the xor ftp server.  I've
>> also included a short python program that interrogates the file.
>> I don't understand why the offset to the first image is almost at the
>> end of the file?
>>
>> http://ftp.xor.aps.anl.gov/sector33_34/34idc/tifread/
>>
>>
>> In [121]: run tiffinfo.py
>> Little endian is II: II
>> Sanity check for endianess should be 42 if a valid tif: (42,)
>> Offset of first image file directory in the tif: (4186124,)
>>
>> Any thoughts?
>>
>> Ross
>>
>
> Sorry for the trouble. I do remember that 32-bit images from a Pilatus
> could not be read with older versions of PIL, but can be read now
> (taking a random one from Nov 2011):
>
> ~> tiffinfo Pilatus_Nov2011.tif
>
> Pilatus_Nov2011.tif: Warning, incorrect count for field "DateTime"
> (32, expecting 20); tag trimmed.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: invalid TIFF
> directory; tags are not sorted in ascending order.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36864 (0x9000) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36865 (0x9001) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36866 (0x9002) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36867 (0x9003) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36868 (0x9004) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36870 (0x9006) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36876 (0x900c) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36877 (0x900d) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36878 (0x900e) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36879 (0x900f) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 36880 (0x9010) encountered.
> TIFFReadDirectory: Warning, Pilatus_Nov2011.tif: unknown field with
> tag 37120 (0x9100) encountered.
> Pilatus_Nov2011.tif: Warning, incorrect count for field "Tag 36864"
> (0, expecting 1); tag ignored.
> Pilatus_Nov2011.tif: Warning, incorrect count for field "Tag 36870"
> (0, expecting 1); tag ignored.
> TIFF Directory at offset 0x82 (130)
>  Subfile Type: (0 = 0x0)
>  Image Width: 487 Image Length: 195
>  Resolution: 69.5714, 69.5714
>  Bits/Sample: 32
>  Sample Format: signed integer
>  Compression Scheme: None
>  Photometric Interpretation: min-is-black
>  Rows/Strip: 195
>  Planar Configuration: single image plane
>  ImageDescription: # Pixel_size 172e-6 m x 172e-6 m
> # Silicon sensor, thickness 0.000320 m
> # Exposure_time 16.700000 s
> # Exposure_period 16.700000 s
> # Tau = 203.0e-09 s
> ....
>
> For this tiff file
>
>>>> import Image
>>>> Image.open('Pilatus_Nov2011.tif').show()
>
> displays fine.  That sure looks like a 32-bit integer TIFF to me,
> though tiffinfo wasn't all that happy with some of the tags.
>
> But I see exactly the same behavior for the tiff files that Mark
> created with simDetector, that Image.open() complains about not being
> able to identify the image file.  I'm not sure why that is....  The
> PIL docs are pretty opaque, but I believe this works:
>
>>>> data = open('32bit_int_1.tiff').read()   # raw file read
>>>> img  = Image.frombuffer('I', (1024, 1024), data, 'raw', 'I')
>>>> img.show()
>
> This shows one small bright spot in upper left, around pixel 200,100,
> with a max intensity of 186.  It's definitely lame that Image.open()
> doesn't auto-recognize this file!  I don't understand why it
> understood the Pilatus file, but not this one (all the
> Image.frombuffer() call is doing is datatype as an integer, and
> explicitly setting the size).
>
> BTW, to convert this to a numpy array, use
>>>> np_data = numpy.array(img.getdata(), dtype='int32').reshape(1024, 1024).transpose()
>
> Finally, if you're interested, I have a start of an Image Display GUI
> (wxPython) for Epics Area Detectors.  This uses exactly this
> 'Image.frombuffer()' method to convert data directly from an
> AreaDetector PV data.  Not much documentation yet, but code is at at
>  https://github.com/pyepics/epicsapps/tree/master/AreaDetector_Display
> and a screenshot is at
>  http://pyepics.github.com/epicsapps/ad_display.html
>
> I've also made a Windows EXE made from this, but haven't posted it online yet.
> Hope that helps,
>
> --Matt Newville <newville at cars.uchicago.edu> 630-252-0431
>



--
--Matt Newville <newville at cars.uchicago.edu> 630-252-0431


Replies:
Re: 32-bit TIFFs and PIL Matt Newville
Re: 32-bit TIFFs and PIL J. Lewis Muir

Navigate by Date:
Prev: BOY: Macro for workspace directory? Ralph Lange
Next: [ASYN-vxi11] can't connect to Tektronix scope on Fedora 16 Pavel Masloff
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017  2018  2019 
Navigate by Thread:
Prev: RE: Macro for workspace directory? Chen, Xihui
Next: Re: 32-bit TIFFs and PIL Matt Newville
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017  2018  2019 
ANJ, 18 Nov 2013 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·