HDF BROWSER (hdfb)

by Ben-chin Cha (02/19/2004)

I. INTRODUCTION

Hdfb provides the IDL user at APS with a mouse driven widget application to browse any HDF / NEXUS file. It was originally developed with IDL 5.0 and now had been further developped and extended to support IDL 6.0 new ITOOLS features. This new version let the user easily and more efficiently access the SDS data set directly. It has been fully tested under IDL 5.5 and IDL 6.0 with the CCD Image files obtained from the Metrology Lab, as well as with the NEXUS files generated from the USER2IDE's 2D/3D MDA scan files.

Currently it supports following basic HDF objects: file ID and descriptions annotations, 24 bits/8 bits raster image binary data, various type of SDS data, Vdata, and Vgroups. User can flexiblely browse through the SDS, Vdata, and Vgroups data sets through using scroll attributes list, push buttons and slider bar provided by the corresponding popup interface dialogs.

By default option it is conveninently geared to extract the SDS dataset from any HDF/NEXUS file. A user can easily extract any 1D/2D/3D SDS data and display the data in ready packaged graphic sub-program. It also provided various output features like PS plots, TIFF/PNG/PICT/ASCII files.

The hdfb also provides a simple command, SDS, which allows the user easily to access any SDS data array as an IDL variable after the normal completion of loading the SDS data sets. This provides the feature a user can easily plug-in his/her own preferred IDL visualization program into the hdfb. This SDS command and plug-in feature are only available in IDL developper verion but not for run-time or IDL vitrual machine version.

II. DOWNLOAD AND SETUP

Normally a user only need to download hdfb.sav to view any HDF/NEXUS file. The hdf.zip provide a set of HDF/NEXUS sample files.

	hdfb.sav Download hdfb for IDLVM 6.0

	hdf.zip Download samples of HDF/NEXUS files for hdfb 

If a user want to get the complete collection of the IDL sources the following link is availble.

	Idl.zip Download the complete set of EPICS/IDL source programs 

	Setup Requirement Contains the installation and IDL setup instruction for UNIX and WIN system

The hdfb frontend system consists of 3 files: os.init, hdfb, and hdfb.pro. All the programs implicitly used by the hdfb is also contained in the installation directory and they will be dynamically loaded into IDL at runtime.

	
os.init          - A system dependent file shared by IDL programs with 
                   specific settings for UNIX/WIN system.
hdfb.pro         - A HDF/NEXUS browser written in IDL widget application 
hdfb             - A UNIX script file, it automatically sets up the 
                   IDL and runs the hdfb.pro

COMMAND SYNTAX

At APS, all the IDL programs are installed in /usr/local/epics/extensions/bin/solaris directory for EPICS 3.13.X. Starting from EPICS 3.14.X the IDL programs are installed in /usr/local/epics/extensions/idllib directory and the UNIX script and IDL saved files are installed in /usr/local/epics/extensions/bin/solaris-sparc directory.

To access the newest version of hdfb, a user has to include the /usr/local/epics/extensions/bin/solaris-sparc directory in his/her command search path.

	UNIX SYSTEM

	1) To access the IDLVM 6.0 version with hdfb.sav on the 
	Unix system just enter the following at the unix prompt:
	
		idlvm hdfb

	2) To access the hdfb developer version on the Unix system 
	just enter the following at the unix prompt:

		hdfb

	WIN SYSTEM

	To access hdfb on the WIN system first invoke IDLDE, then just 
	enter the following at the IDL prompt:

		@os.init
		hdfb

III. USER INTERFACE

File Types Used

Various types of files used by hdfb. They include input file, output file, config file. Additional text or image output files used or saved by subsequent program, e.g. PLOT1D, PLOT2D, EZ_FIT, etc., are also automatically used.

Input Files

The input file supported by hdfb can be either HDF file or NEXUS file. It is assumed that HDF file is ended with '.hdf', the NEXUS file can be ended with '.Nx', or '.nx', or '.nexus'. User can modify the file selection dialog to suit user's own name convention as long as it contains HDF or NEXUS data.

Output Files

Few different types of output files can be generated by hdfb. It provides the user of option override the default file names used. The user is responsible for unique existence of the file name used. It is recommanded that the user has to keep the consistant file types used for easy file management.

The extracted SDS data can be saved in XDR format which is stored at the subdirectory XDR at the current working directory. The xdr file should be ended with '.xdr' suffix. For more knowledge of easy IDL xdr routines please refer the xdr_open.pro.

The extracted SDS data can be saved as TIFF file. The graphic tiff file is stored at the subdirectory TIFF at the current working directory. The tiff file should be ended with '.tiff' suffix.

The temporary text file 'hdf_data.txt' is used by the hdfb for holding the intermediate SDS data or HDF data description. For consistance the text file should be ended with '.txt' suffix.

Config File

Every time a new HDF or NEXUS file is loaded into hdfb, the restart configuration file 'hdfb.config' is automatically created or updated in the current working directory. The configuration file records the last file used and will be used to restart the hdfb next time.

Private Color Table File

The file name 'pvtcolors.dat' is used for save / restore the current IDL color table used by hdfb.

Main Window

The top section consists of various widgets for flexible input HDF / NEXUS file selection area. The middle section consists of data type selection and scrolled text window of attributes display area. The bottom section consists of SDS data graphic display area.

The scroll text area displays the brief HDF file descriptions for the selected HDF object type of the input file. The drawing area shows brief graph of the 1D/2D data extracted from the SDS object. If the SDS "Plot Window" option is set for any HDF / NEXUS file opened, depending on the data object selected from the SDS list, the appropriate EPICS extensions IDL sub-programs PLOT1D, PLOT2D, VIEW3D_2D will be popped up.

File Menu Open... - file selection dialog Quit - exit hdfb

The file selection dialog allows the user to navigate through the file system to pick the desired HDF / NEXUS file. The valid filter strings for file selection include '*.hdf', '*.nexus', '*.Nx','*.nx'. It comes with the defalut file selection type as '*.hdf'. If different file convention used, a user can modifiy the 'Filter' field to suit his/her own condition. At the completion of the file selection dialog, the SDS query dialog will pop up which allows the user easily to access any desired data set from the SDS list. The SDS scroll list contains all the SDS name and dimension attributes found from the file.

Setup Menu
	Save PVTCT - save the IDL color table to 'pvtcolors.dat' 
	Load PVTCT - load the IDL color table from 'pvtcolors.dat'
	Color...   - access the IDL XLOADCT program

The IDL XLOADCT program allows the user to access any of the IDL support default 40 color tables. The Save/Load PVTCT allows the user to save or load a preferred private color table. The default color table used is the 'GRN-RED-BLU-WHT'.

HDF Filename

This text field display the current filename selected by the user. New HDF / NEXUS file can be entered directly through keyboard typing, the new updated QUERY - SDS dialog will pop up as new file name entered.

File Selection Btns
	|>    Select the first file from the file list
	->    Select the next file from the file list
	<-    Select the prev file from the file list
	<|    Select the last file from the file list

This set of file selection buttons allows the user quickly to access the First/Prev/Next/Last SDS file in current data directory.

File Annotation / Raster Image / HDF Info: 
	Droplist: File ID & Desc           
		  8 Bit Raster Image
		  24 Bit Raster Image
		  Global SDS Attributes
		  Global VDATA Attributes
		  Global VGROUP Attributes

The scrolled info text area is updated according to the droplist option selected by the user. For SDS/VDATA/VGROUP selection the corresponding selection dialog is popped up. The default setting for droplist is the 'Global SDS Attributes'. A user can access and view the desired SDS data set through QUERY - SDS dialog.

If any VGROUP defined in the HDF/NEXUS file, the Global VGROUP Attributes shows the Vgroup structure and allows the user to examine how the SDS data sets are grouped. In case of user doubted any problem with SDS data passed to the subprogram, this feature provided another way of examine the grouped SDS data sets.

Clear Button - clear the scrolled info area

Scrolled Info Area - display brief file attributes

Check Options:
	Txt Window  - turns SDS text data window  ON/OFF 
	Plot Window - turns SDS sub-program plot window  ON/OFF

Which plot window pops up depending on the SDS data obtained, PLOT1D is used for plotting 1D data, PLOT2D is used for plotting 2D data, VIEW3D_2D is used for plotting sliced 3D data.

Byte Array as:
	String    - byte array show as string (default setting) 
	Byte      - byte array show as byte value

If the selected SDS data is a byte array, it can be converted to string or kept as a byte array. The default option is converting byte data to string. This takes care the case where string was saved as byte array.

Drawing Area - display coarse SDS raw data 
	For 1D SDS data, the line plot is displayed in the drawing area.
	For 2D SDS data, the drawing area is devided into two regions: 
		the left region for TV image, the right region for surface plot.
	For 3D SDS data, only the first 12 slices of the 3D data are displayed 
		in the drawing area.

For obtaining the real meaningful plot from the sub-program window, a user has to turn the check option for 'Plot Window' to ON.

QUERY - SDS Dialog Window

This dialog allows the user freely to check the SDS data sets found in the file. It allows the user to pass the selected SDS data to sub-program and its simple analysis functions available within sub-program PLOT2D, PLOT1D.

DumpSDSHeader Button Display the text window for dumping the summary of SDS data header found from the input file. First,Next,Prev,Last Btns The buttons allow to access the First/Next/Prev/Last data set from the SDS list. SDS # Field The sequence field to display or specify the desired SDS data set. SDS # Slider The slider-bar to display or specify the desired SDS data set. Multiple SDSs... Construct array of multiple SDS 1D/2D arrays, exists only if axis data found in SDS list. SDS Attributes List The scrolled menu list to display the SDS Attribute Names and dimensions. X Axis Droplist The droplist to select X axis, exists only if axis data found in SDS list. Y Axis Droplist The droplist to select Y axis, exists only if more than one axis data found in SDS list. RAxis Plot... Plot axes with real value instead of index value, exists only if axis data found in SDS list. Close Button The button to close the 'QUERY - SDS' dialog.

The hdfb assumed that SDS data set with name attribute of 'axis1_01' is an axis. The first name attribute with 'axis1_01' is used for X axis, the second name attribute with 'axis1_01' is used for Y axis. A user has to make sure the correct X, Y axes (i.e. dimensions) are picked for 2D image array before pressing the sub-program buttons like 'RAxis...', 'IMAGE2D...'.

Extract SDS Data as Variable

Extract SDS data as an IDL variable is only available for the developer version not for the IDLVM version.

At the close of 'QUERY - SDS' dialog window, the IDL prompt is returned without blocking, a user can enter any valid IDL interactive command at this time. If a user wants to extract the SDS data array directly from the IDL command line, a user can enter the command 'SDS' at the IDL prompt to access the SDS data array.

The following example shows how to extract SDS data array, and pass the extracted 2D data array 'DATA' to the IDL program 'PLOT2D'.

	IDL> SDS, DATA
	IDL> PLOT2D, DATA

If more SDS data sets are desired from the input HDF file, a user can use different data variable name followed with the corresponding specified keyword seqno=i at the 'SDS' command line, where 'i' is the zero based sequence number, e.g.

	IDL> SDS,FLD1,seqno=1  
	IDL> SDS,FLD2,seqno=2  
        ...

This SDS command allows the user easily to access any SDS data from the input file.

HDF SDS Header Dump Window

The following text window is generated by the 'DumpSDSHeader' button from the QUERY - SDS window. The SDS Header is prepared and saved in the intermediate text file 'hdfb_data.txt'. The CW_TERM program is used to display the content of the 'hdfb_data.txt' .

 The SDS header file constists of : 
	SD #    - order of SDS data seqno 
	TYPE    - SDS data type 
	NATTR   - number of attributes defined
	NDIMS   - SDS data dimensions 
	DIMS    - dimensions list, eg (i,j,k)
	NAME    - name attribute string
	FORMAT  - format attribute
	LABEL   - label attribute 
	COORDSYS - coordinate system attribute
	UNIT	- unit attribute	
	DATA    - first 6 numbers from the data array

The attribute is left blank if it is not defined.

Different droplist query mode generates different intermediate text file. The contents of temporary 'hdfb_data.txt' can be easily saved or printed by the user through a set of push buttons.

Print - send contents to a printer
Clear - clear the window contents
Close - close this dialog
Save As... - pop up rename dialog

PLOT2D Image Window

If a seqno of a 2D SDS data picked from the QUERY - SDS dialog, and the "Plot Option" is set on the main window, then a PLOT2D sub-program will pop up to display the data.

The following PLOT2D image window displays the plot window with 2D SDS data obtained from /cdrom/040202_1210/smithe~8.3_a/smithe~42.hdf. The PLOT2D provides various packaged 2D image processing features, please refer PLOT2D user's manual for details. Now a query of zoomed box sub-region is allowed in PLOT2D program. A user can also use Imin,Imax,Jmin,Jmax fields to specify the ROI.

PLOT2D ZOOM ROI Window

The following ZOOM-BOXED-ROI (PLOT2D) window shows the extracted sub-region (the bounding box) from the previous original window. It allows the user query the sub-region of the bounding box of the original window. The X-index, Y-index also display the offset from the original parent window pixel index. When the LMB(left mouse button) is clicked in the ZOOM-BOXED-ROI window the X, Y value fields reflect the original X,Y index values in the parent window.

IV. EXAMPLES

Notes On Access CCD Files

Example of how to access HDF image files generated by Brian Tieman's CCD Image server is given below. The CCD image is tagged with "data" attribute. The procedure of running hdfb is given below:

Insert CCD disk into CD drive
Invoke the IDL hdfb program

     1)	File->Open...   -  dialog to navigate file system to find hdf files
			   pops up QUERY - SDS dialog

     2)	QUERY - SDS Dialog
		Click 'data' from the scroll list to display coarse data 

     3)	Repeat step 1) & 2) to find the desired file 

     4)	Click "Plot Window" in main window Check Options
		pops up PLOT2D window dialog

     5) PLOT2D
	Click "Plot Options..."  - dialog to access various image plot options
		Click "Eq Asp Rt"      - if desired, plot in equal aspect ratio 
		Click "Save PICT ..."  - if desired, dialog to save PICT file
		Click "Save RTIFF..."  - if desired, dialog to save TIFF file
		.
		.
		.
		Click "Close"          - close plot options dialog
	Click "DATA..."  - pop up dialog to generate ascii data 

Access other image file from the CD 
	Repeat the above step 1) and 5)

Quit the hdfb program
	File->Quit     - stop the HDF browser

Nexus 3D Scan Example

Following windows give a typical example of getting a 3D scan data array from a NEXUS file which was generated by the scan2nexus program.

This figure shows the SDS data for the sequence number 131 (RECNO 131:det_16) from the 2xfm_0020.nexus file is a 3D data set, the first twelve 2D images are drawn in the coarse drawing area.

This dialog window shows the 'Query - SDS' dialog, the seqno 131 with attribute name as 'det_16', and dimension as array(1000,21,21). Press the 'RAxis Plot...' button in this dialog, pops up the following dialog window.

This window shows the 'view3d_2d' dialog, the 'Preview' slider bar allows a user to preview any slice for a given rank. Various button allows the user flexible to access any 2D slice from the 3D array. Press the "View3d_2D_Sum Slices..." button pops up the axial sum of the 3D array as given in the following dialog window.

This window contains 3 drawing areas: the axial spectrum ROI at given I,J, TV image of axial spectrum ROI sum, TV line profile at given I,J. Two horizontal slider bars controls the position of the axial ROI vertical lines. Various buttons access various sub-programs for TV image.

This window shows the dialog of constucting Multiple 1D/2D SDSs array. A user can enter multiple 1D seq# to construct a multi-line array and passed it to the plot1d program, or enter multiple 2D seq# to construct a multi-image array and passed it to panimage or image2d sub-programs. In this example, the 2D SDS seq# 47 to 130 is entered with carriage-return then the following panimage window pops up. Press the IMAGE2D... button will pop up the image2d sub-program.

This window displays the panImage result of the multiple selected 2D image array for 2D SDS seq # 47-130 as entered from the previous dialog.

This IMAGE2D window displays the image selected from the constructed 2D image array. In this example the image of 'det_02' is selected from the first Det# scrolled list.