Experimental Physics and
Industrial Control System

We have established that the files do contain these non-UTF8 characters.  The question is, should this matter?  So far the only problem has been observed by Kaz when building on Windows.  The problem is in perl when creating the .d dependency files.

 

The problem does not occur on Linux because by default the dependency files are generated by the compiler, not by perl.  However, I just changed my CONFIG_SITE.linux-x86_64.linux-x86_64 file in base 7.0.1 to have the following line:

HDEPENDS_METHOD=MKMF

 

This causes it to use perl to generate the dependency file.  I then rebuilt the lcms/src directory on my RHEL 7 system.

 

perl -CSD /corvette/usr/local/epics-devel/base-7.0.1/bin/linux-x86_64/mkmf.pl  -m cmstypes.d -I. -I../O.Common -I. -I. -I.. -I../../../../../include/compiler/gcc -I../../../../../include/os/Linux -I../../../../../include      -I/corvette/home/epics/devel/asyn-4-33/include     -I/corvette/home/epics/devel/areaDetector-3-2/ADSupport/include/os/Linux -I/corvette/home/epics/devel/areaDetector-3-2/ADSupport/include   -I/corvette/home/epics/devel/areaDetector-3-2/ADCore/include -I/corvette/usr/local/epics-devel/base-7.0.1/include/compiler/gcc -I/corvette/usr/local/epics-devel/base-7.0.1/include/os/Linux -I/corvette/usr/local/epics-devel/base-7.0.1/include     -I../../../../../supportApp/GraphicsMagickSrc/lcms/include    cmstypes.o ../cmstypes.c

perl -CSD /corvette/usr/local/epics-devel/base-7.0.1/bin/linux-x86_64/mkmf.pl  -m cmssm.d -I. -I../O.Common -I. -I. -I.. -I../../../../../include/compiler/gcc -I../../../../../include/os/Linux -I../../../../../include      -I/corvette/home/epics/devel/asyn-4-33/include     -I/corvette/home/epics/devel/areaDetector-3-2/ADSupport/include/os/Linux -I/corvette/home/epics/devel/areaDetector-3-2/ADSupport/include   -I/corvette/home/epics/devel/areaDetector-3-2/ADCore/include -I/corvette/usr/local/epics-devel/base-7.0.1/include/compiler/gcc -I/corvette/usr/local/epics-devel/base-7.0.1/include/os/Linux -I/corvette/usr/local/epics-devel/base-7.0.1/include     -I../../../../../supportApp/GraphicsMagickSrc/lcms/include    cmssm.o ../cmssm.c

 

Note that perl did not complain, it built the .d file fine.  This is using perl version 5.16.3.

corvette:GraphicsMagickSrc/lcms/src>perl --version

 

This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi

(with 33 registered patches, see perl -V for more detail

 

I then rebuilt using a Fedora 26 system and perl generated the error:

 

perl -CSD /corvette/usr/local/epics-devel/base-7.0.1/bin/linux-x86_64/mkmf.pl  -m cmstypes.d -I. -I../O.Common -I. -I. -I.. -I../../../../../include/compiler/gcc -I../../../../../include/os/Linux -I../../../../../include      -I/corvette/home/epics/devel/asyn-4-33/include     -I/corvette/home/epics/devel/areaDetector-3-2/ADSupport/include/os/Linux -I/corvette/home/epics/devel/areaDetector-3-2/ADSupport/include   -I/corvette/home/epics/devel/areaDetector-3-2/ADCore/include -I/corvette/usr/local/epics-devel/base-7.0.1/include/compiler/gcc -I/corvette/usr/local/epics-devel/base-7.0.1/include/os/Linux -I/corvette/usr/local/epics-devel/base-7.0.1/include     -I../../../../../supportApp/GraphicsMagickSrc/lcms/include    cmstypes.o ../cmstypes.c

Malformed UTF-8 character (fatal) at /corvette/usr/local/epics-devel/base-7.0.1/bin/linux-x86_64/mkmf.pl line 136, <FILE> line 5573.

 

This is perl version 5.24.3.  So whether the characters are a problem depends on the version of perl.  Kaz sees it with perl on Windows, but I do not.  My Windows perl version is 5.14.2.  Kaz, what version of perl are you running on Windows?

 

There is one thing I do not understand is why grep finds the non-UTF8 characters on Fedora 26, but apparent not on RHEL 7?  These are the same physical file on disk.  Can someone explain this?

 

RHEL 7:

corvette:GraphicsMagickSrc/lcms/src>grep -axv '.*' cmstypes.c                                                                                                 

corvette:GraphicsMagickSrc/lcms/src>

 

Fedora 26:

[epics@millenia src]$ grep -axv '.*' cmstypes.c

    //(see clause 4.1 for the definition of ▒aligned▒). Because the Unicode language

//The device representation corresponds to the header▒s ▒color space of data▒ field.

//This representation should be consistent with the ▒number of device components▒

//The PCS representation corresponds to the header▒s PCS field. The PCS representation

//CIE ▒absolute▒ illuminant white point tristimulus values and CIE ▒absolute▒

// The first curve segment always starts at ▒Infinity, and the last curve segment always ends at +Infinity. The

// array = [e11, e12, ▒, e1P, e21, e22, ▒, e2P, ▒, eQ1, eQ2, ▒, eQP, e1, e2, ▒, eQ]

            // Y = (Max ▒ Min) * (X ^ Gamma) + Min

            // a = (Max ▒ Min) ^ ( 1 / Gamma)

 

My conclusion is that the characters are not a problem when using perl 5.14.2 or 5.16.3, but they are a problem when using perl 5.24.3.  Is there a switch to later versions of perl that would suppress this error?

 

Kaz wrote:

Ø  It might be a larger issue than just GraphicsMagick, since Microsoft Unicode characters are in many more files than just GraphicsMagick package. I am finding them in Pilatus detector as well.

 

You have not seen a problem when building ADPilatus.  There could be 3 reasons for this:

-          The ADPilatus Makefile only builds on Linux and Darwin.  Since you only use perl for dependencies on Windows it does not happen.

-          If you switched to using perl on Linux as in my test above you might have the problem, depending on the version of perl.

-          It appears that only some files with non-UTF8 characters have the problem.  In lcms/src lots of files have the characters, but only cmstypes.h generates the perl error.

 

Mark