Re: SPEC "can't get a big memory buffer" issue

[Matt Newville via Tech-talk <tech-talk@aps.anl.gov>]

Hi Gerry, 

I think you need the perl ExtUtils::Embed extension to compile perl 'xs' files.  You should be able to install that with

    perl -MCPAN -e 'install ExtUtils::Embed'

But I think that Apple-supplied perl (/usr/bin/perl) should come with this extension already installed and have EXTERN.h at

 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE/EXTERN.h

If that is already there, I'm not sure why it wouldn't be found.  

--Matt 

On Thu, Jan 17, 2019 at 1:00 AM Mark Rivers via Tech-talk <tech-talk@aps.anl.gov> wrote:

Hi Gerry,

That appears to be a problem with case. It should probably be extern.h not EXTERN.h

I’m passing this to tech-talk to see if there is a known fix.

Mark


Sent from my iPhone

> On Jan 16, 2019, at 10:03 PM, Gerry Swislow <gerry@certif.com> wrote:
>
> Hi Mark,
>
> I'm working on an up-to-date macOS Mojave platform.  The build stops here:
>
> perl -CSD /usr/local/epics/base-7.0.2/bin/darwin-x86/mkmf.pl  -m Cap5.d -I. -I../O.Common -I. -I. -I.. -I/usr/local/epics/base-7.0.2/include/compiler/clang -I/usr/local/epics/base-7.0.2/include/os/Darwin -I/usr/local/epics/base-7.0.2/include -I/usr/local/epics/base-7.0.2/include/compiler/clang -I/usr/local/epics/base-7.0.2/include/os/Darwin -I/usr/local/epics/base-7.0.2/include   -I/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE     Cap5.o Cap5.c
> Installing PERL_MODULES file /usr/local/epics/base-7.0.2/lib/perl/CA.pm
> cc              -DUNIX  -Ddarwin     -O3 -g   -Wall    -g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -fstack-protector    -arch x86_64    -fno-common -fPIC -I. -I../O.Common -I. -I. -I.. -I/usr/local/epics/base-7.0.2/include/compiler/clang -I/usr/local/epics/base-7.0.2/include/os/Darwin -I/usr/local/epics/base-7.0.2/include -I/usr/local/epics/base-7.0.2/include/compiler/clang -I/usr/local/epics/base-7.0.2/include/os/Darwin -I/usr/local/epics/base-7.0.2/include   -I/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE     -c Cap5.c
> ../Cap5.xs:7:10: fatal error: 'EXTERN.h' file not found
> #include "EXTERN.h"
>         ^~~~~~~~~~
> 1 error generated.
> make[5]: *** [Cap5.o] Error 1
> make[4]: *** [install.darwin-x86] Error 2
> make[3]: *** [perl.install] Error 2
> make[2]: *** [src.install] Error 2
> make[1]: *** [ca.install] Error 2
> make: *** [modules.install] Error 2
>
>
> - Gerry
>
>
>
> Begin forwarded message:
>
>> From: Mark Rivers <rivers@cars.uchicago.edu>
>> Subject: Re: SPEC "can't get a big memory buffer" issue
>> Date: January 16, 2019 at 3:29:22 PM EST
>> To: Peter Eng <eng@cars.uchicago.edu>, Zhang Dongzhou <dzhang@hawaii.edu>
>> Cc: "gerry@certif.com" <gerry@certif.com>, Joanne Stubbs <stubbs@cars.uchicago.edu>
>>
>>> We could not link this version with epics base-7.xx.x but it was successful linking with epics base-3.15.5.
>>
>> What error did you get with base-7.0.2?  We should try to fix that.
>>
>> Mark
>>
>> ________________________________
>> From: Peter Eng <eng@cars.uchicago.edu>
>> Sent: Wednesday, January 16, 2019 9:03 AM
>> To: Zhang Dongzhou
>> Cc: Mark Rivers; gerry@certif.com; Joanne Stubbs
>> Subject: Re: SPEC "can't get a big memory buffer" issue
>>
>> Hi Dangzhou,
>> Joanne and I have updated SPEC to version 6.08.01 we used the tar file, spec_6.08.01_ce76_x64_APS.tar.  We could not link this version with epics base-7.xx.x but it was successful linking with epics base-3.15.5.  Our install_data file contante is:
>> ############################
>>
>> geo="psic psic psic"
>> name="psic_idc psic_bmc psic_lab"
>> owner=gpd_user
>> install=/home/specadm/Versions/spec6.08.01/bin
>> aux=/home/specadm/Versions/spec6.08.01/spec.d
>> taco_home=no
>> tango_home=no
>> epics_home=/usr/local/epics/base-3.15.5/lib/linux-x86_64
>> hdf5_libs=no
>> perm=u=rw,g=rw,o=rw
>> editline=-lreadline
>> site_flags=no
>> site_obj=no
>> site_lib=no
>>
>> ############################
>>
>> You can test your python code by running: /home/specadm/Versions/spec6.08.01/bin/psic_bmc
>>
>> We did testing at BMC and were able to run it in both step and trajectory scanning mode.  One thing we noticed but did  not look into was that splot was not working quite correctly - but did not spend any time chasing that down.
>>
>> Let me know if this fixes the problem.
>>
>>   -Peter
>>
>>
>>
>>
>>
>>
>> On Sat, Jan 12, 2019 at 9:35 AM Gerry Swislow <gerry@certif.com<mailto:gerry@certif.com>> wrote:
>> Hi Dongzhou,
>>
>> Thanks for the information.  It seems you've managed to find a combination of commands to trigger the fault.  First off, the segmentation fault occurs because the lazy programmer didn't check for an error return by the function that says it can't get a big memory buffer and the code then tries to dereference a null pointer.  That will be fixed, but even if there was no fault, you would still get the error.  The problem is triggered because of the ascan command send via the server socket where the ascan apparently involves epics_put() calls of arrays or character strings.  I can  explain what happened and the interesting history as to how the code became broken, but it is a bit complicated and probably more than you want to hear.
>>
>> I believe I've fixed the problem for the next release of spec. The version you are running is around two years old.  I have source code control and could conceivably create a patch for your release.  If updating to the current spec release would be very disruptive, we could try the patch.  Which would you prefer, getting the current release with the fix or attempting to patch the old release?  If the latter, I would need to know whether I have to provide a complete distribution, or if you have access to the distribution files and can simply replace a module in one of libraries included in the distribution.  In either case, I'd need to know what Linux platform you are using.
>>
>> Let me know which path you wish to take.  From there, it will take me a little time to get things ready.
>>
>> Best regards,
>>
>> Gerry
>>
>>
>> Begin forwarded message:
>>
>>> From: Zhang Dongzhou <dzhang@hawaii.edu<mailto:dzhang@hawaii.edu>>
>>> Subject: Re: SPEC "can't get a big memory buffer" issue
>>> Date: January 11, 2019 at 10:11:59 PM EST
>>> To: Gerry Swislow <gerry@certif.com<mailto:gerry@certif.com>>
>>> Cc: Peter Eng <eng@cars.uchicago.edu<mailto:eng@cars.uchicago.edu>>, Mark Rivers <rivers@cars.uchicago.edu<mailto:rivers@cars.uchicago.edu>>
>>>
>>> Hi Gerry,
>>>
>>> First thank you very much for your help! I collected the two log files based on your instructions, which are attached in the email. The "gdb_log.txt" is the first log file that you ask, and the "dlog" file is generated using the debug mode of SPEC. Please let me know if you need any further information.
>>>
>>> Best,
>>>
>>> Dongzhou
>>>
>>> On Fri, Jan 11, 2019 at 7:06 PM Gerry Swislow <gerry@certif.com<mailto:gerry@certif.com>> wrote:
>>> Hi Dongzhou,
>>>
>>> Sorry to hear of the problem.  Let's see if we can figure it out.  I'll need more information to understand what is going on.
>>>
>>> First thing I would ask, if the segmentation fault is reproducible, can you use gdb to do a stack back trace?  I would suggest starting psic as usual, then from another terminal window, type "ps -ef | grep psic" to get the process ID (PID) of the spec/psic process. Then in that window type "gdb /usr/local/bin/psic PID".  That may not work, though, if spec is installed either with set-user-id root privileges or setcap was used to give the executable privileged capabilities, unless you run gdb as a privileged user.  If it does work, though, then you can type "c" at the gdb prompt so that spec will continue.  Do the actions that cause the segmentation fault, which will produce a message on the gdb terminal.  At that terminal, type "bt" to get a stack backtrace, which I'd be interested in seeing.
>>>
>>> If your hardware configuration doesn't require the privileged modes of the spec in /usr/local/bin, for testing purposes, you could copy the executable /usr/local/bin/psic to a private directory.  The copied executable will lose the privileges.  If you run that psic from the shell, then you should have no problem connecting via gdb.
>>>
>>> You could also use "gdb /usr/local/bin/psic" directly.  Using gdb will disable the privileges, but it can get messy running gdb directly.  It will catch signals and be otherwise a bit fussy.
>>>
>>> The second thing I would ask is to turn on debugging and create a debugging log file.  A problem with that is if there is a segmentation fault, the most recent and most interesting debugging messages might be left in a memory buffer and not appear in the file.  However, we can try.  Could you do this:
>>>
>>> debug 0xc01040
>>> fon dlog
>>> # ... do the things that cause a crash
>>>
>>> It might also be interesting to see the nearly same things that don't cause a crash.  If you do a bit of that, you can turn off debugging with:
>>>
>>> debug 0
>>> fclose dlog
>>>
>>> If a file named "dlog" or that ends in ".dlog" in "on", debugging messages are sent to the file, but not to the screen.
>>>
>>> The file may get large quickly.  That kind of depends on the specifics of the hardware being used.
>>>
>>> Those two things might be enough for me to know what is happening.  If not, I'll ask for more!
>>>
>>> Best regards,
>>>
>>> Gerry
>>>
>>>
>>>
>>>
>>> Begin forwarded message:
>>>
>>>> From: Zhang Dongzhou <dzhang@hawaii.edu<mailto:dzhang@hawaii.edu>>
>>>> Subject: SPEC "can't get a big memory buffer" issue
>>>> Date: January 11, 2019 at 6:16:28 PM EST
>>>> To: info@certif.com<mailto:info@certif.com>
>>>> Cc: Peter Eng <eng@cars.uchicago.edu<mailto:eng@cars.uchicago.edu>>, Mark Rivers <rivers@cars.uchicago.edu<mailto:rivers@cars.uchicago.edu>>
>>>>
>>>> Hi Gerry,
>>>>
>>>> Hope this email finds you well. I work at GSECARS at the Advanced Photon Source, and I use SPEC to carry out X-ray diffraction. Recently I encountered a strange problem when using SPEC6.04.04 and a python library called SpecClient (https://github.com/mxcube/specclient) together. The spec command I use is "ascan".
>>>>
>>>> So If I type "ascan kphi -2 2 4 1" in the spec terminal using keyboard, the scan works fine and I got the following information:
>>>>
>>>> Scan  11   Fri Jan 11 09:54:54 2019   file = /home/gpd_user/cars5/Data/gpd_user/data/bmc/2019/run1/DAC_setup/test/test1_6.spec    user = gpd_user
>>>> ascan  kphi -2 2  4 1
>>>>
>>>> #      kphi       i1       io    Seconds   Bicron AmpTek_s     ROI1     ROI2     ROI3     IROI    imtot    immax
>>>> 0   -2.0000     1268      803    1.00998     1121     1095        0        0        0        0 -1.773e+04 4.085e+04
>>>> 1   -1.0000     1266      817    1.00998     1121     1104        0        0        0        0 4.884e+04 9.511e+04
>>>> 2    0.0000     1268      812    1.00998     1118     1101        0        0        0        0 -5.041e+04 1.928e+04
>>>> 3    1.0000     1267      750    1.00998     1115     1057        0        0        0        0 -6.73e+04     5887
>>>> 4    2.0000     1269      870    1.00998     1121     1141        0        0        0        0 -6.718e+04     6243
>>>>
>>>> Auto-saving SPEC variables...
>>>>
>>>> However, when I run the following code in python:
>>>> python>>>xtalcmd = SpecCommand.SpecCommandA('','corvette.cars.aps.anl.gov:6780<http://corvette.cars.aps.anl.gov:6780>')
>>>> python>>>eval("xtalcmd.executeCommand(\"%s\")" % "ascan kphi -2 2 4 1")
>>>>
>>>> Basically this python code sends the SPEC command  "ascan kphi -2 2 4 1" to SPEC through socket. Then the SPEC terminal scans two images, and throws out an error:
>>>>
>>>> 54.PSIC_BMC> Total 5 points, 5 seconds
>>>>
>>>> Scan  12   Fri Jan 11 09:59:03 2019   file = /home/gpd_user/cars5/Data/gpd_user/data/bmc/2019/run1/DAC_setup/test/test1_6.spec    user = gpd_user
>>>> ascan  kphi -2 2  4 1
>>>>
>>>> #      kphi       i1       io    Seconds   Bicron AmpTek_s     ROI1     ROI2     ROI3     IROI    imtot    immax
>>>> 0   -2.0000     1264      831    1.00998     1119     1111        0        0        0        0 2.372e+04 6.341e+04
>>>> 1   -1.0000     1264      802    1.00998     1132     1092        0        0        0        0 5.162e+04 1.023e+05
>>>> Can't get a big memory buffer.
>>>> Segmentation fault
>>>>
>>>> Very interestingly, If I am not using area detectors, both typing in the SPEC terminal and running the code in python will work. The "Can't get a big memory buffer" error only happens when an area detector is involved in the scanning. So if I turn the area detector off, and run the spec command "ascan kphi -2 2 4 1" in SPEC terminal, SPEC will do the scan:
>>>>
>>>> Scan   5   Fri Jan 11 14:50:08 2019   file = /home/gpd_user/cars5/Data/gpd_user/data/bmc/2019/run1/test/test/test_1.spec  user = gpd_user
>>>> ascan  phi -2 2  4 1
>>>>
>>>> #       phi       i1       io    Seconds   Bicron AmpTek_s     ROI1     ROI2     ROI3     IROI   transm  filters
>>>> 0   -2.0000     1054      964    1.00998     1256     1235        0        0        0        0 3.81e-06        6
>>>> 1   -1.0000     1054      906    1.00998     1253     1192        0        0        0        0 3.81e-06        6
>>>> 2    0.0000     1053      849    1.00998     1253     1153        0        0        0        0 3.81e-06        6
>>>> 3    1.0000     1052      901    1.00998     1258     1190        0        0        0        0 3.81e-06        6
>>>> 4    2.0000     1051      952    1.00998     1259     1226        0        0        0        0 3.81e-06        6
>>>>
>>>> Auto-saving SPEC variables...
>>>>
>>>> And when I send the same SPEC command through python without the area detector, it is also working fine.
>>>> python>>>xtalcmd = SpecCommand.SpecCommandA('','corvette.cars.aps.anl.gov:6780<http://corvette.cars.aps.anl.gov:6780>')
>>>> python>>>eval("xtalcmd.executeCommand(\"%s\")" % "ascan kphi -2 2 4 1")
>>>>
>>>> 17.PSIC_BMC> Total 5 points, 5 seconds
>>>>
>>>> Scan  12   Fri Jan 11 14:37:55 2019   file = /home/gpd_user/cars5/Data/gpd_user/data/bmc/2019/run1/DAC_setup/test/test1_8.spec  test1  user = gpd_user
>>>> ascan  kphi -2 2  4 1
>>>>
>>>> #      kphi       i1       io    Seconds   Bicron AmpTek_s     ROI1     ROI2     ROI3     IROI   transm  filters
>>>> 0   -2.0000     1051      926    1.00998     1259     1207        0        0        0        0 3.81e-06        6
>>>> 1   -1.0000     1053      925    1.00998     1256     1210        0        0        0        0 3.81e-06        6
>>>> 2    0.0000     1053     1021    1.00998     1258     1273        0        0        0        0 3.81e-06        6
>>>> 3    1.0000     1052     1028    1.00998     1254     1280        0        0        0        0 3.81e-06        6
>>>> 4    2.0000     1049      906    1.00998     1254     1196        0        0        0        0 3.81e-06        6
>>>>
>>>> Auto-saving SPEC variables...
>>>>
>>>> I was wondering whether you have encountered similar issues before? How could we solve the "Can't get a big memory buffer" issue? Looking forward to your advice.
>>>>
>>>> Best,
>>>>
>>>> Dongzhou
>>>>
>>>>
>>>> --
>>>> Dongzhou Zhang
>>>> PX^2 Beamline Scientist
>>>> 9700 S Cass Ave, Bldg 434A
>>>> Argonne, IL 60439, USA
>>>> (630)-252-0444
>>>> dzhang@hawaii.edu<mailto:dzhang@hawaii.edu>
>>>> https://zhangdongzhou.github.io/
>>>
>>>
>>> --
>>> Dongzhou Zhang
>>> PX^2 Beamline Scientist
>>> 9700 S Cass Ave, Bldg 434A
>>> Argonne, IL 60439, USA
>>> (630)-252-0444
>>> dzhang@hawaii.edu<mailto:dzhang@hawaii.edu>
>>> https://zhangdongzhou.github.io/
>>>
>>
>>
>> --
>> Peter J. Eng
>> Research Professor
>> CARS and JFI
>> University of Chicago
>>
>> 773 450-2085