Re: galil softIOC lose the galil controller

[Heinz Junkes <junkes@fhi-berlin.mpg.de>]

Dear Mark,

thank you for your instructions. I followed them, but still about after 4 hours the rio's are no more ping-able.

epics@ioc163:~$ ping 10.0.0.210
PING 10.0.0.210 (10.0.0.210) 56(84) bytes of data.

^C
--- 10.0.0.210 ping statistics ---
7 packets transmitted, 0 received, 100% packet loss, time 6047ms

epics@ioc163:~$ ping 10.0.0.211
PING 10.0.0.211 (10.0.0.211) 56(84) bytes of data.

^C
--- 10.0.0.211 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

epics@ioc163:~$ arp -a
? (10.0.0.211) at 00:50:4c:28:1a:dd [ether] PERM on eth0
? (10.0.0.45) at 18:03:73:d2:93:af [ether] on eth0
? (10.0.0.47) at 90:b1:1c:8b:8b:cd [ether] on eth0
? (10.0.0.2) at 00:0c:29:55:30:01 [ether] on eth0
? (10.0.0.210) at 00:50:4c:28:1a:de [ether] PERM on eth0
? (10.0.0.8) at d8:d3:85:58:e7:4b [ether] on eth0
? (10.0.0.46) at 90:b1:1c:8b:87:50 [ether] on eth0
? (10.0.0.1) at d8:d3:85:a5:13:9e [ether] on eth0

epics@ioc163:~$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         10.0.0.8        0.0.0.0         UG    0      0        0 eth0
10.0.0.0        *               255.255.254.0   U     0      0        0 eth0
224.0.0.0       *               240.0.0.0       U     0      0        0 eth0

they both end up with:

10.0.0.210:

epics> acquireDataRecord:1151 TIMEOUT ERROR.  Galil::record("DR") took longer than 500 ms to read 56 bytes
GalilController::get_double:GalilController::writeReadController:epics[2]=?:1011 TIMEOUT ERROR.  Galil::command("epics[2]=?") took longer than 500 ms to read : response.
acquireDataRecord:1151 TIMEOUT ERROR.  Galil::record("DR") took longer than 500 ms to read 56 bytes
GalilController::get_double:GalilController::writeReadController:epics[1]=?:1011 TIMEOUT ERROR.  Galil::command("epics[1]=?") took longer than 500 ms to read : response.
acquireDataRecord:1151 TIMEOUT ERROR.  Galil::record("DR") took longer than 500 ms to read 56 bytes
GalilController::get_double:GalilController::writeReadController:epics[2]=?:1011 TIMEOUT ERROR.  Galil::command("epics[2]=?") took longer than 500 ms to read : response.
acquireDataRecord:1151 TIMEOUT ERROR.  Galil::record("DR") took longer than 500 ms to read 56 bytes
Disconnected from RIO47100 Rev 1.1h at 10.0.0.210
connect: 5004 OPEN ERROR.  Galil::Galil() failed to open Ethernet host 10.0.0.210

10.0.0.211:

epics> acquireDataRecord:1151 TIMEOUT ERROR.  Galil::record("DR") took longer than 500 ms to read 56 bytes
GalilController::get_double:GalilController::writeReadController:epics[5]=?:1011 TIMEOUT ERROR.  Galil::command("epics[5]=?") took longer than 500 ms to read : response.
acquireDataRecord:1151 TIMEOUT ERROR.  Galil::record("DR") took longer than 500 ms to read 56 bytes
GalilController::get_double:GalilController::writeReadController:epics[6]=?:1011 TIMEOUT ERROR.  Galil::command("epics[6]=?") took longer than 500 ms to read : response.
acquireDataRecord:1151 TIMEOUT ERROR.  Galil::record("DR") took longer than 500 ms to read 56 bytes
acquireDataRecord:1151 TIMEOUT ERROR.  Galil::record("DR") took longer than 500 ms to read 56 bytes
GalilController::get_double:GalilController::writeReadController:epics[3]=?:1011 TIMEOUT ERROR.  Galil::command("epics[3]=?") took longer than 500 ms to read : response.
acquireDataRecord:1151 TIMEOUT ERROR.  Galil::record("DR") took longer than 500 ms to read 56 bytes
Disconnected from RIO47100 Rev 1.1h at 10.0.0.211
connect: 5004 OPEN ERROR.  Galil::Galil() failed to open Ethernet host 10.0.0.211

Any idea?

All the best,
Heinz


On 30.10.2014, at 09:47, Mark Clift <Mark.Clift@synchrotron.org.au> wrote:

> Hi Heinz,
> 
> Please follow this setup procedure for the RIO PLC.
> 
> 1.  Use GalilTools to set the IP address
> 2.  Record the MAC address of the RIO
> 3.  Edit /etc/rc.d/rc.local and add the following:
> 
> # Add multicast route for Galil hardware
> route add -net 224.0.0.0 netmask 240.0.0.0 dev eth0
> 
> # Add arp entry for Galil RIO device
> arp -s <IPADDRESS> <MACADDRESS>
> 
> 4.  Reboot softIOC or type above commands as root, then exit root login
> 5.  Telnet into RIO from softIOC, and issue the following commands
> 
> :IB1   (Possibly IK1 on some firmware)
> :BN   (Burn the changes)
> :^M
> exit
> 
> 6.  From now on only hosts with the arp entry can telnet into RIO
> 
> Then try the IOC again.  I recommend using a single IOC and connection to the RIO.
> 
> Also try the email again.  
> 
> Best wishes,
> 
> Mark.
> 
> Dr. Mark Clift
> Senior Controls Engineer
> Australian Synchrotron
> 800 Blackburn Road
> Clayton 3168
> Ph: +613 8540 4264
> Fax: +613 8540 4200
> Mail: Mark.Clift@Synchrotron.org.au
> ________________________________________
> From: Heinz Junkes [junkes@fhi-berlin.mpg.de]
> Sent: Thursday, 30 October 2014 19:16
> To: Mark Clift
> Cc: tech-talk@aps.anl.gov (tech-talk@aps.anl.gov)
> Subject: Re: galil softIOC lose the galil controller
> 
> Hi techtalk,
> 
> we are running two galil RIO-47120 controllers. We are using the newest versions of galilsuite and galiltools on debian linux.
> 
> ii  galilsuite                          1.0.2                               amd64        Test GalilSuite build
> ii  galiltools                          1.6.3                               amd64        GalilTools
> 
> and the Galil-3-0 motorapp.
> 
> We could'nt get them running stable. Until the day before yesterday I connected to both controllers with one softioc. About every second week they stopped working and the controllers could'nt be ping-ed anymore. We have to reset the controllers. Both stopped at the same time.
> Now I run two softioc's on the same Unix-System. Each of them opens a connection to a single controller ... now they sopped working after about 5-6 hours.
> And again both of them at the same time. I have no info's in any log-file.
> 
> The program on the IOC is very simple:
> 
> dmc-file, rioDIag.dmc :
> 
> 'Galil Motion Control epics@galilmc.com
> 'http://www.galilmc.com/
> #epics
> 'Insert code to run on
> ' GalilStartController() here...
> MA 141,14,141,214;
> MD junkes@fhi-berlin.mpg.de;
> MS rioDiag@fhi-berlin.mpg.de;
> WT 100
> XQ#update,1;'spawn update thread
> EN
> '
> #update;'Example update loop
> AT0
> epics[0]=TIME;'time tick
> epics[1]=_OP0
> epics[2]=_TI0
> OPepics[9]
> IF (epics[10])
> SB8
> WT50
> CB8
> epics[10]=0
> ENDIF
> AT200;'see SCAN field in
> ' galil_array_access.substitutions
> JP#update
> '
> #CMDERR;'runs if an error occurs
> MG "An error occured at line",_ED
> TC 1;'print type of error info
> ZS;'Remove returns from callbck
> EN;'End execution
> '
> #TCPERR
> MG {M} "TCPERR.  Dropped handle", _IA4
> RE
> 
> started with this cmd-file, rioDiag.cmd:
> 
> #Galil Motion Control epics@galilmc.com http://www.galilmc.com/
> 
> #Load Galil Data Record Support
> #dbLoadTemplate("$(TOP)/GalilDiagApp/Db/rioDiag_motor.substitutions")
> 
> #Provide access to Galil Array Table
> dbLoadTemplate("$(TOP)/GalilDiagApp/Db/rioDiag_array_access.substitutions")
> 
> # GalilCreateController command parameters are:
> # 1. Const char *portName - The name of the asyn port to be created for this controller
> # 2. Const char *address  - The address of the controller
> # 3. int numAxes          - The number of axis that will be used on the controller
> # 4. double updatePeriod  - The time in ms between datarecords
> #   (See controller's DR command for minimum).
> #   Async if controller + bus supports it, otherwise is polled/synchronous.
> GalilCreateController("rioDiag", "10.0.0.210", 1, 32)
> #GalilCreateController("rioDiag", "141.14.128.41", 2, 8)
> 
> # GalilCreateAxis command parameters are:
> # 1. char *portName Asyn port for controller
> # 2. char axis A-H,
> GalilCreateAxis("rioDiag","A")
> #GalilCreateAxis("rioDiag","B")
> #GalilCreateAxis("Galil","C")
> #GalilCreateAxis("Galil","D")
> #GalilCreateAxis("Galil","E")
> #GalilCreateAxis("Galil","F")
> #GalilCreateAxis("Galil","G")
> #GalilCreateAxis("Galil","H")
> 
> # GalilStartController command parameters are:
> # 1. char *portName Asyn port for controller
> # 2. char *code file to deliver to the controller, "" for none.
> # 3. char *label to execute on file, "" to not execute.
> GalilStartController("rioDiag", "rioDiag.dmc", "#epics")
> 
> and this st.cmd:
> 
> #!../../bin/linux-x86_64/GalilDiagApp
> #Galil Motion Control epics@galilmc.com http://www.galilmc.com/
> 
> epicsEnvSet("ARCH","linux-x86_64")
> epicsEnvSet("IOC","IOCRIODIAG")
> epicsEnvSet("TOP","/srv/ioc/sys/$(IOC)")
> #epicsEnvSet("SUPPORT","/home/epics/EPICS/support")
> #epicsEnvSet("EPICS_BASE","/home/epics/EPICS/base")
> #epicsEnvSet("ASYN","/home/epics/EPICS/support/asyn/asyn4-22")
> #epicsEnvSet("MOTOR","/home/epics/EPICS/support/motor/motorR6-8")
> epicsEnvSet("DISPLAY",":0")
> epicsEnvSet("LOG_DIR", "/srv/ioc/log")
> 
> 
> ## Register all support components
> dbLoadDatabase("$(TOP)/dbd/GalilDiagApp.dbd",0,0)
> GalilDiagApp_registerRecordDeviceDriver(pdbbase)
> 
> # Configure an example controller
> #< rioMezz.cmd
> < rioDiag.cmd
> 
> #dbLoadRecords ("$(TOP)/db/rioMezz.db","P=FHIFEL:,R=HELPER:,S=GUNRELAY:")
> 
> dbLoadRecords("$(TOP)/db/attenuator.db","P=FHIFEL:,R=DIAG:,S=ATTENUATOR:")
> 
> dbLoadRecords("$(TOP)/db/flipper.db","P=FHIFEL:,R=DIAG:,S=FLIP:")
> 
> #dbLoadRecords("$(TOP)/db/checkAlive.db","RIO=rioMezz:")
> dbLoadRecords("$(TOP)/db/checkAlive.db","RIO=rioDiag:")
> 
> 
> ## autoSaveRestore setup
> save_restoreSet_Debug(0)
> 
> # status-PV prefix, so save_restore can find its status PV's.
> save_restoreSet_status_prefix("${IOC}:")
> 
> # ok to restore a save set that had missing values (no CA connection to PV)?
> # ok to save a file if some CA connections are bad?
> save_restoreSet_IncompleteSetsOk(1)
> 
> #In the restore operation, a copy of the save file will be written.  The
> # file name can look like "auto_settings.sav.bu", and be overwritten every
> # reboot, or it can look like "auto_settings.sav_020306-083522" (this is what
> # is meant by a dated backup file) and every reboot will write a new copy.
> save_restoreSet_DatedBackupFiles(1)
> 
> # specify where save files should go
> set_savefile_path("/srv/ioc/log/as","/save")
> 
> # specify where request files can be found
> set_requestfile_path("/srv/ioc/log/as","/req")
> 
> # Specify what save files should be restored when.
> # Up to eight files can be specified for each pass.
> set_pass0_restoreFile("${IOC}_0.sav")
> set_pass1_restoreFile("${IOC}_1.sav")
> 
> # Number of sequenced backup files (e.g., 'auto_settings.sav0') to write
> save_restoreSet_NumSeqFiles(3)
> 
> save_restoreSet_SeqPeriodInSeconds(600)
> 
> # Time between failed .sav-file write and the retry.
> save_restoreSet_RetrySeconds(60)
> 
> iocInit()
> 
> makeAutosaveFileFromDbInfo("/srv/ioc/log/as/req/${IOC}_0.req", "autosaveFields_pass0")
> makeAutosaveFileFromDbInfo("/srv/ioc/log/as/req/${IOC}_1.req", "autosaveFields_pass1")
> 
> create_monitor_set("${IOC}_0.req", 5, "P=${IOC}:")
> create_monitor_set("${IOC}_1.req", 30, "P=${IOC}:")
> 
> ## Report Installed and Configured I/O-Hardware Information
> dbior 0 1 > ${LOG_DIR}/Database/${IOC}.dbior
> dbhcr     > ${LOG_DIR}/Database/${IOC}.dbhcr
> dbl       > ${LOG_DIR}/Database/${IOC}.dbl
> 
> 
> 
> By the way, the mail-sending on RIO works only once or twice .. then it stops sending any mails
> 
> Best regards,
> Heinz
> 
> 
> On 30.10.2014, at 01:17, Mark Clift <Mark.Clift@synchrotron.org.au> wrote:
> 
>> Hi Qingru,
>> 
>> I would guess that the multi-cast route is missing, which is required for the Galil driver you're using.
>> 
>> We add the following line into /etc/rc.local
>> 
>> route add -net 224.0.0.0 netmask 240.0.0.0 dev eth0
>> 
>> The error simply says it could not poll the first 8 digital inputs with Galil TI (Tell input) command.
>> 
>> Also there is a newer Galil driver designed to replace the one your using located here:
>> 
>> http://motorapp.github.io/Galil-3-0/
>> 
>> Best wishes,
>> 
>> Mark
>> 
>> Dr. Mark Clift
>> Senior Controls Engineer
>> Australian Synchrotron
>> 800 Blackburn Road
>> Clayton 3168
>> Ph: +613 8540 4264
>> Fax: +613 8540 4200
>> Mail: Mark.Clift@Synchrotron.org.au
>

Attachment: smime.p7s
Description: S/MIME cryptographic signature