1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 <2023> 2024 | Index | 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 <2023> 2024 |
<== Date ==> | <== Thread ==> |
---|
Subject: | Re: linux kernel module for mrfioc2 on MTCA |
From: | Han Lee via Tech-talk <tech-talk at aps.anl.gov> |
To: | 문건영(선형가속기팀) <gymun at postech.ac.kr> |
Cc: | 박소희 <sohee5746 at postech.ac.kr>, "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov> |
Date: | Fri, 10 Feb 2023 12:42:53 -0800 |
Dear All,
With the help of Dr. Han, I was able to run mrfioc2.
However, I have one question.
I have one EVG and two EVRs as shown below.
(mTCA-EVM-300, mTCA-EVR-300, mTCA-EVR-300RF)
nat> show_fru
FRU Information:
----------------
FRU Device State Name
==========================================
0 MCH M4 NAT-MCH-CM
3 mcmc1 M4 NAT-MCH-MCMC
5 AMC1 M4 CCT AM G64/471
6 AMC2 M4 mTCA-EVM-300
7 AMC3 M4 mTCA-EVR-300
8 AMC4 M4 mTCA-EVR-300RF
40 CU1 M4 Schroff uTCA CU
50 PM1 M4 NAT-PM-AC600D
60 Clock1 M4 MCH-Clock
61 HubMod1 M4 MCH-PCIe
92 AMC3-RTM M4 mTCA-EVRTM-300
==========================================
I checked it by referring to lspci of mrfioc2 to distinguish 3 cards.
# dmesg | grep mrf
mrf: loading out-of-tree module taints kernel.
mrf-pci 0000:06:00.0: Attaching MRF device w/o PLX bridge (00007011)
mrf-pci 0000:06:00.0: MRF Setup complete
mrf-pci 0000:09:00.0: Attaching MRF device w/o PLX bridge (00007011)
mrf-pci 0000:09:00.0: MRF Setup complete
mrf-pci 0000:0a:00.0: Attaching MRF device w/o PLX bridge (00007011)
mrf-pci 0000:0a:00.0: MRF Setup complete
#
# lspci -v -s 6:00.0
06:00.0 Signal processing controller: Xilinx Corporation Device 7011
Subsystem: Device 1a3e:132c
Flags: bus master, fast devsel, latency 0, IRQ 16
Memory at df200000 (32-bit, non-prefetchable) [size=256K]
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [60] Express Endpoint, MSI 00
Capabilities: [100] Device Serial Number 00-00-00-00-00-00-00-00
Kernel driver in use: mrf-pci
Kernel modules: mrf
# lspci -v -s 9:00.0
09:00.0 Signal processing controller: Xilinx Corporation Device 7011
Subsystem: Device 1a3e:132c
Flags: bus master, fast devsel, latency 0, IRQ 18
Memory at df100000 (32-bit, non-prefetchable) [size=256K]
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [60] Express Endpoint, MSI 00
Capabilities: [100] Device Serial Number 00-00-00-00-00-00-00-00
Kernel driver in use: mrf-pci
Kernel modules: mrf
# lspci -v -s a:00.0
0a:00.0 Signal processing controller: Xilinx Corporation Device 7011
Subsystem: Device 1a3e:232c
Flags: bus master, fast devsel, latency 0, IRQ 16
Memory at df000000 (32-bit, non-prefetchable) [size=512K]
Capabilities: [40] Power Management version 3
Capabilities: [48] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [60] Express Endpoint, MSI 00
Capabilities: [100] Device Serial Number 00-00-00-00-00-00-00-00
Kernel driver in use: mrf-pci
Kernel modules: mrf
https://github.com/epics-modules/mrfioc2/blob/master/documentation/lspci/mtca-evm-300.txt
In the document above, the device name is specified in the subsystem item as shown below, but I do not.
06:00.0 Signal processing controller: Xilinx Corporation XILINX PCI DEVICE
Subsystem: Micro-Research Finland Oy MTCA Event Master 300
Physical Slot: 4
I couldn't distinguish between EVG and EVR, so I had to run them one by one.
epics> mrmEvgSetupPCI("EVG","6:00.0")
Notice: devPCIFindSpec() expect B:D.F in hex
PCI Device not found
epics>
epics> mrmEvgSetupPCI("EVG","9:00.0")
PCI Device not found
epics>
epics>
epics> mrmEvgSetupPCI("EVG","a:00.0")
Device EVG a:0.0
Using IRQ 16
FPGA version: 280e0207
mTCA-EVM-300 #Inputs FP:3 UV:16 RB:0
EVM automatically creating 'EVG:FCT', 'EVG:EVRD', and 'EVG:EVRU'
Found SFP EEPROM
Found SFP EEPROM
Found SFP EEPROM
Found SFP EEPROM
Found SFP EEPROM
Found SFP EEPROM
Found SFP EEPROM
Found SFP EEPROM
Warning: Recommended minimum firmware 2 version is 207.6, found 207.0
Found SFP Strangeness 00000000
Sequencer capability detected
VME 64: Out FP:8 FPUNIV:0 RB:0 IFP:8 GPIO:0
EVR FIFO task start
Warning: Recommended minimum firmware 2 version is 207.6, found 207.0
Found SFP Strangeness 00000000
Sequencer capability detected
VME 64: Out FP:8 FPUNIV:0 RB:0 IFP:8 GPIO:0
EVR FIFO task start
PCI interrupt connected!
epics>
epics> mrmEvrSetup'
Unbalanced quote.
epics> mrmEvrSetupPCI("EVR1","slot=4")
PCI Device not found on slot=4
epics> mrmEvrSetupPCI("EVR1","6:00.0")
Device EVR1 6:0.0 slot=(null)
Using IRQ 16
FWVersion 0x18120207
Found version 519
Found SFP EEPROM
Sequencer capability detected
mTCA: Out FP:4 FPUNIV:0 RB:16 IFP:2 GPIO:0
EVR FIFO task start
Enabling interrupts
epics> mrmEvrSetupPCI("EVR2","9:00.0")
Device EVR2 9:0.0 slot=(null)
Using IRQ 18
FWVersion 0x18100207
Found version 519
Found SFP EEPROM
Sequencer capability detected
mTCA: Out FP:4 FPUNIV:0 RB:16 IFP:2 GPIO:0
EVR FIFO task start
Enabling interrupts
epics>
Subsystem: Device 1a3e:232c
Subsystem: Device 1a3e:132c
Is there no problem if I separate it into the above information?
1a3e:232c is mTCA-EVM-300.
1a3e:132c is mTCA-EVR-300RF or mTCA-EVR-300.
I can distinguish the two EVRs after the IOC runs.
Is there any way to know before the IOC is implemented?
Best regards,
Geonyeong Mun
_________________________________________
Geonyeong Mun
Staff Engineer, Accelerator Division
Pohang Accelerator Laboratory (PAL)
37673 80 Jigokro-127-beongil, Nam-gu,
Pohang, Gyeongbuk, Korea
phone: +82 10 4164 - 2260
office: +82 54 279 - 1098
email: gymun at postech.ac.kr
web: pal.postech.ac.kr
From: Han Lee <jeonglee at lbl.gov>
Sent: Thursday, January 5, 2023 3:00 PM
To: 문건영(선형가속기팀) <gymun at postech.ac.kr>
Cc: tech-talk at aps.anl.gov
Subject: Re: linux kernel module for mrfioc2 on MTCA
The actual kernel driver is located in
There are several methods you may try, here is also very old repo, but you can find "dkms" information there.
Best,
Han
On Wed, Jan 4, 2023 at 9:57 PM Han Lee <jeonglee at lbl.gov> wrote:
Hi Geonyeong,
Please use the kernel driver for mrfiioc2 instead of what you used.
PCIID is not up-to-date, if you want to see through lspci, please check my own archived repo.
HTH,
Han
On Wed, Jan 4, 2023 at 9:41 PM 문건영(선형가속기팀) via Tech-talk <tech-talk at aps.anl.gov> wrote:
Dear All,
I am trying to MTCA-EVG-300 and MTCA-EVR-300U are working on Linux using mrfioc2.
But, the following error is displayed:
epics> mrmEvgSetupPCI("EVG",08,00,0)
Can't open /sys/module/mrf/parameters/interfaceversion in order to read kernel module interface version. Kernel module not loaded or too old.
I used the latest version driver provided by MRF. https://github.com/jpietari/mrf-linux-driver
Do I have to use a driver other than this one?
The mrf directory does not exist In the /sys/module directory.
Instead, pci_mrfevg, pci_mrfevr exists.
# ls /sys/module/pci_mrfev*
/sys/module/pci_mrfevg:
coresize holders initstate refcnt taint
drivers initsize notes sections uevent
/sys/module/pci_mrfevr:
coresize holders initstate refcnt taint
drivers initsize notes sections uevent
# lsmod
Module Size Used by Tainted: G
pci_mrfevr 57344 0
pci_mrfevg 53248 0
nfsv3 28672 1
nfs 135168 2 nfsv3
lockd 77824 2 nfsv3,nfs
grace 16384 1 lockd
sunrpc 196608 6 nfsv3,nfs,lockd
x86_pkg_temp_thermal 16384 0
igb 184320 0
# ls /dev/e*
/dev/ega0 /dev/ega2 /dev/era0 /dev/era2 /dev/erb0 /dev/erb2
/dev/ega1 /dev/ega3 /dev/era1 /dev/era3 /dev/erb1 /dev/erb3
# dmesg | tail
Event Generator PCI/PCIe driver exiting.
Event Receiver PCI/PCIe driver exiting.
Event Generator PCI/PCIe driver init.
mrfevg:BAR0 start df000000 end df07ffff mmaped 41180000
udevd[165]: specified group 'mrf' unknown
Event Receiver PCI/PCIe driver init.
mrfevr:BAR0 start df200000 end df23ffff, mmap 40780000
mrfevr: Device Type: MTCAEVR300
mrfevr:BAR0 start df100000 end df13ffff, mmap 41040000
mrfevr: Device Type: MTCAEVR300
Best regards,
Geonyeong Mun
--Jeong Han Lee, Dr.rer.nat
Staff Scientist and Engineer
Lawrence Berkeley National Laboratory
1 Cyclotron Road Mailstop 46R0161
Berkeley, CA 94720, United States
Tel :+1-510-486-6163
Cell:+1-510-384-3868
--Jeong Han Lee, Dr.rer.nat
Staff Scientist and Engineer
Lawrence Berkeley National Laboratory
1 Cyclotron Road Mailstop 46R0161
Berkeley, CA 94720, United States
Tel :+1-510-486-6163
Cell:+1-510-384-3868