Re: Cross-compiling epics for petalinux

[Donny Domagoj Cosic via Tech-talk <tech-talk at aps.anl.gov>]

Hi Andrew,

Thank you for your reply. I forgot to mention, but you correctly concluded that I am using EPICS 7 base. I do not know exactly starting when, but in recent version of Petalinux, Xilinx does not offer a standalone download for the sdk (I just started using the petalinux tools, maybe I understood the flow incorrectly..). The sdk has to be generated using petalinux tools:

$ petalinux-build --sdk
$ petalinux-package --sysroot
$ source /images/linux/sdk/environment-setup-microblazeel-v11.0-bs-cmp-re-mh-div-fb-xilinx-linux

This creates a sdk directory with the structure (I added it to the end of the email).

The build tools are located in:

/sdk/sysroots/x86_64-petalinux-linux/usr/bin/microblazeel-xilinx-linux/

So if i put GNU_DIR = /sdk/sysroots/ or GNU_DIR = /sdk/sysroots/microblazeel-v11.0-bs-cmp-re-mh-div-fb-xilinx-linux/, it does not find the build tools because they are located in the /sdk/sysroots/x86_64-petalinux-linux directory.

If i put GNU_DIR = /sdk/sysroots/x86_64-petalinux-linux  it also does not find it because it is looking in sdk/sysroots/x86_64-petalinux-linux/bin/microblazeel-xilinx-linux-gcc instead of sdk/sysroots/x86_64-petalinux-linux/usr/bin/microblazeel-xilinx-linux/microblazeel-xilinx-linux-gcc

If I manually set the location of the build tools, i get an error with the include files:

In file included from /home/donny/Xilinx/KC705_Petalinux/images/linux/sdk/sysroots/x86_64-petalinux-linux/usr/lib/microblazeel-xilinx-linux/gcc/microblazeel-xilinx-linux/12.2.0/include-fixed/syslimits.h:7,
                 from /home/donny/Xilinx/KC705_Petalinux/images/linux/sdk/sysroots/x86_64-petalinux-linux/usr/lib/microblazeel-xilinx-linux/gcc/microblazeel-xilinx-linux/12.2.0/include-fixed/limits.h:34,
                 from ../yajl/yajl_parser.c:18:
/home/donny/Xilinx/KC705_Petalinux/images/linux/sdk/sysroots/x86_64-petalinux-linux/usr/lib/microblazeel-xilinx-linux/gcc/microblazeel-xilinx-linux/12.2.0/include-fixed/limits.h:203:75: error: no include path in which to search for limits.h
  203 | #include_next <limits.h>                /* recurse down to the real one */

It seems that the directory structure has changed and I am not sure which include and lib folder should I use for the build, the microblazeel-v11.0-bs-cmp-re-mh-div-fb-xilinx-linux or x86_64-petalinux-linux?

Directory Tree:

/home/donny/Xilinx/KC705_Petalinux/images/linux/sdk/sysroots/
├── microblazeel-v11.0-bs-cmp-re-mh-div-fb-xilinx-linux
│   ├── bin
│   ├── boot
│   ├── etc
│   ├── home
│   ├── lib
│   │   ├── depmod.d
│   │   ├── modules
│   │   └── security
│   ├── media
│   ├── mnt
│   ├── proc
│   ├── run
│   ├── sbin
│   ├── sys
│   ├── tmp
│   ├── usr
│   │   ├── bin
│   │   ├── games
│   │   ├── include
│   │   │   ├── arpa
│   │   │   ├── asm
│   │   │   ├── asm-generic
│   │   │   ├── attr
│   │   │   ├── bits
│   │   │   ├── blkid
│   │   │   ├── c++
│   │   │   ├── drm
│   │   │   ├── elfutils
│   │   │   ├── finclude
│   │   │   ├── gdbm
│   │   │   ├── gnu
│   │   │   ├── iproute2
│   │   │   ├── libfdisk
│   │   │   ├── libmnl
│   │   │   ├── libmount
│   │   │   ├── libsmartcols
│   │   │   ├── linux
│   │   │   ├── lzma
│   │   │   ├── misc
│   │   │   ├── mtd
│   │   │   ├── net
│   │   │   ├── netash
│   │   │   ├── netatalk
│   │   │   ├── netax25
│   │   │   ├── neteconet
│   │   │   ├── netinet
│   │   │   ├── netipx
│   │   │   ├── netiucv
│   │   │   ├── netpacket
│   │   │   ├── netrom
│   │   │   ├── netrose
│   │   │   ├── nfs
│   │   │   ├── openssl
│   │   │   ├── pci
│   │   │   ├── protocols
│   │   │   ├── python3.10
│   │   │   ├── rdma
│   │   │   ├── readline
│   │   │   ├── rpc
│   │   │   ├── rpcsvc
│   │   │   ├── scsi
│   │   │   ├── security
│   │   │   ├── shadow
│   │   │   ├── sound
│   │   │   ├── sys
│   │   │   ├── tcf
│   │   │   ├── tirpc
│   │   │   ├── uuid
│   │   │   ├── video
│   │   │   └── xen
│   │   ├── lib
│   │   │   ├── audit
│   │   │   ├── elfutils
│   │   │   ├── engines-3
│   │   │   ├── gawk
│   │   │   ├── gconv
│   │   │   ├── gdbm
│   │   │   ├── libpam
│   │   │   ├── microblazeel-xilinx-linux
│   │   │   ├── openssl
│   │   │   ├── opkg
│   │   │   ├── ossl-modules
│   │   │   ├── pkgconfig
│   │   │   ├── python3.10
│   │   │   ├── sed
│   │   │   ├── ssl-3
│   │   │   ├── sudo
│   │   │   ├── tc
│   │   │   ├── util-linux
│   │   │   └── zlib
│   │   ├── libexec
│   │   │   ├── awk
│   │   │   └── mtd-utils
│   │   ├── local
│   │   │   ├── include
│   │   │   └── lib
│   │   ├── sbin
│   │   ├── share
│   │   └── src
│   └── var

│   
└── x86_64-petalinux-linux
    ├── bin
    ├── environment-setup.d
    ├── etc
    ├── lib
    ├── sbin
    ├── usr
    │   ├── bin
    │   │   ├── microblazeel-xilinx-linux
    │   │   └── microblazeel-xilinx-linux-musl
    │   ├── include
    │   │   ├── arpa
    │   │   ├── asm
    │   │   ├── asm-generic
    │   │   ├── bits
    │   │   ├── c++
    │   │   ├── drm
    │   │   ├── finclude
    │   │   ├── gdbm
    │   │   ├── gnu
    │   │   ├── google
    │   │   ├── libxml2
    │   │   ├── linux
    │   │   ├── lzma
    │   │   ├── misc
    │   │   ├── mtd
    │   │   ├── nativesdk-bash
    │   │   ├── ncurses
    │   │   ├── ncursesw
    │   │   ├── net
    │   │   ├── netash
    │   │   ├── netatalk
    │   │   ├── netax25
    │   │   ├── neteconet
    │   │   ├── netinet
    │   │   ├── netipx
    │   │   ├── netiucv
    │   │   ├── netpacket
    │   │   ├── netrom
    │   │   ├── netrose
    │   │   ├── nfs
    │   │   ├── openssl
    │   │   ├── protocols
    │   │   ├── python3.10
    │   │   ├── rdma
    │   │   ├── readline
    │   │   ├── rpc
    │   │   ├── rpcsvc
    │   │   ├── scsi
    │   │   ├── sound
    │   │   ├── sys
    │   │   ├── tirpc
    │   │   ├── uuid
    │   │   ├── video
    │   │   └── xen
    │   ├── lib
    │   │   ├── cmake
    │   │   ├── gettext
    │   │   ├── gio
    │   │   ├── locale
    │   │   ├── microblazeel-xilinx-linux
    │   │   ├── ossl-modules
    │   │   ├── perl5
    │   │   ├── pkgconfig
    │   │   ├── pseudo
    │   │   ├── python3.10
    │   │   ├── rpm
    │   │   ├── ssl-3
    │   │   └── x86_64-petalinux-linux
    │   ├── libexec
    │   │   └── microblazeel-xilinx-linux
    │   ├── share
    │   └── var
    └── var

---

Donny Domagoj Cosic

Laboratory for Ion Beam Interactions
Ruđer Bošković Institute
Bijenička cesta 54 | HR-10 000 Zagreb | CROATIA
T: +385 1 4561 041 | @: dcosic at irb.hr | www.irb.hr

Dana 2025-01-07 00:50, Johnson, Andrew N. je napisao(la):

Hi Donny,

 

On 1/5/25, 8:16 AM, "Tech-talk" <tech-talk-bounces at aps.anl.gov> wrote:

 

I started playing with Petalinux on my KC705 board before the holidays
and successfully get Petalinux to boot. Now I am trying to cross compile
EPICS base to run it on the Microblaze but I am running into some
compile problems which are making me think I am doing something wrong.
Any advice or suggestion from your experience would be greatly
appreciated.

 

The existing linux-microblaze target configuration was created many years ago for some systems here at APS that are built using gcc-4.1.2 and followed the standard naming patterns for GCC installations. I haven't tried building EPICS 7 for that target recently but I don't think there have been any changes to EPICS which would have broken our setup. This is a 3-deep tree of the prefix directory that GNU_DIR points to in our installation:

 

microblazeel-unknown-linux-gnu/

├── bin

├── include

│   └── c++

│       └── 4.1.2

├── lib

│   └── gcc

│       └── microblazeel-unknown-linux-gnu

├── libexec

│   └── gcc

│       └── microblazeel-unknown-linux-gnu

├── microblazeel-unknown-linux-gnu

│   ├── bin

│   ├── debug-root

│   ├── lib -> .//sys-root/lib

│   └── sys-root

│       ├── etc

│       ├── lib

│       └── usr

└── share

 

19 directories

 

The top-level bin directory contains all the build tools such as microblazeel-unknown-linux-gnu-gcc, the bin directory under the inner target directory just has a subset of the shorter-named tools "ar as c++ g++ gcc ld nm objdump ranlib strip".

 

More recent Xilinx SDK tool-sets that we have installed have a slightly different internal layout but still work with EPICS when GNU_DIR is pointed at the equivalent prefix directory, e.g.:

 

aarch64-linux/

├── aarch64-linux-gnu

│   ├── bin

│   ├── include

│   ├── lib

│   ├── lib64

│   └── libc

├── bin

├── include

│   └── gdb

├── lib

│   ├── gcc

│   └── ldscripts

├── lib64

├── libexec

│   └── gcc

└── share

    ├── dejagnu

    ├── doc

    ├── gcc-7.3.1

    ├── gdb

    ├── info

    ├── locale

    └── man

 

23 directories

 

As before, the top-level bin directory contains all the build tools with long names such as aarch64-linux-gnu-gcc while the inner bin directory has a smaller subset such as ar and as, but interestingly that now excludes gcc and g++. The GNU_TARGET setting there is aarch64-linux-gnu.

 

I have the 2024.1 version of Petalinux along with the rest of the Xilinx
(AMD) tools installed on a Ubuntu 22.04.5 LTS. I used the provided BSP
which boots successfully and from it I generated the SDK which contains
the sysroots. Sysroots contains 2 folders:
"microblazeel-v11.0-bs-cmp-re-mh-div-fb-xilinx-linux" and
"x86_64-petalinux-linux". The "x86_64-petalinux-linux" folder contains
the compilers.

 

I don't know what your "I generated the SDK which contains the sysroots" step refers to, we normally just install the Xilinx SDK which provides everything, but I could believe that newer GCC versions might need some extra steps.

I edited the CONFIG_SITE in /configure
-> CROSS_COMPILER_TARGET_ARCHS=linux-microblaze

I edited the CONFIG_SITE.Common.linux-microblaze file in /configure/os
-> GNU_DIR =
/home/donny/Xilinx/KC705_Petalinux/images/linux/sdk/sysroots
-> GNU_TARGET=microblazeel-v11.0-bs-cmp-re-mh-div-fb-xilinx-linux

 

That looks like the wrong GNU_TARGET setting to me, it should be just the compiler prefix, i.e.

GNU_TARGET=microblazeel-xilinx-linux

 

It looks like your SDK layout is different to the standard cross-compiler layout that we used to get. If that's the case, you may have more work to get EPICS to cross-build. The EPICS file configure/CONFIG.CrossCommon was written for the standard layout that cross-gcc builds used to install, but your paths below look different, e.g. your <gnu-target>-gcc is installed into a <gnu-target> subdirectory of the bin directory. That's similar to where EPICS installs binaries that it builds, but we don't expect that of GCC installations.

 

Is there some other way you can generate/install the SDK with a more standard layout?

 

Please continue this conversation on tech-talk, others here have experience with this topic and might have ideas or suggestions.

 

- Andrew

 

When I try to compile the linux-x86_64 compiles without a problem
finding the compilers. But when compiling linux-microblaze it does not
find the compilers because there are not in the
"microblazeel-v11.0-bs-cmp-re-mh-div-fb-xilinx-linux" folder. If i
manually add the locations of the compilers in the
CONFIG_SITE.Common.linux-microblaze file:
CC =
/home/donny/Xilinx/KC705_Petalinux/images/linux/sdk/sysroots/x86_64-petalinux-linux/usr/bin/microblazeel-xilinx-linux/microblazeel-xilinx-linux-gcc
CCC =
/home/donny/Xilinx/KC705_Petalinux/images/linux/sdk/sysroots/x86_64-petalinux-linux/usr/bin/microblazeel-xilinx-linux/microblazeel-xilinx-linux-g++
LD =
home/donny/Xilinx/KC705_Petalinux/images/linux/sdk/sysroots/x86_64-petalinux-linux/usr/bin/microblazeel-xilinx-linux/microblazeel-xilinx-linux-ld
AR =
home/donny/Xilinx/KC705_Petalinux/images/linux/sdk/sysroots/x86_64-petalinux-linux/usr/bin/microblazeel-xilinx-linux/microblazeel-xilinx-linux-ar

When i try to make using this configurations i get the following error:
/home/donny/Xilinx/KC705_Petalinux/images/linux/sdk/sysroots/x86_64-petalinux-linux/usr/lib/microblazeel-xilinx-linux/gcc/microblazeel-xilinx-linux/12.2.0/include-fixed/limits.h:203:75:
error: no include path in which to search for limits.h

I think this is because it is using the wrong include folder, it should
be using the one from the microblazeel directory. If i edit
GNU_TARGET_INCLUDE_DIR and GNU_TARGET_LIB_DIR it compiles further but
then cannot find sub directories that are required. This makes me think
that this is not the proper way resolve this problem.

I hope someone has some experience with this as any advice would be very
welcome, I cannot think of anything else to try.

Regards,
--
Donny Domagoj Cosic

 

-- 

Complexity comes for free, Simplicity you have to work for.