[poky] [Errno 18] Invalid cross-device link warning coming from license file copy

[matti kaasinen]

Is there any way of getting rid of warnings of following type:

WARNING: Could not copy license file /YOCTO/poky/LICENSE to
/XXX/work/am335x_evm-poky-linux-gnueabi/xxxx-image/1.0-r0/license-destdir/xxx-image/LICENSE:
[Errno 18] Invalid cross-device link

Yes, sources and results are located on different logical drives, but
should it still be possible to copy files between them. Everything else
works fine in bitbaking.

MAIN-
Build Configuration:
BB_VERSION = "1.26.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "Ubuntu-14.04"
TARGET_SYS = "arm-poky-linux-gnueabi"
MACHINE = "am335x-evm"
DISTRO = "poky"
DISTRO_VERSION = "1.8"
TUNE_FEATURES = "arm armv7a vfp thumb neon callconvention-hard cortexa8"
TARGET_FPU = "vfp-neon"

Thanks,
Matti

[Burton, Ross]

Post by matti kaasinen
WARNING: Could not copy license file /YOCTO/poky/LICENSE to
[Errno 18] Invalid cross-device link
Yes, sources and results are located on different logical drives, but
should it still be possible to copy files between them. Everything else
works fine in bitbaking.

What release? I have my build and checkouts on different disks and I don't
get this warning.

Ross

[matti kaasinen]

Hi!
It is not that easy for me to get rid of these mount points on same drive
as they are made in /etc/fstab file. If I unmount any mountpoint, it does
not have any directory to show anymore. Therefore I need to make some
workaround. Really, system call stat does not recognize mount points on the
same drive. However, df command does recognize. Therefore I'm considering
workaround for this problem using df. Unfortunately it is required to use
pipe for getting results from df. I need to modify
if os.access(src, os.W_OK) and (os.stat(src).st_dev ==
os.stat(destdir).st_dev):
line in copy_license_files function as below:

if os.access(src, os.W_OK) and (os.popen('df -P ' + src).read().split()[11]
== (os.popen('df -P ' + destdir).read().split()[11] ):

I'm pretty sure it's much more slowly, but it makes the important
difference. I suppose this is so seldom problem that poky is not too eager
in patching
poky/meta/classes/license.bbclass with this line.

Is there any way overloading system methods with own methods like recipes
with bbappend? I have not found any, so most likely I just need to patch my
poky with this line locally that makes updates pretty awkward, don't I

Cheers,
Matti

Hi!
This warning/error is coming from copy_license_files (listed below)
#------------------------
import shutil
bb.utils.mkdirhier(destdir)
src = path
dst = os.path.join(destdir, basename)
os.remove(dst)
if os.access(src, os.W_OK) and (os.stat(src).st_dev ==
os.link(src, dst)
shutil.copyfile(src, dst)
bb.warn("Could not copy license file %s to %s: %s" % (src,
dst, e))
#-----------------------
if os.access(src, os.W_OK) and (os.stat(src).st_dev ==
-test is to find out if source directory and destination directories were
on different drives. In my case that seems kind of failing in that task.
"Kind of" because they are in different logical drives that are mounted on
the same physical drive.
Therefore, link is tried to make instead of copy. However, this fails as
those directories are on the different logical drives and "OSError: [Errno
18] Invalid cross-device link" is thrown.
I suppose, there is no other workaround to this than removing mountpoint
from this build directory and instead using it as normal direcotry.
Original idea was that it could have been mapped anywhere - for instance to
nfs drive.
Cheers,
Matti
By manually executing

Ross,
it is still unclear to me what release are you referring to. Could you
please pinpoint it to me.
Thanks,
Matti

I listed my main build configuration in my previous mail. What else do
you need. In fact I have not updated poky recently.
-Matti

Post by Burton, Ross

Post by matti kaasinen
WARNING: Could not copy license file /YOCTO/poky/LICENSE to
[Errno 18] Invalid cross-device link
Yes, sources and results are located on different logical drives, but
should it still be possible to copy files between them. Everything else
works fine in bitbaking.

What release? I have my build and checkouts on different disks and I
don't get this warning.
Ross

[Richard Purdie]

Post by matti kaasinen
WARNING: Could not copy license file /YOCTO/poky/LICENSE
to /XXX/work/am335x_evm-poky-linux-gnueabi/xxxx-image/1.0-r0/license-destdir/xxx-image/LICENSE: [Errno 18] Invalid cross-device link
Yes, sources and results are located on different logical drives, but
should it still be possible to copy files between them. Everything
else works fine in bitbaking.

Can you be a little more clear about what you did please? I assume you
changed TMPDIR to point to a different disk to the one where the poky
checkout is located?

I'm still a little puzzled about why the existing check doesn't catch
the fact they're different disks...

Cheers,

Richard


--

[matti kaasinen]

RIchard,


2015-08-25 17:12 GMT+03:00 Richard Purdie <

Post by matti kaasinen

Post by matti kaasinen
WARNING: Could not copy license file /YOCTO/poky/LICENSE
to

[Errno 18] Invalid cross-device link

Post by matti kaasinen
Yes, sources and results are located on different logical drives, but
should it still be possible to copy files between them. Everything
else works fine in bitbaking.

Can you be a little more clear about what you did please? I assume you
changed TMPDIR to point to a different disk to the one where the poky
checkout is located?

Yes, you are absolutely right.
I have a pretty small (120G) ssd disk I have installed my Ubuntu machine
(/dev/sdb2). I try not to pollute it with checkouts and in particular
builds. For that purpose I have reserved one partition from pretty moderate
(1TB) hdd. Both checkouts (I have played with some distros) and TMPDIR live
in there, but in different mount points. In fact, I do have also a mount
point called /TMPDIR there.

Post by matti kaasinen
I'm still a little puzzled about why the existing check doesn't catch
the fact they're different disks...

Let's study how it works, first the mounts.
As I told before, both the TMPDIR and checkouts (/YOCTO) live in /dev/sdb2
that has been mounted in /mnt/sw. Let's see how they look in df and python
os.stat listings.

==== df shell command
## /mnt is in ssd disk
$ LANG=C df -H /mnt
Filesystem Size Used Avail Use% Mounted on
/dev/sdb2 110G 24G 80G 23% /

## sda2 partition of hdd disk mounted to /mnt/sw
$ LANG=C df -H /mnt/sw
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 572G 148G 395G 28% /mnt/sw

## same results by using /mnt/sw path
$ LANG=C df -H /mnt/sw/TMPDIR
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 572G 148G 395G 28% /mnt/sw

## same results again - in fact as path used same mountpoint /mnt/sw
$ LANG=C df -H /mnt/sw/YOCTO
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 572G 148G 395G 28% /mnt/sw

## let's test mountpoints /TMPDIR and /YOCTO - this is what
copy_license_file tests:

$ LANG=C df -H /TMPDIR
Filesystem Size Used Avail Use% Mounted on
/mnt/sw/TMPDIR 572G 148G 395G 28% /TMPDIR

$ LANG=C df -H /YOCTO
Filesystem Size Used Avail Use% Mounted on
/mnt/sw/YOCTO 572G 148G 395G 28% /YOCTO

## It made difference both in "Filesystem" and "Mounted on" columns
## Let's make the test more close what copy_license_files function tests:

$ LANG=C df -H /YOCTO/poky /TMPDIR/work
Filesystem Size Used Avail Use% Mounted on
/mnt/sw/YOCTO 572G 148G 395G 28% /YOCTO
/mnt/sw/TMPDIR 572G 148G 395G 28% /TMPDIR

==== stat test using ipython
In [9]: os.stat('/mnt').st_dev
Out[9]: 2066L

In [10]: os.stat('/mnt/sw').st_dev
Out[10]: 2050L

In [11]: os.stat('/mnt/sw/TMPDIR/work').st_dev
Out[11]: 2050L

In [12]: os.stat('/mnt/sw/YOCTO/poky').st_dev
Out[12]: 2050L

In [13]: os.stat('/YOCTO/poky').st_dev
Out[13]: 2050L

In [14]: os.stat('/TMPDIR/work').st_dev
Out[14]: 2050L

## Above python tests show that all the tests except os.stat('/mnt').st_dev
return /dev/sda2 mount point number (I guess).

I hope you can get more clear picture of my mounts and how different kind
of tests work.
Cheers,
Matti

[matti kaasinen]

So, it seems that I have following options to choose:
1) change TMPDIR value to point to /mnt/sw/TMPDIR (portable, but config
file templates require editing for next installers).
This would require also bblayers.conf file editing with /mnt/sw
modifications.
2) move checkouts to my ssd: checkouts are not that big - possible.
3) patch copy_license_files function: easy and working, but makes updates
awkward.
4) wait for poky to patch copy_license_files function - won't happen I
suppose?
5) do nothing and live with all those warnings - most likely to happen.
Cheers,
Matti

Post by matti kaasinen
RIchard,
2015-08-25 17:12 GMT+03:00 Richard Purdie <

Post by matti kaasinen

Post by matti kaasinen
WARNING: Could not copy license file /YOCTO/poky/LICENSE
to

[Errno 18] Invalid cross-device link

Post by matti kaasinen
Yes, sources and results are located on different logical drives, but
should it still be possible to copy files between them. Everything
else works fine in bitbaking.

Can you be a little more clear about what you did please? I assume you
changed TMPDIR to point to a different disk to the one where the poky
checkout is located?

Yes, you are absolutely right.
I have a pretty small (120G) ssd disk I have installed my Ubuntu machine
(/dev/sdb2). I try not to pollute it with checkouts and in particular
builds. For that purpose I have reserved one partition from pretty moderate
(1TB) hdd. Both checkouts (I have played with some distros) and TMPDIR live
in there, but in different mount points. In fact, I do have also a mount
point called /TMPDIR there.

Post by matti kaasinen
I'm still a little puzzled about why the existing check doesn't catch
the fact they're different disks...

Let's study how it works, first the mounts.
As I told before, both the TMPDIR and checkouts (/YOCTO) live in /dev/sdb2
that has been mounted in /mnt/sw. Let's see how they look in df and python
os.stat listings.
==== df shell command
## /mnt is in ssd disk
$ LANG=C df -H /mnt
Filesystem Size Used Avail Use% Mounted on
/dev/sdb2 110G 24G 80G 23% /
## sda2 partition of hdd disk mounted to /mnt/sw
$ LANG=C df -H /mnt/sw
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 572G 148G 395G 28% /mnt/sw
## same results by using /mnt/sw path
$ LANG=C df -H /mnt/sw/TMPDIR
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 572G 148G 395G 28% /mnt/sw
## same results again - in fact as path used same mountpoint /mnt/sw
$ LANG=C df -H /mnt/sw/YOCTO
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 572G 148G 395G 28% /mnt/sw
## let's test mountpoints /TMPDIR and /YOCTO - this is what
$ LANG=C df -H /TMPDIR
Filesystem Size Used Avail Use% Mounted on
/mnt/sw/TMPDIR 572G 148G 395G 28% /TMPDIR
$ LANG=C df -H /YOCTO
Filesystem Size Used Avail Use% Mounted on
/mnt/sw/YOCTO 572G 148G 395G 28% /YOCTO
## It made difference both in "Filesystem" and "Mounted on" columns
$ LANG=C df -H /YOCTO/poky /TMPDIR/work
Filesystem Size Used Avail Use% Mounted on
/mnt/sw/YOCTO 572G 148G 395G 28% /YOCTO
/mnt/sw/TMPDIR 572G 148G 395G 28% /TMPDIR
==== stat test using ipython
In [9]: os.stat('/mnt').st_dev
Out[9]: 2066L
In [10]: os.stat('/mnt/sw').st_dev
Out[10]: 2050L
In [11]: os.stat('/mnt/sw/TMPDIR/work').st_dev
Out[11]: 2050L
In [12]: os.stat('/mnt/sw/YOCTO/poky').st_dev
Out[12]: 2050L
In [13]: os.stat('/YOCTO/poky').st_dev
Out[13]: 2050L
In [14]: os.stat('/TMPDIR/work').st_dev
Out[14]: 2050L
## Above python tests show that all the tests except
os.stat('/mnt').st_dev return /dev/sda2 mount point number (I guess).
I hope you can get more clear picture of my mounts and how different kind
of tests work.
Cheers,
Matti