buildiso fails with kernel 5.8.x because of missing nvidia packages

TN2k commented

2020-10-18 16:39:26 +00:00

(Migrated from gitlab2.manjaro.org)

Hi, running buildiso -f -p xfce -b stable -t ~/iso_out

results in

Conf File : /etc/pacman.conf
DB Path   : /var/lib/pacman/
Cache Dirs: /var/cache/pacman/pkg/  
Hook Dirs : /usr/share/libalpm/hooks/  /etc/pacman.d/hooks/  
Lock File : /var/lib/pacman/db.lck
Log File  : /var/log/pacman.log
GPG Dir   : /etc/pacman.d/gnupg/
Targets   : linux54-bbswitch  linux54-broadcom-wl  linux54-zfs  zfs-utils  linux54-virtualbox-guest-modules  linux54-headers  zfs-dkms  libva-intel-driver  libva-mesa-driver  libva-vdpau-driver  lib32-libva-vdpau-driver  libxaw  libxpm  libxvmc  mesa-vdpau  lib32-mesa-vdpau  lib32-libxvmc  lib32-primus  linux54-nvidia-455xx  nvidia-455xx-utils  lib32-nvidia-455xx-utils  opencl-mesa  open-vm-tools  primus  bumblebee  nvidia-prime  gtkmm3  spice-vdagent  virtualbox-guest-utils  vulkan-radeon  lib32-vulkan-radeon  vulkan-intel  lib32-vulkan-intel  xf86-input-vmmouse  xf86-video-amdgpu  xf86-video-ati  xf86-video-dummy  xf86-video-fbdev  xf86-video-intel  xf86-video-nouveau  xf86-video-openchrome  xf86-video-sisusb  xf86-video-vesa  xf86-video-vmware  xf86-video-voodoo
:: Synchronizing package databases...
downloading core.db...
downloading extra.db...
downloading community.db...
downloading multilib.db...
error: target not found: linux54-nvidia-455xx
error: target not found: nvidia-455xx-utils
error: target not found: lib32-nvidia-455xx-utils
error: target not found: linux54-nvidia-455xx
error: target not found: nvidia-455xx-utils
error: target not found: lib32-nvidia-455xx-utils
  -> Copying mhwd package cache ...
building file list ... done

sent 17 bytes  received 12 bytes  58.00 bytes/sec
total size is 0  speedup is 0.00
==> ERROR: File '/var/lib/manjaro-tools/buildiso/xfce/x86_64/mhwdfs/opt/mhwd/pkg/*pkg.tar*' not found.
==> No packages modified, nothing to do.
==> ERROR: A failure occurred in make_image_mhwd().
    Aborting...
 --> overlayfs umount: [/var/lib/manjaro-tools/buildiso/xfce/x86_64/mhwdfs]

Before the kernel update, everything worked. Can you update the script for the new packages?

Hi, running `buildiso -f -p xfce -b stable -t ~/iso_out` results in ```Root : / Conf File : /etc/pacman.conf DB Path : /var/lib/pacman/ Cache Dirs: /var/cache/pacman/pkg/ Hook Dirs : /usr/share/libalpm/hooks/ /etc/pacman.d/hooks/ Lock File : /var/lib/pacman/db.lck Log File : /var/log/pacman.log GPG Dir : /etc/pacman.d/gnupg/ Targets : linux54-bbswitch linux54-broadcom-wl linux54-zfs zfs-utils linux54-virtualbox-guest-modules linux54-headers zfs-dkms libva-intel-driver libva-mesa-driver libva-vdpau-driver lib32-libva-vdpau-driver libxaw libxpm libxvmc mesa-vdpau lib32-mesa-vdpau lib32-libxvmc lib32-primus linux54-nvidia-455xx nvidia-455xx-utils lib32-nvidia-455xx-utils opencl-mesa open-vm-tools primus bumblebee nvidia-prime gtkmm3 spice-vdagent virtualbox-guest-utils vulkan-radeon lib32-vulkan-radeon vulkan-intel lib32-vulkan-intel xf86-input-vmmouse xf86-video-amdgpu xf86-video-ati xf86-video-dummy xf86-video-fbdev xf86-video-intel xf86-video-nouveau xf86-video-openchrome xf86-video-sisusb xf86-video-vesa xf86-video-vmware xf86-video-voodoo :: Synchronizing package databases... downloading core.db... downloading extra.db... downloading community.db... downloading multilib.db... error: target not found: linux54-nvidia-455xx error: target not found: nvidia-455xx-utils error: target not found: lib32-nvidia-455xx-utils error: target not found: linux54-nvidia-455xx error: target not found: nvidia-455xx-utils error: target not found: lib32-nvidia-455xx-utils -> Copying mhwd package cache ... building file list ... done sent 17 bytes received 12 bytes 58.00 bytes/sec total size is 0 speedup is 0.00 ==> ERROR: File '/var/lib/manjaro-tools/buildiso/xfce/x86_64/mhwdfs/opt/mhwd/pkg/*pkg.tar*' not found. ==> No packages modified, nothing to do. ==> ERROR: A failure occurred in make_image_mhwd(). Aborting... --> overlayfs umount: [/var/lib/manjaro-tools/buildiso/xfce/x86_64/mhwdfs] ``` Before the kernel update, everything worked. Can you update the script for the new packages?

oberon commented

2020-10-18 18:13:42 +00:00

(Migrated from gitlab2.manjaro.org)

Please post output of
pacman -Q manjaro-tools-iso
and
buildiso -f -p xfce -b stable -t ~/iso_out -qv

Please post output of `pacman -Q manjaro-tools-iso` and `buildiso -f -p xfce -b stable -t ~/iso_out -qv`

oberon commented

2020-10-18 19:10:03 +00:00

(Migrated from gitlab2.manjaro.org)

nvidia-455xx modules indeed aren't even on stable branch, yet. Not sure how you run into this issue.
When I build against stable branch I get:

Targets   : linux54-bbswitch  linux54-broadcom-wl  linux54-zfs  zfs-utils  linux54-virtualbox-guest-modules  linux54-headers  zfs-dkms  libva-intel-driver  libva-mesa-driver  libva-vdpau-driver  lib32-libva-vdpau-driver  libxaw  libxpm  libxvmc  mesa-vdpau  lib32-mesa-vdpau  lib32-libxvmc  lib32-primus  linux54-nvidia-390xx  linux54-nvidia-450xx  nvidia-390xx-utils  nvidia-450xx-utils  lib32-nvidia-450xx-utils  lib32-nvidia-390xx-utils  opencl-mesa  open-vm-tools  primus  bumblebee  nvidia-prime  gtkmm3  spice-vdagent  virtualbox-guest-utils  vulkan-radeon  lib32-vulkan-radeon  vulkan-intel  lib32-vulkan-intel  xf86-input-vmmouse  xf86-video-amdgpu  xf86-video-ati  xf86-video-dummy  xf86-video-fbdev  xf86-video-intel  xf86-video-nouveau  xf86-video-openchrome  xf86-video-sisusb  xf86-video-vesa  xf86-video-vmware  xf86-video-voodoo

no nvidia-455xx there, like it should be.

**nvidia-455xx** modules indeed aren't even on stable branch, yet. Not sure how you run into this issue. When I build against stable branch I get: ``` Targets : linux54-bbswitch linux54-broadcom-wl linux54-zfs zfs-utils linux54-virtualbox-guest-modules linux54-headers zfs-dkms libva-intel-driver libva-mesa-driver libva-vdpau-driver lib32-libva-vdpau-driver libxaw libxpm libxvmc mesa-vdpau lib32-mesa-vdpau lib32-libxvmc lib32-primus linux54-nvidia-390xx linux54-nvidia-450xx nvidia-390xx-utils nvidia-450xx-utils lib32-nvidia-450xx-utils lib32-nvidia-390xx-utils opencl-mesa open-vm-tools primus bumblebee nvidia-prime gtkmm3 spice-vdagent virtualbox-guest-utils vulkan-radeon lib32-vulkan-radeon vulkan-intel lib32-vulkan-intel xf86-input-vmmouse xf86-video-amdgpu xf86-video-ati xf86-video-dummy xf86-video-fbdev xf86-video-intel xf86-video-nouveau xf86-video-openchrome xf86-video-sisusb xf86-video-vesa xf86-video-vmware xf86-video-voodoo ``` no nvidia-455xx there, like it should be.

TN2k commented

2020-10-18 19:29:07 +00:00

(Migrated from gitlab2.manjaro.org)

Thank for your reply, this is my output:
manjaro-tools-iso 0.15.10-1 for pacman -Q manjaro-tools-iso

and for buildiso -f -p xfce -b stable -t ~/iso_out -qv:

==> manjaro-tools
  -> version: 0.15.10
  -> config: /etc/manjaro-tools/manjaro-tools.conf
==> PROFILE:
  -> gitlab branch: master
  -> build_lists: community|default|manjaro
  -> build_list_iso: xfce
  -> is_build_list: false
==> OPTIONS:
  -> arch: x86_64
  -> branch: stable
  -> kernel: linux54
==> ARGS:
  -> clean_first: true
  -> images_only: false
  -> iso_only: false
  -> persist: false
  -> extra: true
  -> permalink: false
  -> office_installer: 
==> DIST SETTINGS:
  -> dist_name: Manjaro
  -> dist_release: 20.1.1
  -> dist_codename: Mikah
==> ISO INFO:
  -> iso_compression: zstd
==> BUILD QUEUE:
 --> Profile: [xfce]
  -> iso_file: manjaro-xfce-20.1.1-201019-linux54.iso
  -> iso_label: MANJARO_XFCE_2011
  -> autologin: true
  -> nonfree_mhwd: true
  -> multilib: true
  -> extra: true
  -> office_installer: false
  -> permalink: false
  -> netinstall: false
  -> chrootcfg: false
  -> geoip: true
  -> oem_used: false
  -> efi_boot_loader: grub
  -> custom_boot_args: 
  -> hostname: manjaro
  -> username: manjaro
  -> password: manjaro
  -> login_shell: /bin/bash
  -> addgroups: lp,network,power,sys,wheel
  -> enable_systemd: avahi-daemon bluetooth cronie ModemManager NetworkManager org.cups.cupsd tlp haveged ufw apparmor snapd.apparmor snapd fstrim.timer pkgfile-update.timer lightdm
  -> enable_systemd_live: manjaro-live mhwd-live pacman-init mirrors-live
  -> disable_systemd: pacman-init
  -> strict_snaps: 
  -> classic_snaps: 
  -> snap_channel: stable

I automated the whole process a bit (running all needed commands according to https://wiki.manjaro.org/index.php?title=Build_Manjaro_ISOs_with_buildiso), here is my script for that:

#!/bin/bash
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
WORKING_DIR=~/iso-profiles
XFCE_DIR=$WORKING_DIR/manjaro/xfce

echo "Updating...."
sudo pacman -Syu --noconfirm
sudo pacman -S $(tr '\n' ' ' < $SCRIPT_DIR/configs/base_packages) $(tr '\n' ' ' < $SCRIPT_DIR/configs/os_packages) --noconfirm

echo "Cloning directory...."
if [ -d "$WORKING_DIR" ]; then rm -Rf $WORKING_DIR; fi
git clone https://gitlab.manjaro.org/profiles-and-settings/iso-profiles.git ~/iso-profiles

echo "Copying files to edit..."
cp -r ~/iso-profiles/shared/manjaro/live-overlay/ ~/iso-profiles/manjaro/xfce/live-overlay/

echo "Creating directories and copying files for home dir"
mkdir -p $XFCE_DIR/live-overlay/etc/skel/
cp -r $SCRIPT_DIR/* $XFCE_DIR/live-overlay/etc/skel/

mkdir -p $XFCE_DIR/desktop-overlay/etc/skel/
cp -r $SCRIPT_DIR/* $XFCE_DIR/desktop-overlay/etc/skel/

echo "Appending packages..."
cat $SCRIPT_DIR/configs/iso_packages >> $XFCE_DIR/Packages-Desktop

echo "Building iso...."
if [ -d "~/iso_out" ]; then mkdir ~/iso_out; fi
LOG_FILE=~/iso_out/log/$(date -u +"%Y-%m-%dT%H-%M-%S").log
if [ ! -d "~/iso_out/log" ]; then mkdir ~/iso_out/log -p; fi
buildiso -f -p xfce -b stable -t ~/iso_out 2>&1 | tee -a $LOG_FILE
TARGET=$(\
        tail -n 20 $LOG_FILE \
        | grep "Writing to 'stdio:" \
        | cut -d "'" -f2 \
        | cut -d ":" -f2
)

echo
echo $TARGET

echo "Linking iso...."
rm ~/iso_out/out.iso
ln $TARGET ~/iso_out/out.iso

echo "~/iso_out/out.iso"
echo "generating hash..."

echo "Hash: $(sha512sum $TARGET | tee ~/iso_out/sha512hash.txt)"

echo "Log written to $LOG_FILE"
echo "DONE"

Thank for your reply, this is my output: `manjaro-tools-iso 0.15.10-1` for `pacman -Q manjaro-tools-iso` and for `buildiso -f -p xfce -b stable -t ~/iso_out -qv`: ```shell ==> manjaro-tools -> version: 0.15.10 -> config: /etc/manjaro-tools/manjaro-tools.conf ==> PROFILE: -> gitlab branch: master -> build_lists: community|default|manjaro -> build_list_iso: xfce -> is_build_list: false ==> OPTIONS: -> arch: x86_64 -> branch: stable -> kernel: linux54 ==> ARGS: -> clean_first: true -> images_only: false -> iso_only: false -> persist: false -> extra: true -> permalink: false -> office_installer: ==> DIST SETTINGS: -> dist_name: Manjaro -> dist_release: 20.1.1 -> dist_codename: Mikah ==> ISO INFO: -> iso_compression: zstd ==> BUILD QUEUE: --> Profile: [xfce] -> iso_file: manjaro-xfce-20.1.1-201019-linux54.iso -> iso_label: MANJARO_XFCE_2011 -> autologin: true -> nonfree_mhwd: true -> multilib: true -> extra: true -> office_installer: false -> permalink: false -> netinstall: false -> chrootcfg: false -> geoip: true -> oem_used: false -> efi_boot_loader: grub -> custom_boot_args: -> hostname: manjaro -> username: manjaro -> password: manjaro -> login_shell: /bin/bash -> addgroups: lp,network,power,sys,wheel -> enable_systemd: avahi-daemon bluetooth cronie ModemManager NetworkManager org.cups.cupsd tlp haveged ufw apparmor snapd.apparmor snapd fstrim.timer pkgfile-update.timer lightdm -> enable_systemd_live: manjaro-live mhwd-live pacman-init mirrors-live -> disable_systemd: pacman-init -> strict_snaps: -> classic_snaps: -> snap_channel: stable ``` I automated the whole process a bit (running all needed commands according to https://wiki.manjaro.org/index.php?title=Build_Manjaro_ISOs_with_buildiso), here is my script for that: ```shell #!/bin/bash SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" WORKING_DIR=~/iso-profiles XFCE_DIR=$WORKING_DIR/manjaro/xfce echo "Updating...." sudo pacman -Syu --noconfirm sudo pacman -S $(tr '\n' ' ' < $SCRIPT_DIR/configs/base_packages) $(tr '\n' ' ' < $SCRIPT_DIR/configs/os_packages) --noconfirm echo "Cloning directory...." if [ -d "$WORKING_DIR" ]; then rm -Rf $WORKING_DIR; fi git clone https://gitlab.manjaro.org/profiles-and-settings/iso-profiles.git ~/iso-profiles echo "Copying files to edit..." cp -r ~/iso-profiles/shared/manjaro/live-overlay/ ~/iso-profiles/manjaro/xfce/live-overlay/ echo "Creating directories and copying files for home dir" mkdir -p $XFCE_DIR/live-overlay/etc/skel/ cp -r $SCRIPT_DIR/* $XFCE_DIR/live-overlay/etc/skel/ mkdir -p $XFCE_DIR/desktop-overlay/etc/skel/ cp -r $SCRIPT_DIR/* $XFCE_DIR/desktop-overlay/etc/skel/ echo "Appending packages..." cat $SCRIPT_DIR/configs/iso_packages >> $XFCE_DIR/Packages-Desktop echo "Building iso...." if [ -d "~/iso_out" ]; then mkdir ~/iso_out; fi LOG_FILE=~/iso_out/log/$(date -u +"%Y-%m-%dT%H-%M-%S").log if [ ! -d "~/iso_out/log" ]; then mkdir ~/iso_out/log -p; fi buildiso -f -p xfce -b stable -t ~/iso_out 2>&1 | tee -a $LOG_FILE TARGET=$(\ tail -n 20 $LOG_FILE \ | grep "Writing to 'stdio:" \ | cut -d "'" -f2 \ | cut -d ":" -f2 ) echo echo $TARGET echo "Linking iso...." rm ~/iso_out/out.iso ln $TARGET ~/iso_out/out.iso echo "~/iso_out/out.iso" echo "generating hash..." echo "Hash: $(sha512sum $TARGET | tee ~/iso_out/sha512hash.txt)" echo "Log written to $LOG_FILE" echo "DONE" ```

philm commented

2020-10-18 19:43:36 +00:00

(Migrated from gitlab2.manjaro.org)

Stable branch doesn't have the 455 drivers yet. You may change the profiles back to a series supported by the branch. You may want to revert this locally on your end: gitlab.manjaro.org/profiles-and-settings/iso-profiles/-@c1bc88e34b

**Stable** branch doesn't have the 455 drivers yet. You may change the profiles back to a series supported by the branch. You may want to revert this locally on your end: https://gitlab.manjaro.org/profiles-and-settings/iso-profiles/-/commit/c1bc88e34b5612c22ddc1ec9a65c517e84e5aed9

TN2k commented

2020-10-18 19:48:27 +00:00

(Migrated from gitlab2.manjaro.org)

Do you have a clue, when I will be able to use the stable branch?
But thank you so far....

Do you have a clue, when I will be able to use the stable branch? But thank you so far....

oberon commented

2020-10-18 19:48:29 +00:00

(Migrated from gitlab2.manjaro.org)

hah. so building stable branch for me only worked because I didn't update iso-profiles first 😜
Alternatively you may also just build your image against testing branch, btw

hah. so building stable branch for me only worked because I didn't update iso-profiles first :stuck_out_tongue_winking_eye: Alternatively you may also just build your image against testing branch, btw

TN2k commented

2020-10-18 19:50:12 +00:00

(Migrated from gitlab2.manjaro.org)

No, it is ok, I will use your hash so long.... But what about stable? Will it be updated?

TN2k commented

2020-10-18 20:04:39 +00:00

(Migrated from gitlab2.manjaro.org)

Still having the issue with the missing packages, changed the following in my script:

echo "Cloning directory...."
if [ -d "$WORKING_DIR" ]; then rm -Rf $WORKING_DIR; fi
git clone https://gitlab.manjaro.org/profiles-and-settings/iso-profiles.git ~/iso-profiles
dir=$(pwd)
cd ~/iso_profiles
git checkout c1bc88e3
cd $dir

Still having the issue with the missing packages, changed the following in my script: ```shell echo "Cloning directory...." if [ -d "$WORKING_DIR" ]; then rm -Rf $WORKING_DIR; fi git clone https://gitlab.manjaro.org/profiles-and-settings/iso-profiles.git ~/iso-profiles dir=$(pwd) cd ~/iso_profiles git checkout c1bc88e3 cd $dir ```

philm commented

2020-10-18 21:07:27 +00:00

(Migrated from gitlab2.manjaro.org)

With iso-profiles we really don't focus on having a stable version, which works on stable branch. When we do development we use unstable or testing. So it changes more likely. Best would be to use the commits when we prepare a stable release. Everything behind that is again development.

So this might work: gitlab.manjaro.org/profiles-and-settings/iso-profiles/-@8c56062c04

With **iso-profiles** we really don't focus on having a stable version, which works on **stable** branch. When we do development we use **unstable** or **testing**. So it changes more likely. Best would be to use the commits when we prepare a stable release. Everything behind that is again development. So this might work: https://gitlab.manjaro.org/profiles-and-settings/iso-profiles/-/commit/8c56062c04b59d4077625aa60de295967b5da1a2

TN2k commented

2020-10-18 21:26:02 +00:00

(Migrated from gitlab2.manjaro.org)

Even with that hash, I still get the same issue.

TN2k commented

2020-10-18 21:55:10 +00:00

(Migrated from gitlab2.manjaro.org)

Ok, my bad, I misspelled my folder while using your hash.... Script is still running, I will edit this message, when it is done.
Is here a way to determine the last "stable" version (the hash of the commit)?

Ok, my bad, I misspelled my folder while using your hash.... Script is still running, I will edit this message, when it is done. Is here a way to determine the last "stable" version (the hash of the commit)?

philm commented

2020-10-18 22:42:44 +00:00

(Migrated from gitlab2.manjaro.org)

I might try to tag the releases ...

TN2k commented

2020-10-18 23:02:22 +00:00

(Migrated from gitlab2.manjaro.org)

Thank you, using your hash works....
Would be great, if you can tag your releases (or do something different to archive the same result)

Thank you, using your hash works.... Would be great, if you can tag your releases (or do something different to archive the same result)

a.ivanyk commented

2021-05-21 12:02:06 +00:00

(Migrated from gitlab2.manjaro.org)