Host /etc/pacman.d/mirrorlist is used for any branch #96
Labels
No labels
Done
bug
buildiso
buildpkg
buildtree
critical
deployiso
duplicate
enhancement
help wanted
in progress
invalid
manjaro-chroot
note
old-not-relevant
optional
question
sonar
todo
wontfix
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
tools/manjaro-tools#96
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Created by: philmmanjaro
As I'm running mostly unstable on my host I expect to pull stable package into a stable chroot instead of unstable packages.
pacman.log from stable chroot:
Current boxit compare snippet:
:: Different sync package(s) in repository core i686 ------------------------------------------------------------------------------- PACKAGE stable unstable ------------------------------------------------------------------------------- archlinux-keyring 20150212-1 20150523-1 bash 4.3.033-1 4.3.039-1 binutils 2.25-2 2.25-5 btrfs-progs 4.0-1 4.0-2 ca-certificates-mozilla 3.18-3 3.18.1-1 dhcpcd 6.8.2-1 6.9.0-1 dialog 1:1.2_20150225-1 1:1.2_20150513-1 gcc 4.9.2-4 5.1.0-4 gcc-ada 4.9.2-4 5.1.0-4 gcc-fortran 4.9.2-4 5.1.0-4 gcc-go 4.9.2-4 5.1.0-4 gcc-libs 4.9.2-4 5.1.0-4 gcc-objc 4.9.2-4 5.1.0-4Workaround is now to switch to stable branch before building packages:
Created by: udeved
I can't repoduce.
pacman.conf in chroot doesn't contain my commented arch repos of host pacman.conf.
Also mirrorlist points to the right branch.
Hmm.
host:
chroot:
Created by: philmmanjaro
Please change your host to unstable branch and create a new chroot. You'll pull packages from unstable. Check pacman.log from chroot and grep for gcc.
Created by: philmmanjaro
I should have said: uses /etc/pacman.d/mirrorlist from host ...
Created by: udeved
Hmm, the host mirrorlist is set to unstable, I just did a stable openrc test build.
The mirrorlist in chroot is set to stable and uses netzspielplatz as the only mirror.
Created by: philmmanjaro
Well, we might need some sed-magic:
sed -i -e 's|{stable,testing,unstable}|${branch}|g' /chroot/mirrorlistCreated by: philmmanjaro
@udeved: which gcc got installed?
Created by: udeved
gcc-libs-5.1.0-4
hmmm...
If I follow the calls, buildpkg-->mkchroot(with buildmirror arg)-->chroot-run(using buildmirror)
Created by: philmmanjaro
Well, this means it pulls the host/mirrorlist and then when pacman-mirrors kicks in you have your branched mirrorlist. So we have to find a way to force the wanted branch in that mirrorlist.
Created by: udeved
The mirrorlist for stable chroot:
I guess we somehow need to either make basesstrap branch aware, or remove basestrap and replace it with something new.
Created by: philmmanjaro
We call pacman with r-switch and point to a pacman.conf using the host/mirrorlist:
I think there we should change it. I think Include = /etc/pacman.d/mirrorlist is here the wrong way to go. We might rather change it to:
... and change it back later.Created by: udeved
I think its a problem with basestrap, and the order how we provide the conf files is wrong.
I think basestrap overwrites the pacman.conf, it escaped me that it is set true by default.
Created by: philmmanjaro
Ok, then we have to check there ...
Created by: udeved
I think I managed to manipulate pacman.conf according to @philmmanjaro idea.
Before chroot is initialized, pacman-default/multilib.conf will have the core Server build_mirror set to branch.
After chroot init, the pacman.conf in the chroot/root will be set, ie server line removed and Include added.
We need this for buildiso too.
Created by: udeved
ToDo:
Created by: philmmanjaro
I'm fine with that ...
Created by: udeved
Could you take a boxit diff if you have time to confim its working?
gcc-5 is now also in stable afaik.
Created by: philmmanjaro
Sure, we can check it on cmake:
PACKAGE stable unstable ------------------------------------------------------------------------------- cmake 3.2.2-1 3.2.3-1Created by: udeved
Hmm, we also need to set extra repo, cmake is in extra...
Created by: philmmanjaro
currently there is no difference in [core] ...
Created by: udeved
urgs...
I am still confused why the build throws an error with stable config.
The pacman extra db seem to be from unstable, but chroot is stable.
Ie, we need to set for every repo the server instead of include.
Created by: philmmanjaro
Yep. Like the musketiers ;)
Created by: udeved
Back to the idea using pac_conf parser...
Created by: udeved
There is currently no way to build against stable branch from unstable host without setting the host branch too.
I tried with all repo servers, and the build will fail with cmake download. It attemptes to get cmake unstable, but mirror is set to stable, due to the sync db.
In other words, @philmmanjaro idea doesn't work, the sync db before chroot is created are downloaded from host mirror to chroot.
To solve this, we would need to be able to pass a mirrorlist to pacman, or somehow make it work with pacman-mirrors.conf
Could we downgrade the chroot after it was initialized? Ugly idea.
Created by: Kirek
I don't understand this messing around the pacman.conf changin the /etc/pacman.d/mirrorlist line, you only need to create a mirrorlist with a single line (Server = mirror...) and you are done :/
So
Like devtools... https://github.com/manjaro/devtools/blob/master/arch-nspawn.in#L58
Or I'm missing something?
Created by: udeved
Yws, you miss something.
basestrap runs pacman to create initial chroot.
The sysc db in the chroot to create initial base-devel chroot will be downloaded from host mirrorlist.
There is the problem, we have a mismatch of the chroot sync db with the chroot config files.
Eg sync db is from unstable, but all conf points to stable in chroot. Setting a Server url in pacman.conf to create the initial chroot doesn't help, we still have the sync db mismatch.
devtools and manjaroiso did have same problem.
Created by: Kirek
Are you saying that basestrap uses the host pacman.conf? basestrap has the -C option that should point to the chroot pacman.conf (actually you invoke mkchroot with the -C, that passes it to basestrap -C, that passes to pacman --config)
Created by: udeved
Yes, I know, we do this.
Just try out the functions in util.
It is not enough to manipulate the pacman.conf which is used to create the chroot.
As said, I tested this, and builds will fails, for example downloading cmake from stable, because the sync db in the chroot are from unstable, like the host is set. Hence build fails, because it attempts to download cmake higher version(unstable)
Look, before the base-devel group get installed, pacman downloads the sync db in the chroot, and will then use it to pull base-devel.
It appears to solve this, pacman would have to accept a mirrorlist passed as arg, additionally to pacman.conf.
New idea is to make pacman also use local sync in basestrap and manipulaze pacman.conf again.
Created by: Kirek
Okay I think that finally I understand the problem.
Basestrap run pacman outside the chroot, and we pass a pacman.conf, problem is that pacman doesn't know anything about branches, neither pacman.conf says what branch it is. So, basestrap install the initial system in the host branch.
First solution:
Second solution:
we modify pacman to learn about branches add we add a new option in pacman.conf called "Branch=" (this has other implications outside manjaro-tools though)
Created by: Kirek
Fixed it in devtools, this bug has been there since forever...
github.com/manjaro/devtools@6841969076With manjaro-tools should be the same but taking into account the -B option
The workaround is ugly, but unless we change pacman I don't think there is another way.
Created by: udeved
Not fixed.
Your approach is basically the same as the one here.
You just write a temporary pac_conf, I manipulated the pacman.conf in /usr/share/manjaro-tools.
The result is the same, build fails, in my test case sddm-consolekit, just tested with your temporary pac_conf file in mkchroot too.
$ buildpkg -p sddm-consolekit -c -b stable
Again, pacman seems to download from the host mirrorlist the initial pacman sync db files to chroot/var/lib/pacman/sync, before any package will be installed in chroot.
It should pull cmake-3.2.2 from stable, but the extra.db is from unstable like host system.
The build won't fail if I set host to stable too.
Created by: Kirek
Well my patch this fixes the initial sync, because basestrap is the initial sync, when nothing is in the chroot.
If after that it tries to retrieve anything using the host mirrors it could mean: the conf files in the chroot are wrong or it's running a command outside the chroot...
Created by: udeved
That's the code I used in mkchroot.
Is there something wrong with the remote repo db?
Created by: Kirek
Got it
github.com/manjaro/manjaro-tools@780a0c090ethat's why all the fixes never worked, we never passed our custom pacman-*.conf...Created by: udeved
Ok, very good.
Now, do we want a temp pac_conf of do we want to manipulate /usr/share/manjaro-tools?
See initialize_branch(). in util.sh
We would have to deploy pacman-arch.conf with the buiuldmirror.It would be then fixed in the chroot to set Include statement.
Created by: Kirek
We need a temp pac_conf, the one in /usr/share/manjaro-tools should never be modified
I don't get what are you trying to do with initialize_branch, but chroot-run is where we setup the conf files.
Created by: udeved
The functions for modifying pacman.conf in /usr/share/manjaro-tools already exist. Look at util.sh.
We can directly modify these files without temp file.
Created by: Kirek
You don't understand, the files in /usr should be kept alone, we NEVER should modify them.
In fact if you look at my patch, I pass to basestrap the modified .conf, but to chroot-run just the normal .conf in /usr/share/manjaro-tools...
Created by: udeved
I know, I tested your approach too.
I was just asking which approach we take.