Sparc port of an RPM-based distribution
For an example
1. First step is: do an aurora install -- we will use that as
the host for the build chroot; it has hte benefit of being in the native
target architecture of the destination distribution, as well, sowe do ont
have to first solve a cross-arch buildchain build.
ISOs are at: ftp://auroralinux.org/pub/aurora/build-1.0/en/iso
2. Second is set up an FTP server with the ISOs and the
update archive
updates are at: ftp://auroralinux.org/pub/aurora/updates/1.0
[there is a work in process at:
ftp://auroralinux.org/pub/aurora/corona ]
and I place a copy of the README at:
http://www.herrold.com/caos/corona-README.txt
(This document is NOT a competition, nor am I a competitor to 'Spot' --
I was pleased to buy him dinner when he came through town, and to invite
several other people from my local LUG to join us. A good time was had by
all.)
He notes:
- If you find a bug, thats great. Email aurora-sparc-devel@linuxpower.org,
and be specific about it. If you fix the bug, thats even better. Patches
happily accepted in unified format (diff -u).
-- good advice
3. Third is to pull and build a yum-1
http://linux.duke.edu/projects/yum/download/1.0/
11:01 < orc_emac> we are bootstrapping forward
Once we have produced the rpm-4.1.x or later rpm, we will update yum
to its version 2, which has a feature we need for building chroots using
yum
4. Fourth is to verify that updates work with yum - this entails
setting up an FTP server stocked with the base, and the updates for our host
distribution (outside of scope here, but straightforward),
mounting the ISOs -- described at:
http://www.owlriver.com/tips/pxe-install/
and its references to:
ftp://ftp.owlriver.com/pub/local/ORC/k12ltsp/install-from-iso.sh
in paragraph 6 - the mounting helper script is smart enough to automatically
'yum-ify' the ISOs as well
That mount of ISO process is simplified with the mentioned script, and a
helper application this something this simple:
[root@ftp ORC]# rm installCDrc
[root@ftp ORC]# ln -s installCDrc-aurora installCDrc
[root@ftp ORC]# locate aurora
/var/ftp/pub/mirror/aurora
/var/ftp/pub/mirror/aurora/iso
/var/ftp/pub/mirror/aurora/iso/aurora-1.0-sparc-disc1.iso
/var/ftp/pub/mirror/aurora/iso/aurora-1.0-sparc-disc2.iso
/var/ftp/pub/mirror/aurora/iso/aurora-1.0-sparc-disc3.iso
/var/ftp/pub/mirror/aurora/iso/aurora-1.0-sparc-disc4.iso
/var/ftp/pub/mirror/aurora/iso/MD5SUMS
/var/ftp/pub/mirror/redhat/ISO/aurora
/etc/ORC/installCDrc-aurora
[root@ftp ORC]# ls
installCDrc installCDrc-90 installCDrc-LTSPORCsetPXE-RPH-bak
installCDrc-73 installCDrc-9090 installCDrc-template
installCDrc-80 installCDrc-9092 ORCsetPXE
installCDrc-8094 installCDrc-aurora ORCsetPXEinitrd.img
[root@ftp ORC]# ./ORCsetPXE aurora
chirp
4 : 1 2 3 4 : mirror/aurora/iso : aurora-1.0-sparc-disc : .iso
Warning: could not find 0A-aurora - used 0A-PXETEST
[root@ftp ORC]# cat installCDrc-aurora
#
# RHL 7.3 - aurora
#
# /var/ftp/pub/mirror/aurora/iso/aurora-1.0-sparc-disc1.iso
#
CDS="4"
WHERE="mirror/aurora/iso"
FORM="aurora-1.0-sparc-disc"
SUFFIX=".iso"
#
[root@ftp ORC]#
Similarly the mirroring of the updates is trivial with lftp -- this config
file:
[root@ftp root]# cat lftp-aurora.conf
#
mirror -c \
ftp://auroralinux.org/pub/aurora/updates/1.0 \
/var/ftp/pub/mirror/aurora/updates/1.0
#
[root@ftp root]#
will retrieve them for you if run it thus:
lftp -f /root/lftp-aurora.conf
and we also can run a stack of several .conf files with our script:
ftp://ftp.owlriver.com/pub/local/ORC/ORCrebuild/ORC_lftp_mirror
cron is your friend for the ones containing updates; remember to run:
yum-arch .
appropiately
(I always name my ftp server in a given domain 'ftp.domain.tld' -- where
"domain.tld" is variable by site, and is referred to in the value handed out
in the dhcpd.conf to DHCP clients, and in the /etc/resolv.conf for staticly
assigned units:
[herrold@ftp etc]$ sudo grep domain dhcpd.conf
option domain-name-servers 10.16.1.253, 198.30.29.42 ;
option domain-name "first.lan";
[herrold@ftp etc]$
)
(I maintain on that FTP server, a hierarchy, of the form:
/var/ftp/pub/mirror/vendor/product/release/{ISO|updates|SRPMS}
so that finding items is easier)
5. Fifth is to try (and fail) with ORCyum-chroot-caos, duly
run with the option to use the local /etc/yum.conf
(by the way, my /etc/yum.conf looks like this, with that FTP setup:
[main]
cachedir=/var/cache/yum
debuglevel=2
logfile=/var/log/yum.log
pkgpolicy=newest
[base]
name=ISO base
baseurl=ftp://ftp/pub/loop/ftpinstall/
# [updates]
baseurl=ftp://ftp/pub/mirror/aurora/updates/
-- I have a fully functional interior DNS at all sites -- again, DNS setup
is _really_ out of scope here - by naming our FTP server 'ftp, and using
the non-FQDN, the same configuration files works several places)
That chroot builder script, in its most recent version is at:
ftp://ftp.owlriver.com/pub/local/ORC/ORCrebuild/ORCyum-chroot-caos
That script tests for a version of yum-2, because we need to have the
"--installroot=" option, which is not bacported to yum-1 yet. A volunteer
to do this could provide a great boon.
6. Sixth (which I am in process on) is to solve buildchain to
get rpm-4.1 onto aurora; this is not rocket science, but getting there is
somewhat convoluted; once that is done, as mentioned above, also pull, build
install and configure yum-2.
[6a. added 040313]
Being sensible, I looked about a bit. At ftp.rpm.org, there is a
solved rpm-4.1 for sparc, which may get us over that hump with much
less effort:
[root@ultra1 rpm-4.1.x]# pwd
/var/ftp/pub/mirror/rpm/dist/rpm-4.1.x/rpm-4.1.x
[root@ultra1 rpm-4.1.x]# rpm -Fvh *sparc*
error: failed dependencies:
libbz2.so.0 is needed by rpm-4.1-6x
libbz2.so.0 is needed by rpm-build-4.1-6x
libbz2.so.0 is needed by rpm-devel-4.1-6x
libbz2.so.0 is needed by rpm-python-4.1-6x
[root@ultra1 rpm-4.1.x]# locate libbz2
/usr/lib/libbz2.so.1.0.2
/usr/lib/libbz2.so.1
/usr/lib/libbz2.a
/usr/lib/libbz2.so
[root@ultra1 rpm-4.1.x]# rpm -qf /usr/lib/libbz2.so.1.0.2
bzip2-libs-1.0.2-2
[root@ultra1 rpm-4.1.x]# rpm -qi bzip2-libs
Name : bzip2-libs Relocations: (not relocateable)
Version : 1.0.2 Vendor: (none)
Release : 2 Build Date: Sun 21 Apr 2002 07:08:26 PM EDT
Install date: Tue 13 May 2003 03:07:06 PM EDT Build Host: ultra.linux.cz
Group : System Environment/Libraries Source RPM: bzip2-1.0.2-2.src.rpm
Size : 76920 License: BSD
URL : http://sources.redhat.com/bzip2/
Summary : Libraries for applications using bzip2
Description :
Libraries for applications using the bzip2 compression format.
[root@ultra1 rpm-4.1.x]# locate bzip2 | grep src.rpm
/var/ftp/pub/mirror/redhat/fedora/SRPMS/bzip2-1.0.2-12.1.src.rpm
/var/ftp/pub/mirror/aurora/corona/old/SRPMS/SRPMS/bzip2-1.0.2-8.src.rpm
[root@ultra1 rpm-4.1.x]#
So all that we may need to solve is getting bzip2 rebuilt to the
satisfaction of the unit, and we should be able to get the new RPM in.
There is a possible fly in the ointment, that the target for that rpm
version was the earlier RHL-6x series, which carries a much earlier
python and bzip series.
[6b.]
Nope -- the versions prevent us from being able to do a straightforward
rebuild of just one -- let's try a rebuild of that rpm src.rpm instead
[herrold@ultra1 rpm-4.1.x]$ sudo rpm -Fvh *sparc*
/home/herrold/rpmbuild/RPMS/sparc/bzip2-1.0.2-8.sparc.rpm
/home/herrold/rpmbuild/RPMS/sparc/bzip2-devel-1.0.2-8.sparc.rpm
/home/herrold/rpmbuild/RPMS/sparc/bzip2-libs-1.0.2-8.sparc.rpm
error: failed dependencies:
libbz2.so.0 is needed by rpm-4.1-6x
libbz2.so.0 is needed by rpm-build-4.1-6x
libbz2.so.0 is needed by rpm-devel-4.1-6x
libbz2.so.0 is needed by rpm-python-4.1-6x
[herrold@ultra1 rpm-4.1.x]$
Stay tuned.
7. Seventh with be to try and succeed with step 5
8. Once we have a way to make a clean build chroot, the work really
starts.
9. Mirror the SRPMs and hints from your chosen end target,
onto, say, your friendly local FTP server
10. Hand each SRPM in turn to an automated non-root, in pristine
chroot builder
see: ftp://ftp.owlriver.com/pub/local/ORC/ORCrebuild/ORCbuildit
for an example -- the section containing:
#
# now we build as non-root
chroot $BUILDROOT/$TEMP sudo -u builder rpmbuild $NODEPS \
-ba /usr/src/$DIST/SPECS/$j \
>> $BUILDROOT/$TEMP/usr/src/$DIST/$NAME-build.txt \
2>> $BUILDROOT/$TEMP/usr/src/$DIST/$NAME-error.txt || \
touch $BUILDROOT/$TEMP/usr/src/$DIST/$NAME-fail.txt
is building inside the chroot as non-root
11. Big hint: have several SRPM trees, and when moving forward, if
a given package needs LOTS of stuff unraded to build, consider approaching
that goal package by ging forward AS LITTLE AS POSSIBLE. That is -- part
of our method is that 'yum' is smart enough to observe dependeny interlocks,
and ignore content which cannot yet be installed; but it is also smart
enough to ignore stale intermediate build products when it should. Take
your time and just let it build everything in sight, and then upgrade the
build chroot and try again.
12. Lather, rinse, repeat. File reports and hand patches upstream.
13. Once you have a clean buildchain, kernel and bootloader build
going, do a chrooted instll of that, and solve the bootloader issues.
14. Test. We will need to do a complete rebuild from the new version of
all components, to make sure we get the proper glibc, and (possibly) kernel
threading model) in place.
15. Lather, rinse, repeat.
16. It seems obvious, but perhaps it is not; as new binaries are
yielded, it is perfectly possible to get them in use by moving them to the
FTP server, re-running yum.arch appropiately, and add a third stanza to
a new yum.conf file to use the new fruit in bootstrapping the build chroot
foward. Because we are test mode, however, it is not safe to add that
stanza to the main /etc/yum.conf upon which continued viability of the
chroot parent relies.
17. Why is it limited to a sparc port? It is also the case that
this script does not HAVE to be for sparc -- simply demonstrating that
centos-3 is self-hosting is a good 'warming up exercise'. Any RPM based
distribution with SRPMs available is a possible candidate; it is easiest
with an RPM version after the rpm-4.0.x series.
NOTICE AND DISCLAIMER:
=========================================================================
As with all development work, I expect to have to re-install at a moment's
notice, when something goes horribly wrong. Do not develop on production
hosts, or on a host which holds content you cannot afford to lose. If these
instructions cause damage or data loss, please don't bother telling me, as
they are provided 'for what it is worth' 'as is' and with NO WARRANTIES
WHATSOEVER.
----------------------------------
Reports of use, or with questions, or with enhancements are most welcome
to: info@owlriver.com
There is a mailing list at:
http://caosity.org/mailman/listinfo/caos-sparc
which is not limited to sparc alone, upon which this process is a
welcome topic for discussion.
The general topic of automated build systems is a welcome topic of
discussion at:
http://www.colug.net/mailman/listinfo/autobr
(but this has been a inactive list -- this may change)
I am very interested in reports on any arch not listed below:
i386
sparc
and particularly hope to see:
alpha
sparc64
parisc
ppc
arm41 (netwinder)
The master external copy of this document is at:
http://www.herrold.com/caos/frontporting/
040311 RPH initial
040311 RPH - add mention of ORC_lftp_mirror for easing mirror load,
./ORCsetPXE for easing ISO moount and PXE setup process, added
disclaimer
040311 RPH - expand on reasons for each step, clean up capitalization