Bug 1130

Summary: Build Debian SFFS
Product: Libre-SOC's first SoC Reporter: Toshaan Bharvani <toshaan>
Component: MilestonesAssignee: Sadoon Albader <sadoon>
Status: RESOLVED FIXED    
Severity: enhancement CC: libre-soc-bugs, lkcl, programmerjake
Priority: ---    
Version: unspecified   
Hardware: PC   
OS: Linux   
URL: https://libre-soc.org/SFFS/debian_bootstrap/
See Also: https://bugs.libre-soc.org/show_bug.cgi?id=1131
NLnet milestone: NLnet.2022-08-107.ongoing total budget (EUR) for completion of task and all subtasks: 2500
budget (EUR) for this task, excluding subtasks' budget: 2500 parent task for budget allocation: 999
child tasks for budget allocation: The table of payments (in EUR) for this task; TOML format:
sadoon={amount=1500, submitted=2024-02-26, paid=2024-03-08} lkcl = {amount=1000, submitted=2024-02-26, paid=2024-03-08}
Bug Depends on: 860    
Bug Blocks:    
Attachments: sbuild build script

Description Toshaan Bharvani 2023-08-02 20:44:03 BST
We are rebuilding Debian 12 (Bookworm) specifically as it is the most recent version and was just released two months ago.

This task consists not only of the work of rebuilding Debian, but also documentation (https://libre-soc.org/SFFS/debian_bootstrap), and automated scripts (https://git.libre-soc.org/?p=dev-env-setup.git) that guarantee full reproduciblity and only require one to have a POWER8 or newer system to rebuild everything from source using the scripts.
Comment 1 Sadoon Albader 2023-08-04 17:11:37 BST
Debian's build flags are stored in /etc/dpkg/buildflags.conf
This means when working with a schroot (like with sbuild), these flags specifically need to be enabled *inside* the chroot itself, not for the host machine executing sbuild.

cat /srv/chroot/bookworm-ppc64el-sbuild/etc/dpkg/buildflags.conf

APPEND CFLAGS -mcpu=power9 -mno-altivec -mno-vsx -mno-crypto -mno-htm -mlong-double-64
APPEND CXXFLAGS -mcpu=power9 -mno-altivec -mno-vsx -mno-crypto -mno-htm -mlong-double-64

This works according to my test in https://bugs.libre-soc.org/show_bug.cgi?id=1128#c8
Comment 2 Sadoon Albader 2023-08-08 14:44:25 BST
Created attachment 199 [details]
sbuild build script

Created attachment 199 [details]
sbuild build script

Build script for use with sbuild
Requires sbuild, a list of packages "list", an sbuild chroot (schroot) to be set up, and for our use case requires the chroot to have the dpkg-buildflags set as in the comment above.
Will report successful builds in "success" and failures in "failed"

Multiple instances can be run in parallel provided they don't build the same packages. Could be split up by first letter for example, and given different list files. All directories and files are set up as bash variables and the code is self-explanatory (the pkg:0:1 takes the first letter of the package)
Comment 3 Sadoon Albader 2023-08-08 14:46:58 BST
Placeholder for the second script which creates a repository of the built packages.
Comment 4 Sadoon Albader 2023-08-08 14:47:22 BST
Placeholder for the final script to create a debootstrap using these packages
Comment 5 Sadoon Albader 2023-08-08 20:38:31 BST
gpg might need soft floats, fails with -mlong-double-64
Comment 6 Luke Kenneth Casson Leighton 2023-08-23 17:10:40 BST
i've moved the page to a new location, covering "everything SFFS".
at some point an SFFS page itself is needed

https://git.libre-soc.org/?p=libreriscv.git;a=commit;h=6dd2bd753e0f1ebe971fdcd29e4fd371f4ccb672

also updated the wiki page itself to point to this bugreport,
*and* updated the URL of this bugreport to point back to the wiki.
this gives us cross-referencing that makes navigation a hell of
a lot easier.

https://libre-soc.org/irclog/%23libre-soc.2023-08-23.log.html#t2023-08-23T17:12:56

sadoon remember to update comment #0 with a 2-line description
then answer michiel that "yes the actual results may be seen on the
bugreport and verified that they have in fact been done, so you can
answer the EU Auditor and also satisfy NLnet's 'Works for the Public
Good' mandate" https://bugs.libre-soc.org/show_bug.cgi?id=961#c6
Comment 7 Luke Kenneth Casson Leighton 2023-08-25 13:53:14 BST
sadoon i've made some corrections/formatting

* 80 char limit,
* indentation missing,
* $OBJ_DUMP replaced with `$OBJ_DUMP`
* added limited liability on downloads

https://git.libre-soc.org/?p=libreriscv.git;a=commitdiff;h=892206dcaae487109730c2f7dd8dbd7d03f70db2
Comment 8 Sadoon Albader 2023-08-25 15:49:53 BST
glibc will fail to build if attempted with the current buildflags, there is no way around this without patching glibc itself, needs a separate task. We will use vanilla built glibc for now.
Comment 9 Luke Kenneth Casson Leighton 2023-08-25 15:55:29 BST
(In reply to Sadoon Albader from comment #8)
> glibc will fail to build if attempted with the current buildflags, there is
> no way around this without patching glibc itself, needs a separate task.

this is already done as part of NGI POINTER but Toshaan has not yet
made available the patches (which were supposed to have been
released in order to comply with the NGI POINTER conditions).
Comment 10 Luke Kenneth Casson Leighton 2023-08-29 10:57:00 BST
see ideas here for next update, to bootstrap up by using qemu/vsx or POWER9
and recompiling packages individually, natively, one-by-one.
(edit: *within the chroot*. this involves installing build-essential
*from standard debian-ppc64le* and then recompiling even those packages,
ultimately *ending up* with debian-ppc64lesffs but incrementally)

https://libre-soc.org/irclog/%23libre-soc.2023-08-27.log.html#t2023-08-27T10:20:07
Comment 11 Luke Kenneth Casson Leighton 2023-09-01 13:02:11 BST
uploaded the debian-sffs.tar.xz and linked it to the wiki with
the appropriate warning(s)
https://git.libre-soc.org/?p=libreriscv.git;a=commitdiff;h=a90476aec