doc:howto:tce
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
doc:howto:tce [2019/11/14 00:37] stefanbaur [Configuring the Build] |
doc:howto:tce [2021/01/25 00:19] stefanbaur [Starting the Build] This is part two of the patch to create Images for the Raspberry Pi |
||
|---|---|---|---|
| Line 44: | Line 44: | ||
| # we have a successful connect on port 3142, assume | # we have a successful connect on port 3142, assume | ||
| # it's apt-cacher-ng and use it | # it's apt-cacher-ng and use it | ||
| - | + | # | |
| - | if nc -z 127.0.0.1 3142 ; then | + | if nc -z 127.0.0.1 3142 ; then |
| # bad idea with apt-cacher-ng, | # bad idea with apt-cacher-ng, | ||
| - | # export https_proxy=http:// | + | # export https_proxy=http:// |
| # export http_proxy=http:// | # export http_proxy=http:// | ||
| # export ftp_proxy=http:// | # export ftp_proxy=http:// | ||
| Line 54: | Line 54: | ||
| export LB_APT_HTTP_PROXY=http:// | export LB_APT_HTTP_PROXY=http:// | ||
| fi | fi | ||
| + | |||
| + | # set these to true to save source files | ||
| + | #export LB_SOURCE=" | ||
| + | #export LBX2GO_GETSRC=" | ||
| # Select ONE of the following git reposities | # Select ONE of the following git reposities | ||
| Line 71: | Line 75: | ||
| # 32-Bit, smallest memory footprint - not available on buster | # 32-Bit, smallest memory footprint - not available on buster | ||
| # export LBX2GO_ARCH=' | # export LBX2GO_ARCH=' | ||
| - | # EXPERIMENTAL: | + | # For ARM (Raspberry Pi): |
| - | # export LBX2GO_ARCH=' | + | #export LBX2GO_ARCH=' |
| - | # Note that ARM builds are currently | + | #export LBX2GO_ARCH_MODEL=' |
| # If you want to use the stock ISO image as created by this script, add your boot parameters here | # If you want to use the stock ISO image as created by this script, add your boot parameters here | ||
| # export LBX2GO_BOOTAPPEND_LIVE=" | # export LBX2GO_BOOTAPPEND_LIVE=" | ||
| + | |||
| + | if echo -e " | ||
| + | LBX2GO_BOOTAPPEND_LIVE+=" | ||
| + | export LBX2GO_BOOTAPPEND_LIVE | ||
| + | elif echo -e " | ||
| + | LBX2GO_BOOTAPPEND_LIVE+=' | ||
| + | export LBX2GO_BOOTAPPEND_LIVE | ||
| + | fi | ||
| # detect if the selected git repo is meant to build a buster, stretch or jessie image | # detect if the selected git repo is meant to build a buster, stretch or jessie image | ||
| if [ -z " | if [ -z " | ||
| export LBX2GO_DEBVERSION=" | export LBX2GO_DEBVERSION=" | ||
| + | export LBX2GO_BOOTAPPEND_LIVE+=" | ||
| + | elif [ -z " | ||
| + | export LBX2GO_DEBVERSION=" | ||
| + | export LBX2GO_BOOTAPPEND_LIVE+=" | ||
| + | elif [ -z " | ||
| + | export LBX2GO_DEBVERSION=" | ||
| export LBX2GO_BOOTAPPEND_LIVE+=" | export LBX2GO_BOOTAPPEND_LIVE+=" | ||
| elif [ -z " | elif [ -z " | ||
| Line 97: | Line 115: | ||
| # set boot loader type - leave this unchanged unless you really know what you're doing | # set boot loader type - leave this unchanged unless you really know what you're doing | ||
| - | if echo "$LBX2GO_ARCH" | + | if echo $LBX2GO_ARCH | awk ' |
| # This is part of our experimental ARM support | # This is part of our experimental ARM support | ||
| LBX2GO_BOOTLOADERPARAMNAME=" | LBX2GO_BOOTLOADERPARAMNAME=" | ||
| Line 138: | Line 156: | ||
| # This is part of our experimental ARM support | # This is part of our experimental ARM support | ||
| - | if echo "$LBX2GO_ARCH" | + | if echo $LBX2GO_ARCH |
| export LBX2GO_DEFAULTS+=" | export LBX2GO_DEFAULTS+=" | ||
| - | --bootstrap-qemu-static / | + | --bootstrap-qemu-static / |
| + | --apt-options \" | ||
| + | fi | ||
| + | |||
| + | # This is part of our experimental ARM support | ||
| + | # This makes sure the resulting disk image is at least 1GB in size, even though our build currently requires way less. | ||
| + | # It's unlikely that anyone will need to boot from a smaller partition; but if we let live-build pick the minimum size automatically, | ||
| + | # we will not have enough space left to copy the firmware blobs into the right location. | ||
| + | if echo $LBX2GO_ARCH | grep -q ' | ||
| + | export LBX2GO_DEFAULTS+=" | ||
| + | --hdd-size 1024" | ||
| fi | fi | ||
| Line 150: | Line 178: | ||
| # This is to optimize squashfs size, based on a suggestion by intrigeri from the TAILS team | # This is to optimize squashfs size, based on a suggestion by intrigeri from the TAILS team | ||
| # note that this will permanently change / | # note that this will permanently change / | ||
| - | sed -i -e ' | + | if dpkg --print-architecture | grep -q ' |
| + | # on arm, these parameters must not be used; if they' | ||
| + | if grep -q -- '-Xbcj x86 -b 1024K -Xdict-size 1024K' / | ||
| + | apt install --reinstall live-build | ||
| + | fi | ||
| + | else | ||
| + | | ||
| + | fi | ||
| # This removes documentation, | # This removes documentation, | ||
| Line 159: | Line 194: | ||
| # This patches the squashfs file into the initrd. Only parsed when image type " | # This patches the squashfs file into the initrd. Only parsed when image type " | ||
| # Will require boot parameter live-media=/ | # Will require boot parameter live-media=/ | ||
| - | # Both TFTP client and TFTP server must support file transfers >32MB for this to work, if you want to deploy this initrd via TFTP, | + | # Both TFTP client and TFTP server must support file transfers >32MB for this to work, if you want to deploy this initrd via TFTP, |
| # so e.g. atftpd will not work - tftpd-hpa, however, seems to have no problem with larger files. | # so e.g. atftpd will not work - tftpd-hpa, however, seems to have no problem with larger files. | ||
| # When using iPXE, you can use http instead of TFTP. | # When using iPXE, you can use http instead of TFTP. | ||
| Line 174: | Line 209: | ||
| # /!\ the options below are NOT RECOMMENDED unless you use live-build from Debian Buster /!\ | # /!\ the options below are NOT RECOMMENDED unless you use live-build from Debian Buster /!\ | ||
| # to create an image that can be written to a hard disk (for older live-build versions, this | # to create an image that can be written to a hard disk (for older live-build versions, this | ||
| - | # always results in a "build failed" | + | # always results in a "build failed" |
| # live-build from Buster and things will work): | # live-build from Buster and things will work): | ||
| # export LBX2GO_IMAGETYPE=' | # export LBX2GO_IMAGETYPE=' | ||
| # to create a tar file only (seems to be broken in older live-build versions - Buster works): | # to create a tar file only (seems to be broken in older live-build versions - Buster works): | ||
| # export LBX2GO_IMAGETYPE=' | # export LBX2GO_IMAGETYPE=' | ||
| + | |||
| + | # This is part of our experimental ARM support | ||
| + | if echo " | ||
| + | # enforce hdd image for arm at the moment (might need to support netboot later on too) | ||
| + | if ! [ " | ||
| + | echo " | ||
| + | LBX2GO_IMAGETYPE=" | ||
| + | fi | ||
| + | fi | ||
| if [ " | if [ " | ||
| Line 206: | Line 250: | ||
| # Create Timestamp | # Create Timestamp | ||
| LBX2GO_TIMESTAMP=$(date +" | LBX2GO_TIMESTAMP=$(date +" | ||
| + | |||
| + | exec > >(tee "/ | ||
| # Set Directory name | # Set Directory name | ||
| - | LBX2GO_TCEDIR=" | + | LBX2GO_TCEDIR=" |
| if [ -z " | if [ -z " | ||
| + | ( echo " | ||
| [ -z " | [ -z " | ||
| [ -z " | [ -z " | ||
| Line 222: | Line 269: | ||
| echo -e "One or more of the following variables is unset:" | echo -e "One or more of the following variables is unset:" | ||
| echo -e " | echo -e " | ||
| + | echo " | ||
| echo -e " | echo -e " | ||
| echo -e " | echo -e " | ||
| Line 238: | Line 286: | ||
| cd $LBX2GO_TCEDIR | cd $LBX2GO_TCEDIR | ||
| + | X2GO_LBCONFIG_STRING=$(cat << | ||
| lb config $LBX2GO_ARCH $LBX2GO_SPACE $LBX2GO_DEFAULTS \ | lb config $LBX2GO_ARCH $LBX2GO_SPACE $LBX2GO_DEFAULTS \ | ||
| | | ||
| | | ||
| | | ||
| + | X2GOLBCONFIGSTRING | ||
| + | ) | ||
| + | # Our previous way of doing this had issues with newlines and multiple blanks. So we're now doing a bit | ||
| + | # of sanitizing, then we eval the variable. | ||
| + | X2GO_LBCONFIG_STRING=$(echo " | ||
| + | eval " | ||
| + | |||
| # This will copy any patches we have prepared | # This will copy any patches we have prepared | ||
| if [ -d " | if [ -d " | ||
| cp -a ../patch/* config/ | cp -a ../patch/* config/ | ||
| fi | fi | ||
| + | |||
| # This will copy any patches we have prepared for minidesktop | # This will copy any patches we have prepared for minidesktop | ||
| if [ -d " | if [ -d " | ||
| cp -a ../ | cp -a ../ | ||
| fi | fi | ||
| + | |||
| + | # This checks if a bootloader directory is present (e.g. because of a custom splash.svg) | ||
| + | # and adds all other files that might be missing (live-build won't add them automatically | ||
| + | # if the directory already exists) | ||
| + | if [ -d config/ | ||
| + | rsync -aPH --ignore-existing --exclude=" | ||
| + | fi | ||
| + | # When enabled, this silences the audible beep at syslinux/ | ||
| + | # Note that this is an accessibility feature for blind users, so use with care. | ||
| + | sed -e " | ||
| + | |||
| # This enables an i386-only package in the sources.list file when an i386 build is requested | # This enables an i386-only package in the sources.list file when an i386 build is requested | ||
| - | if echo "$LBX2GO_ARCH" | + | if echo $LBX2GO_ARCH | grep -q -i " |
| sed -i -e 's/# for i386 only #//' config/ | sed -i -e 's/# for i386 only #//' config/ | ||
| fi | fi | ||
| + | |||
| + | # This is part of our experimental ARM support | ||
| + | # It adds required arm64-only packages when an arm64 build is requested | ||
| + | if echo $LBX2GO_ARCH | grep -q " | ||
| + | |||
| + | # firmware for wifi | ||
| + | echo " | ||
| + | |||
| + | if [ " | ||
| + | # modules required for Raspberry Pi 3 LAN | ||
| + | echo " | ||
| + | echo " | ||
| + | echo " | ||
| + | echo " | ||
| + | echo " | ||
| + | echo " | ||
| + | echo " | ||
| + | |||
| + | # firmware for basic raspi functions - required for boot on Pi3 | ||
| + | echo " | ||
| + | # standard linux kernel - for Pi3 | ||
| + | echo " | ||
| + | |||
| + | elif [ " | ||
| + | # firmware for basic raspi functions - required for boot on Pi4 | ||
| + | echo " | ||
| + | echo " | ||
| + | |||
| + | # newer linux kernel - required for pi4/pi400 | ||
| + | echo " | ||
| + | |||
| + | else | ||
| + | echo " | ||
| + | fi | ||
| + | |||
| + | fi | ||
| + | |||
| + | |||
| # This is for minidesktop builds only | # This is for minidesktop builds only | ||
| if [ -f config/ | if [ -f config/ | ||
| Line 284: | Line 390: | ||
| # at files not being downloaded, disable these three entries. | # at files not being downloaded, disable these three entries. | ||
| export https_proxy=$LB_APT_HTTP_PROXY | export https_proxy=$LB_APT_HTTP_PROXY | ||
| - | export http_proxy=$LB_APT_HTTP_PROXY | + | |
| export ftp_proxy=$LB_APT_FTP_PROXY | export ftp_proxy=$LB_APT_FTP_PROXY | ||
| fi | fi | ||
| - | # This is a crude hack to detect crossbuilds | + | # This is part of our experimental ARM support |
| - | # It makes some necessary changes, and also tries to speed up squashfs creation. | + | # It is used when building |
| - | | + | # It makes some necessary changes, and also tries to speed up squashfs creation |
| - | | + | |
| + | | ||
| - | # This command removes all references to fuseext and x2gothinclient from the | + | # This command removes all references to fuseext, freerdp-nightly, |
| - | # package list files. | + | # package list files. |
| - | echo " | + | echo " |
| - | sed -e ' | + | sed -e ' |
| - | # This command removes the X2Go repository from the directory where additional | + | |
| - | # archives are stored. | + | # archives are stored. |
| # packages, but Debian Buster does - so that's what we're falling back to. | # packages, but Debian Buster does - so that's what we're falling back to. | ||
| echo " | echo " | ||
| rm ./ | rm ./ | ||
| - | # The following is a hack to reduce squashfs creation time. We're replacing mksquashfs | + | |
| - | # in the changeroot | + | # We're replacing mksquashfs in the changeroot with a wrapper script that drops the |
| - | # mksquashfs call into a file. | + | # original |
| - | # We need to do this as a background task, waiting for the mksquashfs executable to | + | |
| - | # appear in the changeroot; as the changeroot will only be created later on, once | + | |
| - | # lb build is called. | + | # appear in the changeroot; as the changeroot will only be created later on, once |
| - | + | # lb build is called. | |
| - | # The other background task waits until the command file has been created, then | + | |
| - | # it applies some necessary patches to it, and starts the mksquashfs command natively | + | |
| - | # on the build host, rather than in the changeroot environment. | + | |
| - | # This is because in the changeroot, we'd be running the ARM mksquashfs in a qemu | + | |
| - | # software emulation of the ARM architecture, | + | |
| - | # native, raw CPU power and cores available to us. | + | |
| - | | + | |
| - | # our own PID along to the background tasks, and tell them to terminate if our PID | + | # it applies some necessary patches |
| - | # disappears while they're still in their waiting/ | + | # on the build host, rather than in the changeroot environment. |
| + | # This is because in the changeroot, we'd be running the ARM mksquashfs | ||
| + | # software emulation of the ARM architecture, | ||
| + | # native, raw CPU power and cores available to us. | ||
| - | MASTERPID=$$ | + | # To make sure we don't have any lingering processes in the background, we're passing |
| + | # our own PID along to the background tasks, and tell them to terminate if our PID | ||
| + | # disappears while they' | ||
| - | # Replace mksquashfs in chroot with script | + | |
| - | # (script will undo this upon completion) | + | |
| - | ( | + | |
| - | # wait until the chroot has been populated or until our parent process dies | + | |
| - | while ! [ -x ./ | + | |
| - | ps $MASTERPID | + | |
| - | sleep 1 | + | |
| - | done | + | |
| - | # make sure we don't overwrite the real executable if it has already been | + | |
| - | # moved out of the way | + | |
| - | if ! [ -x ./ | + | |
| - | cp ./ | + | |
| - | fi | + | |
| - | echo '# | + | |
| - | # log the name we've been called with and all parameters into this file | + | |
| - | echo 'echo "$0 $@" >/ | + | |
| - | # once the native mksquashfs is complete, we will remove this file | + | |
| - | echo 'while [ -f / | + | |
| - | echo ' | + | |
| - | echo ' | + | |
| - | # so let's wait until it has been removed before deleting ourselves ... | + | |
| - | echo 'rm / | + | |
| - | # ... and moving the real executable back into its place | + | |
| - | echo 'mv / | + | |
| - | chmod 755 ./ | + | |
| - | ) & | + | |
| - | | + | |
| - | ( | + | # |
| - | # wait until the trigger file has been created | + | ( |
| - | while ! [ -f ./chroot/tmp/filesystem.squashfs.temp | + | |
| - | ps $MASTERPID >/ | + | while ! [ -x ./chroot/usr/bin/ |
| - | sleep 1 | + | ps $MASTERPID >/ |
| - | done | + | sleep 1 |
| - | # using any of the available filters (x86, arm, armthumb) for the | + | done |
| - | # -Xbcj command results in an unusable squashfs on arm, so we drop the | + | # make sure we don't overwrite |
| - | # parameter completely | + | # moved out of the way |
| - | # also, all absolute paths (detected by beginning with " | + | if ! [ -x ./ |
| - | # prefixed | + | cp ./ |
| - | # to look for the corresponding paths/ | + | fi |
| - | | + | echo '#!/bin/ |
| - | ./chroot/tmp/filesystem.squashfs.temp | + | |
| - | # now let's make this executable | + | echo ' |
| - | chmod 755 ./chroot/tmp/filesystem.squashfs.temp | + | |
| + | echo 'while [ -f / | ||
| + | | ||
| + | echo ' | ||
| + | # so let's wait until it has been removed before deleting ourselves ... | ||
| + | echo 'rm / | ||
| + | # | ||
| + | echo 'mv / | ||
| + | | ||
| + | ) & | ||
| - | | + | # start the native mksquashfs after patching the parameters |
| - | # in the squashfs - no idea why we don't need them while inside the chroot ... | + | ( |
| - | echo ' | + | # wait until the trigger file has been created or until our parent process dies |
| - | echo ' | + | while ! [ -f ./ |
| - | echo ' | + | ps $MASTERPID >/ |
| - | # now let's execute the script and, if it terminates without an error, | + | sleep 1 |
| - | # we'll move the newly created squashfs into the chroot where the chrooted | + | done |
| - | # mksquashfs command would have created it; if that worked as well, we' | + | # using any of the available filters (x86, arm, armthumb) for the |
| - | # remove the script file so our dummy mksquashfs inside the chroot knows | + | # -Xbcj command results in an unusable squashfs on arm, so we drop the |
| - | # it's time to terminate itself. | + | # parameter completely if it's there. |
| - | ./ | + | # also, all absolute paths (detected by beginning with " /") need to be |
| - | mv ./ | + | # prefixed with " |
| - | rm ./ | + | # to look for the corresponding paths/ |
| - | ) & | + | sed -e 's/ -Xbcj x86/ /g' -e 's# /# ./ |
| + | ./ | ||
| + | #needs switch from e.g. / | ||
| + | sed -e " | ||
| + | ./ | ||
| + | # now let's make this executable | ||
| + | chmod 755 ./ | ||
| + | |||
| + | | ||
| + | # in the squashfs - no idea why we don't need them while inside the chroot ... | ||
| + | echo ' | ||
| + | echo ' | ||
| + | echo ' | ||
| + | # now let's execute the script and, if it terminates without an error, | ||
| + | # we'll move the newly created squashfs into the chroot where the chrooted | ||
| + | # mksquashfs command would have created it; if that worked as well, we' | ||
| + | # remove the script file so our dummy mksquashfs inside the chroot knows | ||
| + | # it's time to terminate itself. | ||
| + | ./ | ||
| + | mv ./ | ||
| + | rm ./ | ||
| + | ) & | ||
| + | fi | ||
| fi | fi | ||
| Line 393: | Line 505: | ||
| if [ " | if [ " | ||
| - | ln ./ | + | ln ./ |
| - | | + | |
| fi | fi | ||
| + | |||
| + | # This is part of our experimental ARM support | ||
| + | if [ " | ||
| + | # after the build, let's determine the name of our image file ... | ||
| + | IMAGEFILE=" | ||
| + | |||
| + | # ... and change the partition type to reflect the file system actually in use for partition 1 | ||
| + | # (" | ||
| + | sfdisk --part-type $IMAGEFILE 1 b | ||
| + | |||
| + | # next, we need to patch two things inside the image, so we need to set up a loop device for it. | ||
| + | FREELOOP=$(losetup -f) # note that this could become a TOCTOU issue if more than 1 process tries to use loop devices | ||
| + | |||
| + | # as the image is a full disk image containing a partition, we need to jump to the position where the first partition starts | ||
| + | losetup -o 1048576 $FREELOOP $IMAGEFILE | ||
| + | |||
| + | # now let's mount it | ||
| + | mkdir -p ./tempmount | ||
| + | mount $FREELOOP ./tempmount | ||
| + | |||
| + | # purge this dir, so we have enough space; we'll return to fill it later | ||
| + | rm ./ | ||
| + | |||
| + | # first, we copy the contents of the boot/ | ||
| + | # see if inplace helps against out of space errors | ||
| + | rsync -aP --inplace ./ | ||
| + | |||
| + | mkdir -p ./ | ||
| + | rsync -aP ./ | ||
| + | |||
| + | # next, we replace the " | ||
| + | sed -e ' | ||
| + | |||
| + | # here comes the cleanup part | ||
| + | sync | ||
| + | umount $FREELOOP | ||
| + | losetup -d $FREELOOP | ||
| + | rmdir ./tempmount | ||
| + | fi | ||
| + | |||
| if [ " | if [ " | ||
| if [ " | if [ " | ||
| Line 407: | Line 559: | ||
| if [ " | if [ " | ||
| genisoimage -o ./ | genisoimage -o ./ | ||
| - | if [ -e ./ | + | if [ -e ./ |
| - | ln ./ | + | ln ./ |
| - | | + | |
| - | elif [ -e ./ | + | elif [ -e ./ |
| - | ln ./ | + | ln ./ |
| - | | + | |
| fi | fi | ||
| mv ./ | mv ./ | ||
| Line 443: | Line 595: | ||
| cd .. | cd .. | ||
| fi | fi | ||
| + | |||
| </ | </ | ||
| Line 516: | Line 669: | ||
| === Required unless using the X2Go Session Broker: Adding the x2go-tce.sessions session configuration file to your HTTP or FTP Server === | === Required unless using the X2Go Session Broker: Adding the x2go-tce.sessions session configuration file to your HTTP or FTP Server === | ||
| Again, this is assuming you already have an existing, working HTTP or FTP server setup. | Again, this is assuming you already have an existing, working HTTP or FTP server setup. | ||
| - | * run X2GoClient on any computer you like, and configure a session the same way it should appear on the ThinClient <note tip>when using a Windows client, run x2goclient.exe --portable, or it will store the session information in the registry, rather than in a " | + | * run X2GoClient on any computer you like, and configure a session the same way it should appear on the ThinClient |
| + | <note tip>when using a Windows client, run x2goclient.exe --portable, or it will store the session information in the registry, rather than in a " | ||
| * locate the " | * locate the " | ||
| * copy it to x2go-tce.sessions | * copy it to x2go-tce.sessions | ||
| Line 1155: | Line 1310: | ||
| FIXME Scripts triggered by if-up should check if a new download is really necessary. | FIXME Scripts triggered by if-up should check if a new download is really necessary. | ||
| - | FIXME Feature request: In TCE (not MMD), use < | + | ===== List of closed ToDos/ |
| + | * Feature request: In TCE (not MMD), use < | ||
| while ! (grep " | while ! (grep " | ||
| echo s >/ | echo s >/ | ||
| Line 1162: | Line 1318: | ||
| echo o >/ | echo o >/ | ||
| </ | </ | ||
| - | ===== List of closed ToDos/ | + | |
| * '' | * '' | ||
| * It would be nice to have a boot parameter '' | * It would be nice to have a boot parameter '' | ||
doc/howto/tce.txt · Last modified: 2024/01/26 19:49 by stefanbaur
Imprint & Legal Disclaimer -
Data Protection & Privacy Statement
Except where otherwise noted, content on this wiki is licensed under the following license: GNU Free Documentation License 1.3