[build] refactoring of 1st stage rfs split build

[k-v1]

Why I did it

Fix #16944 to enable DPKG cache for 1st stage of RFS

Work item tracking

• Microsoft ADO (number only):

How I did it

1. Make sonic-build-hooks deb package reproducible
2. Enable pigz by default to speed up build (especially when compress docker files)
3. Move 1st stage of rootfs to the file build_rfs.sh and enable DPKG cache for this stage
4. Fix dependencies for 1st stage of rootfs

1st stage (build_rfs.sh):

1. debootstrap
2. makedev
3. install kernel and initramfs
4. install docker
5. install deb packages from debian mirror
6. install wheel packages from pypi
7. compress rootfs to squashfs file

2nd stage (build_debian.sh):

1. unsquash rootfs from 1st stage
2. copy and create config files
3. optional actions (e.g. install kubernetes or sign linux kernel)
4. create users and groups
5. sonic_debian_extension.sh
6. update initramfs
7. cleanup rootfs and collect versions info
8. compress rootfs and docker files together

How to verify it

Build SONiC image with enabled DPKG cache for RFS

Which release branch to backport (provide reason below if selected)

• 201811
• 201911
• 202006
• 202012
• 202106
• 202111
• 202205
• 202211
• 202305

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

[k-v1]

@xumia @liushilongbuaa
Could you please help with failed armhf build?
It failed because:

2023-11-06T07:27:57.8190938Z ERROR: Cannot install sonic-utilities==1.2 because these package versions have conflicting dependencies.
2023-11-06T07:27:57.8191680Z 
2023-11-06T07:27:57.8191915Z The conflict is caused by:
2023-11-06T07:27:57.8192484Z     sonic-utilities 1.2 depends on cryptography==3.3.2
2023-11-06T07:27:57.8193121Z     The user requested (constraint) cryptography==3.3.1

sonic-utilitities requires cryptography==3.3.2:
https://github.com/sonic-net/sonic-utilities/blob/253b7975df061a69a52a87fbd98ec6430a728339/setup.py#L230

But at the same time we must install cryptography==3.3.1 for armhf for azure-storage:

sonic-buildimage/files/build_templates/sonic_debian_extension.j2

Lines 483 to 491 in 97db5f5

	if [[ $CONFIGURED_ARCH == armhf ]]; then
	# The azure-storage package depends on the cryptography package. Newer
	# versions of cryptography require the rust compiler, the correct version
	# for which is not readily available in buster. Hence we pre-install an
	# older version here to satisfy the azure-storage dependency.
	# Note: This is not a problem for other architectures as pre-built versions
	# of cryptography are available for those. This sequence can be removed
	# after upgrading to debian bullseye.
	sudo https_proxy=$https_proxy LANG=C chroot $FILESYSTEM_ROOT pip3 install cryptography==3.3.1

It seems like incorrect and probably should be fixed.

[k-v1]

This sequence can be removed after upgrading to debian bullseye.

In theory we can remove this code for armhf and install python3-cryptography from debian bullseye mirror (bullseye version is 3.3.2-1).

[k-v1]

kvmtest-t1-lag failed in other PRs too.
Maybe it's broken now.

[k-v1]

@saiarcot895
Could you please also review this PR?
Also I'm not sure should it be merged before or after bookworm branch merged.

[liushilongbuaa]

@k-v1 , Is there any way to disable LOAD_FROM_CACHE?
It's better to set it disabled as default value.

[k-v1]

@k-v1 , Is there any way to disable LOAD_FROM_CACHE? It's better to set it disabled as default value.

@liushilongbuaa
I think it's possible to add additional option to enable/disable DPKG cache for RFS.
But I'm not sure we should disable it by default.
If we find any additional issues after merging then we can temporarily disable cache, of course.

[saiarcot895]

Also I'm not sure should it be merged before or after bookworm branch merged.

Because of the scope of changes, I would prefer having this go in after the bookworm branch is merged.

[k-v1]

Waiting for #16959
When it's merged I'll start to update for bookworm.

[k-v1]

Also I've found one issue with DPKG cache.
As I understand, we don't validate items from this cache.
I mean when we install deb package/wheel package/docker image/rfs squashfs from DPKG cache we don't know status of this file. It can be corrupted. Maybe we need to store a checksum of this file in its name and compare this checksum with real of checksum of the file when install it from DPKG cache.