[Marvell] Build infrastructure enhancements #18143

pavannaregundi · 2024-02-21T07:23:37Z

Why I did it

This change is about marvell build infrastructure enhancement by merging architecture specific marvell platform(marvell-arm64 and marvell-armhf) folders under 'marvell'.

Work item tracking

Microsoft ADO (number only):

How I did it

This is archived with following changes divided into following commits,

Move mrvl-prestera and sonic-platform-marvell submodule
'mrvl-prestera' and 'sonic-platform-marvell' submodules are moved from 'marvell-arm64'to 'marvell'. Additionaly submodules are changed to support multi-architecture build.
Create sonic-platform-nokia from marvell-arm64/armhf
'sonic-platform-nokia' is created from merge of 'marvell-armhf' and 'marvell-arm64' in 'marvell'. Additionally changes are done in debian/rules to support multi-architecture build.
Change marvell platform makefile to support all arch
Combine all the architecture targets in makefiles. Added CONFIGURED_ARCH check to build for specific architecture.
Updated sai deb which matches latest SAI commit.
Add support for architecture specific platform.conf
Changed build_image.sh and onie-mk-demo.sh to read architecture specific platform.conf.
Moved marvell-arm64/platform.conf to marvell/platform_arm64.conf.
Moved marvell-armhf/platform.conf to marvell/platform_armhf.conf.
[arm64] Adjust the path to read sonic_fit.its
Use CONFIGURED_PLATFORM based path while reading sonic_fit.its.
[arm64/armhf] Add architecure suffix for output image
This change is needed to generate the output image with architecture suffix for arm64/armhf platform.
For arm64, defined a marvell specific file to avoid affect on other vendors.
Change platform_asic name to marvell
Change 'marvell-arm64' and 'marvell-armhf' platform_asic name to 'marvell'.
Remove marvell-armhf and marvell-arm64 folders
Removed 'marvell-armhf' and 'marvell-arm64' as infra is combined with 'marvell' folder.
Update readme to reflect new build commands
Change azure pipeline commands for marvell

New build commands:
AMD64 - make configure PLATFORM=marvell && make target/sonic-marvell.bin
ARM64 - make configure PLATFORM=marvell PLATFORM_ARCH=arm64 && make target/sonic-marvell-arm64.bin
ARMHF - make configure PLATFORM=marvell PLATFORM_ARCH=armhf && make target/sonic-marvell-armhf.bin

How to verify it

Built images for all the platform and installed on hardware to verify the port up.
Verified onie and sonic-sonic install.

Which release branch to backport (provide reason below if selected)

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

pavannaregundi · 2024-02-21T10:05:32Z

@Pavan-Nokia please review.

saiarcot895 · 2024-02-26T23:14:04Z

.azure-pipelines/azure-pipelines-image-template.yml

@@ -49,7 +49,11 @@ jobs:
          sudo modprobe overlay
          sudo apt-get install -y acl
          sudo bash -c "echo 1 > /proc/sys/vm/compact_memory"
-          ENABLE_DOCKER_BASE_PULL=y make PLATFORM=$(PLATFORM_AZP) PLATFORM_ARCH=$(PLATFORM_ARCH) $(BUILD_OPTIONS) configure
+          if [ ${PLATFORM_AZP} == marvell-arm64 ] || [ ${PLATFORM_AZP} == marvell-armhf ]; then


Instead of the if-checks here, can the top-level pipeline build files be updated to point to marvell instead?

@saiarcot895 If we look at the changes in the build commands, only configure command is different in "PLATFORM=marvell" string. Other commands are same. So I opted this change.
If we change name in top-level pipeline( ex: name: marvell-armhf) we need to change or checks multiple places such as how the build pipelines are fetched in "Azure.sonic-buildimage.official.marvell-armhf?branchName=master&label=Marvell-armhf" or while running a make command(make target/sonic-marvell-armhf.bin) for build. I thought this is less change, and does not break other arm64 platform builds.

hmm, @xumia, thoughts?

How about define a new parameter in azure-pipelines-build.yml for marvell-armhf and marvell-amd64.

sonic-buildimage/.azure-pipelines/azure-pipelines-build.yml

Line 100 in 7f454a7

PLATFORM_ARCH: arm64

For example:

PLATFORM: marvell

If the variable PLATFORM is not defined (empty), then use PLATFORM_AZP

[ -z "$PLATFORM" ] && PLATFORM=$(PLATFORM_AZP) ENABLE_DOCKER_BASE_PULL=y make PLATFORM=$PLATFORM PLATFORM_ARCH=$(PLATFORM_ARCH) $(BUILD_OPTIONS) configure // There is a little different $PLATFORM and $(PLATFORM), be careful.

Any other platforms have the similar requirements, only need to override the variable, no logic change.

@xumia Have pushed the changes as recommended by you. Please review.

xumia · 2024-03-01T04:58:23Z

.azure-pipelines/build-template.yml

+      if [ ${{ parameters.platform }} == marvell-arm64 ] || [ ${{ parameters.platform }} == marvell-armhf ]; then
+        ENABLE_DOCKER_BASE_PULL=y make configure PLATFORM=marvell PLATFORM_ARCH=${{ parameters.platform_arch }}
+      else
+        ENABLE_DOCKER_BASE_PULL=y make configure PLATFORM=${{ parameters.platform }} PLATFORM_ARCH=${{ parameters.platform_arch }}
+      fi


The file has been deprecated, and it is no use. The change can be skipped.

Ok. Reverting this change.

pavannaregundi · 2024-03-05T03:43:48Z

/azpw run Azure.sonic-buildimage

mssonicbld · 2024-03-05T03:43:51Z

/AzurePipelines run Azure.sonic-buildimage

azure-pipelines · 2024-03-05T03:43:57Z

Pull request contains merge conflicts.

Blueve · 2024-03-06T12:54:54Z

@liushilongbuaa

Blueve · 2024-03-07T06:16:34Z

Hi @pavannaregundi , can you trigger a new build by typing /azpw run Azure.sonic-buildimage in comments again?

pavannaregundi · 2024-03-07T06:18:25Z

/azpw run Azure.sonic-buildimage

mssonicbld · 2024-03-07T06:18:27Z

/AzurePipelines run Azure.sonic-buildimage

azure-pipelines · 2024-03-07T06:18:38Z

Azure Pipelines successfully started running 1 pipeline(s).

liushilongbuaa · 2024-03-18T09:26:05Z

.azure-pipelines/azure-pipelines-image-template.yml

@@ -49,7 +49,8 @@ jobs:
          sudo modprobe overlay
          sudo apt-get install -y acl
          sudo bash -c "echo 1 > /proc/sys/vm/compact_memory"
-          ENABLE_DOCKER_BASE_PULL=y make PLATFORM=$(PLATFORM_AZP) PLATFORM_ARCH=$(PLATFORM_ARCH) $(BUILD_OPTIONS) configure
+          [ -z "$PLATFORM" ] && PLATFORM=$(PLATFORM_AZP)


Why? When PLATFORM is set?

@liushilongbuaa This is a new optional variable introduced in 'azure-pipelines-build.yml' to select the platform if it is different from default PLATFORM_AZP. Basically, pick platform name from PLATFORM_AZP when PLATFORM variable is not set.

Details of this change are already present in comments above.

liushilongbuaa · 2024-03-18T09:28:48Z

build_debian.sh

@@ -781,7 +781,7 @@ if [[ $TARGET_BOOTLOADER == uboot ]]; then
            ## Overwriting the initrd image with uInitrd
            sudo LANG=C chroot $FILESYSTEM_ROOT mv /boot/u${INITRD_FILE} /boot/$INITRD_FILE
        else
-            sudo cp -v $PLATFORM_DIR/${sonic_asic_platform}-${CONFIGURED_ARCH}/sonic_fit.its $FILESYSTEM_ROOT/boot/
+            sudo cp -v $PLATFORM_DIR/$CONFIGURED_PLATFORM/sonic_fit.its $FILESYSTEM_ROOT/boot/


centec-arm64 is also impacted.

Yes. But $CONFIGURED_PLATFORM for "centec-arm64" is "centec-arm64". So change will work.

liushilongbuaa · 2024-03-18T10:32:22Z

can you trigger a new build by typing /azpw run Azure.sonic-buildimage in comments again?

pavannaregundi · 2024-03-18T10:38:57Z

/azpw run Azure.sonic-buildimage

mssonicbld · 2024-03-18T10:38:59Z

/AzurePipelines run Azure.sonic-buildimage

azure-pipelines · 2024-03-18T10:39:12Z

Azure Pipelines successfully started running 1 pipeline(s).

liushilongbuaa · 2024-03-19T09:58:27Z

can you trigger a new build by typing /azpw run Azure.sonic-buildimage in comments again?

Done. Build still failing.

Can you debug locally to fix sonic-config-engine UT?

liushilongbuaa · 2024-03-20T09:34:19Z

Hi @neethajohn , can you help on the UT?

pavannaregundi · 2024-03-20T11:57:41Z

can you trigger a new build by typing /azpw run Azure.sonic-buildimage in comments again?

Done. Build still failing.

Can you debug locally to fix sonic-config-engine UT?

@liushilongbuaa We will try looking at this. However, someone who has experience in this area can help us will be good.

radha-danda · 2024-04-03T02:07:49Z

/azpw run Azure.sonic-buildimage

Blueve · 2024-04-08T09:30:45Z

Potential fix of build ut issue: #18566

The marvell build pipeline continuously fail, seems related to the new added PLATFORM variable. And above PR can clean up the variable before build py-wheel

liushilongbuaa · 2024-04-08T10:20:40Z

@pavannaregundi , can you rerun PR checker by comment '/azpw run Azure.sonic-buildimage'?

'mrvl-prestera' and 'sonic-platform-marvell' submodules are moved from 'marvell-arm64'to 'marvell'. Additionaly submodules are changed to support multi-architecture build. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

sonic-platform-nokia is created from merge of 'marvell-armhf' and 'marvell-arm64' in 'marvell'. Additionaly changes are done to support multi-arch build. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

Combine all the architecture targets in makefiles. Added CONFIGURED_ARCH check to build for specific architecture. Updated sai deb which matches latest SAI commit. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

Changed build_image.sh and onie-mk-demo.sh to read architecture specific platform.conf. Moved marvell-arm64/platform.conf to marvell/platform_arm64.conf. Moved marvell-armhf/platform.conf to marvell/platform_armhf.conf. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

Use CONFIGURED_PLATFORM based path while reading sonic_fit.its. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

This change is needed to generate the output image with architecture suffix for arm64/armhf platform. For arm64, defined a marvell specific file to avoid affect on other vendors. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

Change 'marvell-arm64' and 'marvell-armhf' platform_asic name to 'marvell'. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

Removed 'marvell-armhf' and 'marvell-arm64' as infra is combined with 'marvell' folder. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

make configure is changed as below for 'marvell-arm64' and 'marvell-armhf'. arm64 - make configure PLATFORM=marvell PLATFORM_ARCH=arm64 armhf - make configure PLATFORM=marvell PLATFORM_ARCH=armhf Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

pavannaregundi · 2024-04-09T05:02:59Z

@Blueve @liushilongbuaa I have pushed a change by changing the azure new variable for platform to "PLATFORM_NAME", instead of "PLATFORM". It could be that yml is setting this as env variable.

pavannaregundi · 2024-04-09T10:48:00Z

@Blueve @liushilongbuaa I have pushed a change by changing the azure new variable for platform to "PLATFORM_NAME", instead of "PLATFORM". It could be that yml is setting this as env variable.

Builds are passing now.

liushilongbuaa

LGTM

pavannaregundi · 2024-04-12T07:02:58Z

@prgeor All comments are addressed. Can we merge this PR.

prgeor · 2024-04-14T07:47:29Z

@prgeor All comments are addressed. Can we merge this PR.

@lguohan @yxieca can you merge?

pavannaregundi requested a review from lguohan as a code owner February 21, 2024 07:23

pavannaregundi force-pushed the master branch 3 times, most recently from 90b6ff5 to 2174f65 Compare February 22, 2024 02:50

saiarcot895 reviewed Feb 26, 2024

View reviewed changes

saiarcot895 requested review from prgeor and xumia February 28, 2024 04:11

pavannaregundi force-pushed the master branch from 2174f65 to 7f454a7 Compare February 29, 2024 08:45

xumia reviewed Mar 1, 2024

View reviewed changes

pavannaregundi force-pushed the master branch 3 times, most recently from 6d0f3b5 to 56d0df8 Compare March 4, 2024 03:38

pavannaregundi force-pushed the master branch from 56d0df8 to 5a0b93e Compare March 5, 2024 04:46

Blueve requested a review from liushilongbuaa March 6, 2024 12:55

pavannaregundi force-pushed the master branch from 5a0b93e to cb9c724 Compare March 13, 2024 03:44

liushilongbuaa reviewed Mar 18, 2024

View reviewed changes

krismarvell mentioned this pull request Apr 8, 2024

Fix for sonic-config-engine build failure - test_buffers_dell6100_render_template #18566

Open

9 tasks

pavannaregundi added 9 commits April 9, 2024 04:27

Move mrvl-prestera and sonic-platform-marvell submodule

531a0e6

'mrvl-prestera' and 'sonic-platform-marvell' submodules are moved from 'marvell-arm64'to 'marvell'. Additionaly submodules are changed to support multi-architecture build. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

Create sonic-platform-nokia from marvell-arm64/armhf

8bfd373

sonic-platform-nokia is created from merge of 'marvell-armhf' and 'marvell-arm64' in 'marvell'. Additionaly changes are done to support multi-arch build. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

Change marvell platform makefile to support all arch

355c8a2

Combine all the architecture targets in makefiles. Added CONFIGURED_ARCH check to build for specific architecture. Updated sai deb which matches latest SAI commit. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

[arm64] Adjust the path to read sonic_fit.its

d663b53

Use CONFIGURED_PLATFORM based path while reading sonic_fit.its. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

Change platform_asic name to marvell

295a7ab

Change 'marvell-arm64' and 'marvell-armhf' platform_asic name to 'marvell'. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

Remove marvell-armhf and marvell-arm64 folders

a0cc78d

Removed 'marvell-armhf' and 'marvell-arm64' as infra is combined with 'marvell' folder. Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

Update readme to reflect new build commands

e9423ab

Signed-off-by: Pavan Naregundi <pnaregundi@marvell.com>

pavannaregundi force-pushed the master branch 3 times, most recently from 793f219 to 44e9f1a Compare April 9, 2024 04:58

pavannaregundi force-pushed the master branch from 44e9f1a to 4611de1 Compare April 9, 2024 05:00

Blueve approved these changes Apr 9, 2024

View reviewed changes

saiarcot895 requested a review from liushilongbuaa April 10, 2024 01:28

liushilongbuaa approved these changes Apr 10, 2024

View reviewed changes

prgeor approved these changes Apr 14, 2024

View reviewed changes

yxieca merged commit 006f4b2 into sonic-net:master Apr 15, 2024
20 checks passed

[Marvell] Build infrastructure enhancements #18143

[Marvell] Build infrastructure enhancements #18143

Conversation

pavannaregundi commented Feb 21, 2024 • edited Loading

Why I did it

Work item tracking

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

pavannaregundi commented Feb 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xumia Mar 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pavannaregundi commented Mar 5, 2024

mssonicbld commented Mar 5, 2024

azure-pipelines bot commented Mar 5, 2024

Blueve commented Mar 6, 2024

Blueve commented Mar 7, 2024

pavannaregundi commented Mar 7, 2024

mssonicbld commented Mar 7, 2024

azure-pipelines bot commented Mar 7, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liushilongbuaa commented Mar 18, 2024

pavannaregundi commented Mar 18, 2024

mssonicbld commented Mar 18, 2024

azure-pipelines bot commented Mar 18, 2024

liushilongbuaa commented Mar 19, 2024

liushilongbuaa commented Mar 20, 2024

pavannaregundi commented Mar 20, 2024

radha-danda commented Apr 3, 2024

Blueve commented Apr 8, 2024

liushilongbuaa commented Apr 8, 2024

pavannaregundi commented Apr 9, 2024

pavannaregundi commented Apr 9, 2024

liushilongbuaa left a comment

Choose a reason for hiding this comment

pavannaregundi commented Apr 12, 2024

prgeor commented Apr 14, 2024

pavannaregundi commented Feb 21, 2024 •

edited

Loading

xumia Mar 1, 2024 •

edited

Loading