Bump ci rocm6.2 #3340

TedThemistokleous · 2024-08-02T22:47:22Z

Updating CI to ROCm 6.2 and other appropriate dockers

TedThemistokleous · 2024-08-02T22:48:22Z

Still waiting info for torch sha-1 used dont see it on repo.radeaon under the usual manylinux link. Will let this run over weekend in the meantime.

codecov · 2024-08-02T22:58:23Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.04%. Comparing base (7c2fdf5) to head (0bb51fc).
Report is 14 commits behind head on develop.

Additional details and impacted files

@@            Coverage Diff            @@
##           develop    #3340    +/-   ##
=========================================
  Coverage    92.04%   92.04%            
=========================================
  Files          505      506     +1     
  Lines        20699    20872   +173     
=========================================
+ Hits         19052    19212   +160     
- Misses        1647     1660    +13

Flag	Coverage Δ
	`92.04% <100.00%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

ahsan-ca · 2024-08-03T02:07:14Z

Looks good besides the CI failures.

TedThemistokleous · 2024-08-03T03:53:28Z

Looks good besides the CI failures.

Yeah. something I'll have to deal with I think. @pfultz, should I be including some of your changes to the CPP checks from the 6.1.3 CI update PR in CMakeLists.txt
9291c01

pfultz2 · 2024-08-05T14:01:58Z

Yeah. something I'll have to deal with I think. @pfultz, should I be including some of your changes to the CPP checks from the 6.1.3 CI update PR in CMakeLists.txt

There are lots of changes needed from there, but it seems like hip-config.cmake is broken in 6.2, so we probably need to do some patching of the files.

causten

You've got some more work to clear CI but the files I was expecting to see changes in are here.

TedThemistokleous · 2024-08-08T18:12:20Z

Confirmed items are upstream on repo.radeon finally for the Onnxruntime wheel

…well

TedThemistokleous · 2024-09-19T02:22:26Z

Got past the hipcc errors and the hip-config by modifying hip-clang.cmake usind sed to set the HIP_PLATFORM to "Amd"

Now hitting this

-- Generating embedding library 'migraphx_kernels'
-- Performing Test HAS_HIP_LAMBDA_HOST_DEVICE
-- Performing Test HAS_HIP_LAMBDA_HOST_DEVICE - Failed
-- Build with rocMLIR::rockCompiler 2.0.0
-- MIGraphX is using HIP Clang
CMake Error at cmake/TargetFlags.cmake:42 (get_target_property):
  get_target_property() called with non-existent target "hip::device".
Call Stack (most recent call first):
  cmake/TargetFlags.cmake:61 (get_target_property2)
  cmake/TargetFlags.cmake:100 (append_flags)
  src/targets/gpu/CMakeLists.txt:250 (target_flags)


-- Hip compiler flags: " "
-- Looking for miopenHiddenSetConvolutionFindMode in MIOpen
CMake Error at /data/build/CMakeFiles/CMakeTmp/CMakeLists.txt:20 (add_executable):
  Target "cmTC_cab74" links to target "hip::host" but the target was not
  found.  Perhaps a find_package() call is missing for an IMPORTED target, or
  an ALIAS target is missing?


CMake Error at /usr/share/cmake-3.22/Modules/CheckLibraryExists.cmake:72 (try_compile):
  Failed to generate test project build system.
Call Stack (most recent call first):
  src/targets/gpu/CMakeLists.txt:291 (check_library_exists)

Had to add these as part of ignore for tiday -bugprone-empty-catch -performance-enum-size -readaility-enum-size -readability-avoid-nested-conditional-operator -readability-redundant-member-init

TedThemistokleous · 2024-09-19T12:41:43Z

Before we close this out we should make issues for the following tidy warnings that I just stubbed out

bug-prone-empty-catch
performance-enum-size

The other readability ones may not need issues

readability-redundant-member-init
readability-avoid-nested-conditional-operator

pfultz2 · 2024-09-20T13:43:47Z

hip-clang.docker

@@ -28,6 +28,7 @@ RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --allow-
    wget \
    rocm-device-libs \
    hip-dev \
+    hipcc \


This should be added to the install_preqs.sh.

Moved via #3469

migraphx-bot · 2024-09-21T00:34:55Z

Test	Batch	Rate new 0bb51f	Rate old 32a84a	Diff	Compare
torchvision-resnet50	64	3,251.39	3,250.93	0.01%	✅
torchvision-resnet50_fp16	64	6,981.90	6,986.29	-0.06%	✅
torchvision-densenet121	32	2,433.85	2,433.76	0.00%	✅
torchvision-densenet121_fp16	32	4,111.85	4,110.78	0.03%	✅
torchvision-inceptionv3	32	1,635.90	1,635.58	0.02%	✅
torchvision-inceptionv3_fp16	32	2,738.41	2,737.29	0.04%	✅
cadene-inceptionv4	16	776.74	775.61	0.15%	✅
cadene-resnext64x4	16	808.50	808.58	-0.01%	✅
slim-mobilenet	64	7,453.48	7,452.87	0.01%	✅
slim-nasnetalarge	64	208.35	208.18	0.09%	✅
slim-resnet50v2	64	3,434.90	3,434.69	0.01%	✅
bert-mrpc-onnx	8	1,150.56	1,150.44	0.01%	✅
bert-mrpc-tf	1	310.83	311.37	-0.17%	✅
pytorch-examples-wlang-gru	1	428.30	426.16	0.50%	✅
pytorch-examples-wlang-lstm	1	390.88	386.06	1.25%	✅
torchvision-resnet50_1	1	773.02	816.95	-5.38%	🔴
cadene-dpn92_1	1	397.30	401.62	-1.08%	✅
cadene-resnext101_1	1	381.22	380.63	0.16%	✅
onnx-taau-downsample	1	344.52	345.07	-0.16%	✅
dlrm-criteoterabyte	1	35.06	35.06	-0.00%	✅
dlrm-criteoterabyte_fp16	1	58.05	58.01	0.07%	✅
agentmodel	1	8,046.15	7,975.82	0.88%	✅
unet_fp16	2	58.03	58.07	-0.07%	✅
resnet50v1_fp16	1	934.55	921.74	1.39%	✅
resnet50v1_int8	1	935.51	953.21	-1.86%	✅
bert_base_cased_fp16	64	1,152.88	1,152.80	0.01%	✅
bert_large_uncased_fp16	32	355.81	355.84	-0.01%	✅
bert_large_fp16	1	211.62	211.95	-0.16%	✅
distilgpt2_fp16	16	2,162.07	2,157.57	0.21%	✅
yolov5s	1	536.73	528.76	1.51%	✅
tinyllama	1	43.42	43.37	0.11%	✅
vicuna-fastchat	1	171.25	168.85	1.42%	✅
whisper-tiny-encoder	1	414.85	418.42	-0.85%	✅
whisper-tiny-decoder	1	428.64	425.66	0.70%	✅

This build is not recommended to merge 🔴

migraphx-bot · 2024-09-21T00:34:57Z

✅ bert-mrpc-onnx: PASSED: MIGraphX meets tolerance

✅ bert-mrpc-tf: PASSED: MIGraphX meets tolerance

✅ pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

✅ pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

✅ torchvision-resnet50_1: PASSED: MIGraphX meets tolerance

✅ cadene-dpn92_1: PASSED: MIGraphX meets tolerance

✅ cadene-resnext101_1: PASSED: MIGraphX meets tolerance

✅ dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

✅ agentmodel: PASSED: MIGraphX meets tolerance

✅ unet: PASSED: MIGraphX meets tolerance

✅ resnet50v1: PASSED: MIGraphX meets tolerance

✅ bert_base_cased_fp16: PASSED: MIGraphX meets tolerance

🔴bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

✅ bert_large: PASSED: MIGraphX meets tolerance

✅ yolov5s: PASSED: MIGraphX meets tolerance

✅ tinyllama: PASSED: MIGraphX meets tolerance

✅ vicuna-fastchat: PASSED: MIGraphX meets tolerance

✅ whisper-tiny-encoder: PASSED: MIGraphX meets tolerance

✅ whisper-tiny-decoder: PASSED: MIGraphX meets tolerance

✅ distilgpt2_fp16: PASSED: MIGraphX meets tolerance

TedThemistokleous · 2024-09-23T19:47:18Z

CMakeLists.txt

@@ -196,6 +198,7 @@ rocm_enable_clang_tidy(
        clang-analyzer-alpha.cplusplus.MismatchedIterator
        clang-analyzer-alpha.cplusplus.MisusedMovedObject
        -bugprone-switch-missing-default-case
+        -bugprone-empty-catch


created #3470

TedThemistokleous · 2024-09-23T20:51:12Z

CMakeLists.txt

        -bugprone-signed-char-misuse
        -bugprone-unchecked-optional-access
+	-bugprone-unused-local-non-trivial-variable


Upgrade CI to 6.1.3

9291c01

TedThemistokleous added the Continous Integration Pull request updates parts of continous integration pipeline label Aug 2, 2024

TedThemistokleous self-assigned this Aug 2, 2024

TedThemistokleous requested a review from causten as a code owner August 2, 2024 22:47

TedThemistokleous linked an issue Aug 2, 2024 that may be closed by this pull request

Bump CI to 6.2 once released #3318

Closed

TedThemistokleous requested review from pfultz2, ahsan-ca and CharlieL7 August 2, 2024 22:48

ahsan-ca approved these changes Aug 3, 2024

View reviewed changes

pfultz2 added 3 commits August 3, 2024 15:43

Build with clang to avoid linker errors with ubsan

65cac2b

Fix asan odr issues

c7d148f

Use same debug flags

f0ece94

causten approved these changes Aug 5, 2024

View reviewed changes

pfultz2 added 7 commits August 5, 2024 07:32

Disable warning

2903290

Add asan options

79585c0

Just disable odr detection

398adad

Move to docker

be8b78a

Set to 0

de9b6b5

Disable global reporting

0cf5d32

Use gcc for ASAN

2065d63

pfultz2 added 4 commits August 12, 2024 13:26

Switch back to clang

00a658b

Remove static since get_type_name is already static

47a4480

Try using a copy

b9386da

Format

ff70eb7

TedThemistokleous and others added 10 commits September 13, 2024 20:44

Update hipcc line to env instead

763c39b

Workaround for hipcc

94f10e7

Merge branch 'develop' into bump_ci_rocm6.2

ffcd9c0

Sed delete out hipconfig as well as apply change to other dockers as …

942945e

…well

remove workaround for hipcc from sles docker for now

6fe2a02

Remove entire WIN32 block in hip-clang.cmake

c2bf0d2

Add or true to sles workaround

93e28e8

Add hip-runtime-amd to hip-clang docker to solve CI issue

cbcb054

Try additional libs to see if this gets past error with hipcc

af25026

replace if with HIP_PLATFORM being set already as workaround

6f82c0c

TedThemistokleous force-pushed the bump_ci_rocm6.2 branch from 27631be to 6f82c0c Compare September 18, 2024 22:09

Ted Themistokleous added 2 commits September 18, 2024 22:39

Write in updated HIP_PLATFORM statement

14268da

Remove libs that are not needed

b7ec938

Ted Themistokleous added 2 commits September 19, 2024 03:54

Manually install hipcc remove workaround

af7a9fb

Update ignore from tidy

355f663

Had to add these as part of ignore for tiday -bugprone-empty-catch -performance-enum-size -readaility-enum-size -readability-avoid-nested-conditional-operator -readability-redundant-member-init

workaround not needed in Dockerfile

5053237

pfultz2 reviewed Sep 20, 2024

View reviewed changes

Chris Austen added 2 commits September 20, 2024 10:17

disable tidy issues

aba657c

turn lint off from line 27

0bb51fc

causten merged commit ab5da3c into develop Sep 21, 2024
48 checks passed

causten deleted the bump_ci_rocm6.2 branch September 21, 2024 00:50

TedThemistokleous mentioned this pull request Sep 23, 2024

Uncomment - bugprone-empty-catch #3470

Open

3 tasks

TedThemistokleous commented Sep 23, 2024

View reviewed changes

TedThemistokleous mentioned this pull request Sep 23, 2024

Uncomment - -bugprone-unused-local-non-trivial-variable #3471

Open

3 tasks

TedThemistokleous commented Sep 23, 2024

View reviewed changes

TedThemistokleous mentioned this pull request Sep 23, 2024

Uncomment -bugprone-multi-level-implicit-pointer-conversion #3472

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bump ci rocm6.2 #3340

Bump ci rocm6.2 #3340

TedThemistokleous commented Aug 2, 2024

TedThemistokleous commented Aug 2, 2024

codecov bot commented Aug 2, 2024 •

edited

Loading

ahsan-ca commented Aug 3, 2024

TedThemistokleous commented Aug 3, 2024 •

edited

Loading

pfultz2 commented Aug 5, 2024

causten left a comment

TedThemistokleous commented Aug 8, 2024 •

edited

Loading

TedThemistokleous commented Sep 19, 2024

TedThemistokleous commented Sep 19, 2024

pfultz2 Sep 20, 2024

TedThemistokleous Sep 23, 2024

migraphx-bot commented Sep 21, 2024

migraphx-bot commented Sep 21, 2024

TedThemistokleous Sep 23, 2024

TedThemistokleous Sep 23, 2024

Bump ci rocm6.2 #3340

Bump ci rocm6.2 #3340

Conversation

TedThemistokleous commented Aug 2, 2024

TedThemistokleous commented Aug 2, 2024

codecov bot commented Aug 2, 2024 • edited Loading

Codecov Report

ahsan-ca commented Aug 3, 2024

TedThemistokleous commented Aug 3, 2024 • edited Loading

pfultz2 commented Aug 5, 2024

causten left a comment

Choose a reason for hiding this comment

TedThemistokleous commented Aug 8, 2024 • edited Loading

TedThemistokleous commented Sep 19, 2024

TedThemistokleous commented Sep 19, 2024

pfultz2 Sep 20, 2024

Choose a reason for hiding this comment

TedThemistokleous Sep 23, 2024

Choose a reason for hiding this comment

migraphx-bot commented Sep 21, 2024

migraphx-bot commented Sep 21, 2024

TedThemistokleous Sep 23, 2024

Choose a reason for hiding this comment

TedThemistokleous Sep 23, 2024

Choose a reason for hiding this comment

codecov bot commented Aug 2, 2024 •

edited

Loading

TedThemistokleous commented Aug 3, 2024 •

edited

Loading

TedThemistokleous commented Aug 8, 2024 •

edited

Loading