int4: disable const_folding for unpack_int4

[lakhinderwalia]

disable const_folding for unpack_int4

[codecov]

Codecov Report

Attention: Patch coverage is 86.07595% with 11 lines in your changes missing coverage. Please review.

Project coverage is 92.02%. Comparing base (1cd2854) to head (55e0f05).
Report is 1 commits behind head on develop.

Files with missing lines	Patch %	Lines
src/quantize_int4.cpp	91.66%	4 Missing ⚠️
src/shape.cpp	0.00%	4 Missing ⚠️
src/quantization.cpp	0.00%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #3322      +/-   ##
===========================================
- Coverage    92.04%   92.02%   -0.02%     
===========================================
  Files          506      508       +2     
  Lines        20872    20948      +76     
===========================================
+ Hits         19212    19278      +66     
- Misses        1660     1670      +10     

Flag	Coverage Δ
	92.02% <86.07%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[lakhinderwalia]

@pfultz2, @CharlieL7, this PR is being moved out of the draft mode. Some of its necessary functionality is in other PRs. For example, block_quantization support. Also int4 (i.e. signed support) for the basic pack/unpack. Parse int4 graph.
At this stage, we are able to compile the test graphs that are checked-in, and also other full quark-generated graph. We also need some of these other PRs to merge first. Some other tests will be added as the review progresses. Thanks.

[pfultz2]

This is still missing unit test for simplify_qdq. It really should be addressed either before merging or after merging.

[lakhinderwalia]

This is still missing unit test for simplify_qdq. It really should be addressed either before merging or after merging.

Added.

[pfultz2]

There is an mlir failure on jenkins:

[2024-09-19T21:19:12.937Z] [   RUN    ] int4_unpack_conv

[2024-09-19T21:19:12.937Z] terminate called after throwing an instance of 'migraphx::version_1::exception'

[2024-09-19T21:19:12.937Z]   what():  /home/jenkins/workspace/AMDMIGraphX_PR-3322/src/targets/gpu/mlir.cpp:787: run_backend_pipeline: MLIR backend compilation failed: Error: The size of rock.alloc should be greather than zero.

[2024-09-19T21:19:12.937Z] Note: see current operation: %43 = "rock.alloc"() : () -> memref<16xi4, #gpu.address_space<private>>

Do you see this same error locally?

[lakhinderwalia]

There is an mlir failure on jenkins:

[2024-09-19T21:19:12.937Z] [   RUN    ] int4_unpack_conv

[2024-09-19T21:19:12.937Z] terminate called after throwing an instance of 'migraphx::version_1::exception'

[2024-09-19T21:19:12.937Z]   what():  /home/jenkins/workspace/AMDMIGraphX_PR-3322/src/targets/gpu/mlir.cpp:787: run_backend_pipeline: MLIR backend compilation failed: Error: The size of rock.alloc should be greather than zero.

[2024-09-19T21:19:12.937Z] Note: see current operation: %43 = "rock.alloc"() : () -> memref<16xi4, #gpu.address_space<private>>

Do you see this same error locally?

No.

[pfultz2]

If this needs a change from mlir to work, then just comment out the broken test for now so we can merge this in.

[lakhinderwalia]

Thanks. Actually, MLIR has a fix just commited-in. But I can disable the test. However, we do need one more approval :-)

[causten]

I've got #3467 which is pulling from mlir tip. I"m planning to merge it asap

[lakhinderwalia]

I've got #3467 which is pulling from mlir tip. I"m planning to merge it asap

@causten, please include 1b99123a4e86629b07b1c2815668f6222d093d70 or newer. Thanks.

[migraphx-bot]

Test	Batch	Rate new 151126	Rate old e230c0	Diff	Compare
torchvision-resnet50	64	3,257.30	3,249.77	0.23%	✅
torchvision-resnet50_fp16	64	6,992.53	6,987.71	0.07%	✅
torchvision-densenet121	32	2,434.16	2,431.48	0.11%	✅
torchvision-densenet121_fp16	32	4,083.73	4,103.92	-0.49%	✅
torchvision-inceptionv3	32	1,635.86	1,637.67	-0.11%	✅
torchvision-inceptionv3_fp16	32	2,747.11	2,744.19	0.11%	✅
cadene-inceptionv4	16	779.17	779.19	-0.00%	✅
cadene-resnext64x4	16	808.07	808.74	-0.08%	✅
slim-mobilenet	64	7,457.20	7,462.54	-0.07%	✅
slim-nasnetalarge	64	208.33	208.50	-0.08%	✅
slim-resnet50v2	64	3,441.47	3,435.17	0.18%	✅
bert-mrpc-onnx	8	1,151.63	1,150.08	0.14%	✅
bert-mrpc-tf	1	322.39	314.23	2.60%	✅
pytorch-examples-wlang-gru	1	426.46	420.51	1.42%	✅
pytorch-examples-wlang-lstm	1	377.66	495.59	-23.79%	🔴
torchvision-resnet50_1	1	813.07	770.67	5.50%	🔆
cadene-dpn92_1	1	398.84	402.30	-0.86%	✅
cadene-resnext101_1	1	380.22	381.59	-0.36%	✅
onnx-taau-downsample	1	344.47	343.63	0.24%	✅
dlrm-criteoterabyte	1	35.03	35.05	-0.06%	✅
dlrm-criteoterabyte_fp16	1	58.16	58.08	0.14%	✅
agentmodel	1	7,894.72	8,076.83	-2.25%	✅
unet_fp16	2	57.99	57.92	0.11%	✅
resnet50v1_fp16	1	1,006.42	935.45	7.59%	🔆
resnet50v1_int8	1	975.69	956.44	2.01%	✅
bert_base_cased_fp16	64	1,172.53	1,153.21	1.68%	✅
bert_large_uncased_fp16	32	362.79	355.68	2.00%	✅
bert_large_fp16	1	210.74	211.87	-0.53%	✅
distilgpt2_fp16	16	2,205.73	2,159.18	2.16%	✅
yolov5s	1	531.90	533.70	-0.34%	✅
tinyllama	1	43.38	43.69	-0.70%	✅
vicuna-fastchat	1	168.05	172.04	-2.32%	✅
whisper-tiny-encoder	1	417.71	417.90	-0.05%	✅
whisper-tiny-decoder	1	424.38	424.90	-0.12%	✅

This build is not recommended to merge 🔴

[migraphx-bot]

     ✅ bert-mrpc-onnx: PASSED: MIGraphX meets tolerance

     ✅ bert-mrpc-tf: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

     ✅ torchvision-resnet50_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-dpn92_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-resnext101_1: PASSED: MIGraphX meets tolerance

     ✅ dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

     ✅ agentmodel: PASSED: MIGraphX meets tolerance

     ✅ unet: PASSED: MIGraphX meets tolerance

     ✅ resnet50v1: PASSED: MIGraphX meets tolerance

     ✅ bert_base_cased_fp16: PASSED: MIGraphX meets tolerance

🔴bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

     ✅ bert_large: PASSED: MIGraphX meets tolerance

     ✅ yolov5s: PASSED: MIGraphX meets tolerance

     ✅ tinyllama: PASSED: MIGraphX meets tolerance

     ✅ vicuna-fastchat: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-encoder: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-decoder: PASSED: MIGraphX meets tolerance

     ✅ distilgpt2_fp16: PASSED: MIGraphX meets tolerance