24.3.5 Backport of PR #63502 : Support running function for parameterized view value assignment

[shiyer7474]

Original PR : ClickHouse#63502.

Changelog category (leave one):

• Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Support executing function during assignment of parameterized view value

[altinity-robot]

This is an automated comment for commit 095ada5 with description of existing statuses. It's updated for the latest CI running

❌ Click here to open a full report in a separate page

Check name	Description	Status
CI running	A meta-check that indicates the running CI. Normally, it's in success or pending state. The failed status indicates some problems with the PR	⏳ pending
ClickHouse build check	Builds ClickHouse in various configurations for use in further steps. You have to fix the builds that fail. Build logs often has enough information to fix the error, but you might have to reproduce the failure locally. The cmake options can be found in the build log, grepping for cmake. Use these options and follow the general build process	⏳ pending
Integration tests	The integration tests report. In parenthesis the package type is given, and in square brackets are the optional part/total tests	❌ failure
Mergeable Check	Checks if all other necessary checks are successful	❌ failure
Stateless tests	Runs stateless functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc	❌ failure

Successful checks

Check name	Description	Status
Compatibility check	Checks that clickhouse binary runs on distributions with old libc versions. If it fails, ask a maintainer for help	✅ success
Docker keeper image	The check to build and optionally push the mentioned image to docker hub	✅ success
Docker server image	The check to build and optionally push the mentioned image to docker hub	✅ success
Install packages	Checks that the built packages are installable in a clear environment	✅ success
Ready for release	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Stateful tests	Runs stateful functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc	✅ success
Stress test	Runs stateless functional tests concurrently from several clients to detect concurrency-related errors	✅ success

[shiyer7474]

Retried the failed jobs a couple of times. I looked at all the current failures and this is the summary :-

Many suites are failing with due to error in test_storage_s3_queue/test.py::test_upgrade

IntegrationTestsAnalyzerAsan / Integration tests (asan, old analyzer)-1
INFO:root:Killing all alive docker containers
INFO:root:Removing all docker containers
iptables: Index of deletion too big.
INFO:root:All iptables rules cleared, 0 iterations, last error: Command 'sudo iptables -D DOCKER-USER 1' returned non-zero exit status 1.
INFO:root:Executing cmd: cd /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration && timeout --signal=KILL 2h ./runner --disable-net-host --analyzer --docker-compose-images-tags=altinityinfra/dotnet-client:11de0b29a15d --docker-compose-images-tags=altinityinfra/integration-helper:2cffe1eae894 --docker-compose-images-tags=altinityinfra/integration-test:1d1c87eee2db  --docker-image-version=9d492c2eec24 --docker-compose-images-tags=altinityinfra/kerberized-hadoop:ce74919e88f5 --docker-compose-images-tags=altinityinfra/kerberos-kdc:9391ecdee8d7 --docker-compose-images-tags=altinityinfra/mysql-golang-client:9bec2a638e6e --docker-compose-images-tags=altinityinfra/mysql-java-client:766bff31cfe4 --docker-compose-images-tags=altinityinfra/mysql-js-client:41ba7c2ec2a1 --docker-compose-images-tags=altinityinfra/mysql-php-client:88be89c1e3b6 --docker-compose-images-tags=altinityinfra/nginx-dav:b55ac9cd7519 --docker-compose-images-tags=altinityinfra/postgresql-java-client:a4eff5c7f4d6 --docker-compose-images-tags=altinityinfra/python-bottle:a2d3dc777d0c  -t test_storage_s3_queue/test.py::test_upgrade  -- -rfEps --run-id=2 --color=no --durations=0  | tee /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/parallel3_2.nfo
INFO:root:Tests with ERROR state (0): []
INFO:root:Tests with PASSED state (0): []
INFO:root:Tests with FAILED state (1): ['test_storage_s3_queue/test.py::test_upgrade']

3 suites are failing because of errors in these tests :-

TieredStorage(minio)
TieredStorage(s3amazon)
TieredStorage(s3gcs)

3. ssl_server is failing in 2 suites with an unclear error (I saw some FIPS related messages also)

RegressionTestsRelease / Common (ssl_server)
➤  Aug 24,2024 5:48:17 /ssl server/zookeeper
➤  Aug 24,2024 5:48:40 /ssl server/zookeeper/secure connection
✔ 31s 552ms [   OK   ] /ssl server/zookeeper/secure connection
➤  Aug 24,2024 5:49:12 /ssl server/zookeeper/secure connection to invalid zookeeper port
✔ 2m 56s    [   OK   ] /ssl server/zookeeper/secure connection to invalid zookeeper port
➤  Aug 24,2024 5:52:09 /ssl server/zookeeper/secure connection with empty truststore
✔ 2m 56s    [   OK   ] /ssl server/zookeeper/secure connection with empty truststore
➤  Aug 24,2024 5:55:05 /ssl server/zookeeper/secure connection 

with unsigned client certificate
✔ 2m 59s    [   OK   ] /ssl server/zookeeper/secure connection with unsigned client certificate
➤  Aug 24,2024 5:58:04 /ssl server/zookeeper/secure connection without client certificate
Error: The operation was canceled.

Below failure is unrelated to this PR

RegressionTestsAarch64 / Common (memory)
Aug 24,2024 2:31:45 /memory
➤  Aug 24,2024 2:33:37 /memory/memory
➤  Aug 24,2024 2:33:37 /memory/memory/check leak with system memory dump log
✄ 67ms      [  Skip  ] /memory/memory/check leak with system memory dump log
✔ 69ms      [   OK   ] /memory/memory
➤  Aug 24,2024 2:33:37 /memory/memory leak
➤  Aug 24,2024 2:33:37 /memory/memory leak/check create and drop tables
✘ 57m 33s   [  Fail  ] /memory/memory leak/check create and drop tables
    AssertionError
✘ 57m 33s   [  Fail  ] /memory/memory leak
    AssertionError
✘ 59m 32s   [  Fail  ] /memory
    AssertionError
    Traceback (most recent call last):
      File "/home/ubuntu/_work/ClickHouse/ClickHouse/memory/regression.py", line 66, in <module>
        regression()
      File "/home/ubuntu/_work/ClickHouse/ClickHouse/memory/../helpers/argparser.py", line 145, in capture_cluster_args
        return func(self, cluster_args=cluster_args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/ubuntu/_work/ClickHouse/ClickHouse/memory/regression.py", line 62, in regression
        Feature(run=load("memory.tests.test_memory_leak", "feature"))
      File "/home/ubuntu/_work/ClickHouse/ClickHouse/memory/../memory/tests/test_memory_leak.py", line 217, in feature
        scenario()
      File "/home/ubuntu/_work/ClickHouse/ClickHouse/memory/../memory/tests/test_memory_leak.py", line 206, in check_create_and_drop_tables
        current_memory.resident <= max_memory
    AssertionError: Memory usage 242157 larger than expected 220000

Passing

✔ [ Skip ] '/memory/memory/check leak with system memory dump log' (67ms)
✔ [ OK ] '/memory/memory' (69ms)

Known

✘ [ XFail ] '/memory/I clean up/I clean up/collect service logs/getting log for clickhouse1' (11ms)
✘ [ XFail ] '/memory/I clean up/I clean up/collect service logs/getting log for clickhouse2' (9ms)
✘ [ XFail ] '/memory/I clean up/I clean up/collect service logs/getting log for clickhouse3' (9ms)
✘ [ XFail ] '/memory/I clean up/I clean up/collect service logs/getting log for clickhouse4' (8ms)

Failing

✘ [ Fail ] '/memory/memory leak/check create and drop tables' (57m 33s)
✘ [ Fail ] '/memory/memory leak' (57m 33s)
✘ [ Fail ] '/memory' (59m 32s)

Below failure also unrelated to PR

FunctionalStatelessTestMsan / Stateless tests (msan)-4

2024-08-23 18:39:55 01721_join_implicit_cast_long:                                          [ FAIL ] 40.04 sec. - return code: 241
2024-08-23 18:39:55 Received exception from server (version 24.3.5):
2024-08-23 18:39:55 Code: 241. DB::Exception: Received from localhost:9000. DB::Exception: Memory limit (total) exceeded: would use 28.71 GiB (attempt to allocate chunk of 0 bytes), maximum: 27.54 GiB. OvercommitTracker decision: Memory overcommit has freed not enough memory.: While executing AggregatingTransform. (MEMORY_LIMIT_EXCEEDED)

[filimonov]

What's about ClickHouse#63502 ? I guess we also need it...

[shiyer7474]

Yes, this PR is for backporting PR 63502 to Altinity 24.3.5. Once this is approved & merged, I will backport my fix in PR ClickHouse#67654.