Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bugfix](wgcore) map at only get reference and it will core in multithread #31702

Merged
merged 2 commits into from
Mar 3, 2024

Conversation

yiguolei
Copy link
Contributor

@yiguolei yiguolei commented Mar 3, 2024

Proposed changes

  1. map.at method only get a reference of the task group.
  2. in multi thread env, the task group maybe erased by another thread.
  3. map.at()->stop_task_schedulers will core.

==1697844==ERROR: AddressSanitizer: heap-use-after-free on address 0x6060231f4f00 at pc 0x5568f726c610 bp 0x7f876c41d6f0 sp 0x7f876c41d6e8
READ of size 8 at 0x6060231f4f00 thread T2671
#0 0x5568f726c60f in std::__cxx11::basic_string, std::allocator>::_M_data() const /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187:28
#1 0x5568f726cc8a in std::__cxx11::basic_string, std::allocator>::_M_is_local() const /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:222:16
#2 0x5568f726c64a in std::__cxx11::basic_string, std::allocator>::_M_dispose() /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:231:7
#3 0x5568f7265329 in std::__cxx11::basic_string, std::allocator>::~basic_string() /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:658:9
#4 0x5568f727dcec in doris::Status::ErrMsg::~ErrMsg() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/common/status.h:499:12
#5 0x5568f727db2c in std::default_delete::operator()(doris::Status::ErrMsg*) const /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:85:2
#6 0x5568f727e503 in std::__uniq_ptr_impl>::reset(doris::Status::ErrMsg*) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:182:4
#7 0x5568f727e41e in std::__uniq_ptr_impl>::operator=(std::__uniq_ptr_impl>&&) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:167:2
#8 0x5568f727e3d2 in std::__uniq_ptr_data, true, true>::operator=(std::__uniq_ptr_data, true, true>&&) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:212:61
#9 0x5568f727e392 in std::unique_ptr>::operator=(std::unique_ptr>&&) /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:371:51
#10 0x5568f72693e2 in doris::Status::operator=(doris::Status&&) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/common/status.h:347:22
#11 0x5568fb5c7368 in doris::ThreadPool::shutdown() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/util/threadpool.cpp:284:18
#12 0x5568faef0d02 in doris::taskgroup::TaskGroup::try_stop_schedulers() /home/zcp/repo_center/doris_branch-2.1/doris/be/src/runtime/task_group/task_group.cpp:433:32
#13 0x5568faf16076 in doris::taskgroup::TaskGroupManager::delete_task_group_by_ids(std::set, std::allocator>) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/runtime/task_group/task_group_manager.cpp:94:33
#14 0x5568fb04b337 in doris::WorkloadGroupListener::handle_topic_info(std::vector> const&) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/agent/workload_group_listener.cpp:65:38
#15 0x5568fb0432ad in doris::TopicSubscriber::handle_topic_info(doris::TPublishTopicRequest const&) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/agent/topic_subscriber.cpp:46:35
#16 0x5568faff92d6 in doris::BackendService::publish_topic_info(doris::TPublishTopicResult&, doris::TPublishTopicRequest const&) /home/zcp/repo_center/doris_branch-2.1/doris/be/src/service/backend_service.h:97:48
#17 0x5568fb8076ea in doris::BackendServiceProcessor::process_publish_topic_info(int, apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, void*) /home/zcp/repo_center/doris_branch-2.1/doris/gensrc/build/gen_cpp/BackendService.cpp:6690:13
#18 0x5568fb7eab88 in doris::BackendServiceProcessor::dispatchCall(apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, std::__cxx11::basic_string, std::allocator> const&, int, void*) /home/zcp/repo_center/doris_branch-2.1/doris/gensrc/build/gen_cpp/BackendService.cpp:5492:3
#19 0x5568f72a2f3b in apache::thrift::TDispatchProcessor::process(std::shared_ptr, std::shared_ptr, void*) /home/zcp/repo_center/doris_branch-2.1/doris/thirdparty/installed/include/thrift/TDispatchProcessor.h:121:12
#20 0x55692b217991 in apache::thrift::server::TConnectedClient::run() (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x599ea991) (BuildId: b0fb5c12e333cb73)
#21 0x55692b218c1c in apache::thrift::server::TThreadedServer::TConnectedClientRunner::run() (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x599ebc1c) (BuildId: b0fb5c12e333cb73)
#22 0x55692b21da95 in apache::thrift::concurrency::Thread::threadMain(std::shared_ptr) (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x599f0a95) (BuildId: b0fb5c12e333cb73)
#23 0x55692b21d93b in void std::__invoke_impl), std::shared_ptr>(std::__invoke_other, void (&&)(std::shared_ptr), std::shared_ptr&&) (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x599f093b) (BuildId: b0fb5c12e333cb73)
#24 0x55692b21d88e in std::__invoke_result), std::shared_ptr>::type std::__invoke), std::shared_ptr>(void (
&&)(std::shared_ptr), std::shared_ptr&&) (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x599f088e) (BuildId: b0fb5c12e333cb73)
#25 0x55692b21d7fe in void std::thread::_Invoker), std::shared_ptr>>::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x599f07fe) (BuildId: b0fb5c12e333cb73)
#26 0x55692b21d783 in std::thread::_Invoker), std::shared_ptr>>::operator()() (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x599f0783) (BuildId: b0fb5c12e333cb73)
#27 0x55692b21d727 in std::thread::_State_impl), std::shared_ptr>>>::_M_run() (/mnt/hdd01/ci/branch21-deploy/be/lib/doris_be+0x599f0727) (BuildId: b0fb5c12e333cb73)
#28 0x55692daa4c6f in execute_native_thread_routine /data/gcc-11.1.0/build/x86_64-pc-linux-gnu/libstdc++-v3/src/c++11/../../../../../libstdc++-v3/src/c++11/thread.cc:82:18
#29 0x7f93673ed608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
#30 0x7f936769a132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@yiguolei
Copy link
Contributor Author

yiguolei commented Mar 3, 2024

run buildall

Copy link
Contributor

github-actions bot commented Mar 3, 2024

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
Copy link
Contributor

github-actions bot commented Mar 3, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.43% (8554/24146)
Line Coverage: 27.20% (69410/255227)
Region Coverage: 26.41% (36006/136337)
Branch Coverage: 23.25% (18394/79100)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0becdae596d1f437ff39b455192fbad358337469_0becdae596d1f437ff39b455192fbad358337469/report/index.html

Copy link
Contributor

github-actions bot commented Mar 3, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Mar 3, 2024
Copy link
Contributor

github-actions bot commented Mar 3, 2024

PR approved by anyone and no changes requested.

@yiguolei yiguolei merged commit d8b4edb into apache:master Mar 3, 2024
25 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed workload-group
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants