Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] be http server not ok #27671

Closed
3 tasks done
xingyingone opened this issue Nov 28, 2023 · 4 comments · Fixed by #27752
Closed
3 tasks done

[Bug] be http server not ok #27671

xingyingone opened this issue Nov 28, 2023 · 4 comments · Fixed by #27752

Comments

@xingyingone
Copy link
Contributor

xingyingone commented Nov 28, 2023 •
edited
Loading

Search before asking

  • I had searched in the issues and found no similar issues.

Version

2.0.3

What's Wrong?

be进程存在，且心跳正常，执行查询ok。但是http服务貌似停止了，无法正常采集metric
这里可以看到心跳正常
image
这里可以看到http拒绝服务
image
doris manager显示这个be无效了
image

What You Expected?

正常采集metric

How to Reproduce?

不知

Anything Else?

无

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@xingyingone
Copy link
Contributor Author

pstack打印的be堆栈

135102.log

@xingyingone
Copy link
Contributor Author

导入会失败，一直hang住
image

@xingyingone
Copy link
Contributor Author

xingyingone commented Nov 28, 2023 •
edited
Loading

一些线索：

  1. 在故障前，在执行高并发stream load导入操作。100个线程池，每个线程池100个线程，即最高1w并发，实际上应该有几百导入并发
  2. be stack中，http server的线程池一直在等stream load结束，所以阻塞了
    image
    对应代码：
    image
  3. stream load任务貌似阻塞了
    image
    对应代码：
    image
  4. 目前写入停止了，没有任何负载，但是http服务还是没有恢复

@xingyingone
Copy link
Contributor Author

11.27号晚上发现http服务出问题了
image

@xingyingone xingyingone mentioned this issue Nov 29, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[fix](stream_load)fix bug when stream without content-length or chunk… xingyingone/doris
1 participant
@xingyingone