-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Benchmarking sharded cluster mongodb with YCSB #1673
Comments
Your database size is about 10 GB, which fits completely inside the RAM of one server. In that scenario, no performance benefit from sharding is possible. This is assuming you left fieldcount and fieldlength with default values. You should experiment with larger record counts, like 40,000,000 for example. You should also run the db.stats() command to examine the size of your database. |
@twblamer Sorry for the late reply. But single db still fater than sharded cluster... Besides, my database is new and I will clean data before I run ycsb. |
With the sharding you used ( I recommend doing the pre-split on a clean, unused cluster. Then load and run the benchmark. Your virtual machines are probably a little undersized CPU wise (2 vCPUs) given you are throwing 500 threads of concurrency at the cluster probably from a Windows desktop with 16 or more real core/threads... |
@allanbank
I also ran the
Seems like usertable still not have chunk after ran the script. |
That certainly does not look like the split worked. I think the bug is in the script that the lines:
Should be:
Can you edit the script and see if that does a better job? |
@allanbank Thanks for the reply.
And I collect some weird log as below, hope it could help.
|
I tried upgrade my mongo db to 6.0.3 and redeploy sharded cluster db.
|
Do you have data in the database? If so the moves can take minutes to 10's of minutes to run. (The first move could move about 99% of the data, The second, 98%. etc. Data migration is intentionally not a speedy process. If not then the issue might be that your config server cluster is not healthy so the move is not progressing. If you want to leave the data in the script then do the splitAt(...) in one loop and then the moveChunk(...) in a separate loop after all of the splitAt(...) commands have completed. |
Hi, I am new with mongodb and YCSB.
I have 2 type of mongo db:
I assume that sharded cluster db would have better performance than single db.
But after test with YCSB, I got the test result as below:
Seems like Shard cluster didn't have better performance than single mongodb.
I don't know why it happened, did I miss something?
Below are my environment information, let me know if you want to know more information.
Environment info :
The way I test single db
The way I test cluster db
The text was updated successfully, but these errors were encountered: