-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow a server to host segments on multi storage tiers #8843
Comments
found another related issue: #6343 |
hi, this issue had a design doc earlier, but now i see it's been removed and I don't seem to have permissions to access it anymore. Can you provide access? |
Proposed some ideas about how to tackle this issue in design doc. Feel free to comment. |
The major functionality is completed. What's remaining is about how to cleanup orphan segments, which is a related problem but orthogonal, because orphan segments can happen in many situations other than tier migration, as detailed in the design doc above. I'll address that separately. |
As of today, a table with multi-tiers has to use different server pools to support multi storage tiers, and relies on SegmentRelocator to move the segments among those server pools to put them on the right storage tier. An example cluster setup may be like using a pool of servers with SSD as a tier and another pool of servers with HDD as a second tier.
But this model doesn't work when a server have both SSD and HDD, and wants to define multi tiers backed by them.
We'd like to address this issue. And one possible way is to implement a SegmentDirectoryLoader that's aware of multi data dirs; and enhance current way of tier migration to move segments across tiers even if they are not relocated to a different server, i.e. to be done on the same server locally.
The text was updated successfully, but these errors were encountered: