HOME
Ptubes is a database disaster recovery product based on PITR (Point In Time Recovery), which can be used to restore the entire database to a specific point in time to help users improve the reliability and security of the database. The product consists of three core components, Reader, Storage and SDK.
| Module name | Function introduction |
|---|---|
| SDK | 1. A set of programming interfaces for interacting with Reader components. After receiving the Binlog sent by the Reader, use various types of database clients (supporting multiple relational databases, KV databases, message systems, RPC services, etc.) to efficiently write the Binlog to the downstream database. |
| Reader | 1. Real-time capture of the online change log (hereinafter referred to as: Binlog) generated by the source database (Mysql, Postgresql, etc.). 2. Manage table structure (hereinafter referred to as: Schema), parse Binlog and submit it to Storage according to Schema. 3. Accept the network connection request from SDK, and send Binlog to SDK through network connection. 4. A Reader cluster can accommodate multiple Reader tasks, and a Reader task is associated with a database cluster. |
| Storage | 1. Realize data persistence and support various storage modes such as disk, memory, and S3. |
Ptubes provides functions such as secure backup and efficient distribution of database change events, and can be widely used in scenarios such as data backup, data playback, data recovery, event-driven, and database multi-activity.
Ptubes act as a transmission pipeline between heterogeneous database systems, backing up the Binlog generated by the source database to the target database system in near real time.
On the one hand, when the source database is destroyed, the data of the target database can be used to continue to provide services.
On the other hand, the query pressure of the source database can be relieved, and the results expected by the user can be obtained by reading the target database.
It can also expand the computing power of the source database, and provide more diversified data processing capabilities through a variety of target database engines.
Taking Mysql as an example, Ptubes can periodically generate a complete data image (Snapshot), and use the adjacent Snapshot to superimpose the incremental Binlog to construct complete data at any time.
Ptubes has the ability to parse and reverse Binlog, and can convert Binlog into a reverse SQL operation to restore the database to any time in history.
When a specific data change is found, Ptubes acts as a message service, which can send the change as a message and ensure that the message is not lost, and trigger the execution of one or more services in the subsequent process.
It can be widely used in scenarios such as eventual consistency between services and event notification.
Through Ptubes, the data written by the user into the database can be synchronized to another database thousands of kilometers away in near real time, so that the database has the capability of disaster recovery across computer rooms and even across cities.