Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(source): PostgreSQL/MySQL CDC source vs table relationship #2449

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix(source): PostgreSQL/MySQL CDC source vs table relationship #2449

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/guides/ingest-from-mysql-cdc.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,11 +149,11 @@ If you are running RisingWave locally from binaries and intend to use the native

## Create a table using the native CDC connector in RisingWave

To ensure all data changes are captured, you must create a table and specify primary keys. See the [`CREATE TABLE`](/sql/commands/sql-create-table.md) command for more details.
To ensure all data changes are captured, you must create a table and specify primary keys.

### Syntax

Syntax for creating a CDC source.
Syntax for creating a shared CDC source.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to clarify the logic first: need to create a source before creating table

Suggested change
Syntax for creating a shared CDC source.
Before creating a table, you need to first create a source to the upstream MySQL database using the following syntax. This source can be shared by multiple tables from the same MySQL database.


```sql
CREATE SOURCE [ IF NOT EXISTS ] source_name WITH (
Expand All @@ -162,7 +162,7 @@ CREATE SOURCE [ IF NOT EXISTS ] source_name WITH (
);
```

Syntax for creating a CDC table. Note that a primary key is required and must be consistent with the upstream table.
Syntax for creating a CDC table based on the shared source. Note that a primary key is required and must be consistent with the upstream table.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better use a complete sentence.

Suggested change
Syntax for creating a CDC table based on the shared source. Note that a primary key is required and must be consistent with the upstream table.
Below is the syntax for creating a CDC table based on the shared source. Note that a primary key is required and must be consistent with the upstream table.


```sql
CREATE TABLE [ IF NOT EXISTS ] table_name (
Expand All @@ -173,7 +173,7 @@ CREATE TABLE [ IF NOT EXISTS ] table_name (
WITH (
snapshot='true'
)
FROM source TABLE table_name;
FROM source_name TABLE table_name;
```

### Connector parameters
Expand Down
8 changes: 4 additions & 4 deletions docs/guides/ingest-from-postgres-cdc.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,11 +151,11 @@ If you are running RisingWave locally from binaries and intend to use the native

## Create a table using the native CDC connector

To ensure all data changes are captured, you must create a table or source and specify primary keys. See the [`CREATE TABLE`](/sql/commands/sql-create-table.md) command for more details.
To ensure all data changes are captured, you must create a shared source first and create tables based on that source.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's ok as of now in this PR, but I feel it would be more useful if we have a topic or section about "shared sources" that we can now point to. There are a few RisingWave-specific concepts that we may want to elaborate somewhere. In addition, the "Sources" term in Glossary should be updated. cc: @st1page @WanYixian

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. The shared source is still in development by @xxchan and it is expected to release in v2.1. I think we will add the doc that time.


### Syntax

Syntax for creating a CDC source.
Syntax for creating a shared CDC source.

```sql
CREATE SOURCE [ IF NOT EXISTS ] source_name WITH (
Expand All @@ -164,7 +164,7 @@ CREATE SOURCE [ IF NOT EXISTS ] source_name WITH (
);
```

Syntax for creating a CDC table. Note that a primary key is required and must be consistent with the upstream table.
Syntax for creating a CDC table based on the shared source. Note that a primary key is required and must be consistent with the upstream table.

```sql
CREATE TABLE [ IF NOT EXISTS ] table_name (
Expand All @@ -175,7 +175,7 @@ CREATE TABLE [ IF NOT EXISTS ] table_name (
WITH (
snapshot='true'
)
FROM source TABLE table_name;
FROM source_name TABLE table_name;
```

To check the progress of backfilling historical data, find the corresponding internal table using the [`SHOW INTERNAL TABLES`](/sql/commands/sql-show-internal-tables.md) command and query from it.
Expand Down
Loading