Skip to content

Commit

Permalink
[vectorized](jdbc) fix external table of oracle have keyworld column (#…
Browse files Browse the repository at this point in the history
…15487)

if column name is keyword of oracle, the query will report error
  • Loading branch information
zhangstar333 authored and morningman committed Dec 31, 2022
1 parent 1170848 commit 389e323
Show file tree
Hide file tree
Showing 4 changed files with 12 additions and 2 deletions.
2 changes: 2 additions & 0 deletions docs/en/docs/ecosystem/external-table/jdbc-of-doris.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,8 @@ Parameter Description:
```
select * from mysql_table where k1 > 1000 and k3 ='term';
```
Because it is possible to use keywords in the database as column name, in order to solve this problem, escape characters will be automatically added to field names and table names in SQL statements according to the standards of each database. For example, MYSQL (``), PostgreSQL (""), SQLServer ([]), and ORACLE (""), But this may cause case sensitivity of field names. You can check the query statements issued to each database after escape through explain SQL.

### Data write

After the JDBC external table is create in Doris, the data can be written directly by the `insert into` statement, the query results of Doris can be written to the JDBC external table, or the data can be imported from one JDBC table to another.
Expand Down
1 change: 1 addition & 0 deletions docs/zh-CN/docs/ecosystem/external-table/jdbc-of-doris.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ PROPERTIES (
```
select * from mysql_table where k1 > 1000 and k3 ='term';
```
由于可能存在使用数据库内部的关键字作为字段名,为解决这种状况下仍能正确查询,所以在SQL语句中,会根据各个数据库的标准自动在字段名与表名上加上转义符。例如 MYSQL(``)、PostgreSQL("")、SQLServer([])、ORACLE(""),所以此时可能会造成字段名的大小写敏感,具体可以通过explain sql,查看转义后下发到各个数据库的查询语句。

### 数据写入

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,11 @@ private static String psqlProperName(String name) {
return list.stream().map(s -> "\"" + s + "\"").collect(Collectors.joining("."));
}

private static String oracleProperName(String name) {
List<String> list = Arrays.asList(name.split("\\."));
return list.stream().map(s -> "\"" + s.toUpperCase() + "\"").collect(Collectors.joining("."));
}

public static String databaseProperName(TOdbcTableType tableType, String name) {
switch (tableType) {
case MYSQL:
Expand All @@ -104,6 +109,8 @@ public static String databaseProperName(TOdbcTableType tableType, String name) {
return mssqlProperName(name);
case POSTGRESQL:
return psqlProperName(name);
case ORACLE:
return oracleProperName(name);
default:
return name;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1312,7 +1312,7 @@ public void testPushDownOfOdbcTable() throws Exception {
// this table is Oracle ODBC table, so abs(k1) should not be pushed down
queryStr = "explain select * from odbc_oracle where k1 > 10 and abs(k1) > 10";
explainString = getSQLPlanOrErrorMsg(queryStr);
Assert.assertTrue(explainString.contains("k1 > 10"));
Assert.assertTrue(explainString.contains("\"K1\" > 10"));
Assert.assertTrue(!explainString.contains("abs(k1) > 10"));
}

Expand Down Expand Up @@ -1352,7 +1352,7 @@ public void testOdbcSink() throws Exception {
String explainString = getSQLPlanOrErrorMsg(queryStr);
Assert.assertTrue(explainString.contains("TABLENAME IN DORIS: odbc_oracle"));
Assert.assertTrue(explainString.contains("TABLE TYPE: ORACLE"));
Assert.assertTrue(explainString.contains("TABLENAME OF EXTERNAL TABLE: tbl1"));
Assert.assertTrue(explainString.contains("TABLENAME OF EXTERNAL TABLE: \"TBL1\""));

// enable transaction of ODBC Sink
Deencapsulation.setField(connectContext.getSessionVariable(), "enableOdbcTransaction", true);
Expand Down

0 comments on commit 389e323

Please sign in to comment.