From b31d48387ebae0da9cda53af08bc2575df003af2 Mon Sep 17 00:00:00 2001 From: Dmitry Mayorov Date: Fri, 7 Sep 2018 14:02:34 +0200 Subject: [PATCH 1/3] database guidelines This is the collection of guidelines for database work. futher information can be found here https://github.com/owncloud/documentation/issues/3805 --- .../pages/app/fundamentals/database.adoc | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/modules/developer_manual/pages/app/fundamentals/database.adoc b/modules/developer_manual/pages/app/fundamentals/database.adoc index a4a1e05b9a..46d467454a 100644 --- a/modules/developer_manual/pages/app/fundamentals/database.adoc +++ b/modules/developer_manual/pages/app/fundamentals/database.adoc @@ -42,6 +42,30 @@ class AuthorDAO { } ---- +Database programming guidelines +------------------------------- + +* Always use the Query Builder +* Don't update more than 1 million rows within a transaction due to DB limitations +* Don't add more than 1000 conditions in a `WHERE ... IN ...` statement but chunk it into separate queries +* Don't add more than 999 conditions in a `WHERE ... IN ...` statement but chunk it into separate queries when using SQLite +* When processing big tables, always do this in chunks, don't store the whole table in memory +* Oracle compatibility specifics: +** For Oracle `null` and empty strings are the same thing. Need special handling to catch these cases +** When reading values, make sure to convert null to empty string when expected +** Keep this in mind when designing DB schema and values expectations. +** When using a condition based on empty strings, use `is not null` with Oracle instead +** Oracle can only compare the first 4000 bytes of a `CLOB` column +** Make sure to properly escape column names when using custom functions with `createFunction`. the escaping is usually done automatically by the query builder otherwise. Oracle is the most likely to complain about unquoted columns while other databases will work fine +* Always add the table name when calling `lastInsertId($tableName)`, it is required by Oracle to return correct values +* In general don't specify a value for autoincrement column. if you have to, keep in mind that Oracle's autoincrement trigger will get in the way on `INSERT`, so you'll need a subsequent `UPDATE` to properly adjust the value +* *Always* make sure there are unit tests for the database operations with queries to verify the result, this will help find out whether the database related code works on all databases and often times might reveal database quirks +* Running unit tests with specific databases: `make test-php TEST_PHP_SUITE=path/to/test/file.php TEST_DATABASE=$databasetype` where "$databasetype" is one of "sqlite", "mysql", "mariadb", "pgsql", "oci" and "mysqlmb4" +* String concatenation should be done like this: +** `CONCAT(str1, str2, ... strN)` for Mysql +** `str1 || str1 ... || strN` Sqlite/PgSQL/Oracle +* Use `IQueryBuilder::createPositionalParameter` instead of `IQueryBuilder::createNamedParameter` when using `like()` + [[mappers]] Mappers ------- From 4b173102ef8fa1d1678fd814ef655692ee94fbcb Mon Sep 17 00:00:00 2001 From: Dmitry Mayorov Date: Tue, 11 Sep 2018 16:36:22 +0200 Subject: [PATCH 2/3] sql lite fixes --- modules/developer_manual/pages/app/fundamentals/database.adoc | 1 - 1 file changed, 1 deletion(-) diff --git a/modules/developer_manual/pages/app/fundamentals/database.adoc b/modules/developer_manual/pages/app/fundamentals/database.adoc index 46d467454a..21a7567697 100644 --- a/modules/developer_manual/pages/app/fundamentals/database.adoc +++ b/modules/developer_manual/pages/app/fundamentals/database.adoc @@ -47,7 +47,6 @@ Database programming guidelines * Always use the Query Builder * Don't update more than 1 million rows within a transaction due to DB limitations -* Don't add more than 1000 conditions in a `WHERE ... IN ...` statement but chunk it into separate queries * Don't add more than 999 conditions in a `WHERE ... IN ...` statement but chunk it into separate queries when using SQLite * When processing big tables, always do this in chunks, don't store the whole table in memory * Oracle compatibility specifics: From a076865c001b63be7ae7ccc3fbfcde9a9eba3d61 Mon Sep 17 00:00:00 2001 From: Dmitry Mayorov Date: Wed, 12 Sep 2018 14:35:05 +0200 Subject: [PATCH 3/3] fixes --- .../pages/app/fundamentals/database.adoc | 33 +++++++++---------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/modules/developer_manual/pages/app/fundamentals/database.adoc b/modules/developer_manual/pages/app/fundamentals/database.adoc index 21a7567697..704767fd2d 100644 --- a/modules/developer_manual/pages/app/fundamentals/database.adoc +++ b/modules/developer_manual/pages/app/fundamentals/database.adoc @@ -45,25 +45,24 @@ class AuthorDAO { Database programming guidelines ------------------------------- -* Always use the Query Builder -* Don't update more than 1 million rows within a transaction due to DB limitations -* Don't add more than 999 conditions in a `WHERE ... IN ...` statement but chunk it into separate queries when using SQLite -* When processing big tables, always do this in chunks, don't store the whole table in memory +* Always use the Query Builder. +* Don't update more than 1 million rows within a transaction due to DB limitations. +* Don't add more than 999 conditions in a `WHERE ... IN ...` statement but chunk it into separate queries when using SQLite. +* When processing big tables, always do this in chunks, don't store the whole table in memory. * Oracle compatibility specifics: -** For Oracle `null` and empty strings are the same thing. Need special handling to catch these cases -** When reading values, make sure to convert null to empty string when expected -** Keep this in mind when designing DB schema and values expectations. -** When using a condition based on empty strings, use `is not null` with Oracle instead -** Oracle can only compare the first 4000 bytes of a `CLOB` column -** Make sure to properly escape column names when using custom functions with `createFunction`. the escaping is usually done automatically by the query builder otherwise. Oracle is the most likely to complain about unquoted columns while other databases will work fine -* Always add the table name when calling `lastInsertId($tableName)`, it is required by Oracle to return correct values -* In general don't specify a value for autoincrement column. if you have to, keep in mind that Oracle's autoincrement trigger will get in the way on `INSERT`, so you'll need a subsequent `UPDATE` to properly adjust the value -* *Always* make sure there are unit tests for the database operations with queries to verify the result, this will help find out whether the database related code works on all databases and often times might reveal database quirks -* Running unit tests with specific databases: `make test-php TEST_PHP_SUITE=path/to/test/file.php TEST_DATABASE=$databasetype` where "$databasetype" is one of "sqlite", "mysql", "mariadb", "pgsql", "oci" and "mysqlmb4" +** For Oracle, `null` and empty strings are the same thing. Special handling is required to catch these cases. +** When reading values, make sure to convert nulls to empty strings when expected. +** When using a condition based on empty strings, use `is not null` with Oracle instead. +** Oracle can only compare the first 4000 bytes of a `CLOB` column. +** Make sure to properly escape column names when using custom functions with `createFunction`. The escaping is usually done automatically by the query builder. Oracle is the most likely to complain about unquoted columns while other databases will work fine. +** Always add the table name when calling `lastInsertId($tableName)`, as it is required by Oracle to return correct values. +* In general, don't specify a value for an `autoincrement` column. If you have to, keep in mind that Oracle's `autoincrement` trigger will get in the way on `INSERT`. As a result, you'll need a subsequent `UPDATE` to properly adjust the value. +* *Always* make sure there are unit tests for the database operations with queries to verify the result. This will help find out whether the database related code works on all databases and often times might reveal database quirks. +* Running unit tests with specific databases: `make test-php TEST_PHP_SUITE=path/to/test/file.php TEST_DATABASE=$databasetype` where "$databasetype" is one of "sqlite", "mysql", "mariadb", "pgsql", "oci" and "mysqlmb4". * String concatenation should be done like this: -** `CONCAT(str1, str2, ... strN)` for Mysql -** `str1 || str1 ... || strN` Sqlite/PgSQL/Oracle -* Use `IQueryBuilder::createPositionalParameter` instead of `IQueryBuilder::createNamedParameter` when using `like()` +** `CONCAT(str1, str2, ... strN)` for MYSQL. +** `str1 || str1 ... || strN` SQLite/pgSQL/Oracle. +* Use `IQueryBuilder::createPositionalParameter` instead of `IQueryBuilder::createNamedParameter` when using `like()`. [[mappers]] Mappers