Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Usage of bind variables for volatile filter conditions (2) #2087

Closed
wants to merge 44 commits into from
Closed

Usage of bind variables for volatile filter conditions (2) #2087

wants to merge 44 commits into from

Conversation

rammpeter
Copy link
Contributor

Bind variables are used for more volatile columns owner and table name.
For our larger OLTP systems it is essential to run them with cursor_sharing=exact.
In this case dictionary access without bind variables leads to significant longer CI pipeline runtime as well as production load.

rammpeter and others added 6 commits December 7, 2020 09:55
This is useful especially in case of cursor_sharing=exact

(cherry picked from commit f8b7a1a)
(cherry picked from commit f8021f8)
* Set JDBC statement cache size in database config

(cherry picked from commit 1c52814)

* Fix rubocop findings

(cherry picked from commit e099b4f)
@yahonda
Copy link
Collaborator

yahonda commented Dec 30, 2020

Thanks for the pull request.
column_definitions and pk_and_sequence_for methods uses SQL literals intentionally since #1713 to fix #1678.
Would you restore original behavior for these methods? Thank you.

@rammpeter
Copy link
Contributor Author

Excuse me please for inconvenience in this case.
If it would be ok for you, I‘d add a test case first for the problem described in #1678 to raise a test error under current pull request state and fix this than.

yegayazilim and others added 3 commits January 1, 2021 13:47
* Fix write_lobs "invalid byte sequence in UTF-8"

write_lobs checks to see whether the value is blank before executing the SQL statement. If the LOB is binary blank? call causes "invalid byte sequence in UTF-8" exception. We can instead use unless value, which is applicable both to strings and binary data.

* check serialized columns to be nil

ActiveRecord persists empty Array ([]) and Hash ({}) as nil. So, after serializing the value need to check again.

* Save non UTF-8 string in binary column

* Fix layout offense

* Fix space offenses

* copy test string in order to force encoding

* Use unary plus to unfreeze
* Without this commit

```ruby
$ ruby active_record_gem_spec.rb
Fetching https://github.com/rails/rails.git
... snip ...
Using rails 6.2.0.alpha from https://github.com/rails/rails.git (at master@afc79e3)
$
```

* With this commit

```ruby
$ ruby active_record_gem_spec.rb
Fetching https://github.com/rails/rails.git
... snip ...
Using rails 6.2.0.alpha from https://github.com/rails/rails.git (at master@afc79e3)
-- create_table(:posts, {:force=>true})
D, [2021-01-01T17:27:35.727438 #554514] DEBUG -- :    (20.9ms)  DROP TABLE "POSTS"
D, [2021-01-01T17:27:35.733266 #554514] DEBUG -- :    (5.4ms)  DROP SEQUENCE "POSTS_SEQ"
D, [2021-01-01T17:27:35.748060 #554514] DEBUG -- :    (14.4ms)  CREATE TABLE "POSTS" ("ID" NUMBER(38) NOT NULL PRIMARY KEY)
D, [2021-01-01T17:27:35.751081 #554514] DEBUG -- :    (2.8ms)  CREATE SEQUENCE "POSTS_SEQ" START WITH 1
   -> 0.1797s
-- create_table(:comments, {:force=>true})
D, [2021-01-01T17:27:35.869071 #554514] DEBUG -- :    (21.4ms)  DROP TABLE "COMMENTS"
D, [2021-01-01T17:27:35.875660 #554514] DEBUG -- :    (6.3ms)  DROP SEQUENCE "COMMENTS_SEQ"
D, [2021-01-01T17:27:35.884094 #554514] DEBUG -- :    (8.0ms)  CREATE TABLE "COMMENTS" ("ID" NUMBER(38) NOT NULL PRIMARY KEY, "POST_ID" NUMBER(38))
D, [2021-01-01T17:27:35.887143 #554514] DEBUG -- :    (2.8ms)  CREATE SEQUENCE "COMMENTS_SEQ" START WITH 1
   -> 0.1359s
D, [2021-01-01T17:27:36.043216 #554514] DEBUG -- :   ActiveRecord::InternalMetadata Load (1.4ms)  SELECT "AR_INTERNAL_METADATA".* FROM "AR_INTERNAL_METADATA" WHERE "AR_INTERNAL_METADATA"."KEY" = :a1 FETCH FIRST :a2 ROWS ONLY  [["key", "environment"], ["LIMIT", 1]]
D, [2021-01-01T17:27:36.688610 #554514] DEBUG -- :   Post Create (8.4ms)  INSERT INTO "POSTS" ("ID") VALUES (:a1)  [["id", 1]]
D, [2021-01-01T17:27:37.203028 #554514] DEBUG -- :   Comment Create (7.5ms)  INSERT INTO "COMMENTS" ("ID") VALUES (:a1)  [["id", 1]]
D, [2021-01-01T17:27:37.214316 #554514] DEBUG -- :   Comment Update (3.7ms)  UPDATE "COMMENTS" SET "POST_ID" = :a1 WHERE "COMMENTS"."ID" = :a2  [["post_id", 1], ["id", 1]]
D, [2021-01-01T17:27:37.221641 #554514] DEBUG -- :    (5.0ms)  SELECT COUNT(*) FROM "COMMENTS" WHERE "COMMENTS"."POST_ID" = :a1  [["post_id", 1]]
D, [2021-01-01T17:27:37.226691 #554514] DEBUG -- :    (4.2ms)  SELECT COUNT(*) FROM "COMMENTS"
D, [2021-01-01T17:27:37.231298 #554514] DEBUG -- :   Comment Load (3.5ms)  SELECT "COMMENTS".* FROM "COMMENTS" ORDER BY "COMMENTS"."ID" ASC FETCH FIRST :a1 ROWS ONLY  [["LIMIT", 1]]
D, [2021-01-01T17:27:37.236094 #554514] DEBUG -- :   Post Load (3.2ms)  SELECT "POSTS".* FROM "POSTS" WHERE "POSTS"."ID" = :a1 FETCH FIRST :a2 ROWS ONLY  [["id", 1], ["LIMIT", 1]]
.

Finished in 1.67 seconds (files took 0.1944 seconds to load)
1 example, 0 failures

$
```

Refer
https://relishapp.com/rspec/rspec-core/v/3-6/docs/command-line/run-with-ruby-command
Require `rspec/autorun` to run one line spec file
@yahonda
Copy link
Collaborator

yahonda commented Jan 2, 2021

I‘d add a test case first for the problem described in #1678 to raise a test error under current pull request state and fix this than.

Sounds good. As long as the issue described in #1678 is fixed with bind variables it should be fine.

…binds even if primary SQL is executed with connection.unprepared_statement (like behavior of AR.to_sql).

Preserves the function of fix for issue #1687.
@rammpeter
Copy link
Contributor Author

Issue described in #1678 is fixed now with bind variables.

yahonda and others added 6 commits January 9, 2021 17:42
This cop should work with RuboCop 1.8 which includes rubocop/rubocop#9291

```ruby
$ bundle exec rubocop -a
Inspecting 71 files
...........................C...........................................

Offenses:

lib/active_record/connection_adapters/oracle_enhanced_adapter.rb:495:39: C: [Corrected] Layout/SpaceBeforeBrackets: Remove the space before the opening brackets.
          @do_not_prefetch_primary_key [table_name] = do_not_prefetch = !has_primary_key?(table_name, owner, desc_table_name)
                                      ^

71 files inspected, 1 offense detected, 1 offense corrected

Tip: Based on detected gems, the following RuboCop extension libraries might be helpful:
  * rubocop-rake (http://github.com/rubocop-hq/rubocop-rake)
  * rubocop-rspec (http://github.com/rubocop-hq/rubocop-rspec)

You can opt out of this message by adding the following to your config (see https://docs.rubocop.org/rubocop/extensions.html#extension-suggestions for more options):
  AllCops:
    SuggestExtensions: false
$
```
Refer rubocop/rubocop#9305
Due to a bug, the SQL describing them was being discarded
…usion-bug

Ensure FKs are properly included in structure dumps
…kets_cop

Enable `Layout/SpaceBeforeBrackets` cop
…ease61

Address Travis CI warnings and bump Ubuntu version to 20.04
Tidy up Travis CI configuration
…nyms

Since #2055 added /*+ OPTIMIZER_FEATURES_ENABLE('11.2.0.2') */ hint
to most of the SCHEMA queries and there are performance regressions
reported for queries to all_synonyms

Fix #2090
Remove /*+ OPTIMIZER_FEATURES_ENABLE('11.2.0.2') */ hint for all_synonyms
# with conn.unprepared_statement (like AR.to_sql)
def select_values_forcing_binds(arel, name, binds)
# remove possible force of unprepared SQL during dictionary access
unprepared_statement_forced = prepared_statements_disabled_cache.include?(object_id)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not know prepared_statements_disabled_cache method implemented via rails/rails@206345b9890

@@ -491,7 +491,7 @@ def prefetch_primary_key?(table_name = nil)
do_not_prefetch = @do_not_prefetch_primary_key[table_name]
if do_not_prefetch.nil?
owner, desc_table_name = @connection.describe(table_name)
@do_not_prefetch_primary_key [table_name] = do_not_prefetch = !has_primary_key?(table_name, owner, desc_table_name)
@do_not_prefetch_primary_key[table_name] = do_not_prefetch = !has_primary_key?(table_name, owner, desc_table_name)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you rebase this pull request based on the current master branch? because #2108 addresses this unnecessary spece.

yahonda and others added 9 commits January 14, 2021 17:37
Prepare release v6.1.1 [skip ci]
...and filter by owner, rather than using user_tab_columns. Ensures column
comments are dumped for the current schema
…s-across-schemas

structure dump: read column comments from all_tab_cols
This is useful especially in case of cursor_sharing=exact

(cherry picked from commit f8b7a1a)
(cherry picked from commit f8021f8)
…binds even if primary SQL is executed with connection.unprepared_statement (like behavior of AR.to_sql).

Preserves the function of fix for issue #1687.
…eter_bind_usage

# Conflicts:
#	lib/active_record/connection_adapters/oracle_enhanced/structure_dump.rb
@rammpeter
Copy link
Contributor Author

Sorry, I'm not sure enough if everything worked well with rebase.
I close this pull request and reopen it again based on current head.

@rammpeter rammpeter closed this Jan 15, 2021
@rammpeter rammpeter deleted the rammpeter_bind_usage branch January 15, 2021 08:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants