Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move annotation logic into Query Runner #4113

Merged
merged 7 commits into from
Sep 2, 2019
Merged

Conversation

arikfr
Copy link
Member

@arikfr arikfr commented Sep 1, 2019

What type of PR is this? (check all applicable)

  • Refactor

Description

This is a small refactor towards allowing controlling the query annotation in the query runner (a relevant use case is coming up in a follow up PR).

In this pull request, we move the annotation logic into the query runner (with a default implementation in BaseQueryRunner).

@arikfr arikfr added the Backend label Sep 1, 2019
@arikfr arikfr requested a review from rauchy September 1, 2019 14:11
@@ -38,10 +38,8 @@ def default(self, o):

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing changed here, just auto formatting.

@rauchy
Copy link
Contributor

rauchy commented Sep 1, 2019

What's the benefit in implementing this as a mixin, as opposed to an actual method on BaseQueryRunner?

@arikfr
Copy link
Member Author

arikfr commented Sep 1, 2019

What's the benefit in implementing this as a mixin, as opposed to an actual method on BaseQueryRunner?

Because the one implemented in BaseQueryRunner is the implementation that does annotate, so we need the mixin to override it.

Although now I wonder if we might get bitten by the order in which the methods get overriden.

@rauchy
Copy link
Contributor

rauchy commented Sep 2, 2019

Yeah, I got that, but still -

(a) seems like most query runners have no annotations (in this PR, you mix it in about 35 times).
(b) adding a mixin to remove functionality feels counterintuitive.

Don't you think it'll be easier to understand if it were the other way around? (an AnnotationMixin)

@arikfr
Copy link
Member Author

arikfr commented Sep 2, 2019

seems like most query runners have no annotations (in this PR, you mix it in about 35 times).

You should improve your grep condition :-) You probably included the import statements in this count. It's being used 27 times. We have 56 Query Runners, so that's 48% ;-) So we still have a majority that use annotations.

I still prefer the annotation to be the default, but here's an alternative approach: add a kill switch instead of mixin, so it becomes:

class JSON(BaseHTTPQueryRunner):
    add_query_annotation = False

Might need a better name for the flag.

@rauchy
Copy link
Contributor

rauchy commented Sep 2, 2019

You should improve your grep condition :-)

damnit you're right 😠 I still find a "mixins add behavior" approach simpler, but 🤷‍♂

Regarding the alternative - how about simply annotate = True.
Also, did you consider an @annotated decorator?

@arikfr
Copy link
Member Author

arikfr commented Sep 2, 2019

I agree that it makes more sense for a mixin to add behavior. Although one could claim that this adds behavior too (of doing nothing). 🤪

Regarding the alternative - how about simply annotate = True.

Feels a bit unclear, like "annotate what"? Ideally it would be annotate_query, but that's the method name :-(

@rauchy
Copy link
Contributor

rauchy commented Sep 2, 2019

Feels a bit unclear, like "annotate what"?

True dat.

Also, did you consider an @Annotated decorator?

@arikfr
Copy link
Member Author

arikfr commented Sep 2, 2019

Also, did you consider an @Annotated decorator?

Feel like an overkill for something that can be a simple boolean class property.

…ries (#4114)

* Add scheduled status to query job metadata.

* Add: option to set query group for adhoc/scheduled Redshift queries

* Scheduled might not be set for already enqueued queries.
@arikfr arikfr merged commit 4c56900 into master Sep 2, 2019
@arikfr arikfr deleted the redshift-query-group branch September 2, 2019 13:02
arikfr added a commit that referenced this pull request Sep 18, 2019
* Migrate Parameters component to React (#4006)

* Start Parameters Migration

* Add dirtyCount

* Use workaround with setState

* Apply Changes

* Add EditSettingsDialog

* Add Cmd/Ctrl + Enter behavior

* Remove isApplying

* Delete Angular version of parameters

* Update tests

* Remove angular stuff

* Update jest

* Drag placeholder

* Update events

* Use old button styling and move css

* Reviewing code

* Add parameter rearrange test

* Add Parameter Settings title change test

* Update Parameter Settings button styling

* Move parameter url logic back to Parameters

* Disable url update when query is new

* Styling changes (#4019)

* Ran's title width styling

* Update drag test

* Improve sizing for Number inputs

Co-Authored-By: Ran Byron <ranbena@gmail.com>

* Fix issue with dragged parameter wrapping

Co-Authored-By: Ran Byron <ranbena@gmail.com>

* Don't reevaluate dirtyParamCount

* Allow multiple values :)

* Fix parameter alignments

* Fix Select width on search

* Update client/app/components/Parameters.less

Co-Authored-By: Ran Byron <ranbena@gmail.com>

* Humanize param.name

* Make sure angular updates Execute disabled status

* Add more flake8 tests and fail build if any test fails (#4055)

* Add more flake8 tests and fail build if any test fails

Run all flake8 E9xx + F63x + F7xx + F82x tests.

* long = long in Python 2

* Fix: MySQL connections without SSL are failing (#4090)

* Move connection logic into a single method & make sure not to pass ssl value if not used.

* Remove wildcard import and format file.

* Removed redash-newstyle.less (#4017)

* Make sure we always pass a list to _get_column_lists (#4095)

(some data sources might return None as the columns list)

* [Data Sources] Add: Azure Data Explorer (Kusto) query runner (#4091)

* [Data Sources] Add: Azure Data Explorer (Kusto) query runner

* CodeClimate fixes

* Remove TODO

* Fixed configuration properties names for Azure Kusto

* Azure Kusto: get_schema in one query

* azure-kusto-data update to 0.0.32

* Add Kusto to the default query runners list

* [Qubole] - Adding support to process Quantum query types. (#4066)

* [Qubole] - Adding support to process Quantum query types.

Quantum is a serverless interactive service that offers
direct SQL access to user's data lake. Changes are made
to accept `quantum` query type from user which makes
`Cluster Label` as optional.

* -Making quantum as defult query.
-Dictionary safe access to connection parmeters

* keeping pep8 standards

* Maintainig pep8 std

* Use latest version of qds-sdk

* Use qds-sdk v1.13.0

* Use qds-sdk v1.12.0

* Use qds-sdk v1.13.0

* Updating SDK with verified version

* hive as default query type

* qds-sdk : Locking most recent release version

* qds-sdk : Locking recent release version

* falling back to original version of qds-sdk

* Dashboard: when updating parameters, run only relevant queries (#3804)

* refresh only affected queries in dashboard when parameters are changed

* rename pendingParameters to updatedParameters

* select which widgets to update according to their mapping as a dashboard-level parameter

* use lodash's include

* Migrate with SQL statements. (#4105)

* Update badge in README.md to link to CircleCI (#4104)

* Update README.md

* Update README.md

* Update README.md

Co-Authored-By: Ran Byron <ranbena@gmail.com>

* Update README.md

* Fix widget bottom element alignment (#4110)

* Fix: allow users with view only acces to use the queries in Query Results (#4112)

* Fix: allow users with view only acces to access the queries

* Add tests

* Update error message

* Update error message. Take 2

* Fix Dropdown parameter options appearing behind Dialog (#4109)

* Add ability to use Ant's Table loading property when using ItemsTable (#4117)

* Display data source icon in query editor (#4119)

* Move annotation logic into Query Runner (#4113)

* Code formatting

* Move annotation logic into query runner, so it can be overriden in the query runner.

* Add mixin to __all__

* Switch to flag instead of mixin

* Feature (Redshift): option to set query group for adhoc/scheduled queries  (#4114)

* Add scheduled status to query job metadata.

* Add: option to set query group for adhoc/scheduled Redshift queries

* Scheduled might not be set for already enqueued queries.

* Fix number param value normlization (#4116)

* Use ng-src for data source icons (#4123)

* hive_ds: show a user friendly error message when possible (#4121)

* Update botocore, to get pass pip warning (#4122)

* Widget table scroll-x visible (#4101)

* Table viz horizontal scroll made visible

* Added tests

* Fixed snapshot pre-condition

* Perhaps this would trigger percy

* Upgrade Sentry-SDK and enable additional integratoins (#4127)

* Update sentry-sdk version

* Add additional Sentry integrations

* Migrate Counter visualization to React (#4106)

* Migrate Counter to React: Renderer

* Migrate Counter to React: Editor

* Cleanup

* Review and fix rows indexing algorithm

* Counter not properly scaled in editor

* Fix wrong label for/input id pair

* Tests

* Tests

* Fix vendor prefixes

* Remove unnecessary useEffect dependencies

* Update tests

* Fix Percy snapshot names

* Sync botocor eversions across requirements files. (#4128)

* Decrease size of widget pagination (#4120)

* Added tests

* Perhaps this would trigger percy

* Decrease size of widget pagination

* Removed unused attr

* Updated tests

* Allow the user to decide how to handle null values in charts (#4071)

* #2629 Refactor Chart visualization, add option for handling NULL values (keep/convert to 0.0)

* Handle null values in line/area stacking code; some cleanup

* Handle edge case: line/area stacking when last value of one of series is missing

* Mjnor update to line/area stacking code

* Fix line/area normalize to percents feature

* Unit tests

* Refine tests; add tests for prepareLayout function

* Tests for prepareData (heatmap) function

* Tests for prepareData (pie) function

* Tests for prepareData (bar, line, area) function

* Tests for prepareData (scatter, bubble) function

* Tests for prepareData (box) function

* Remove unused file

* Alerts: Add more condition comparison options (#4134)

* #4132 Add more condition comparison options

* Add arguments to fallback lambda

* Remove duplicate messages method (#4131)

* Migrate Chart visualization to React Part 1: Renderer (#4130)

* Migrate Chart visualization: Renderer

* Refine PlotlyChart component; move stylesheets to visualization's folder

* Migrate Custom JS Chart to React

* Cleanup

* Add jsconfig settings with '@' webpack alias (#4135)

* Counter Editor: move components to own files (#4138)

* Allow users to share aggregated usage information with us (#4108)

* Initial commit of BeaconConsent component

* Add comment about being able to change setting

* Use <Text> correctly

* Final version of consent screen

* Show beacon consent message on homepage only if it wasn't enabled already.

* Add consent setting to organization settings screen.

* Add support for custom message in OrgSetting.save.

* Implmenet consent saving.

* If consent given, send extra data

* Add HelpTrigger

* Make CodeClimate happy

* Wrap everything with DynamicComponent

* CHANGELOG for V8-beta. (#4057)

* CHANGELOG for V8-beta.

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Bug fix: Query view doesn't sync parameters when selecting and deleting (#4146)

* Query Snippets: Use onClick instead of link for 'Click here' option (#4144)

* Snippets: Don't change url when not needed

* Revert "Snippets: Don't change url when not needed"

This reverts commit 2f346f3.

* Query Snippets: use onClick instead of link

* Color picker component (#4136)

* Widget filters overlapped by visualization (#4137)

* Fix: widget filters overlapped by visualization

* Fix tests

* Fix tests

* CHANGELOG for v8.0.0-beta.2 (#4145)

* Stop building tarballs.

* Update version reference.

* CHANGELOG for 8.0.0-beta.2
arikfr added a commit that referenced this pull request Oct 27, 2019
* Code formatting

* Move annotation logic into query runner, so it can be overriden in the query runner.

* Add mixin to __all__

* Switch to flag instead of mixin

* Feature (Redshift): option to set query group for adhoc/scheduled queries  (#4114)

* Add scheduled status to query job metadata.

* Add: option to set query group for adhoc/scheduled Redshift queries

* Scheduled might not be set for already enqueued queries.
harveyrendell pushed a commit to pushpay/redash that referenced this pull request Nov 14, 2019
* Code formatting

* Move annotation logic into query runner, so it can be overriden in the query runner.

* Add mixin to __all__

* Switch to flag instead of mixin

* Feature (Redshift): option to set query group for adhoc/scheduled queries  (getredash#4114)

* Add scheduled status to query job metadata.

* Add: option to set query group for adhoc/scheduled Redshift queries

* Scheduled might not be set for already enqueued queries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants