Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Be more permissive when parameters are safe #3383

Merged
merged 38 commits into from
Feb 26, 2019
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
cd2cee7
use the textless endpoint (/api/queries/:id/results) for pristine
Jan 30, 2019
e8ceeb7
reverse conditional. not not is making me the headaches.
Jan 31, 2019
06917ce
add ParameterizedQuery#is_safe with an inital naive implementation which
Jan 31, 2019
31e2238
allow getting new query results even if user has only view permissions
Jan 31, 2019
c93ed69
Merge branch 'master' into use-textless-endpoint-for-pristine-queries
Feb 3, 2019
d11e4e7
Merge branch 'use-textless-endpoint-for-pristine-queries' into be-mor…
Feb 3, 2019
f858ca0
fix lint error - getDerivedStateFromProps should be placed after state
Feb 3, 2019
9bd2308
Merge branch 'fix-time-ago-lint' into be-more-permissive-when-paramet…
Feb 3, 2019
04a1adf
Merge branch 'master' into use-textless-endpoint-for-pristine-queries
Feb 3, 2019
9fc9fcb
Revert "use the textless endpoint (/api/queries/:id/results) for pris…
Feb 3, 2019
ab5eb8f
move execution preparation to a different function, which will be soon
Feb 3, 2019
df4e9e4
go to textless /api/queries/:id/results by default
Feb 3, 2019
49c9a68
let the query view decide if text or textless endpoint is needed
Feb 3, 2019
65e742c
Merge branch 'use-textless-endpoint-for-pristine-queries' into be-mor…
Feb 3, 2019
8716626
Merge branch 'master' into be-more-permissive-when-parameters-are-safe
Feb 3, 2019
d308a4d
allow safe queries to be executed in the UI even if the user has no
Feb 3, 2019
8807e6a
Merge branch 'master' into be-more-permissive-when-parameters-are-safe
Feb 3, 2019
6c385c0
change `run_query`'s signature to accept a ParameterizedQuery instead of
Feb 3, 2019
4682368
use dict#get instead of a None guard
Feb 3, 2019
da45021
use ParameterizedQuery in queries handler as well
Feb 5, 2019
5493f97
Merge branch 'master' into be-more-permissive-when-parameters-are-safe
Feb 5, 2019
ae139fa
test that /queries/:id/results allows execution of safe queries even if
Feb 6, 2019
c1b2d12
Merge branch 'master' into be-more-permissive-when-parameters-are-safe
Feb 6, 2019
4518d12
lint
Feb 6, 2019
2202403
raise HTTP 400 when receiving invalid parameter values. Fixes #3394
Feb 6, 2019
926b679
Merge branch 'master' into be-more-permissive-when-parameters-are-safe
Feb 10, 2019
df62d8d
Merge branch 'master' into be-more-permissive-when-parameters-are-safe
Feb 12, 2019
ca08af3
remove unused methods
Feb 12, 2019
c698a03
avoid cyclic imports by importing only when needed
Feb 12, 2019
97df888
Merge branch 'master' into be-more-permissive-when-parameters-are-safe
Feb 17, 2019
fca4f1e
verify that a ParameterizedQuery without any parameters is considered
Feb 17, 2019
70ea5b9
Merge branch 'master' into be-more-permissive-when-parameters-are-safe
Feb 21, 2019
c66cfab
Merge branch 'master' into be-more-permissive-when-parameters-are-safe
Feb 22, 2019
dc3f221
Merge branch 'master' into be-more-permissive-when-parameters-are-safe
Feb 24, 2019
8027ed2
Merge branch 'be-more-permissive-when-parameters-are-safe' of github.…
Feb 24, 2019
65b6b2a
Merge branch 'master' into be-more-permissive-when-parameters-are-safe
Feb 26, 2019
5bdab05
introduce query.parameter_schema
Feb 26, 2019
9d5e231
encapsulate ParameterizedQuery creation inside Query
Feb 26, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion client/app/pages/queries/view.js
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ function QueryViewCtrl(
$scope.canEdit = currentUser.canEdit($scope.query) || $scope.query.can_edit;
$scope.canViewSource = currentUser.hasPermission('view_source');

$scope.canExecuteQuery = () => currentUser.hasPermission('execute_query') && !$scope.dataSource.view_only;
$scope.canExecuteQuery = () => $scope.query.is_safe || (currentUser.hasPermission('execute_query') && !$scope.dataSource.view_only);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the user changes the query and it's no longer safe?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then he will get a 403 back from the server. I'll say this is an edge case, which while not handled in the most elegant way, is a good enough solution.

We might want to consider a better error message there, though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should prevent editing query source when the user doesn't have Full Access permission to the data source. We might actually be doing this already.

Then this becomes a non issue. We can open a separate issue for this.


$scope.canForkQuery = () => currentUser.hasPermission('edit_query') && !$scope.dataSource.view_only;

Expand Down
4 changes: 3 additions & 1 deletion redash/handlers/queries.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
require_permission, view_only)
from redash.utils import collect_parameters_from_request
from redash.serializers import QuerySerializer
from redash.utils.parameterized_query import ParameterizedQuery


# Ordering map for relationships
Expand Down Expand Up @@ -411,8 +412,9 @@ def post(self, query_id):
require_access(query.groups, self.current_user, not_view_only)

parameter_values = collect_parameters_from_request(request.args)
parameterized_query = ParameterizedQuery(query.query_text)

return run_query(query.data_source, parameter_values, query.query_text, query.id)
return run_query(parameterized_query, parameter_values, query.data_source, query.id)


class QueryTagsResource(BaseResource):
Expand Down
44 changes: 19 additions & 25 deletions redash/handlers/query_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
from redash.tasks import QueryTask
from redash.tasks.queries import enqueue_query
from redash.utils import (collect_parameters_from_request, gen_query_hash, json_dumps, utcnow)
from redash.utils.parameterized_query import ParameterizedQuery, dropdown_values
from redash.utils.parameterized_query import ParameterizedQuery, InvalidParameterError, dropdown_values


def error_response(message):
Expand Down Expand Up @@ -64,7 +64,7 @@ def run_query_sync(data_source, parameter_values, query_text, max_age=0):
return None


def run_query(data_source, parameter_values, query_text, query_id, max_age=0, parameter_schema=None):
def run_query(query, parameters, data_source, query_id, max_age=0):
if data_source.paused:
if data_source.pause_reason:
message = '{} is paused ({}). Please try later.'.format(data_source.name, data_source.pause_reason)
Expand All @@ -73,7 +73,10 @@ def run_query(data_source, parameter_values, query_text, query_id, max_age=0, pa

return error_response(message)

query = ParameterizedQuery(query_text, parameter_schema).apply(parameter_values)
try:
query.apply(parameters)
except InvalidParameterError as e:
abort(400, message=e.message)

if query.missing_params:
return error_response(u'Missing parameter value for: {}'.format(u", ".join(query.missing_params)))
Expand All @@ -86,7 +89,10 @@ def run_query(data_source, parameter_values, query_text, query_id, max_age=0, pa
if query_result:
return {'query_result': query_result.to_dict()}
else:
job = enqueue_query(query.text, data_source, current_user.id, metadata={"Username": current_user.email, "Query ID": query_id})
job = enqueue_query(query.text, data_source, current_user.id, metadata={
"Username": current_user.email,
"Query ID": query_id
})
return {'job': job.to_dict()}


Expand All @@ -112,6 +118,8 @@ def post(self):
query_id = params.get('query_id', 'adhoc')
parameters = params.get('parameters', collect_parameters_from_request(request.args))

parameterized_query = ParameterizedQuery(query)

data_source = models.DataSource.get_by_id_and_org(params.get('data_source_id'), self.current_org)

if not has_access(data_source.groups, self.current_user, not_view_only):
Expand All @@ -125,7 +133,7 @@ def post(self):
'query_id': query_id,
'parameters': parameters
})
return run_query(data_source, parameters, query, query_id, max_age)
return run_query(parameterized_query, parameters, data_source, query_id, max_age)


ONE_YEAR = 60 * 60 * 24 * 365.25
Expand Down Expand Up @@ -159,23 +167,6 @@ def options(self, query_id=None, query_result_id=None, filetype='json'):

return make_response("", 200, headers)

def _fetch_rows(self, query_id):
query = models.Query.get_by_id_and_org(query_id, self.current_org)
require_access(query.data_source.groups, self.current_user, view_only)

query_result = models.QueryResult.get_by_id_and_org(query.latest_query_data_id, self.current_org)

return json_loads(query_result.data)["rows"]

def _convert_queries_to_enums(self, definition):
if definition["type"] == "query":
definition["type"] = "enum"

rows = self._fetch_rows(definition.pop("queryId"))
definition["enumOptions"] = [row.get('value', row.get(row.keys()[0])) for row in rows if row]

return definition

@require_permission('execute_query')
def post(self, query_id):
"""
Expand All @@ -195,10 +186,13 @@ def post(self, query_id):
query = get_object_or_404(models.Query.get_by_id_and_org, query_id, self.current_org)
parameter_schema = query.options.get("parameters", [])

if not has_access(query.data_source.groups, self.current_user, not_view_only):
return {'job': {'status': 4, 'error': 'You do not have permission to run queries with this data source.'}}, 403
parameterized_query = ParameterizedQuery(query.query_text, parameter_schema)
allow_executing_with_view_only_permissions = parameterized_query.is_safe

if has_access(query.data_source.groups, self.current_user, allow_executing_with_view_only_permissions):
return run_query(parameterized_query, parameters, query.data_source, query_id, max_age)
else:
return run_query(query.data_source, parameters, query.query_text, query_id, max_age, parameter_schema=parameter_schema)
return {'job': {'status': 4, 'error': 'You do not have permission to run queries with this data source.'}}, 403

@require_permission('view_query')
def get(self, query_id=None, query_result_id=None, filetype='json'):
Expand Down
2 changes: 2 additions & 0 deletions redash/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
from redash import models
from redash.permissions import has_access, view_only
from redash.utils import json_loads
from redash.utils.parameterized_query import ParameterizedQuery


def public_widget(widget):
Expand Down Expand Up @@ -100,6 +101,7 @@ def serialize_query(query, with_stats=False, with_visualizations=False, with_use
'options': query.options,
'version': query.version,
'tags': query.tags or [],
'is_safe': ParameterizedQuery(query.query_text, query.options.get("parameters", [])).is_safe,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

query.options.get("parameters", [])

Not the first time I see this. Maybe we should have a parameters property on the Query model?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first I thought 5bdab05, but then I thought 9d5e231. WDYT?

}

if with_user:
Expand Down
7 changes: 6 additions & 1 deletion redash/utils/parameterized_query.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import pystache
from functools import partial
from flask_login import current_user
from redash.authentication.org_resolving import current_org
from numbers import Number
from redash import models
from redash.utils import mustache_render, json_loads
Expand All @@ -19,6 +18,7 @@ def _pluck_name_and_value(default_column, row):


def _load_result(query_id):
from redash.authentication.org_resolving import current_org
query = models.Query.get_by_id_and_org(query_id, current_org)
require_access(query.data_source.groups, current_user, view_only)
query_result = models.QueryResult.get_by_id_and_org(query.latest_query_data_id, current_org)
Expand Down Expand Up @@ -118,6 +118,11 @@ def _valid(self, name, value):

return validate(value)

@property
def is_safe(self):
text_parameters = filter(lambda p: p["type"] == "text", self.schema)
return not any(text_parameters)

@property
def missing_params(self):
query_parameters = set(_collect_query_parameters(self.template))
Expand Down
14 changes: 14 additions & 0 deletions tests/handlers/test_query_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,20 @@ def test_execute_new_query(self):
self.assertEquals(rv.status_code, 200)
self.assertIn('job', rv.json)

def test_prevents_execution_of_unsafe_queries_on_view_only_data_sources(self):
ds = self.factory.create_data_source(group=self.factory.org.default_group, view_only=True)
query = self.factory.create_query(data_source=ds, options={"parameters": [{"name": "foo", "type": "text"}]})

rv = self.make_request('post', '/api/queries/{}/results'.format(query.id), data={"parameters": {}})
self.assertEquals(rv.status_code, 403)

def test_allows_execution_of_safe_queries_on_view_only_data_sources(self):
ds = self.factory.create_data_source(group=self.factory.org.default_group, view_only=True)
query = self.factory.create_query(data_source=ds, options={"parameters": [{"name": "foo", "type": "number"}]})

rv = self.make_request('post', '/api/queries/{}/results'.format(query.id), data={"parameters": {}})
self.assertEquals(rv.status_code, 200)

def test_access_with_query_api_key(self):
ds = self.factory.create_data_source(group=self.factory.org.default_group, view_only=False)
query = self.factory.create_query()
Expand Down
12 changes: 12 additions & 0 deletions tests/utils/test_parameterized_query.py
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,18 @@ def test_raises_on_unexpected_param_types(self):
with pytest.raises(InvalidParameterError):
query.apply({"bar": "baz"})

def test_is_not_safe_if_expecting_text_parameter(self):
schema = [{"name": "bar", "type": "text"}]
query = ParameterizedQuery("foo", schema)

self.assertFalse(query.is_safe)

def test_is_safe_if_not_expecting_text_parameter(self):
schema = [{"name": "bar", "type": "number"}]
query = ParameterizedQuery("foo", schema)

self.assertTrue(query.is_safe)
rauchy marked this conversation as resolved.
Show resolved Hide resolved

@patch('redash.utils.parameterized_query._load_result', return_value={
"columns": [{"name": "id"}, {"name": "Name"}, {"name": "Value"}],
"rows": [{"id": 5, "Name": "John", "Value": "John Doe"}]})
Expand Down