Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Report function name for unknown exceptions during execution #14987

Merged
merged 14 commits into from
Oct 25, 2023
Merged

Report function name for unknown exceptions during execution #14987

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
eda8e4f
provide function name when unknown exceptions are encountered
kgyrtkirk Sep 14, 2023
9f59055
fix keywords/etc
kgyrtkirk Sep 14, 2023
017ff4a
fix keywrod order - regex excercise
kgyrtkirk Sep 14, 2023
08f1f5e
add test
kgyrtkirk Sep 14, 2023
43c72fd
add check&fix keywords
kgyrtkirk Sep 15, 2023
3423f6a
Merge remote-tracking branch 'apache/master' into function-druid-exce…
kgyrtkirk Sep 15, 2023
e922c82
decoupledIgnore
kgyrtkirk Sep 18, 2023
55ad208
Revert "decoupledIgnore"
kgyrtkirk Sep 18, 2023
f71bdef
unpatch Function
kgyrtkirk Sep 19, 2023
3567584
move to a different location
kgyrtkirk Sep 19, 2023
e81fbe7
checkstyle
kgyrtkirk Sep 19, 2023
54d1beb
Merge remote-tracking branch 'apache/master' into function-druid-exce…
kgyrtkirk Sep 28, 2023
0931995
Merge remote-tracking branch 'apache/master' into function-druid-exce…
kgyrtkirk Oct 3, 2023
0507c52
Merge remote-tracking branch 'apache/master' into function-druid-exce…
kgyrtkirk Oct 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion processing/src/main/java/org/apache/druid/math/expr/FunctionalExpr.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@

import com.google.common.base.Preconditions;
import com.google.common.collect.ImmutableList;
import org.apache.druid.error.DruidException;
import org.apache.druid.java.util.common.StringUtils;
import org.apache.druid.math.expr.vector.ExprVectorProcessor;

Expand Down Expand Up @@ -186,7 +187,15 @@ public String toString()
@Override
public ExprEval eval(ObjectBinding bindings)
{
return function.apply(args, bindings);
try {
return function.apply(args, bindings);
}
catch (DruidException | ExpressionValidationException e) {
throw e;
}
catch (Exception e) {
throw DruidException.defensive().build(e, "Invocation of function '%s' encountered exception.", name);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some other category would make more sense than "defensive". Defensive is sort of like an assert (it signals that we believe this should never happen unless there's a bug) but we do in fact expect functions to throw exceptions under "normal" conditions sometimes. (Like if they receive arguments of the wrong type, etc)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case the invoked function have already thrown out some reasonable exception (DruidException) - that's not touched at all - because a catch above caught it.

I wanted to add this to catch cases in which some unexpected issue happened while the function was processing....so that we don't let out a NullPointerException ; instead say that "Invocation of function x encountered ... " - it may give some help to the user what might have been the problem ; and it could give better error message to support instead of just refering to it as an NPE.

or do you think it would be better to repackage also normal DruidExcpetion-s and re-packaged? - and rethrown them with a bit more context?
so it will be something like:

DruidException: while invoking function x
caused by:
DruidException: Can't process an empty set because it Sunday!
``

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. I thought this was wrapping all exceptions, but misread it.

In the case I'm wondering if the "user-caused" exceptions (bad input) thrown by functions are all proper DruidExceptions? In that case a defensive might make sense here. We just want to avoid defensive for anything that isn't a code bug on our end.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes - other exceptions are good; the case was like: an unexpected issue happened - and the NPE bubbled up to the top. Giving little-to-no detail to the user about what happened; as the message didn't contained anything usefull.

I think if the user could see a bit friendlier message - it might give a starting point to either start narrowing it down the issue; or try to work it around

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should surface as runtime_failure IMO. Both defensive and runtime_failure map to same error code.

}
}

@Override
Expand Down
36 changes: 36 additions & 0 deletions processing/src/test/java/org/apache/druid/math/expr/FunctionTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,12 @@
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
import org.apache.druid.common.config.NullHandling;
import org.apache.druid.error.DruidException;
import org.apache.druid.guice.NestedDataModule;
import org.apache.druid.java.util.common.IAE;
import org.apache.druid.java.util.common.Pair;
import org.apache.druid.java.util.common.StringUtils;
import org.apache.druid.math.expr.Expr.ObjectBinding;
import org.apache.druid.segment.column.TypeStrategies;
import org.apache.druid.segment.column.TypeStrategiesTest;
import org.apache.druid.segment.column.TypeStrategy;
Expand All @@ -38,11 +40,15 @@
import org.junit.Test;

import javax.annotation.Nullable;

import java.math.BigDecimal;
import java.math.RoundingMode;
import java.nio.ByteBuffer;
import java.util.Set;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;

public class FunctionTest extends InitializedNullHandlingTest
{
private Expr.ObjectBinding bestEffortBindings;
Expand Down Expand Up @@ -119,6 +125,36 @@ public void setup()
allBindings = new Expr.ObjectBinding[]{bestEffortBindings, typedBindings};
}

@Test
public void testUnknownErrorsAreWrappedAndReported()
{
final Expr expr = Parser.parse("abs(x)", ExprMacroTable.nil());

ObjectBinding bind = new ObjectBinding()
{

@Override
public ExpressionType getType(String name)
{
return ExpressionType.LONG_ARRAY;
}

@Override
public Object get(String name)
{
throw new RuntimeException("nested-exception");
}
};
DruidException e = Assert.assertThrows(DruidException.class,
() -> {
expr.eval(bind);
});

assertEquals("Invocation of function 'abs' encountered exception.", e.getMessage());
assertNotNull(e.getCause());
assertEquals("nested-exception", e.getCause().getMessage());
}

@Test
public void testCaseSimple()
{
Expand Down