forked from apache/druid
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update from master-for-merge-to-release (apache#318)
* IMPLY-6556 remove offending settings.xml for intellij inspections * GCS lookup support (apache#11026) * GCS lookup support * checkstyle fix * review comments * review comments * remove unused import * remove experimental from Kinesis with caveats (apache#10998) * remove experimental from Kinesis with caveats * add suggested known issue * spelling fixes * Bump aliyun SDK to 3.11.3 (apache#11044) * Update reset-cluster.md (apache#10990) fixed Error: Could not find or load main class org.apache.druid.cli.Main * Make imply-view-manager non-experimental (apache#316) * Make druid.indexer.task.ignoreTimestampSpecForDruidInputSource default to true, for backwards compat (apache#315) * Add explicit EOF and use assert instead of exception (apache#11041) * Add Calcite Avatica protobuf handler (apache#10543) * bump to latest of same version node and npm versions, bump frontend-maven-plugin (apache#11057) * request logs through kafka emitter (apache#11036) * request logs through kafka emitter * travis fixes * review comments * kafka emitter unit test * new line * travis checks * checkstyle fix * count request lost when request topic is null * IMPLY-6556 map local repository instead .m2 * remove outdated info from faq (apache#11053) * remove outdated info from faq * Add an option for ingestion task to drop (mark unused) all existing segments that are contained by interval in the ingestionSpec (apache#11025) * Auto-Compaction can run indefinitely when segmentGranularity is changed from coarser to finer. * Add option to drop segments after ingestion * fix checkstyle * add tests * add tests * add tests * fix test * add tests * fix checkstyle * fix checkstyle * add docs * fix docs * address comments * address comments * fix spelling * Allow list for JDBC connection properties to address CVE-2021-26919 (apache#11047) * Allow list for JDBC connection properties to address CVE-2021-26919 * fix tests for java 11 * Fix compile issue from dropExisting in ingest-service (apache#320) Co-authored-by: Slava Mogilevsky <triggerwoods91@gmail.com> Co-authored-by: Parag Jain <pjain1@apache.org> Co-authored-by: Charles Smith <38529548+techdocsmith@users.noreply.github.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: frank chen <frank.chen021@outlook.com> Co-authored-by: Tushar Raj <43772524+tushar-1728@users.noreply.github.com> Co-authored-by: Jonathan Wei <jon-wei@users.noreply.github.com> Co-authored-by: Jihoon Son <jihoonson@apache.org> Co-authored-by: Lasse Krogh Mammen <lkm@bookboon.com> Co-authored-by: Clint Wylie <cwylie@apache.org> Co-authored-by: Maytas Monsereenusorn <maytasm@apache.org>
- Loading branch information
12 people
authored
Apr 2, 2021
1 parent
a191871
commit 9603203
Showing
126 changed files
with
4,832 additions
and
333 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
63 changes: 63 additions & 0 deletions
63
core/src/main/java/org/apache/druid/utils/ConnectionUriUtils.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one | ||
| * or more contributor license agreements. See the NOTICE file | ||
| * distributed with this work for additional information | ||
| * regarding copyright ownership. The ASF licenses this file | ||
| * to you under the Apache License, Version 2.0 (the | ||
| * "License"); you may not use this file except in compliance | ||
| * with the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, | ||
| * software distributed under the License is distributed on an | ||
| * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| * KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
|
|
||
| package org.apache.druid.utils; | ||
|
|
||
| import com.google.common.base.Preconditions; | ||
|
|
||
| import java.util.Set; | ||
|
|
||
| public final class ConnectionUriUtils | ||
| { | ||
| // Note: MySQL JDBC connector 8 supports 7 other protocols than just `jdbc:mysql:` | ||
| // (https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-reference-jdbc-url-format.html). | ||
| // We should consider either expanding recognized mysql protocols or restricting allowed protocols to | ||
| // just a basic one. | ||
| public static final String MYSQL_PREFIX = "jdbc:mysql:"; | ||
| public static final String POSTGRES_PREFIX = "jdbc:postgresql:"; | ||
|
|
||
| /** | ||
| * This method checks {@param actualProperties} against {@param allowedProperties} if they are not system properties. | ||
| * A property is regarded as a system property if its name starts with a prefix in {@param systemPropertyPrefixes}. | ||
| * See org.apache.druid.server.initialization.JDBCAccessSecurityConfig for more details. | ||
| * | ||
| * If a non-system property that is not allowed is found, this method throws an {@link IllegalArgumentException}. | ||
| */ | ||
| public static void throwIfPropertiesAreNotAllowed( | ||
| Set<String> actualProperties, | ||
| Set<String> systemPropertyPrefixes, | ||
| Set<String> allowedProperties | ||
| ) | ||
| { | ||
| for (String property : actualProperties) { | ||
| if (systemPropertyPrefixes.stream().noneMatch(property::startsWith)) { | ||
| Preconditions.checkArgument( | ||
| allowedProperties.contains(property), | ||
| "The property [%s] is not in the allowed list %s", | ||
| property, | ||
| allowedProperties | ||
| ); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| private ConnectionUriUtils() | ||
| { | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one | ||
| * or more contributor license agreements. See the NOTICE file | ||
| * distributed with this work for additional information | ||
| * regarding copyright ownership. The ASF licenses this file | ||
| * to you under the Apache License, Version 2.0 (the | ||
| * "License"); you may not use this file except in compliance | ||
| * with the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, | ||
| * software distributed under the License is distributed on an | ||
| * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| * KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
|
|
||
| package org.apache.druid.utils; | ||
|
|
||
| public final class Throwables | ||
| { | ||
| public static boolean isThrowable(Throwable t, Class<? extends Throwable> searchFor) | ||
| { | ||
| if (t.getClass().isAssignableFrom(searchFor)) { | ||
| return true; | ||
| } else { | ||
| if (t.getCause() != null) { | ||
| return isThrowable(t.getCause(), searchFor); | ||
| } else { | ||
| return false; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| private Throwables() | ||
| { | ||
| } | ||
| } |
90 changes: 90 additions & 0 deletions
90
core/src/test/java/org/apache/druid/utils/ConnectionUriUtilsTest.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one | ||
| * or more contributor license agreements. See the NOTICE file | ||
| * distributed with this work for additional information | ||
| * regarding copyright ownership. The ASF licenses this file | ||
| * to you under the Apache License, Version 2.0 (the | ||
| * "License"); you may not use this file except in compliance | ||
| * with the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, | ||
| * software distributed under the License is distributed on an | ||
| * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| * KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
|
|
||
| package org.apache.druid.utils; | ||
|
|
||
| import com.google.common.collect.ImmutableSet; | ||
| import org.junit.Rule; | ||
| import org.junit.Test; | ||
| import org.junit.experimental.runners.Enclosed; | ||
| import org.junit.rules.ExpectedException; | ||
| import org.junit.runner.RunWith; | ||
|
|
||
| @RunWith(Enclosed.class) | ||
| public class ConnectionUriUtilsTest | ||
| { | ||
| public static class ThrowIfURLHasNotAllowedPropertiesTest | ||
| { | ||
| @Rule | ||
| public ExpectedException expectedException = ExpectedException.none(); | ||
|
|
||
| @Test | ||
| public void testEmptyActualProperties() | ||
| { | ||
| ConnectionUriUtils.throwIfPropertiesAreNotAllowed( | ||
| ImmutableSet.of(), | ||
| ImmutableSet.of("valid_key1", "valid_key2"), | ||
| ImmutableSet.of("system_key1", "system_key2") | ||
| ); | ||
| } | ||
|
|
||
| @Test | ||
| public void testThrowForNonAllowedProperties() | ||
| { | ||
| expectedException.expect(IllegalArgumentException.class); | ||
| expectedException.expectMessage("The property [invalid_key] is not in the allowed list [valid_key1, valid_key2]"); | ||
|
|
||
| ConnectionUriUtils.throwIfPropertiesAreNotAllowed( | ||
| ImmutableSet.of("valid_key1", "invalid_key"), | ||
| ImmutableSet.of("system_key1", "system_key2"), | ||
| ImmutableSet.of("valid_key1", "valid_key2") | ||
| ); | ||
| } | ||
|
|
||
| @Test | ||
| public void testAllowedProperties() | ||
| { | ||
| ConnectionUriUtils.throwIfPropertiesAreNotAllowed( | ||
| ImmutableSet.of("valid_key2"), | ||
| ImmutableSet.of("system_key1", "system_key2"), | ||
| ImmutableSet.of("valid_key1", "valid_key2") | ||
| ); | ||
| } | ||
|
|
||
| @Test | ||
| public void testAllowSystemProperties() | ||
| { | ||
| ConnectionUriUtils.throwIfPropertiesAreNotAllowed( | ||
| ImmutableSet.of("system_key1", "valid_key2"), | ||
| ImmutableSet.of("system_key1", "system_key2"), | ||
| ImmutableSet.of("valid_key1", "valid_key2") | ||
| ); | ||
| } | ||
|
|
||
| @Test | ||
| public void testMatchSystemProperties() | ||
| { | ||
| ConnectionUriUtils.throwIfPropertiesAreNotAllowed( | ||
| ImmutableSet.of("system_key1.1", "system_key1.5", "system_key11.11", "valid_key2"), | ||
| ImmutableSet.of("system_key1", "system_key2"), | ||
| ImmutableSet.of("valid_key1", "valid_key2") | ||
| ); | ||
| } | ||
| } | ||
| } |
48 changes: 48 additions & 0 deletions
48
core/src/test/java/org/apache/druid/utils/ThrowablesTest.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one | ||
| * or more contributor license agreements. See the NOTICE file | ||
| * distributed with this work for additional information | ||
| * regarding copyright ownership. The ASF licenses this file | ||
| * to you under the Apache License, Version 2.0 (the | ||
| * "License"); you may not use this file except in compliance | ||
| * with the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, | ||
| * software distributed under the License is distributed on an | ||
| * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| * KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
|
|
||
| package org.apache.druid.utils; | ||
|
|
||
| import org.junit.Assert; | ||
| import org.junit.Test; | ||
|
|
||
| public class ThrowablesTest | ||
| { | ||
| @Test | ||
| public void testIsThrowableItself() | ||
| { | ||
| Assert.assertTrue(Throwables.isThrowable(new NoClassDefFoundError(), NoClassDefFoundError.class)); | ||
| } | ||
|
|
||
| @Test | ||
| public void testIsThrowableNestedThrowable() | ||
| { | ||
| Assert.assertTrue( | ||
| Throwables.isThrowable(new RuntimeException(new NoClassDefFoundError()), NoClassDefFoundError.class) | ||
| ); | ||
| } | ||
|
|
||
| @Test | ||
| public void testIsThrowableNonTarget() | ||
| { | ||
| Assert.assertFalse( | ||
| Throwables.isThrowable(new RuntimeException(new ClassNotFoundException()), NoClassDefFoundError.class) | ||
| ); | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.