Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spark Direct Readers and Writers for Druid. #10920

Closed
wants to merge 34 commits into from
Closed

Spark Direct Readers and Writers for Druid. #10920

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
887d969
Spark Direct Readers and Writers for Druid.
JulianJaffePinterest Feb 24, 2021
072271b
Reworked configuration system.
JulianJaffePinterest Mar 2, 2021
e98b453
Add support for specifying dimension objects when writing.
JulianJaffePinterest Mar 4, 2021
f03149b
Refactor and add columnar reads.
JulianJaffePinterest Apr 12, 2021
331e0c9
Allow specifying SQL-compatible null handling for writing as well.
JulianJaffePinterest Apr 14, 2021
e2d9949
Better default handling and comments in DruidClient.
JulianJaffePinterest Apr 16, 2021
07fa3d2
Add package stage; improve documentation.
JulianJaffePinterest Apr 28, 2021
6a8a5ef
Add partitioners and semi-typed configuration.
JulianJaffePinterest May 27, 2021
98e2c88
Add more documentation for deploying.
JulianJaffePinterest Jun 8, 2021
4f538bb
Merge remote-tracking branch 'apache/master' into spark_druid_connector
JulianJaffePinterest Jun 29, 2021
9154073
Rebase changes and add license information.
JulianJaffePinterest Jun 29, 2021
c180ea9
Rebase fix-up, spelling, odds & ends.
JulianJaffePinterest Jul 1, 2021
f30157a
Fix dependency/spelling/spotbugs/rat checks.
JulianJaffePinterest Jul 1, 2021
4388b38
Fix spelling & licenses.
JulianJaffePinterest Jul 2, 2021
a5def27
Add licenses & de-version surefire.
JulianJaffePinterest Jul 2, 2021
41b5308
Merge remote-tracking branch 'apache/master' into spark_druid_connector
JulianJaffePinterest Jul 2, 2021
a1492ff
Update partioner docs.
JulianJaffePinterest Jul 9, 2021
74310ab
Merge remote-tracking branch 'apache/master' into spark_druid_connector
JulianJaffePinterest Jul 9, 2021
506943f
Merge upstream.
JulianJaffePinterest Jul 9, 2021
cae355c
Explicitly set surefire version in plugin section.
JulianJaffePinterest Jul 13, 2021
32a7e79
Merge remote-tracking branch 'apache/master' into spark_druid_connector
JulianJaffePinterest Jul 13, 2021
5778180
Java 11/15 compatibility.
JulianJaffePinterest Jul 13, 2021
dedbb64
Merge remote-tracking branch 'apache/master' into spark_druid_connector
JulianJaffePinterest Jul 14, 2021
b6682a2
Shade guava.
JulianJaffePinterest Jul 14, 2021
c3fe24d
Move the spark connectors to a top-level module.
JulianJaffePinterest Jul 14, 2021
9795452
Bump spark version to 2.4.8.
JulianJaffePinterest Jul 14, 2021
8c94e77
Improve logging.
JulianJaffePinterest Jul 14, 2021
34e6162
Add logging to failing travis test.
JulianJaffePinterest Jul 14, 2021
f83bfb7
Remove debugging logging.
JulianJaffePinterest Jul 15, 2021
abda5f8
Merge remote-tracking branch 'apache/master' into spark_druid_connector
JulianJaffePinterest Jul 15, 2021
7c53f1f
Merge remote-tracking branch 'apache/master' into spark_druid_connector
JulianJaffePinterest Jul 16, 2021
b0489fa
Fix typo in pom.
JulianJaffePinterest Jul 16, 2021
6fbc73d
Better Azure support; replace PasswordProvider.
JulianJaffePinterest Jul 17, 2021
8392f87
Add spellcheck suppressions.
JulianJaffePinterest Jul 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 139 additions & 0 deletions codestyle/scalastyle_config.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
<!--
~ Licensed to the Apache Software Foundation (ASF) under one
~ or more contributor license agreements. See the NOTICE file
~ distributed with this work for additional information
~ regarding copyright ownership. The ASF licenses this file
~ to you under the Apache License, Version 2.0 (the
~ "License"); you may not use this file except in compliance
~ with the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing,
~ software distributed under the License is distributed on an
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
~ KIND, either express or implied. See the License for the
~ specific language governing permissions and limitations
~ under the License.
-->
<scalastyle commentFilter="enabled">
<name>Apache Druid Scalastyle configuration</name>
<check level="warning" class="org.scalastyle.file.FileTabChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.file.FileLengthChecker" enabled="true">
<parameters>
<parameter name="maxFileLength"><![CDATA[800]]></parameter>
</parameters>
</check>
<check level="error" class="org.scalastyle.file.HeaderMatchesChecker" enabled="true">
<parameters>
<parameter name="header"><![CDATA[/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/]]></parameter>
</parameters>
</check>
<check level="warning" class="org.scalastyle.scalariform.SpacesAfterPlusChecker" enabled="true"/>
<check level="warning" class="org.scalastyle.file.WhitespaceEndOfLineChecker" enabled="true"/>
<check level="warning" class="org.scalastyle.scalariform.SpacesBeforePlusChecker" enabled="true"/>
<check level="warning" class="org.scalastyle.scalariform.SpaceAfterCommentStartChecker" enabled="true"/>
<check level="warning" class="org.scalastyle.file.FileLineLengthChecker" enabled="true">
<parameters>
<parameter name="maxLineLength"><![CDATA[120]]></parameter>
<parameter name="tabSize"><![CDATA[4]]></parameter>
</parameters>
</check>
<check level="warning" class="org.scalastyle.scalariform.ClassNamesChecker" enabled="true">
<parameters>
<parameter name="regex"><![CDATA[[A-Z][A-Za-z]*]]></parameter>
</parameters>
</check>
<check level="warning" class="org.scalastyle.scalariform.ObjectNamesChecker" enabled="true">
<parameters>
<parameter name="regex"><![CDATA[[A-Z][A-Za-z]*]]></parameter>
</parameters>
</check>
<check level="warning" class="org.scalastyle.scalariform.PackageObjectNamesChecker" enabled="true">
<parameters>
<parameter name="regex"><![CDATA[^[a-z][A-Za-z]*$]]></parameter>
</parameters>
</check>
<check level="warning" class="org.scalastyle.scalariform.EqualsHashCodeChecker" enabled="true"/>
<check level="warning" class="org.scalastyle.scalariform.IllegalImportsChecker" enabled="true">
<parameters>
<parameter name="illegalImports"><![CDATA[sun._,java.awt._]]></parameter>
</parameters>
</check>
<check level="warning" class="org.scalastyle.scalariform.ParameterNumberChecker" enabled="true">
<parameters>
<parameter name="maxParameters"><![CDATA[8]]></parameter>
</parameters>
</check>
<check level="warning" class="org.scalastyle.scalariform.NoWhitespaceBeforeLeftBracketChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.NoWhitespaceAfterLeftBracketChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.ReturnChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.NullChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.NoCloneChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.NoFinalizeChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.CovariantEqualsChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.StructuralTypeChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.NumberOfTypesChecker" enabled="true">
<parameters>
<parameter name="maxTypes"><![CDATA[30]]></parameter>
</parameters>
</check>
<check level="warning" class="org.scalastyle.scalariform.UppercaseLChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.SimplifyBooleanExpressionChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.IfBraceChecker" enabled="true">
<parameters>
<parameter name="singleLineAllowed"><![CDATA[true]]></parameter>
<parameter name="doubleLineAllowed"><![CDATA[false]]></parameter>
</parameters>
</check>
<check level="warning" class="org.scalastyle.scalariform.MethodLengthChecker" enabled="true">
<parameters>
<parameter name="maxLength"><![CDATA[50]]></parameter>
</parameters>
</check>
<check level="warning" class="org.scalastyle.scalariform.MethodNamesChecker" enabled="true">
<parameters>
<parameter name="regex"><![CDATA[^[a-z][A-Za-z0-9]*$]]></parameter>
</parameters>
</check>
<check level="warning" class="org.scalastyle.scalariform.NumberOfMethodsInTypeChecker" enabled="true">
<parameters>
<parameter name="maxMethods"><![CDATA[30]]></parameter>
</parameters>
</check>
<check level="error" class="org.scalastyle.scalariform.PublicMethodsHaveTypeChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.file.NewLineAtEofChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.RedundantIfChecker" enabled="true"></check>
<check customId="println" level="error" class="org.scalastyle.scalariform.TokenChecker" enabled="true">
<parameters>
<parameter name="regex"><![CDATA[println]]></parameter>
</parameters>
</check>
<check level="warning" class="org.scalastyle.scalariform.DeprecatedJavaChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.EmptyClassChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.ClassTypeParameterChecker" enabled="true">
<parameters>
<parameter name="regex"><![CDATA[^[A-Z_]$]]></parameter>
</parameters>
</check>
<check level="error" class="org.scalastyle.scalariform.UnderscoreImportChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.LowercasePatternMatchChecker" enabled="true"></check>
<check level="warning" class="org.scalastyle.scalariform.ImportGroupingChecker" enabled="true"></check>
</scalastyle>
Loading