Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upstream: 43b325b6aaf3a96671ae81a64f727f262ade18eb #505

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Commits on Oct 18, 2024

  1. Move SqlTaskStatus and SqlTaskStausTest from msq module to sql module. (

    apache#17380)
    
    - This is a non-functional change that moves SqlTaskStatus and its unit test SqlTaskStatusTest from the msq module to the sql module to help class reuse in other places.
    - This refactor is extracted from this PR to facilitate easier review.
    - Fix a minor spacing issue in the TaskStartTimeoutFault error message.
    abhishekrb19 authored Oct 18, 2024
    Configuration menu
    Copy the full SHA
    9a16d4e View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2024

  1. Fail MSQ compaction if multi-valued partition dimensions are found (a…

    …pache#17344)
    
    MSQ currently supports only single-valued string dimensions as partition keys.
    This patch adds a check to ensure that partition keys are single-valued in case
    this info is available by virtue of segment download for schema inference.
    
    During compaction, if MSQ finds multi-valued dimensions (MVDs) declared as part
    of `range` partitionsSpec, it switches partitioning type to dynamic, ending up in
    repeated compactions of the same interval. To avoid this scenario, the segment
    download logic is also updated to always download segments if info on multi-valued
    dimensions is required.
    gargvishesh authored Oct 19, 2024
    Configuration menu
    Copy the full SHA
    5da9949 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2024

  1. Add BrokerClient implementation (apache#17382)

    This patch is extracted from PR 17353.
    
    Changes:
    
    - Added BrokerClient and BrokerClientImpl to the sql package that leverages the ServiceClient functionality; similar to OverlordClient and CoordinatorClient implementations in the server module.
    - For now, only two broker API stubs are added: submitSqlTask() and fetchExplainPlan().
    - Added a new POJO class ExplainPlan that encapsulates explain plan info.
    - Deprecated org.apache.druid.discovery.BrokerClient in favor of the new BrokerClient in this patch.
    - Clean up ExplainAttributesTest a bit and added serde verification.
    abhishekrb19 authored Oct 21, 2024
    Configuration menu
    Copy the full SHA
    187e21a View commit details
    Browse the repository at this point in the history
  2. Web console: bump dependencies including d3 and typescript (apache#17381

    )
    
    * bump deps including d3
    
    * better clean script
    vogievetsky authored Oct 21, 2024
    Configuration menu
    Copy the full SHA
    6cf372a View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. Remove unused coordinator dynamic configs mergeSegmentsLimit, mergeBy…

    …tesLimit (apache#17384)
    
    * Remove unused coordinator dynamic configs
    
    * Update docs and web-console
    kfaraz authored Oct 22, 2024
    Configuration menu
    Copy the full SHA
    9dfb378 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2024

  1. abstract common base of SQL micro-benchmarks to reduce boilerplate an…

    …d standardize parameters (apache#17383)
    
    changes:
    * adds `SqlBenchmarkDatasets` which contains commonly used benchmark data generator schemas
    * adds `SqlBaseBenchmark` which contains common benchmark segment generation methods for any benchmark using `SqlBenchmarkDatasets`
    * adds `SqlBaseQueryBenchmark` and `SqlBasePlanBenchmark` for benchmarks measuring queries and planning respectively
    * migrate all existing SQL jmh benchmarks to extend `SqlBaseQueryBenchmark`, quite dramatically reducing the boilerplate needed to create benchmarks, and allowing the use of multiple datasources within a benchmark file
    * adjustments to data generator stuff to allow passing in an ObjectMapper so that the same mapper can be used for both benchmark queries and segment generation, avoiding the need to register stuff with both mappers for benchmarks
    * adds `SqlProjectionsBenchmark` and `SqlComplexMetricsColumnsBenchmark` for measuring projections and measuring complex metric compression respectively
    clintropolis authored Oct 23, 2024
    Configuration menu
    Copy the full SHA
    1157ecd View commit details
    Browse the repository at this point in the history
  2. SeekableStreamSupervisor: Use workerExec as the client connectExec. (a…

    …pache#17394)
    
    * SeekableStreamSupervisor: Use workerExec as the client connectExec.
    
    This patch uses the already-existing per-supervisor workerExec as the
    connectExec for task clients, rather than using the process-wide default
    ServiceClientFactory pool.
    
    This helps prevent callbacks from backlogging on the process-wide pool.
    It's especially useful for retries, where callbacks may need to establish
    new TCP connections or perform TLS handshakes.
    
    * Fix compilation, tests.
    
    * Fix style.
    gianm authored Oct 23, 2024
    Configuration menu
    Copy the full SHA
    60dadde View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    43b325b View commit details
    Browse the repository at this point in the history