Skip to content

Commit

Permalink
Merge branch 'master' into superset_additions
Browse files Browse the repository at this point in the history
* master: (466 commits)
  chore: bump pandas to latest stable version (#11018)
  fix: dashboard edit button (again) (#11029)
  style(explore): use tertiary button against gray background (#11011)
  docs: add security vulnerability GH issue template (#11023)
  fix: [dashboard] should not show edit button when user has no edit permit (#11024)
  fix: timer component, fixes #10849, closes #11002 (#11004)
  fix: enable several pylint rules partially in db_engines_specs module (#11000)
  fix: pylint checks in connectors/sqla/models.py (#10974)
  fix: reenable pylint rule `unused-import` in charts and connectors modules (#11014)
  Enabled pylint rules in `db_engines` module: (#11016)
  fix: changes a pylint check in dashboard module (#10978)
  fix: menu shows a 0 when there are not settings (#11009)
  fix: query search low privileged user search access denied (#11017)
  chore: downgrade expected exception from error to info (#10994)
  fix: Add Item Overflow on Dataset Editor (#10983)
  Bring back import menu (#11007)
  feat(listview): feature flag config to set default viewing mode (#10986)
  build: add react-hooks linting (#11006)
  fix: unbreak ci (#11003)
  fix: enable pylint rules in db_engine_specs module (#10998)
  ...

# Conflicts:
#	requirements.txt
#	superset/app.py
#	superset/models/schedules.py
#	superset/tasks/schedules.py
#	superset/translations/messages.pot
  • Loading branch information
rorymillersoft committed Oct 16, 2020
2 parents 31f527f + 6181994 commit 320ab1e
Show file tree
Hide file tree
Showing 1,299 changed files with 110,794 additions and 46,930 deletions.
54 changes: 54 additions & 0 deletions .asf.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

# https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories
---
github:
description: "Apache Superset is a Data Visualization and Data Exploration Platform"
homepage: https://superset.apache.org/
labels:
- superset
- apache
- apache-superset
- data-visualization
- data-viz
- analytics
- business-intelligence
- data-science
- data-engineering
- asf
- bi
- business-analytics
- data-analytics
- data-analysis
- data-science
- python
- react
- sql-editor
- flask
features:
# Enable issues management
issues: true
# Enable projects for project management boards
projects: true
# Enable wiki for documentation
wiki: true

enabled_merge_buttons:
squash: true
merge: false
rebase: false
47 changes: 47 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

# EditorConfig is awesome: https://EditorConfig.org

# top-most EditorConfig file
root = true

# Unix-style newlines with a newline ending every file
[*]
end_of_line = lf
insert_final_newline = true
charset = utf-8

# 4 space indentation for Python files
[*.py]
indent_style = space
indent_size = 4
max_line_length=88

# 2 space indentation for Frontend files
[*.{js,jsx,ts,tsx,html,less,css}]
indent_style = space
indent_size = 2

# 2 space indentation for json and yaml files
[*.{json,yml}]
indent_style = space
indent_size = 2

# Tab indentation
[Makefile]
indent_style = tab
8 changes: 3 additions & 5 deletions .fossa.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,9 @@ analyze:
type: npm
target: superset-frontend
path: superset-frontend
- name: docs
type: pip
target: docs
path: docs
- name: .
- name: base
type: pip
target: .
path: .
options:
requirements: ./requirements/base.txt
12 changes: 12 additions & 0 deletions .github/ISSUE_TEMPLATE/security_vulnerability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
name: Security vulnerability
about: Report a security vulnerability or issue
labels: "#security"

---

## DO NOT REPORT SECURITY VULNERABILITIES HERE

Please report security vulnerabilities to private@superset.apache.org.

In the event a community member discovers a security flaw in Superset, it is important to follow the [Apache Security Guidelines](https://www.apache.org/security/committers.html) and release a fix as quickly as possible before public disclosure. Reporting security vulnerabilities through the usual GitHub Issues channel is not ideal as it will publicize the flaw before a fix can be applied.
8 changes: 0 additions & 8 deletions .github/prlint.json

This file was deleted.

68 changes: 34 additions & 34 deletions .github/workflows/bashlib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ set -e
GITHUB_WORKSPACE=${GITHUB_WORKSPACE:-.}
ASSETS_MANIFEST="$GITHUB_WORKSPACE/superset/static/assets/manifest.json"

# Rounded job start time, used to create a unique Cypress build id for
# parallelization so we can manually rerun a job after 20 minutes
NONCE=$(echo "$(date "+%Y%m%d%H%M") - ($(date +%M)%20)" | bc)

# Echo only when not in parallel mode
say() {
if [[ $(echo "$INPUT_PARALLEL" | tr '[:lower:]' '[:upper:]') != 'TRUE' ]]; then
Expand All @@ -29,38 +33,36 @@ say() {

# default command to run when the `run` input is empty
default-setup-command() {
pip-install
apt-get-install
pip-upgrade
}

# install python dependencies
pip-install() {
cd "$GITHUB_WORKSPACE"

# Don't use pip cache as it doesn't seem to help much.
# cache-restore pip
apt-get-install() {
say "::group::apt-get install dependencies"
sudo apt-get update && sudo apt-get install --yes \
libsasl2-dev
say "::endgroup::"
}

say "::group::Install Python pacakges"
pip install -r requirements.txt
pip install -r requirements-dev.txt
pip install -e ".[postgres,mysql]"
pip-upgrade() {
say "::group::Upgrade pip"
pip install --upgrade pip
say "::endgroup::"

# cache-save pip
}

# prepare (lint and build) frontend code
npm-install() {
cd "$GITHUB_WORKSPACE/superset-frontend"

cache-restore npm
# cache-restore npm

say "::group::Install npm packages"
echo "npm: $(npm --version)"
echo "node: $(node --version)"
npm ci
say "::endgroup::"

cache-save npm
# cache-save npm
}

build-assets() {
Expand All @@ -71,16 +73,6 @@ build-assets() {
say "::endgroup::"
}

build-assets-cached() {
cache-restore assets
if [[ -f "$ASSETS_MANIFEST" ]]; then
echo 'Skip frontend build because static assets already exist.'
else
build-assets
cache-save assets
fi
}

build-instrumented-assets() {
cd "$GITHUB_WORKSPACE/superset-frontend"

Expand Down Expand Up @@ -127,6 +119,7 @@ testdata() {
say "::group::Load test data"
# must specify PYTHONPATH to make `tests.superset_test_config` importable
export PYTHONPATH="$GITHUB_WORKSPACE"
pip install -e .
superset db upgrade
superset load_test_users
superset load_examples --load-test-data
Expand All @@ -139,7 +132,7 @@ codecov() {
local codecovScript="${HOME}/codecov.sh"
# download bash script if needed
if [[ ! -f "$codecovScript" ]]; then
curl -s https://codecov.io/bash > "$codecovScript"
curl -s https://codecov.io/bash >"$codecovScript"
fi
bash "$codecovScript" "$@"
say "::endgroup::"
Expand All @@ -160,19 +153,22 @@ cypress-install() {
# Run Cypress and upload coverage reports
cypress-run() {
cd "$GITHUB_WORKSPACE/superset-frontend/cypress-base"

local page=$1
local group=${2:-Default}
local cypress="./node_modules/.bin/cypress run"
local browser=${CYPRESS_BROWSER:-chrome}

export TERM="xterm"

say "::group::Run Cypress for [$page]"
if [[ -z $CYPRESS_RECORD_KEY ]]; then
$cypress --spec "cypress/integration/$page" --browser "$browser"
else
# additional flags for Cypress dashboard recording
$cypress --spec "cypress/integration/$page" --browser "$browser" --record \
--group "$group" --tag "${GITHUB_REPOSITORY},${GITHUB_EVENT_NAME}"
$cypress --spec "cypress/integration/$page" --browser "$browser" \
--record --group "$group" --tag "${GITHUB_REPOSITORY},${GITHUB_EVENT_NAME}" \
--parallel --ci-build-id "${GITHUB_SHA:0:8}-${NONCE}"
fi

# don't add quotes to $record because we do want word splitting
Expand All @@ -185,15 +181,17 @@ cypress-run-all() {
# so errors can print to stderr.
local flasklog="${HOME}/flask.log"
local port=8081
export CYPRESS_BASE_URL="http://localhost:${port}"

nohup flask run --no-debugger -p $port > "$flasklog" 2>&1 < /dev/null &
nohup flask run --no-debugger -p $port >"$flasklog" 2>&1 </dev/null &
local flaskProcessId=$!

cypress-run "*/*"
cypress-run "*/**/*"

# Upload code coverage separately so each page can have separate flags
# -c will clean existing coverage reports, -F means add flags
codecov -cF "cypress"
# || true to prevent CI failure on codecov upload
codecov -cF "cypress" || true

# After job is done, print out Flask log for debugging
say "::group::Flask log for default run"
Expand All @@ -205,11 +203,13 @@ cypress-run-all() {

# Restart Flask with new configs
kill $flaskProcessId
nohup flask run --no-debugger -p $port > "$flasklog" 2>&1 < /dev/null &
nohup flask run --no-debugger -p $port >"$flasklog" 2>&1 </dev/null &
local flaskProcessId=$!

cypress-run "sqllab/*" "Backend persist"
codecov -cF "cypress"

# || true to prevent CI failure on codecov upload
codecov -cF "cypress" || true

say "::group::Flask log for backend persist"
cat "$flasklog"
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/caches.js
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,11 @@ const assetsConfig = {
module.exports = {
pip: {
path: [`${homeDirectory}/.cache/pip`],
hashFiles: [`${workspaceDirectory}/requirements*.txt`],
hashFiles: [`${workspaceDirectory}/requirements/*.txt`],
},
npm: {
path: [`${homeDirectory}/.npm`],
hashFiles: ['superset-frontend/package-lock.json'],
hashFiles: [`${workspaceDirectory}/superset-frontend/package-lock.json`],
},
assets: assetsConfig,
// use separate cache for instrumented JS files and regular assets
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/license-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
FOSSA_API_KEY: ${{ secrets.FOSSA_API_KEY }}
run: |
set -eo pipefail
if [[ "${{github.event_name}}" != "pull_request" ]]; then
if [[ "${{github.event_name}}" != "pull_request" ]]; then
./scripts/fossa.sh
exit 0
fi
Expand All @@ -30,10 +30,10 @@ jobs:
cat<<EOF
CHANGED FILES:
$FILES
EOF
if [[ "${FILES}" =~ (.*package*\.json|requirements.*\.txt|setup\.py) ]]; then
if [[ "${FILES}" =~ (.*package*\.json|requirements\/[a-z_-]+\.txt|setup\.py) ]]; then
echo "Detected dependency changes... running fossa check"
./scripts/fossa.sh
Expand Down
22 changes: 22 additions & 0 deletions .github/workflows/pr-lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: PR Lint

on:
pull_request:
# By default, a workflow only runs when a pull_request's activity type is opened, synchronize, or reopened. We
# explicity override here so that PR titles are re-linted when the PR text content is edited.
#
# Possible values: https://help.github.com/en/actions/reference/events-that-trigger-workflows#pull-request-event-pull_request
types: [opened, edited, reopened, synchronize]

jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: morrisoncole/pr-lint-action@v1.3.0
with:
title-regex: "^(build|chore|ci|docs|feat|fix|perf|refactor|style|test|other)((.+))?:\\s.+"
on-failed-regex-fail-action: true
on-failed-regex-create-review: true
on-failed-regex-comment:
"Please format your PR title to match: `%regex%`!"
repo-token: "${{ secrets.GITHUB_TOKEN }}"
Loading

0 comments on commit 320ab1e

Please sign in to comment.