Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incompatible_disallow_data_transition: remove outdated "cfg = "data'" syntax #6153

Closed
gregestren opened this issue Sep 14, 2018 · 17 comments
Closed
Assignees
Labels
incompatible-change Incompatible/breaking change P2 We'll consider working on this in future. (Assignee optional)

Comments

@gregestren
Copy link
Contributor

Description of the problem / feature request:

cfg = "data" is outdated syntax that was turned into a no-op in 10a6130.

The functionality this was originally intended for will be provided by Starlark build configuration.

Feature requests: what underlying problem are you trying to solve with this feature?

Remove outdated code & clean up technical debt.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Support can already be toggled with --incompatible_disallow_data_transition:

$ cat testapp/rules.bzl
def _foo_rule_impl(ctx):
    return []

foo_rule = rule(
    implementation = _foo_rule_impl,
    attrs = {
        "data": attr.label_list(cfg = "data"),
    },
)

$ cat testapp/BUILD
load("//testapp:rules.bzl", "foo_rule")

foo_rule(
    name = "g",
)

$ bazel build //testapp:g --incompatible_disallow_data_transition=false  [default]
INFO: Build completed successfully, 1 total action

$ bazel build //testapp:g --incompatible_disallow_data_transition
ERROR: /workspace/testapp/rules.bzl:7:17: Traceback (most recent call last):
	File "/workspace/testapp/rules.bzl", line 4
		rule(implementation = _foo_rule_impl, a...")})
	File "/workspace/testapp/rules.bzl", line 7, in rule
		attr.label_list(cfg = "data")
Using cfg = "data" on an attribute is a noop and no longer supported. Please remove it.
You can use --incompatible_disallow_data_transition=false to temporarily disable this
check.
ERROR: Skipping '//testapp:g': error loading package 'testapp': Extension file
'testapp/rules.bzl' has errors
FAILED: Build did NOT complete successfully (1 packages loaded)
@gregestren
Copy link
Contributor Author

Any recommendations on a deprecation plan?

@c-parsons
Copy link
Contributor

vladmos@, might be a good candidate for our upcoming formatter/migrator tool. This one's a pretty easy one -- we just need to remove all instances of cfg = "data". They're a no-op.

@vladmos
Copy link
Member

vladmos commented Sep 14, 2018

c-parsons@ I've already implemented this particular fix, it searches for nodes that match the pattern attr.xxxx(..., cfg = "data", ...) and report/remove the cfg = "data" part.

@gregestren
Copy link
Contributor Author

How does this catch all uses across all repositories?

i.e. how do we know when its safe to enforce the deprecation?

@laurentlb
Copy link
Contributor

We will never know: not all the users of Bazel publish the BUILD/bzl files.

To provide smooth upgrade for users:

  • Give a notice (with the incompatible flag + note in the release announcement)
  • Make sure that all projects tested on the CI are not broken by the change
  • Provide a tool

Once the tool has a few more fixes (for other kinds of cleanup), we can advertise it widely.

@laurentlb laurentlb added the P2 We'll consider working on this in future. (Assignee optional) label Sep 17, 2018
@c-parsons
Copy link
Contributor

vladmos@: awesome! Is there a good documentation source that we can point users to for using the tool to fix this?
I can draft a notice, but I want to make sure to link to the tool :)

@dslomov dslomov changed the title Starlark: remove outdated "cfg = "data'" syntax incompatible_disallow_data_transition: remove outdated "cfg = "data'" syntax Oct 31, 2018
@dslomov dslomov added the incompatible-change Incompatible/breaking change label Oct 31, 2018
@dslomov
Copy link
Contributor

dslomov commented Oct 31, 2018

What is missing for migration: migration docs, length of migration window. After these are done, please add "migration-ready" label.

@vladmos
Copy link
Member

vladmos commented Nov 15, 2018

Can be automatically fixed with buildifier --lint=fix --warnings=attr-cfg

bazel-io pushed a commit that referenced this issue Nov 30, 2018
Baseline: 7bf7f03

Cherry picks:

   + fd52341:
     update bazel-toolchains pin to latest release Part of changes to
     allow bazelci to use 0.19.0 configs. RBE toolchain configs at or
     before 0.17.0 are not compatible with bazel 0.19.0 or above.
   + 241f28d:
     Revert "Toggle --incompatible_disable_late_bound_option_defaults
     flag."
   + f7e5aef:
     Add cc_toolchain targets for the new entries in the default
     cc_toolchain_suite.
   + d2920e3:
     Revert "WindowsFileSystem: open files with delete-sharing"

[Breaking changes in 0.20](https://github.com/bazelbuild/bazel/issues?q=is%3Aissue+label%3Abreaking-change-0.20)

  - [--incompatible_remove_native_http_archive](#6570).
  - [--incompatible_remove_native_git_repository](#6569).
  - [--incompatible_disable_cc_toolchain_label_from_crosstool_proto](#6434).
  - [--incompatible_disable_depset_in_cc_user_flags](#6384).
  - [--incompatible_disable_cc_configuration_make_variables](#6381).
  - [--incompatible_disallow_conflicting_providers](#5902).
  - [--incompatible_range_type](#5264).

[0.20 is a migration window for the following changes](https://github.com/bazelbuild/bazel/issues?q=is%3Aissue+label%3Amigration-0.20)

  - [--incompatible_use_jdk10_as_host_javabase](#6661)
  - [--incompatible_use_remotejdk_as_host_javabase](#6656)
  - [--incompatible_disable_sysroot_from_configuration](#6565)
  - [--incompatible_provide_cc_toolchain_info_from_cc_toolchain_suite](#6537)
  - [--incompatible_disable_depset_in_cc_user_flags](#6383)
  - [--incompatible_package_name_is_a_function](#5827)

[Breaking changes in the next release (0.21)](https://github.com/bazelbuild/bazel/issues?q=is%3Aissue+label%3Abreaking-change-0.21)

  - [--incompatible_use_jdk10_as_host_javabase](#6661)
  - [--incompatible_use_remotejdk_as_host_javabase](#6656)
  - [--incompatible_disable_sysroot_from_configuration](#6565)
  - [--incompatible_provide_cc_toolchain_info_from_cc_toolchain_suite](#6537)
  - [--incompatible_disable_depset_in_cc_user_flags](#6383)
  - [--incompatible_disallow_data_transition](#6153)
  - [--incompatible_package_name_is_a_function](#5827)
  - [--incompatible_disallow_slash_operator](#5823)
  - [--incompatible_static_name_resolution](#5637)

Incompatible changes:

  - the --experimental_no_dotd_scanning_with_modules command line
    argument is not supported anymore.
  - The --prune_cpp_modules command line option is not supported
    anymore.
  - the --experimental_prune_cpp_input_discovery command line option
    is not supported anymore.

New features:

  - Added support for Android NDK r18.

Important changes:

  - The 'default' parameter of attr.output and attr.output_list is
    removed. This is controlled by
    --incompatible_no_output_attr_default
  - A number of platform-related Starlark APIs which were previously
    marked "experimental" are now disabled by default, and may be
    enabled via --experimental_platforms_api
  - Make legacy-test-support ("legacy_test-<api-level>") from
    android_sdk_repository neverlink. The legacy test support
    libraries shouldn't be built into test binaries. To make them
    available at runtime, developers should declare them via
    uses-library:
    https://developer.android.com/training/testing/set-up-project#andr
    oid-test-base
  - query remote server Capabilities (per REAPI v2)
  - CppRules: All cc_toolchains depended on from
    cc_toolchain_suite.toolchains are now analyzed when not using
    platforms in order to select the right cc_toolchain.
  - removed obsolete --explicit_jre_deps flag.
  - Incompatible flag
    --incompatible_disable_legacy_cpp_toolchain_skylark_api was
    flipped.
  - Improve error messaging when unsupport proguard options are
    specified at the library level.
  - Incompatible flag
    --incompatible_disable_legacy_cpp_toolchain_skylark_api was
    flipped.
  - Incompatible flag
    --incompatible_disable_legacy_cpp_toolchain_skylark_api was
    flipped.
  - The --incompatible_disable_late_bound_option_defaults flag has
    been flipped (#6384)
  - Incompatible flag
    --incompatible_disable_legacy_flags_cc_toolchain_api was flipped
    (#6434)
  - Fixed issue where ctx.resolve_command created conflicting
    intermediate files when resolve_command was called multiple times
    within the same rule invocation with a long command attribute.
  - Incompatible flag
    --incompatible_disable_cc_configuration_make_variables was
    flipped (#6381)
  - If the --javabase flag is unset, it Bazel locates a JDK using
    the JAVA_HOME environment variable and searching the PATH. If no
    JDK is found --javabase will be empty, and builds targeting Java
    will not
    be supported. Previously Bazel would fall back to using the
    embedded
    JDK as a --javabase, but this is no longer default behaviour. A
    JDK should
    be explicitly installed instead to enable Java development
  - Bazel will now shut down when idle for 5 minutes and the system
    is low on RAM (linux only).
  - CROSSTOOL file is now read from the package of cc_toolchain, not
    from
    the package of cc_toolchain_suite. This is not expected to break
    anybody since
    cc_toolchain_suite and cc_toolchain are commonly in the same
    package.
  - All overrides of Starlark's ctx.new_file function are now
    deprecated.
      Try the `--incompatible_new_actions_api` flag to ensure your
    code is forward-compatible.
  - --incompatible_disable_cc_toolchain_label_from_crosstool_proto
    was flipped.
  - Introduce --(no)shutdown_on_low_sys_mem startup flag to toggle
    idle low-memory shutdown, disabled by default.
  - --incompatible_disable_cc_toolchain_label_from_crosstool_proto
    was flipped.
  - --incompatible_disable_cc_toolchain_label_from_crosstool_proto
    was flipped.
  - CppRules: All cc_toolchains depended on from
    cc_toolchain_suite.toolchains are now analyzed when not using
    platforms in order to select the right cc_toolchain.
  - The function `attr.license` is deprecated and will be removed.
      It can be disabled now with `--incompatible_no_attr_license`.
  - `range()` function now returns a lazy value
    (`--incompatible_range_type` is now set by default).
  - The code coverage report now includes the actual paths to header
    files instead of the ugly,
    Bazel generated, virtual includes path.
  - `--incompatible_disallow_conflicting_providers` has been switched
    to true
  - Add new flag `--incompatible_disable_systool_from_configration` to
    disable loading the systool from CppConfiguration.
  - Add new flag `--incompatible_disable_sysroot_from_configuration`
    to
    disable loading the systool from CppConfiguration.
  - Sorting remote Platform properties for remote execution. May
    affect cache keys!
  - Use different server log files per Bazel server process; java.log
    is
    now a symlink to the latest log.

This release contains contributions from many people at Google, as well as a7g4 <a7g4@a7g4.net>, Alan <alan.agius@betssongroup.com>, Asaf Flescher <asafflesch@gmail.com>, Benjamin Peterson <bp@benjamin.pe>, Ed Schouten <ed.schouten@prodrive-technologies.com>, George Gensure <ggensure@uber.com>, George Kalpakas <kalpakas.g@gmail.com>, Greg <gregestren@users.noreply.github.com>, Irina Iancu <iirina@users.noreply.github.com>, Keith Smiley <keithbsmiley@gmail.com>, Loo Rong Jie <loorongjie@gmail.com>, Mark Zeren <mzeren@vmware.com>, Petros Eskinder <petroseskinder@users.noreply.github.com>, rachcatch <rachelcatchpoole@hotmail.com>, Robert Brown <robert.brown@gmail.com>, Robert Gay <robert.gay@redfin.com>, Salty Egg <2281521+zhouhao@users.noreply.github.com>.
ahirreddy pushed a commit to databricks/bazel that referenced this issue Dec 1, 2018
Baseline: 7bf7f03

Cherry picks:

   + fd52341:
     update bazel-toolchains pin to latest release Part of changes to
     allow bazelci to use 0.19.0 configs. RBE toolchain configs at or
     before 0.17.0 are not compatible with bazel 0.19.0 or above.
   + 241f28d:
     Revert "Toggle --incompatible_disable_late_bound_option_defaults
     flag."
   + f7e5aef:
     Add cc_toolchain targets for the new entries in the default
     cc_toolchain_suite.
   + d2920e3:
     Revert "WindowsFileSystem: open files with delete-sharing"

[Breaking changes in 0.20](https://github.com/bazelbuild/bazel/issues?q=is%3Aissue+label%3Abreaking-change-0.20)

  - [--incompatible_remove_native_http_archive](bazelbuild#6570).
  - [--incompatible_remove_native_git_repository](bazelbuild#6569).
  - [--incompatible_disable_cc_toolchain_label_from_crosstool_proto](bazelbuild#6434).
  - [--incompatible_disable_depset_in_cc_user_flags](bazelbuild#6384).
  - [--incompatible_disable_cc_configuration_make_variables](bazelbuild#6381).
  - [--incompatible_disallow_conflicting_providers](bazelbuild#5902).
  - [--incompatible_range_type](bazelbuild#5264).

[0.20 is a migration window for the following changes](https://github.com/bazelbuild/bazel/issues?q=is%3Aissue+label%3Amigration-0.20)

  - [--incompatible_use_jdk10_as_host_javabase](bazelbuild#6661)
  - [--incompatible_use_remotejdk_as_host_javabase](bazelbuild#6656)
  - [--incompatible_disable_sysroot_from_configuration](bazelbuild#6565)
  - [--incompatible_provide_cc_toolchain_info_from_cc_toolchain_suite](bazelbuild#6537)
  - [--incompatible_disable_depset_in_cc_user_flags](bazelbuild#6383)
  - [--incompatible_package_name_is_a_function](bazelbuild#5827)

[Breaking changes in the next release (0.21)](https://github.com/bazelbuild/bazel/issues?q=is%3Aissue+label%3Abreaking-change-0.21)

  - [--incompatible_use_jdk10_as_host_javabase](bazelbuild#6661)
  - [--incompatible_use_remotejdk_as_host_javabase](bazelbuild#6656)
  - [--incompatible_disable_sysroot_from_configuration](bazelbuild#6565)
  - [--incompatible_provide_cc_toolchain_info_from_cc_toolchain_suite](bazelbuild#6537)
  - [--incompatible_disable_depset_in_cc_user_flags](bazelbuild#6383)
  - [--incompatible_disallow_data_transition](bazelbuild#6153)
  - [--incompatible_package_name_is_a_function](bazelbuild#5827)
  - [--incompatible_disallow_slash_operator](bazelbuild#5823)
  - [--incompatible_static_name_resolution](bazelbuild#5637)

Incompatible changes:

  - the --experimental_no_dotd_scanning_with_modules command line
    argument is not supported anymore.
  - The --prune_cpp_modules command line option is not supported
    anymore.
  - the --experimental_prune_cpp_input_discovery command line option
    is not supported anymore.

New features:

  - Added support for Android NDK r18.

Important changes:

  - The 'default' parameter of attr.output and attr.output_list is
    removed. This is controlled by
    --incompatible_no_output_attr_default
  - A number of platform-related Starlark APIs which were previously
    marked "experimental" are now disabled by default, and may be
    enabled via --experimental_platforms_api
  - Make legacy-test-support ("legacy_test-<api-level>") from
    android_sdk_repository neverlink. The legacy test support
    libraries shouldn't be built into test binaries. To make them
    available at runtime, developers should declare them via
    uses-library:
    https://developer.android.com/training/testing/set-up-project#andr
    oid-test-base
  - query remote server Capabilities (per REAPI v2)
  - CppRules: All cc_toolchains depended on from
    cc_toolchain_suite.toolchains are now analyzed when not using
    platforms in order to select the right cc_toolchain.
  - removed obsolete --explicit_jre_deps flag.
  - Incompatible flag
    --incompatible_disable_legacy_cpp_toolchain_skylark_api was
    flipped.
  - Improve error messaging when unsupport proguard options are
    specified at the library level.
  - Incompatible flag
    --incompatible_disable_legacy_cpp_toolchain_skylark_api was
    flipped.
  - Incompatible flag
    --incompatible_disable_legacy_cpp_toolchain_skylark_api was
    flipped.
  - The --incompatible_disable_late_bound_option_defaults flag has
    been flipped (bazelbuild#6384)
  - Incompatible flag
    --incompatible_disable_legacy_flags_cc_toolchain_api was flipped
    (bazelbuild#6434)
  - Fixed issue where ctx.resolve_command created conflicting
    intermediate files when resolve_command was called multiple times
    within the same rule invocation with a long command attribute.
  - Incompatible flag
    --incompatible_disable_cc_configuration_make_variables was
    flipped (bazelbuild#6381)
  - If the --javabase flag is unset, it Bazel locates a JDK using
    the JAVA_HOME environment variable and searching the PATH. If no
    JDK is found --javabase will be empty, and builds targeting Java
    will not
    be supported. Previously Bazel would fall back to using the
    embedded
    JDK as a --javabase, but this is no longer default behaviour. A
    JDK should
    be explicitly installed instead to enable Java development
  - Bazel will now shut down when idle for 5 minutes and the system
    is low on RAM (linux only).
  - CROSSTOOL file is now read from the package of cc_toolchain, not
    from
    the package of cc_toolchain_suite. This is not expected to break
    anybody since
    cc_toolchain_suite and cc_toolchain are commonly in the same
    package.
  - All overrides of Starlark's ctx.new_file function are now
    deprecated.
      Try the `--incompatible_new_actions_api` flag to ensure your
    code is forward-compatible.
  - --incompatible_disable_cc_toolchain_label_from_crosstool_proto
    was flipped.
  - Introduce --(no)shutdown_on_low_sys_mem startup flag to toggle
    idle low-memory shutdown, disabled by default.
  - --incompatible_disable_cc_toolchain_label_from_crosstool_proto
    was flipped.
  - --incompatible_disable_cc_toolchain_label_from_crosstool_proto
    was flipped.
  - CppRules: All cc_toolchains depended on from
    cc_toolchain_suite.toolchains are now analyzed when not using
    platforms in order to select the right cc_toolchain.
  - The function `attr.license` is deprecated and will be removed.
      It can be disabled now with `--incompatible_no_attr_license`.
  - `range()` function now returns a lazy value
    (`--incompatible_range_type` is now set by default).
  - The code coverage report now includes the actual paths to header
    files instead of the ugly,
    Bazel generated, virtual includes path.
  - `--incompatible_disallow_conflicting_providers` has been switched
    to true
  - Add new flag `--incompatible_disable_systool_from_configration` to
    disable loading the systool from CppConfiguration.
  - Add new flag `--incompatible_disable_sysroot_from_configuration`
    to
    disable loading the systool from CppConfiguration.
  - Sorting remote Platform properties for remote execution. May
    affect cache keys!
  - Use different server log files per Bazel server process; java.log
    is
    now a symlink to the latest log.

This release contains contributions from many people at Google, as well as a7g4 <a7g4@a7g4.net>, Alan <alan.agius@betssongroup.com>, Asaf Flescher <asafflesch@gmail.com>, Benjamin Peterson <bp@benjamin.pe>, Ed Schouten <ed.schouten@prodrive-technologies.com>, George Gensure <ggensure@uber.com>, George Kalpakas <kalpakas.g@gmail.com>, Greg <gregestren@users.noreply.github.com>, Irina Iancu <iirina@users.noreply.github.com>, Keith Smiley <keithbsmiley@gmail.com>, Loo Rong Jie <loorongjie@gmail.com>, Mark Zeren <mzeren@vmware.com>, Petros Eskinder <petroseskinder@users.noreply.github.com>, rachcatch <rachelcatchpoole@hotmail.com>, Robert Brown <robert.brown@gmail.com>, Robert Gay <robert.gay@redfin.com>, Salty Egg <2281521+zhouhao@users.noreply.github.com>.
@meteorcloudy
Copy link
Member

Flipping this flag will causing many projects to fail in downstream, please help the downstream projects to migrate (filing issues, pinging the project owner, etc)
https://buildkite.com/bazel/bazel-at-release-plus-incompatible-flags/builds/13#37dd9512-9d32-4430-befb-ce3d371014e7

This is flag was planned to release in 0.21.0, but it's not flipped when 0.21.0 was cut, so looks like we should extend the migration window.

@meteorcloudy
Copy link
Member

This flag will break many projects in downstream, you can see the details at
https://buildkite.com/bazel/bazel-at-release-plus-incompatible-flags/builds/48

Please help them migrate (by filing issues and sending PRs and pinging the owners), thanks!

@gregestren
Copy link
Contributor Author

I'm happy to help migrate. Is anyone else signed up for helping? Any tracking issue to follow the TODOs?

@vladmos
Copy link
Member

vladmos commented Jan 7, 2019

I tried to migrate earlier today, many of the projects from the link above have already been fixed, I've sent bazelbuild/intellij#485 for review. Would it be possible to rerun the global presubmit with --incompatible_disallow_data_transition again?

@gregestren
Copy link
Contributor Author

So sounds like you're on top of it. Let me know if you want any help.

@meteorcloudy
Copy link
Member

@vladmos Yes, you can rerun the pipeline by clicking "New Build" at https://buildkite.com/bazel/bazel-at-release-plus-incompatible-flags

@laurentlb
Copy link
Contributor

I should have updated this thread before. I've been sending PRs over the last days/weeks. My Tensorflow PR was merged this night. I think IntelliJ/CLion is the last one.
Thanks Vlad!

@meteorcloudy
Copy link
Member

Thanks everyone!
@laurentlb TensorFlow's own build is still red (due to Windows build). Because we test downstream at last green commit, so your fix is not yet in downstream. I'm asking TF to fix their Windows build, then it will also be green in downstream with incompatible changes.

@philwo
Copy link
Member

philwo commented Feb 6, 2019

I verified that this flag was flipped and thus will be included in the Bazel 0.23.0 release.

bazel-io pushed a commit that referenced this issue Mar 26, 2019
#6153

RELNOTES: None.
PiperOrigin-RevId: 240442015
luca-digrazia pushed a commit to luca-digrazia/DatasetCommitsDiffSearch that referenced this issue Sep 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
incompatible-change Incompatible/breaking change P2 We'll consider working on this in future. (Assignee optional)
Projects
None yet
Development

No branches or pull requests

7 participants