Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement replace-all expression #2059

Closed
wants to merge 15 commits into from

Conversation

ZeLonewolf
Copy link
Contributor

@ZeLonewolf ZeLonewolf commented Jan 12, 2023

Unblocks osm-americana/openstreetmap-americana#680

This PR implements a replace-all expression for strings. This functionality is needed to efficiently support name delimiters as described in the osm-americana ticket "raise semi-colon limit on lists".

The expression works as follows:

['replace-all', 'cats;dogs', ';', ' / ']

This takes the string cats;dogs and replaces the ; character, replacing it with / and producing cats / dogs.

This is my first maplibre PR, so please help me out if I haven't done anything correctly!

Launch Checklist

  • Confirm your changes do not include backports from Mapbox projects (unless with compliant license) - if you are not sure about this, please ask!
  • Briefly describe the changes in this PR.
  • Link to related issues.
  • Write tests for all new functionality.
  • Document any changes to public APIs.
  • Add an entry to CHANGELOG.md under the ## main section.

@ZeLonewolf ZeLonewolf marked this pull request as ready for review January 12, 2023 01:17
@github-actions
Copy link
Contributor

github-actions bot commented Jan 12, 2023

Bundle size report:

Size Change: +22 B
Total Size Before: 207 kB
Total Size After: 207 kB

Output file Before After Change
maplibre-gl.js 198 kB 198 kB +22 B
maplibre-gl.css 9.1 kB 9.1 kB 0 B
ℹ️ View Details
Source file Before After Change
src/util/evented.ts 525 B 3.97 kB +3.45 kB
node_modules/kdbush/src/index.js 396 B 593 B +197 B
node_modules/gl-matrix/esm/vec4.js 392 B 543 B +151 B
src/render/uniform_binding.ts 572 B 721 B +149 B
src/style/create_style_layer.ts 174 B 234 B +60 B
src/symbol/symbol_size.ts 579 B 637 B +58 B
src/style-spec/expression/definitions/interpolate.ts 1.22 kB 1.27 kB +43 B
src/style-spec/expression/definitions/index.ts 1.63 kB 1.66 kB +27 B
src/data/bucket/pattern_bucket_features.ts 324 B 350 B +26 B
src/util/actor.ts 846 B 871 B +25 B
src/data/bucket/fill_attributes.ts 92 B 109 B +17 B
src/util/ajax.ts 2.09 kB 2.11 kB +12 B
node_modules/csscolorparser/csscolorparser.js 2.05 kB 2.06 kB +7 B
src/style/style_layer/circle_style_layer.ts 531 B 538 B +7 B
src/style/style_layer/background_style_layer_properties.g.ts 109 B 115 B +6 B
src/data/dem_data.ts 834 B 840 B +6 B
node_modules/@mapbox/vector-tile/lib/vectortilefeature.js 1.01 kB 1.01 kB +5 B
src/style/style_layer/fill_extrusion_style_layer.ts 925 B 930 B +5 B
src/symbol/quads.ts 1.88 kB 1.89 kB +5 B
src/style/style_layer/line_style_layer_properties.g.ts 275 B 279 B +4 B
src/style/format_section_override.ts 353 B 357 B +4 B
src/style/style_layer/fill_style_layer.ts 275 B 279 B +4 B
src/data/feature_index.ts 1.74 kB 1.75 kB +4 B
node_modules/@mapbox/point-geometry/index.js 630 B 633 B +3 B
src/style-spec/util/interpolate.ts 230 B 233 B +3 B
src/style/evaluation_parameters.ts 386 B 389 B +3 B
src/data/bucket/circle_attributes.ts 92 B 95 B +3 B
src/util/color_ramp.ts 432 B 435 B +3 B
src/data/bucket/fill_extrusion_bucket.ts 1.43 kB 1.43 kB +3 B
src/util/config.ts 83 B 85 B +2 B
src/style-spec/util/color.ts 316 B 318 B +2 B
src/style-spec/validate/validate.ts 433 B 435 B +2 B
src/util/is_char_in_unicode_block.ts 878 B 880 B +2 B
src/style/style_layer/circle_style_layer_properties.g.ts 226 B 228 B +2 B
src/data/bucket/heatmap_bucket.ts 82 B 84 B +2 B
src/util/webp_supported.ts 374 B 375 B +1 B
src/style-spec/expression/scope.ts 194 B 195 B +1 B
src/style-spec/expression/definitions/assertion.ts 624 B 625 B +1 B
src/style-spec/expression/evaluation_context.ts 310 B 311 B +1 B
src/style-spec/expression/definitions/let.ts 434 B 435 B +1 B
src/style-spec/expression/definitions/index_of.ts 489 B 490 B +1 B
src/style-spec/expression/definitions/case.ts 459 B 460 B +1 B
src/style-spec/expression/definitions/comparison.ts 837 B 838 B +1 B
src/style-spec/expression/definitions/length.ts 332 B 333 B +1 B
src/util/transferable_grid_index.ts 1.02 kB 1.02 kB +1 B
src/style/zoom_history.ts 179 B 180 B +1 B
src/data/segment.ts 450 B 451 B +1 B
node_modules/murmurhash-js/murmurhash2_gc.js 279 B 280 B +1 B
node_modules/gl-matrix/esm/common.js 181 B 182 B +1 B
node_modules/@mapbox/vector-tile/index.js 89 B 90 B +1 B
src/symbol/collision_feature.ts 386 B 387 B +1 B
src/style/style_layer/symbol_style_layer_properties.g.ts 655 B 656 B +1 B
src/util/throttled_invoker.ts 213 B 214 B +1 B
node_modules/@mapbox/whoots-js/index.mjs 273 B 274 B +1 B
src/util/vectortile_to_geojson.ts 250 B 251 B +1 B
src/style-spec/validate/validate_constants.ts 113 B 112 B -1 B
src/style-spec/expression/types/collator.ts 204 B 203 B -1 B
src/style-spec/expression/compound_expression.ts 734 B 733 B -1 B
src/style-spec/expression/definitions/at.ts 377 B 376 B -1 B
src/style-spec/expression/definitions/match.ts 751 B 750 B -1 B
src/style-spec/expression/index.ts 1.66 kB 1.66 kB -1 B
src/style-spec/validate/validate_formatted.ts 62 B 61 B -1 B
src/util/web_worker_transfer.ts 1.03 kB 1.03 kB -1 B
src/util/script_detection.ts 1.65 kB 1.65 kB -1 B
node_modules/murmurhash-js/index.js 66 B 65 B -1 B
src/data/program_configuration.ts 2.62 kB 2.62 kB -1 B
src/data/extent.ts 32 B 31 B -1 B
src/style/query_utils.ts 485 B 484 B -1 B
node_modules/@mapbox/vector-tile/lib/vectortile.js 186 B 185 B -1 B
src/style/style_layer/symbol_style_layer.ts 1.16 kB 1.16 kB -1 B
src/data/bucket/symbol_bucket.ts 4.26 kB 4.26 kB -1 B
src/util/resolve_tokens.ts 97 B 96 B -1 B
src/data/bucket/pattern_attributes.ts 111 B 109 B -2 B
src/style-spec/expression/types.ts 517 B 514 B -3 B
src/style-spec/expression/definitions/number_format.ts 529 B 526 B -3 B
src/data/bucket/line_bucket.ts 2.42 kB 2.41 kB -3 B
src/style/style_layer/heatmap_style_layer.ts 364 B 361 B -3 B
src/style/style_layer/raster_style_layer.ts 66 B 63 B -3 B
src/source/tile_id.ts 1.33 kB 1.33 kB -3 B
src/util/browser.ts 450 B 446 B -4 B
src/style/style_layer/fill_extrusion_style_layer_properties.g.ts 187 B 183 B -4 B
src/style/parse_glyph_pbf.ts 370 B 366 B -4 B
src/style/style_layer/hillshade_style_layer.ts 142 B 138 B -4 B
src/style/style_layer/line_style_layer.ts 831 B 826 B -5 B
src/style/properties.ts 1.91 kB 1.9 kB -6 B
src/style/style_layer/hillshade_style_layer_properties.g.ts 163 B 157 B -6 B
src/style/style_layer/custom_style_layer.ts 526 B 520 B -6 B
src/style-spec/expression/definitions/step.ts 643 B 636 B -7 B
src/render/image_atlas.ts 838 B 831 B -7 B
node_modules/gl-matrix/esm/mat4.js 2.8 kB 2.79 kB -8 B
src/data/bucket/fill_extrusion_attributes.ts 134 B 123 B -11 B
node_modules/earcut/src/earcut.js 2.82 kB 2.79 kB -30 B
src/symbol/symbol_layout.ts 3.83 kB 3.79 kB -32 B
src/util/performance.ts 732 B 699 B -33 B
src/style-spec/util/color_spaces.ts 804 B 757 B -47 B
src/source/rtl_text_plugin.ts 968 B 920 B -48 B
src/util/tile_request_cache.ts 936 B 887 B -49 B
src/util/util.ts 2.58 kB 2.38 kB -197 B
src/data/array_types.g.ts 2.9 kB 2.65 kB -257 B
src/style-spec/error/validation_error.ts 3.56 kB 123 B -3.44 kB

@1ec5 1ec5 mentioned this pull request Feb 5, 2023
6 tasks
@HarelM
Copy link
Collaborator

HarelM commented Feb 7, 2023

This PR and #2064 will be discussed tomorrow in the TSC meeting, feel free to join and present your point of view.

@1ec5
Copy link
Contributor

1ec5 commented Feb 8, 2023

Thanks to the committee for hearing out the OSM Americana project’s perspective at the meeting today. To recap:

  • This expression operator is a conspicuous gap in an otherwise substantial suite of existing string manipulation operators in the style specification.
  • String manipulation is particularly useful in runtime styling, where the expression may need to vary based on dynamic user inputs. Yet the same manipulations are also useful to styles based on tilesets authored by different third parties.
  • Americana’s current use case is to unique a feature’s names after concatenating its name in the user’s dynamically selected language with its names in the local languages. Other use cases have been raised in connection with Add find-and-replace expression operator mapbox/mapbox-gl-js#4100.
  • A built-in operator would eliminate the need for unwieldy, limited, and unperformant workarounds, such as the ones detailed in Add split and join expressions #2064 (comment), as well as in future situations requiring string replacement.

Some counterpoints raised during the call; please correct me if I’ve misrepresented anything:

  • Procedurally, the PR should’ve been preceded by an issue or discussion laying out the case for this feature. The mapbox-gl-js issue did not give MapLibre stakeholders enough opportunity to influence the feature’s design. There is a need for a planning process around the style specification.
  • Style authors are expected to have control of the contents of the tiles used by the style; they should generate their own tilesets that hard-code presentational modifications into the feature properties.
  • Dynamic client-side features necessitate serving tiles from an API that can perform postprocessing on the fly. However, an extension hook for runtime styling could be considered: Expression engine extension points #1295.
  • Style expressions are already too complex. Adding more string manipulation operators would encourage style authors to make them even more complex.
  • Measuring the performance of a given expression across the whole map is already difficult enough without implementing additional expression operators. (There was a mention about Mapbox crashes that I didn’t follow.)

There were some other points made about arrays in vector tiles that were more germane to #2064 and nyurik/future-mvt#1. More details in the meeting minutes.

@HarelM
Copy link
Collaborator

HarelM commented Mar 1, 2023

We are moving the spec to its own repo.
I would recommends moving this PR there.
I would also wait a bit to let the dust settle with the docs generation etc as I've seen that the style-spec docs are not updated when updating the style and I think there's a need to improve on that.
Sorry for the late delay in the response.
Same goes for #2064.

@ZeLonewolf
Copy link
Contributor Author

Closing per #2059 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants