Merge pull request #129 from mapbox/localised-vs-nonlocalised

Localize all or nothing

CHANGELOG.md

@@ -1,3 +1,7 @@
+# 2.0.0
+
+- Updates the `localize` function to return features with either all properties localized or features with no properties localized; nothing in between [#129](https://github.com/mapbox/vtcomposite/pull/129).
+
 # 1.1.0
 
 - Updates the `localize` function to translate `_mbx_class` to `class` when `_mbx_worldview` is provided along with matching worldview filtering

CONTRIBUTING.md

@@ -31,6 +31,8 @@ npm install -g documentation
 npm run docs
 ```
 
+* Note for MacOS: if you're having issue building on MacOS, try commenting out [`make_global_settings` in binding.gyp](https://github.com/mapbox/vtcomposite/blob/main/binding.gyp#L5-L9) (for more info see [this](https://github.com/mapbox/node-cpp-skel/pull/169#issuecomment-1068127191)).
+
 # Benchmarks
 
 Benchmarks can be run with the bench/bench.js script to test vtcomposite against common real-world fixtures (provided by mvt-fixtures) and to test vtcomposite against its predecessor compositing library node-mapnik. When making changes in a pull request, please provide the benchmarks from the master branch and the HEAD of your current branch.  You can control the `concurrency`, `iterations`, and `package` of the benchmarks with the following command:
@@ -96,4 +98,4 @@ These commands are set from within [the Makefile](./Makefile).
 
 # Releasing
 
-In short, you'll need to push a commit with the log line containing `[publish binary]` to build the binary, followed by an `npm publish`. See [node-cpp-skel](https://github.com/mapbox/node-cpp-skel/blob/d2848ed5bcc5a798ff39a2ac139b70844043ff11/docs/publishing-binaries.md) for all the details.
+In short, you'll need to push a commit with the log line containing `[publish binary]` to build the binary, followed by an `npm publish`. See [node-cpp-skel](https://github.com/mapbox/node-cpp-skel/blob/d2848ed5bcc5a798ff39a2ac139b70844043ff11/docs/publishing-binaries.md) for all the details.

README.md

@@ -73,40 +73,86 @@ A filtering function for modifying a tile's features and properties to support l
 #### Parameters
 
 - `params` **Object**
-  - `params.buffer` **Buffer** a vector tile buffer, gzip compressed or not
-  - `params.compress` **Boolean** a boolean value indicating whether or not to return a compressed buffer. Default is to return an uncompressed buffer. (optional, default `false`)
-  - `params.language` **String** the IETF BCP 47 language code used to search for matching translations available in a feature's properties. All language-related properties must match the following format: `{language_prefix}{language_property}_{language}`. Default properties are `_mbx_name_{language}`. Example `_mbx_name_jp` property includes the Japanese translation for the value in `name`.
-    - If a feature has a matching translation, the feature redefines the `{language_property}` to the value of the discovered translation.
-    - If a feature has a matching translation, the original value from `{language_property}` is retained in a newly created value `{language_property}_local`.
-    - If a translation is from a property _without_ a `{language_prefix}`, the property is retained in the final tile.
-    - If a translation is from a property _with_ a `{language_prefix}`, the property is dropped in the final tile. This allows users to add as many languages as desired to the vector tile and eventually drop them before using the tile.
-    - If a feature does not have a matching translation, no fields are modified.
-    - In any case, all fields with a property that matches `{language_prefix}` are dropped from the final tile, even if they do not pertain to language translations.
-  - `params.language_property` **String** the primary property in features that identifies the feature in a language. Default `name`. This values is used to search for additional translations that match the following format `{language_property}_{language}`
-  - `params.language_prefix` **String** prefix for any additional translation properties. The value is used to search for additional translations that match the following format: `{language_prefix}{language_property}_{language}`.
-  - `params.worldview` **Array<String>** array of ISO 3166-1 alpha-2 country codes used to filter out features of different worldviews. Worldview data must be included in the vector tile. See `params.worldview_property` for more details on encoding data.
-    - If a feature matches one of the requested worldviews, the feature is kept. It will have a property `worldview` equal to the matching worldview value and the `params.worldview_property` property will be dropped. If the original feature contained a `worldview` property, it is overwritten.
-    - If a feature has a worldview value of `all` it is considered a match and `worldview: all` is added to the feature's properties and the `params.worldview_property` property is dropped. If the original feature contains a `worldview` property, it is ovewritten.
-    - If a feature does not match the request worldview the entire feature is dropped.
-    - If a feature does not have a `params.worldview_property` property it is retained.
-  - `params.worldview_property` **String** the name of the property that specifies in which worldview a feature belongs. The vector tile encoded property must contain a comma-separated string of ISO 3166-1 alpha-2 country codes that define which worldviews the feature represents (example: `US,RU,IN`). Default value: `_mbx_worldview`.
-    - If a feature contains multiple values that match multiple given values in the `params.worldview` array, it will be split into multiple features in the final vector tile, one for each matching worldview.
-  - `params.class_property` **String** the name of the property that specifies the class of a feature (examples: `sea`, `canal`, `village`). Default value: `_mbx_class`.
-- `callback` **Function** callback function that returns `err` and `buffer` parameters
+  - `params.buffer` **Buffer** a vector tile buffer, gzip compressed or not.
+  - `params.compress` **Boolean** a boolean value indicating whether or not to return a compressed buffer.
+    - Default value: `false` (i.e. return an uncompressed buffer).
+  - `params.hidden_prefix` **String** prefix for any additional properties that will be used to override non-prefixed properties.
+    - Default value: `_mbx_`.
+    - Any property that starts with this prefix are considered hidden properties and thus will be dropped.
+  - `params.languages` **Array<String>** array of IETF BCP 47 language codes used to search for matching translations available in a feature's properties.
+    - Optional parameter.
+    - All language-related properties must match the following format: `{hidden_prefix}{language_property}_{language}`.
+    - Default properties are `_mbx_name_{language}`; for example, the `_mbx_name_jp` property contains the Japanese translation for the value in `name`.
+  - `params.language_property` **String** the primary property in features that identifies the feature in a language.
+    - Default value: `name`.
+    - This values is used to search for additional translations that match the following format `{language_property}_{language}`.
+  - `params.worldviews` **Array<String>** array of ISO 3166-1 alpha-2 country codes used to filter out features of different worldviews.
+    - Optional parameter.
+    - For now, only the first item in the array will be processed; the rest are discarded (*TODO in the future*: expand support for more than one worldviews).
+  - `params.worldview_property` **String** the name of the property that specifies in which worldview a feature belongs.
+    - Default value: `worldview`.
+    - The vector tile encoded property must contain a single ISO 3166-1 alpha-2 country code or a comma-separated string of country codes that define which worldviews the feature represents (for example: `JP`, `IN,RU,US`).
+  - `params.worldview_default` **String** default worldview to assume when `params.worldviews` is not provided.
+    - Default value: `US`.
+  - `params.class_property` **String** the name of the property that specifies the class category of a feature.
+    - Default value: `class`.
+  - `callback` **Function** callback function that returns `err` and `buffer` parameters
+
+The existence of the parameters `params.languages` and `params.worldviews` determines the type of features that will be returned:
+
+- Non-localized feature: when `params.languages` and `params.worldviews` both do not exist.
+  - No new language property.
+  - The property `{language_property}` retains its original value.
+  - Properties like `{language_property}_{language}` are kept.
+  - Properties like `{hidden_prefix}{language_property}_{language}` are dropped.
+  - All features with `{worldview_property}` are kept.
+  - All features with `{hidden_prefix}{worldview_property}` are dropped except for those that have the value `all`.
+  - Property `{class_property}` retains its original value.
+  - Property `{hidden_prefix}{class_property}` is dropped.
+
+- Localized feature: when either `params.languages` or `params.worldviews` exists.
+  - A new `{language_property}_local` property is created to keep the original value of `{language_property}`
+  - The value of `{language_property}` is replaced with the first translation found by looping through `params.languages`.
+    - First searches for `{language_property}_{language}` and then `{hidden_prefix}{language_property}_{language}` before moving on to the next language in `params.languages`.
+  - Properties like `{language_property}_{language}` are dropped.
+  - Properties like `{hidden_prefix}{language_property}_{language}` are dropped.
+  - All features with `{worldview_property}` are dropped except for those that have the value `all`.
+  - Features with `{hidden_prefix}{worldview_property}` are kept if their `{hidden_prefix}{worldview_property}` value is
+    - `all`, or
+    - a comma-separated list that contains the first item of `parmas.worldviews`, in which a property `{worldview_property}` is created from that one single worldview country code and the property `{hidden_prefix}{worldview_property}` is dropped.
+  - If `{hidden_prefix}{class_property}` exists,
+    - Property `{class_property}` is replaced with the value in `{hidden_prefix}{class_property}`.
+    - Property `{hidden_prefix}{class_property}` is dropped.
 
 #### Example
 
+Example 1: a tile of non-localized features
+
+```js
+const { localize } = require('@mapbox/vtcomposite');
+
+const params = {
+  // REQUIRED
+  buffer: require('fs').readFileSync('./path/to/tile.mvt'),
+};
+
+localize(params, function(err, result) {
+  if (err) throw err;
+  console.log(result); // tile buffer
+});
+```
+
+Example 2: a tile of localized features in Japan worldview
+
 ```js
 const { localize } = require('@mapbox/vtcomposite');
 
 const params = {
   // REQUIRED
   buffer: require('fs').readFileSync('./path/to/tile.mvt'),
   // OPTIONAL (defaults)
-  language: null,
-  worldview: [],
-  worldview_property: '_mbx_worldview',
-  class_property: '_mbx_class',
+  languages: ['ja']
+  worldviews: ['JP'],
   compress: true
 };
 

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mapbox/vtcomposite",
-  "version": "1.1.0",
+  "version": "2.0.0",
   "description": "Compositing operations on Vector Tiles (c++ bindings using N-API)",
   "url": "http://github.com/mapbox/vtcomposite",
   "main": "./lib/index.js",

src/module_utils.hpp

@@ -29,6 +29,12 @@ inline std::vector<std::string> split(std::string const& input)
     return values;
 }
 
+// checks if a string starts with a given substring
+inline bool startswith(std::string const& astring, std::string const& substring)
+{
+    return substring.length() <= astring.length() && std::equal(substring.begin(), substring.end(), astring.begin());
+}
+
 // finds the intersection of two vectors of strings
 // and assigns the intersection to a new vector passed by reference
 // results are returned in alphabetically ascending order
@@ -44,4 +50,4 @@ void inline intersection(
                           v2.begin(), v2.end(),
                           std::back_inserter(result));
 }
-} // namespace utils
+} // namespace utils