Content family and org #545

tomwayson · 2021-06-15T16:58:27Z

This PR gets closer to resolving #322. In essence, everything we need should be there if the content was fetched from the Hub API. If the content is coming from portal, we still have to add logic to enrichContent() to fetch additional enrichments such as server, layers, recordCount, etc.

This PR also makes a few fixes to the way itemToContent and datasetToContent were handling hubId and isPortal. TLDR: we should only set hubId if we are fairy confident that it is in the index. Also, isPortal is not returned by the API, we'll need to set it (and the proxied url) in enrichContent() before calling fetchEnrichments().

In service of those, the PR also moves several low level utils from the Hub app to Hub.js (getFamily, getItemHubId, etc), and it also re-orders how properties are set in datasetToConent() to logically group them (common, server, layer). So it may seem like a lot of 🧀 was moved.

…ontent() affects: @esri/hub-common, @esri/hub-content, @esri/hub-search ISSUES CLOSED: #355

affects: @esri/hub-common, @esri/hub-content

… not returned by the API affects: @esri/hub-content

…, & getItemHubId affects: @esri/hub-common

affects: @esri/hub-content always set content.item.modified to the item's modified date content.modified should prefer the enriched date and fall back to item.modified

…ments affects: @esri/hub-content also enrichContent is now tolerant of content not having errors

dbouwman · 2021-06-15T17:12:37Z

packages/common/src/types.ts

@@ -319,11 +348,25 @@ export interface IHubContent extends IHubResource, IItem {
  data?: {
    [propName: string]: any;
  };
+  // service types: Feature Service, Map Service
+  /** service information (currentVersion, capabilities, maxRecordCount etc) */
+  server?: Partial<IFeatureServiceDefinition>;


Should this be service? vs server?

the Hub API returns server, so we rollin' w/ that.

codecov · 2021-06-15T17:13:46Z

Codecov Report

Merging #545 (02d08c6) into master (655cca6) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master      #545   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          430       430           
  Lines         5929      5982   +53     
  Branches       912       935   +23     
=========================================
+ Hits          5929      5982   +53

Impacted Files	Coverage Δ
packages/common/src/collections.ts	`100.00% <100.00%> (ø)`
packages/common/src/content.ts	`100.00% <100.00%> (ø)`
packages/content/src/content.ts	`100.00% <100.00%> (ø)`
packages/content/src/enrichments.ts	`100.00% <100.00%> (ø)`
packages/content/src/hub.ts	`100.00% <100.00%> (ø)`
packages/content/src/portal.ts	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 655cca6...02d08c6. Read the comment docs.

dbouwman · 2021-06-15T17:21:08Z

packages/common/src/types.ts

+  layers?: Array<Partial<ILayerDefinition>>;
+  // layer types: Feature Layers, Raster Layers
+  /** layer information (geometryType, fields, etc) */
+  layer?: Partial<ILayerDefinition>;


Thoughts on just using layers and if there is only one entry... treating it like a layer?

so, layer represents the one layer out of all the ones on the server that you are currently looking at. If this we an instance of a class, we'd have a layerId property and a layer getter like this.layers.filter(layer => layer.id === this.layerId).

dbouwman · 2021-06-15T17:21:57Z

packages/common/src/types.ts

+  // layer types: Feature Layers, Raster Layers
+  /** layer information (geometryType, fields, etc) */
+  layer?: Partial<ILayerDefinition>;
+  recordCount?: number;


how does recordCount work w/ layers?

recordCount only refers the the "current" layer (see #545 (comment)). and is not returned for service types (Feature Service, Map Service), only for the Feature Layer pseudo type.

i.e it must be re-fetched if you switch layers

mjuniper · 2021-06-15T17:23:53Z

packages/content/src/hub.ts

@@ -269,8 +297,13 @@ export function datasetToItem(dataset: DatasetResource): IItem {
    id: itemId,
    owner: owner as string,
    orgId,
-    created: (created || createdAt) as number,
-    modified: (modified || updatedAt) as number,
+    created: created as number,


I sorta thought createdAt came from the hub api but i don't see it. Are we sure we will never have createdAt?

Hmm thinking more about this, i guess created at was from our composer service.

it comes from the model ~~serializer~~ adapter (or composer). I've tested this locally sending models that come out of composer in the app's datasetModelToContent() and it does the right thing.

I have a massive spreadsheet that maps all the attributes returned from the API for various content types and maps them to the dataset model, which is a superset w/ CPs, and such.

dbouwman

Not sure I quite follow all of that, but seems legit :)

rweber-esri

Looks good to me, only minor comments.

rweber-esri · 2021-06-15T17:32:24Z

packages/common/src/content.ts

+
+/**
+ * return the layerId if we can tell that item is a single layer service
+ * @param {*} item from AGO


Minor, comment formatting. Outside scope of this PR, but have we ever explored adding eslint-plugin-jsdoc or similar for enforcing consistent commenting/docing styles?

so you are saying that the doc comments shouldn't have types, right? b/c that is taken care of by typedoc?

I hadn't considered that. I guess it'd just be nice if our inline comments were aligned w/ the docs generated by typedoc.

I'm not sure what's out of alignment, either specifically w/ this comment (other than I think it should not have the types), or in general, but if there are improvements w/ can make w/ eslint-plugin-jsdoc I say open an issue.

packages/content/src/hub.ts

tomwayson added 6 commits June 12, 2021 13:28

feat(content): add getFamily() and populate content.family in itemToC…

a2558fa

…ontent() affects: @esri/hub-common, @esri/hub-content, @esri/hub-search ISSUES CLOSED: #355

fix(content): should not set hubId when in portal

a46924f

affects: @esri/hub-common, @esri/hub-content

fix(content): datasetToContent should not return isProxied as that is…

c088bfd

… not returned by the API affects: @esri/hub-content

feat(common): add isFeatureService, getLayerIdFromUrl, getItemLayerId…

cd81e8f

…, & getItemHubId affects: @esri/hub-common

fix(content): fix how content.modified and content.item.modified are set

8333f3c

affects: @esri/hub-content always set content.item.modified to the item's modified date content.modified should prefer the enriched date and fall back to item.modified

feat(content): datasetToContent returns server, layer, and org enrich…

a689e1d

…ments affects: @esri/hub-content also enrichContent is now tolerant of content not having errors

tomwayson requested review from dbouwman and mjuniper June 15, 2021 16:58

tomwayson changed the title ~~F/content family and identifier~~ Content family and org Jun 15, 2021

dbouwman reviewed Jun 15, 2021

View reviewed changes

tomwayson requested a review from rweber-esri June 15, 2021 17:17

dbouwman reviewed Jun 15, 2021

View reviewed changes

mjuniper approved these changes Jun 15, 2021

View reviewed changes

dbouwman approved these changes Jun 15, 2021

View reviewed changes

rweber-esri approved these changes Jun 15, 2021

View reviewed changes

address PR feedback

02d08c6

tomwayson merged commit c41b183 into master Jun 15, 2021

tomwayson deleted the f/content-family-and-identifier branch June 15, 2021 18:14

tomwayson mentioned this pull request Jun 18, 2021

Add IHubContent.org and populate it in getContent()? #536

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Content family and org #545

Content family and org #545

tomwayson commented Jun 15, 2021 •

edited

Loading

dbouwman Jun 15, 2021

tomwayson Jun 15, 2021

codecov bot commented Jun 15, 2021 •

edited

Loading

dbouwman Jun 15, 2021

tomwayson Jun 15, 2021

dbouwman Jun 15, 2021

tomwayson Jun 15, 2021

tomwayson Jun 15, 2021

mjuniper Jun 15, 2021

mjuniper Jun 15, 2021

tomwayson Jun 15, 2021 •

edited

Loading

tomwayson Jun 15, 2021

dbouwman left a comment

rweber-esri left a comment

rweber-esri Jun 15, 2021

tomwayson Jun 15, 2021

rweber-esri Jun 15, 2021

tomwayson Jun 15, 2021

Content family and org #545

Content family and org #545

Conversation

tomwayson commented Jun 15, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jun 15, 2021 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tomwayson Jun 15, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dbouwman left a comment

Choose a reason for hiding this comment

rweber-esri left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tomwayson commented Jun 15, 2021 •

edited

Loading

codecov bot commented Jun 15, 2021 •

edited

Loading

tomwayson Jun 15, 2021 •

edited

Loading