Add pagination setup code-snippet for Data Stream integration

Flow Actions/Data Stream/README.md

@@ -0,0 +1,20 @@
+# Pagination Setup for Data Stream Integration
+
+This script is used to manage pagination for a Data Stream integration in ServiceNow.
+Use this script to set pagination variables or manipulate variables extracted by XPath or JSON Path.
+
+## Code Overview
+
+The script calculates the next offset based on the current page's limit and offset, and determines whether or not to retrieve the next page of data based on the total count of records available.
+
+### Key Variables:
+- **`limit`**: The number of records to retrieve per page.
+- **`offset`**: The starting point for the next page of records.
+- **`getNextPage`**: A boolean flag indicating whether to continue fetching more pages.
+
+### How it works:
+- The script compares the `nextOffset` (calculated as `currentOffset + limit`) to the total number of records (`totalCount`).
+- If the `nextOffset` is less than the `totalCount`, the `getNextPage` variable is set to `true` to fetch the next page.
+- If there are no more records to fetch, the `getNextPage` variable is set to `false`.
+
+This setup is particularly useful for handling large datasets in paginated API requests, ensuring efficient and scalable data retrieval.

Flow Actions/Data Stream/pagination-setup.js

@@ -0,0 +1,25 @@
+(function paginate(variables, pageResponse) {
+    // Parse the limit (maximum number of records per page) from the variables
+    var limit = parseInt(variables.limit);
+
+    // Extract the total number of records from the response headers (usually provided by the external API)
+    var totalCount = parseInt(pageResponse.response_headers['X-Total-Count']);
+
+    // Parse the current offset (starting point for the current page of records)
+    var currentOffset = parseInt(variables.offset);
+
+    // Calculate the offset for the next page by adding the limit to the current offset
+    var nextOffset = currentOffset + limit;
+
+    // Check if the next offset is still within the total number of records
+    if (nextOffset < totalCount) {
+        // If there are still more records to retrieve, set the flag to true to get the next page
+        variables.getNextPage = true;
+
+        // Update the offset for the next page, converting the number back to a string
+        variables.offset = nextOffset.toString();
+    } else {
+        // If there are no more records to retrieve, set the flag to false to stop further pagination
+        variables.getNextPage = false;
+    }
+})(variables, pageResponse);