Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Support to only build certain dependencies #442

Merged
merged 9 commits into from
Jul 23, 2021
53 changes: 52 additions & 1 deletion lib/cli/commands/build.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Build

const baseMiddleware = require("../middlewares/base.js");
const buildHelper = require("../../utils/buildHelper");

const build = {
command: "build",
Expand Down Expand Up @@ -37,6 +38,38 @@ build.builder = function(cli) {
default: false,
type: "boolean"
})
.option("include-dependency", {
describe: "A list of dependencies to be included into the build process. You can use the asterisk '*' as" +
" an alias for including all dependencies into the build process. The listed dependencies cannot be" +
" overruled by dependencies defined in 'exclude-dependency'.",
type: "array"
})
.option("include-dependency-regexp", {
describe: "A list of regular expressions defining dependencies to be included into the build process." +
" This list is prioritized like 'include-dependency'.",
type: "array"
})
.option("include-dependency-tree", {
describe: "A list of dependencies to be included into the build process. Transitive dependencies are" +
" implicitly included and do not need to be part of this list. These dependencies overrule" +
" the selection of 'exclude-dependency-tree' but can be overruled by 'exclude-dependency'.",
type: "array"
})
.option("exclude-dependency", {
describe: "A list of dependencies to be excluded from the build process. The listed dependencies can" +
" be overruled by dependencies defined in 'include-dependency'.",
type: "array"
})
.option("exclude-dependency-regexp", {
describe: "A list of regular expressions defining dependencies to be excluded from the build process." +
" This list is prioritized like 'exclude-dependency'.",
type: "array"
})
.option("exclude-dependency-tree", {
describe: "A list of dependencies to be excluded from the build process. Transitive dependencies are" +
" implicitly included and do not need to be part of this list.",
type: "array"
})
.option("dest", {
describe: "Path of build destination",
default: "./dist",
Expand Down Expand Up @@ -98,11 +131,29 @@ async function handleBuild(argv) {
}

const tree = await normalizer.generateProjectTree(normalizerOptions);
const buildSettings = (tree.builder && tree.builder.settings) || {};

const {includedDependencies, excludedDependencies} = buildHelper.createDependencyLists({
tree: tree,
larskissel marked this conversation as resolved.
Show resolved Hide resolved
includeDependency: argv["include-dependency"],
includeDependencyRegExp: argv["include-dependency-regexp"],
includeDependencyTree: argv["include-dependency-tree"],
excludeDependency: argv["exclude-dependency"],
excludeDependencyRegExp: argv["exclude-dependency-regexp"],
excludeDependencyTree: argv["exclude-dependency-tree"],
defaultIncludeDependency: buildSettings.includeDependency,
defaultIncludeDependencyRegExp: buildSettings.includeDependencyRegExp,
defaultIncludeDependencyTree: buildSettings.includeDependencyTree
});
const buildAll = buildHelper.alignWithBuilderApi(argv.all, includedDependencies, excludedDependencies);

Copy link
Member

@codeworrior codeworrior Jul 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's hard to understand from the code what the overall logic (idea) regarding priorities is. Please add a comment describing exactly this (what filter has priority over what other filter)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have changed the implementation. The new code has a lot more comments for describing priority.

await builder.build({
tree: tree,
destPath: argv.dest,
cleanDest: argv["clean-dest"],
buildDependencies: argv.all,
buildDependencies: buildAll,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just out of curiosity: why does the builder still require include/exclude? We're already traversing the tree, looking at the new filtering options. Why can't we resolve the filters in the CLI layer and just give a list of projects (includes maybe) to the builder?

Copy link
Member

@RandomByte RandomByte Jul 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does the builder still require include/exclude?

For consumers like Kristian, who use the Node.js API of the builder directly.

I would try to keep the CLI layer as simple as possible and just pass the includes/excludes to the builder mostly untouched.

includedDependencies: includedDependencies,
excludedDependencies: excludedDependencies,
dev: command === "dev",
selfContained: command === "self-contained",
jsdoc: command === "jsdoc",
Expand Down
216 changes: 216 additions & 0 deletions lib/utils/buildHelper.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,216 @@
const log = require("@ui5/logger").getLogger("cli:utils:buildHelper");

/**
* Creates an object containing the flattened project dependency tree. Each dependency is defined as an object key while
* its value is an array of all of its transitive dependencies.
*
* @param {object} tree Project tree as generated by the [@ui5/project.normalizer]{@link module:@ui5/project.normalizer}
* @returns {object<string, string[]>} An object with dependency names as key and each with an array of its transitive
* dependencies as value
*/
function getFlattenedDependencyTree(tree) {
const dependencyInfo = {};

larskissel marked this conversation as resolved.
Show resolved Hide resolved
function _getTransitiveDependencies(project, dependencies) {
project.dependencies.forEach((dep) => {
if (!dependencies.includes(dep.metadata.name)) {
dependencies.push(dep.metadata.name);
_getTransitiveDependencies(dep, dependencies);
}
});
return dependencies;
}
function _processDependencies(project) {
project.dependencies.forEach((dep) => {
if (!dependencyInfo[dep.metadata.name]) {
dependencyInfo[dep.metadata.name] = _getTransitiveDependencies(dep, []);
_processDependencies(dep);
}
});
}

_processDependencies(tree);
return dependencyInfo;
}

/**
* Creates dependency lists for 'includedDependencies' and 'excludedDependencies'. Regular expressions are directly
* applied to a list of all project dependencies so that they don't need to be evaluated in later processing steps.
* Generally, includes are handled with a higher priority than excludes. Additionally, operations for processing
* transitive dependencies are handled with a lower priority than explicitly mentioned dependencies. The default
* dependencies set in the build settings are appended in the end.
*
* The priority of the various dependency lists is applied in the following order, but note that a later list can't
* overrule earlier ones:
* <ol>
* <li>includeDependency, includeDependencyRegExp</li>
* <li>excludeDependency, excludeDependencyRegExp</li>
* <li>includeDependencyTree</li>
* <li>excludeDependencyTree</li>
* <li>defaultIncludeDependency, defaultIncludeDependencyRegExp, defaultIncludeDependencyTree</li>
* </ol>
*
* @param {object} parameters Parameters
* @param {object} parameters.tree Project tree as generated by the
* [@ui5/project.normalizer]{@link module:@ui5/project.normalizer}
* @param {string[]} parameters.includeDependency The dependencies to be considered in 'includedDependencies'; the
* "*" character can be used as wildcard for all dependencies and is an alias for the CLI option "--all"
* @param {string[]} parameters.includeDependencyRegExp Strings which are interpreted as regular expressions
* to describe the selection of dependencies to be considered in 'includedDependencies'
* @param {string[]} parameters.includeDependencyTree The dependencies to be considered in 'includedDependencies';
* transitive dependencies are also appended
* @param {string[]} parameters.excludeDependency The dependencies to be considered in 'excludedDependencies'
* @param {string[]} parameters.excludeDependencyRegExp Strings which are interpreted as regular expressions
* to describe the selection of dependencies to be considered in 'excludedDependencies'
* @param {string[]} parameters.excludeDependencyTree The dependencies to be considered in 'excludedDependencies';
* transitive dependencies are also appended
* @param {string[]} parameters.defaultIncludeDependency Same as 'includeDependency' parameter; used for build
* settings
* @param {string[]} parameters.defaultIncludeDependencyRegExp Same as 'includeDependencyRegExp' parameter; used
* for build settings
* @param {string[]} parameters.defaultIncludeDependencyTree Same as 'includeDependencyTree' parameter; used for
* build settings
* @returns {{includedDependencies:string[],excludedDependencies:string[]}} An object containing the
* 'includedDependencies' and 'excludedDependencies'
*/
function createDependencyLists({
tree,
includeDependency = [], includeDependencyRegExp = [], includeDependencyTree = [],
excludeDependency = [], excludeDependencyRegExp = [], excludeDependencyTree = [],
defaultIncludeDependency = [], defaultIncludeDependencyRegExp = [], defaultIncludeDependencyTree = []
}) {
if (
!includeDependency.length && !includeDependencyRegExp.length && !includeDependencyTree.length &&
!excludeDependency.length && !excludeDependencyRegExp.length && !excludeDependencyTree.length &&
!defaultIncludeDependency.length && !defaultIncludeDependencyRegExp.length &&
!defaultIncludeDependencyTree.length
) {
return {includedDependencies: [], excludedDependencies: []};
}

const flattenedDependencyTree = getFlattenedDependencyTree(tree);
RandomByte marked this conversation as resolved.
Show resolved Hide resolved

function isExcluded(excludeList, depName) {
return excludeList && excludeList.has(depName);
}
function processDependencies({targetList, dependencies, dependenciesRegExp = [], excludeList, handleSubtree}) {
if (handleSubtree && dependenciesRegExp.length) {
throw new Error("dependenciesRegExp can't be combined with handleSubtree:true option");
}
dependencies.forEach((depName) => {
if (depName === "*") {
targetList.add(depName);
} else if (flattenedDependencyTree[depName]) {
if (!isExcluded(excludeList, depName)) {
targetList.add(depName);
}
if (handleSubtree) {
flattenedDependencyTree[depName].forEach((dep) => {
if (!isExcluded(excludeList, dep)) {
targetList.add(dep);
}
});
}
} else {
log.warn(
`Could not find dependency "${depName}" for project ${tree.metadata.name}. Dependency filter is ` +
`ignored`);
}
});
dependenciesRegExp.map((exp) => new RegExp(exp)).forEach((regExp) => {
for (const depName in flattenedDependencyTree) {
if (regExp.test(depName) && !isExcluded(excludeList, depName)) {
targetList.add(depName);
}
}
});
}

const includedDependencies = new Set();
const excludedDependencies = new Set();

// add dependencies defined in includeDependency and includeDependencyRegExp to the list of includedDependencies
processDependencies({
targetList: includedDependencies,
dependencies: includeDependency,
dependenciesRegExp: includeDependencyRegExp
});
// add dependencies defined in excludeDependency and excludeDependencyRegExp to the list of excludedDependencies
processDependencies({
targetList: excludedDependencies,
dependencies: excludeDependency,
dependenciesRegExp: excludeDependencyRegExp
});
// add dependencies defined in includeDependencyTree with their transitive dependencies to the list of
// includedDependencies; due to prioritization only those dependencies are added which are not excluded
// by excludedDependencies
processDependencies({
targetList: includedDependencies,
dependencies: includeDependencyTree,
excludeList: excludedDependencies,
handleSubtree: true
});
// add dependencies defined in excludeDependencyTree with their transitive dependencies to the list of
// excludedDependencies; due to prioritization only those dependencies are added which are not excluded
// by includedDependencies
processDependencies({
targetList: excludedDependencies,
dependencies: excludeDependencyTree,
excludeList: includedDependencies,
handleSubtree: true
});
// due to the lowest priority only add the dependencies defined in build settings if they are not excluded
// by any other dependency defined in excludedDependencies
processDependencies({
targetList: includedDependencies,
dependencies: defaultIncludeDependency,
dependenciesRegExp: defaultIncludeDependencyRegExp,
excludeList: excludedDependencies
});
processDependencies({
targetList: includedDependencies,
dependencies: defaultIncludeDependencyTree,
excludeList: excludedDependencies,
handleSubtree: true
});

return {
includedDependencies: Array.from(includedDependencies),
excludedDependencies: Array.from(excludedDependencies)
};
}

/**
* Returns whether project dependencies have to be built influenced by <code>includedDependencies</code> and
* <code>excludedDependencies</code>.
* If only selected dependencies (via <code>includedDependencies</code>) have to be built, the "*" character
* is added to the <code>excludedDependencies</code> to make sure that all other dependencies are
* excluded.
* In case a "*" character is included in <code>includedDependencies</code>, it is removed and the
* <code>buildAll</code> flag is set to <code>true</code> as it behaves as an alias.
*
* @param {boolean} buildAll The value of the <code>all</code> command line parameter to decide if project
* dependencies have to be built
* @param {string[]} includedDependencies The list of included dependencies
* @param {string[]} excludedDependencies The list of excluded dependencies
* @returns {boolean} Whether it is required to build project dependencies
*/
function alignWithBuilderApi(buildAll, includedDependencies, excludedDependencies) {
if ((!buildAll && !includedDependencies.includes("*")) && includedDependencies.length) {
excludedDependencies.push("*");
}
if (includedDependencies.includes("*")) {
buildAll = true;
includedDependencies.splice(includedDependencies.indexOf("*"), 1);
}
if (!buildAll && includedDependencies.length) {
buildAll = true;
}
return buildAll;
}

module.exports = {
getFlattenedDependencyTree,
createDependencyLists,
alignWithBuilderApi
};
Loading