Question - How to add folder per-function basis when packaging individually

[Vadorequest]

I'm packaging my functions individually, but I want to customise additional folders per-functions. (those are pre-built folders, from other apps)

I tried following the https://serverless.com/framework/docs/providers/aws/guide/packaging/ documentation, but to my understanding, using serverless-webpack bypasses totally those options, right? (include/exclude)

serverless-webpack is great to include modules, but with folders or single files I'm a bit stuck. I ended up adding a Copy plugin to copy the folders I want, but they get copied for every function and I don't see a workaround.

const slsw = require("serverless-webpack");
const nodeExternals = require("webpack-node-externals");
const CopyWebpackPlugin = require('copy-webpack-plugin');

const plugins = [
  new CopyWebpackPlugin([
    {
      from: '.next/**',
    },
    {
      from: 'static/**',
    },
  ])
];

module.exports = {
  entry: slsw.lib.entries,
  target: "node",

  // Necessary for __dirname and __filename to work correctly when bundling with Webpack for the dev environment.
  // XXX See https://github.com/webpack/webpack/issues/1599
  node: {
    __dirname: true,
    __filename: true,
  },
  plugins,
  // Generate sourcemaps for proper error messages
  devtool: 'source-map',
  // We use webpack-node-externals to excludes all node deps. (like aws-sdk)
  // You can manually set the externals too.
  externals: [nodeExternals()],
  // Run babel on all .js files and skip those in node_modules
  module: {
    rules: [
      {
        test: /\.js$/,
        loader: "babel-loader",
        include: __dirname,
        exclude: /node_modules/
      }
    ]
  },
};

[HyperBrain]

Hi @Vadorequest, thanks for the question 👍

using serverless-webpack bypasses totally those options, right?

Yes, that's right, because serverless webpack currently only has an include/exclude for the dependent modules (as they are auto-detected by webpack and might need a small manual "correction" in rare cases).

The workaround right now to include arbitrary files into the package(s) is to use the copy webpack plugin, like you did - with the side effect of multiplying the files.

A final solution to the problem could be that the plugin would respect any package: include properties for functions and apply the copy webpack plugin configuration per lambda by itself. What must not happen is, that the webpack plugin itself copies the files.

But with such a configuration approach it would leave that part to the tools designed for that. The per function configured copy plugin would even run in the correct context automatically.

[Vadorequest]

Thanks for the quick answer @HyperBrain. I'm having a hard time figuring out what workaround you recommend though =D

Indeed, serverless-webpack could do the work for us and allow to use the natural include/exclude SLS way, would be just easier/more natural.

Meanwhile though, is there any way to dynamically know what function we're building to apply some plugins conditionally? I don't know if that's a good way to resolve the issue, but it may be possible this way.

[HyperBrain]

The workaround I recommend for now is to include the files with the copy plugin and accept the drawback of having the files everywhere 😄

There is currently no way to have the built function dynamically available in the webpack configuration, so this way seems blocked. But since version 4.3.0 the plugin exposes hooks that can be hooked by local plugins, so as a workaround that only copies where needed, you could write a small local plugin that hooks after:webpack:package:packExternalModules and just copies the files to the function's output folder. The standard path to the compiled function is .webpack/<function>.

For the hooks doc, see the "for developers" section in the README. Hooking after packExternalModules has the advantage that right after your hook exits, the whole function folder is packaged into the ZIP.
I don't know if your project setup qualifies for the efforts, but it's a feasible alternative.

[Vadorequest]

Thanks again. My project doesn't qualifies for "such" effort since it's only a POC but it was interesting to have an alternative solution since other people who run in the same issue may actually need it.

https://github.com/Vadorequest/serverless-with-next/

[ejweiler]

@HyperBrain I'm trying to accomplish this using the hooks as you suggest, I'm struggling to figure how how to hook into those events from the readme. In a plugin, is it similar to how you'd hook into webpack event hooks?

//works
compiler.plugin('done', () => {
    console.log('done event was called');
})

//never fires
compiler.plugin('after:webpack:package:packExternalModules',  () => {
    console.log('packExternalModules event was called');
})

[HyperBrain]

@ejweiler You can hook any Serverless plugin you write into the serverless-webpack plugin, i.e. you create a standard Serverless plugin and define your hooks as follows (the hooks are not accessible from webpack plugins, but Serverless):

// In the plugin constructor
this.hooks = {
  ...
  'after:webpack:package:packExternalModules': () => BbPromise.bind(this)
    .then(() => {
      // Do anything you want here
      return BbPromise.resolve();
    }),
  ...
};

Then it is important that in Serverless' plugin declaration (in serverless.yml) your plugin comes after the serverless-webpack plugin to guarantee the calling order of any hooks. Also, you should use promises whenever possible, as the Serverless framework will await any hooks with their .then() method.

I think there are also some tutorials out there which explain in detail how to write Serverless plugins. FYI: Serverless plugins can either be global or local to the project.

[ejweiler]

Thank you! I was mistakenly trying to make a webpack plugin

Before using serverless-webpack I had used the serverless configurations for includes/excludes. I was able to handle most of my issues but had a similar scenario to this thread. One lambda uses phantomjs to resize images so we rely on that executable plus a javascript script that isn't required in the lambda (we call these both from a child process).

Here's the code for my plugin:

'use strict';

const fs = require('fs-extra')

class PhantomBundlerPlugin {
    constructor(serverless, options) {
        this.serverless = serverless
        this.options = options

        this.hooks = {
            'after:webpack:package:packExternalModules': this.copyPhantomFiles
        }
    }

    copyPhantomFiles() {
        return fs.copy('./src/screenshotAddPhantom.js', './.webpack/screenshotAdd/src/screenshotAddPhantom.js')
            .then(() => {
                return fs.copy('./src/phantomjs_linux-x86_64.sh', './.webpack/screenshotAdd/src/phantomjs_linux-x86_64')  
            })
            .catch(err => {
                console.log(err)
            })
    }
}

module.exports = PhantomBundlerPlugin

Serverless plugin guide

[HyperBrain]

I think it should work that way. However I'm not sure if the copy of screenshotAddPhantom.js is needed. Webpack may bundle it already and you might only need the library. But it won't hurt to copy more than needed anyway 😀

[hmtri1011]

@HyperBrain Hi, I'm also facing this problem, do we have any good news for this :)