Function Scaffolding #1589
Replies: 12 comments 14 replies
-
ExampleBelow is an example walkthrough of the root of the proposal, using a highly simplified Go language runtime for illustration. The
Creating a new Go function writes the template files as normal:
Notice how the function written has no dependencies on a function language
We can now run this Funciton locally (outside of a container) just as though
To see how this happened we can take a look in the ephemeral directory
The main file starts the functions language middleware. Note that here is where the dependency is inverted: the functions middleware "the platform"
The go.mod file registers the symplink as containing the function's source code.
The This directory then can be the Service which is built into a container |
Beta Was this translation helpful? Give feedback.
-
I really like this proposal, and it's nice that it will work with interpreted languages such as Node.js and Python. As an aside, I do like having https://github.com/knative-sandbox/func-go primarily for visibility and potential continuity with other runtimes. Based on what you write here,
using a more sophisticated scaffolding than in your example, the We also should be considering how this will be surfaced in Language Packs. Do these now have two top level directories at the repository root containing scaffolding and functions separately? Essentially the same as the
Or do they instead have scaffolding defined per-template or per-runtime within the current structure?
The latter option would make /cc @knative/func-reviewers @knative/func-writers @knative/functions-wg-leads @knative/technical-oversight-committee |
Beta Was this translation helpful? Give feedback.
-
In the scaffolding example where is the yaml file(s) of what is deployed to Kubernetes namespace? How can I customize it? |
Beta Was this translation helpful? Give feedback.
-
I am sorry, I don't follow why we need to increment the directory? |
Beta Was this translation helpful? Give feedback.
-
I am curious, can't we somehow skip these scripts and run the functionality directly from |
Beta Was this translation helpful? Give feedback.
-
Quick question: how func repositories will work with scaffolding? Can I provide some part of scaffolding in tempalte repository that will override (merge) into existing scaffolding? |
Beta Was this translation helpful? Give feedback.
-
For scaffolding can we also have an option to scaffold Kubernetes artifacts that are deployed? I think right now they are only in func code and not possible to modify or overwrite without changing source code? |
Beta Was this translation helpful? Give feedback.
-
Can user put .func scaffolding into git repo? What would happen? What id scaffolding was modified or outdated or our-of-sync (from other version of func)? |
Beta Was this translation helpful? Give feedback.
-
Writing down points from WG meeting: there is contract that func expects (funcion signature etc) - we may have some way of checking it and giving developers warnings or validation. also some kind of version checking about what is required version of func to use scaffolding if it were modified? |
Beta Was this translation helpful? Give feedback.
-
This system is now available for the Go runtime behind the feature-flag |
Beta Was this translation helpful? Give feedback.
-
See the Epic-issue #1703 which enumerates the (larger) issues related to implementing this MVP of the Scaffolded-functions. |
Beta Was this translation helpful? Give feedback.
-
Reopening in leau of having actual documentation just yet |
Beta Was this translation helpful? Give feedback.
-
I would appreciate any comments
Function Scaffolding
Proposed is a method by which we can deploy Functions with no dependencies
on the underlying Functions infrastructure.
Introduced is the term "Function Scaffolding", which refers to both the code and
stage of deployment process where a Function is wrapped in the code necessary
to run it as a process.
Background
Functions enable a developer to focus almost entirely on the logic of their
program by providing a simple interface with the underlying infrastructure which
will run their code. This interface is the function signature in their language
of choice.
To enable this simplicity, it is necessary to augment the developer's function
with the plumbing which exposes their function as a process, listening on
a port, etc.
This code, which exists to adapt the developer's Function to the operating
system's process scheduler within a cluster context, is its scaffolding.
It is imperative that this scaffolding not be part of a Function developer's
source code by default, since it is logically part of the platform which runs
Functions, rather than the Function itself.
This is a requirement for providing a function platform as a service because it
decouples the Function developer's source code from the method by which this
code is run. A Function's source code repository which contains none of this
scaffolding can remain unchanged even as the platform is updated.
As an analogy, just as we need not rebuild a virtual machine when our cloud provider
updates their hypervisor, nor do we need to rebuild our containers when our
container engine is updated, so too a Function developer should not need to
mutate their Function's source code when the Functions Runtime is updated. From
the perspective of the Function developer, a restart (likely automated) should
be sufficient.
Three examples of situations in which the Function as a Service platform should
be able to update without requiring a Function Developer's code to change:
into a Function invocation call has a patch released.
improve platform visibility or health.
The proposal below outlines how to separate this platform code from
the Function's source repository, enabling the flexibility expected of a
platform, while importantly not losing the ability for a Function developer to
run their code directly, locally, outside of a container, or even opt in to
managing this scaffolding code themselves for more advanced cases.
Proposal
The following primary steps are executed in order to produce a Function,
running as a service in the currently connected Function Platform (cluster):
When a Function is deployed, scaffolding necessary to run the Function is first
written which imports the developer's Function as a dependency. This scaffolded
Function is now ready to be built into a container and deployed because it is
now a full Service.
This scaffolding code is transient, kept in the non-source-controlled
.func
directory until the run or deploy task completes.
The
func scaffold
SubcommandIn addition to this scaffolding process being a default part of the deployment
process, the act of scaffolding is also exposed using a new command:
func scaffold
.This command allows a user to write the scaffolding code into their project,
effectively opting to take on the responsibility of updates and maintenance
themselves, but enabling them to accomplish more advanced tasks, such as
altering the code used for health and readiness endpoints, etc.
This subcommand is also useful in CI/CD contexts such as Tekton Pipelines
which do not support Functions natively, as they can run
func scaffold
in anyFunction project retreived from source control, and immediately have a full
Service on which to operate.
Expanding the
func run
SubcommandThe implementation of
func run
would also use this same scaffoldng step,because scaffolding is part of building the function container.
However, rather than performing the containerization task by default, it is
proposed that the default behavior of
func run
be to invoke a developer'sfunction locally on the host, just as if the function scaffolding code existed
in their project. From the developer's perspective, their host OS is now
capable of running Functions directly.
Containerizing and running the Funciton in a local container runtime would then
be accomplished using the command
func run --container
.Implementation
In general, the suggested implementation of Function Scaffolding is to roughly
mirror the process of Function Templating. Each core supported language
includes its scaffolding code embedded in the library, with the ephemeral
.func
runtime directory used for output.Language scaffolding code would be kept in the
./scaffolding
subdirectoryof the
func
repository and encoded as source code on generate in the samemanner as templates. This code is expected to be quite small, since all logic
is kept in dedicated functions runtime libraries for each supported language.
When a Function is built, the contents of the scaffolding is written into
.func/build/0/
. This directory increments with any concurrent calls tobuild
in the same way
run
increments its runtime directory today, and the directoryis by default removed when the process completes. This scaffolding code
consists of everything necessary to run the Function as a Service except the
function code itself.
For example, in the case of Go, this would be
main.go
etc. This scaffoldingimports the user's Function code locally, as a library of the expected
method signature and residing at the relative path
./f
(
myFunctionProject/.func/runs/0/f
). This path would be a link./f -> ../../../
. Together, this creates a complete Service consisting ofthe process boundary (main), Functions tooling (imported), and the Function
source code itself.
Included in the scaffolding is the command necessary to invoke the service
as
run.sh
, and for languages which require compilation, abuild.sh
.run.sh
then becomes the entrypoint for the final container.If a developer chooses to
func scaffold
their project and take the sourcecode, by default it would be written to
./dist
, andscaffolded:true
writtento
func.yaml
. On subsequent builds, the scaffolding step would be skippedand the commands
./dist/build.sh
and./dist/run.sh
used instead.Running Locally with
func run
Similarly to deployment,
func run
scaffolds the project, but uses the localpath
.func/runs/0
. By default, the run script is executed as a subprocessto run the scaffolded function (service). Running can optionally be from
within a containerized environment using
func run --container
whichcontinues on to create the container and invoke it rather than the code in
.func/runs/0
.Additional Configuration
appended to the parameters used in the default build and run scripts OR
--keep
can be added when running or deploying to cause theretention of the ephemeral
.func
artifacts for educational or debuggingpurposes.
Summary
By adding the "scaffolding" step as a distinct phase of a Function's lifecycle,
we allow our Functions to be free of platform dependencies. This enables us
to create a clear distinction between a Function and a Function Platform, and
is a prerequisite to providing services expected of a platform such as
independent update cycles, as well as more advanced Function features which
will depend on there being a distinct phase at which a Function is scaffolded.
I would appreciate any comments or questions
Beta Was this translation helpful? Give feedback.
All reactions