Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Function starting aborted after about 30 seconds. #1760

Closed
rabejens opened this issue May 25, 2023 · 2 comments
Closed

Function starting aborted after about 30 seconds. #1760

rabejens opened this issue May 25, 2023 · 2 comments
Labels
kind/bug Categorizes issue or PR as related to a bug. kind/good-first-issue Denotes an issue ready for a new contributor. kind/needs-validation More info needed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale.

Comments

@rabejens
Copy link

rabejens commented May 25, 2023

I have a function template which takes longer than 30 seconds to become ready. Is it possible to increase the timeout in func.yaml to mitigate this?

Followup 1:

I configured my knative-serving to set the scale-to-zero-grace-period to 600s (10 minutes) but this did not change anything. It is still scaling to zero after 30 seconds.

Followup 2:

Even disabling scale-to-zero completely has it killed after about 30 seconds.

Followup 3:

As a workaround, I currently made liveness and readiness instantly return 200 OK even though the function is still initializing. While this works for the time being, this is not optimal.

@rabejens rabejens changed the title How to adjust readiness delay? Function starting aborted after about 30 seconds. May 25, 2023
@lance
Copy link
Member

lance commented May 26, 2023

I have tried to replicate this using a simple function. I was not able to replicate it with a 60 second delay, but a 2 minute delay resulted in the failure message, "Initial scale was never achieved". Is this what you are seeing too?

Node.js function with liveness and readiness probes with 2 minute delay
const handle = async (context, body) => {
  return { "message": "Hello world!" };
}

// Wait until the delay completes before returning 200 OK
const readiness = (request, reply) => {
  if (ready) {
    reply.statusCode = 200;
    return 'Yeah boyeee';
  } else {
    reply.statusCode = 503;
    return 'Not ready';
  }
}

const liveness = readiness;

// Utility function to simulate a delay
const sleep = async (ms) => {
  return new Promise(resolve => setTimeout(resolve, ms));
}

let ready = false;
// 120 second delay to simulate a long startup time
sleep(120000).then( _ => { ready = true; }).catch( e => { console.error(e); })

// Export the function
module.exports = { handle, readiness, liveness };

Running this function yields

kn service list
NAME   URL                                     LATEST      AGE   CONDITIONS   READY   REASON
foo    http://foo.default.127.0.0.1.sslip.io   foo-00005   43h   1 OK / 3     False   RevisionFailed : Revision "foo-00006" failed with message: Initial scale was never achieved.

@lkingland lkingland added kind/bug Categorizes issue or PR as related to a bug. kind/needs-validation More info needed kind/good-first-issue Denotes an issue ready for a new contributor. labels Aug 23, 2023
Copy link
Contributor

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

@github-actions github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug. kind/good-first-issue Denotes an issue ready for a new contributor. kind/needs-validation More info needed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale.
Projects
Status: ✅ Done
Development

No branches or pull requests

3 participants