Backend takes time to start

[wesleybl]

When I run the command:

make backend-docker-start

The backend takes time to start. It first installs plone.restapi and then starts Plone. Every time I stop the container and start it again this happens. This is correct? Is there a way to start the backend faster? There used to be an api folder that had a Plone buildout, but it was removed.

[stevepiercy]

@wesleybl read on from this comment #6110 (review) for alternatives. You can either use buildout or generate the latest Plone backend snapshot from the cookiecutter.

[wesleybl]

@stevepiercy @davisagli Volto is very dependent on Plone. I think it would be important to have a script to "install Plone quickly" in the Volto repository. Saying "use cookiecutter or use buildout" may be common for experienced developers but can be a hindrance for beginner developers or those who don't know Plone but want to help with Volto.

Is there any documentation on "How to install Plone for the Volto core"?

Another point is that the Volto CI does not always use the same versions as a Vanilla Plone. For example, a more actual version of pone.restapi is generally used. If these pinns were integrated, it would be easier. It could be that something doesn't work in your installation and works in the CI, or vice versa, precisely because of pinns differences.

[stevepiercy]

@wesleybl Develop Volto core.

[wesleybl]

@wesleybl Develop Volto core.

@stevepiercy In this documentation, the command to start the backend is exactly the command I'm talking about in this issue:

make backend-docker-start

Another problem with this command is that the data is not persisted. If we start the backend, stop it and then start it again, the data created is no longer there.

[stevepiercy]

@wesleybl I don't have an answer for you, but I put this on the agenda for next week's @plone/volto-team meeting to discuss and get back to you. Alternatively, you could post to the Community Forum to ask for an answer.

[stevepiercy]

Possibly related issue: https://community.plone.org/t/troubleshooting-volto-setup-frontend-configuration-issue/19656

[stevepiercy]

@plone/volto-team it appears our documentation of Develop Volto core is broken. Can someone please help out? I have not verified the issue, and I wouldn't know what to do if I did.

[ichim-david]

@wesleybl Develop Volto core.

@stevepiercy In this documentation, the command to start the backend is exactly the command I'm talking about in this issue:

make backend-docker-start

Another problem with this command is that the data is not persisted. If we start the backend, stop it and then start it again, the data created is no longer there.

@wesleybl this is true regarding the persistence of data and it's a pity. Perhaps we should modify the make command as to mount a data folder inside the volto monorepo and use it thus persisting the data.

@plone/volto-team if we tell users that this is the way to develop using docker we should make sure we persist their demoing, making it easier to come back and work on other issues.

EDIT:
Added #6168 to add the data folder to packages/volto. Alongside some mentions in the docs I think it would be helpful to have this data given that we mention the use of this backend-docker-start image as a way to start the backend for developing
https://6.docs.plone.org/volto/contributing/developing-core.html#start-the-backend-and-volto

[ichim-david]

When I run the command:

make backend-docker-start

The backend takes time to start. It first installs plone.restapi and then starts Plone. Every time I stop the container and start it again this happens. This is correct? Is there a way to start the backend faster? There used to be an api folder that had a Plone buildout, but it was removed.

@wesleybl you could test the backend changes that modelled the cookiecutter output from here
c61a549

I would be interested in hearing how that works for you, if api was still better or if this is more flexible for you.
If that is the case then maybe we can add back the backend code that was made ready either way but was deemed as no longer needed to have the non docker way of installing the backend and as such it got deleted.

[wesleybl]

@wesleybl you could test the backend changes that modelled the cookiecutter output from here
c61a549

@ichim-david I didn't understand this commit. Is the link correct? Do you want me to test this PR?

[ichim-david]

@wesleybl you could test the backend changes that modelled the cookiecutter output from here
c61a549

@ichim-david I didn't understand this commit. Is the link correct? Do you want me to test this PR?

@wesleybl you can test it #6168 I already know it will solve your backend-docker-start data persistency issue.

Regarding that commit that I meant to link to aade85d which was the previous commit before the one that removed the backend folder. See the commit after my comment I left here #6110 (comment)

Here is the link to how backend folder was modelled before it was deemed as something that can be removed altogether instead of replacing the api folder
https://github.com/plone/volto/tree/aade85d02b8b63d5a514a35434aff5bbfaa609d0/backend
This is the thing that I thought might have been useful for you to test and give feedback if it was a better system rather than the api folder backend.

[wesleybl]

@ichim-david I tested the backend folder. It worked for me. I prefer her format. It will only take a while for the first installation. Starting will be faster from then on. I will be able to change a file for some testing, something that would not be possible with Docker. I can debug.

But I respect the decision to use docker. For most situations it will resolve. We have the persistence problem that you are verifying. We also have the problem of taking a long time to start. I think it was due to the installation of the "custom version" of plone.restapi. I don't see a solution for this using docker. Maybe we have our dockerfile? But I think the use of a customized version has not been used for a long time. Or is it? This delay may not be that annoying. We usually start the backend and leave it running all the time.

[ichim-david]

@ichim-david I tested the backend folder. It worked for me. I prefer her format. It will only take a while for the first installation. Starting will be faster from then on. I will be able to change a file for some testing, something that would not be possible with Docker. I can debug.

But I respect the decision to use docker. For most situations it will resolve. We have the persistence problem that you are verifying. We also have the problem of taking a long time to start. I think it was due to the installation of the "custom version" of plone.restapi. I don't see a solution for this using docker. Maybe we have our dockerfile? But I think the use of a customized version has not been used for a long time. Or is it? This delay may not be that annoying. We usually start the backend and leave it running all the time.

@wesleybl it's fast if you have a mac with a fast ssd :), it takes around 10s before starting the plone instance :). This happens due to the use of plone.restapi 9.7.0 for the teaser dynamic data fetching. Once a Plone version is released with this version that step should no longer be necessary to get rid of this bottleneck.
Maybe something can be done for the docker image, not my area of expertise so we shall see, maybe ask @ericof for some input on this issue.

Regarding the backend working I'm glad it works for you, I guess you can use this solution since it's more flexible in modifying the plone instance todo actual changes as opposed to the docker image.
Maybe once cookicutter docs and story is finished it can bring the alternative to the docker image for the Plone instance for the people that need the extra hands-down flexibility.
You could also try to generate the Plone backend using it and add any feedback https://github.com/plone/cookieplone-templates.

[wesleybl]

You could also try to generate the Plone backend using it and add any feedback https://github.com/plone/cookieplone-templates.

I have reservations about this. To see: #6157 (comment)

But it's ok :)

[sneridagh]

Let me add my 5c to this.

The Plone versions used in Volto core should be "latest", since both backend and frontend have to be "in sync" for features. eg. if main has a feature not yet in an official Plone core release (as it's the case during these days) it needs to pull in the packages that provide these features (in this case latest p.restapi). This causes the build to pip install again, and this takes some extra time. This will be "solved" when 6.0.12 will be out.

Volto core HAS to work like this, this is unavoidable. Using docker is just faster and much more convenient, and less error prone. From my point of view, you don't need anything else than the latest Plone to develop Volto features.

In the case you are developing something in the backend as well that pairs with the PR that you are working on, no matter what package it is, you should use buildout.coredev. There's no reason for not doing it while developing the Plone backend. It's not only the recommended way to develop backend Plone, but the way to go. In the past, you could go away with the api folder, yes, but it was wrong and that build was terribly outdated, and in the upcoming months was going to get obsolete.

[stevepiercy]

As far as updating Develop Volto core, is there anything we should do at this time?

1. Do we provide sufficient context for using this method to install Volto?
2. Are the instructions correct for its specific context?
3. Do we refer to other methods for other contexts?

IMO, the answer is "no" to all three questions, and we need those questions answered.

There is an open PR that may help resolve Question 3 plone/documentation#1671.

[stevepiercy]

@wesleybl I created a PR that will close this issue. See #6184. I would appreciate your review. Thank you!

[wesleybl]

@sneridagh

plone/plone-backend#147

It will make the backend start faster, correct?