Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assets: implement migration for old uploaded files #12326

Open
etj opened this issue Jun 10, 2024 · 2 comments · May be fixed by #12411
Open

Assets: implement migration for old uploaded files #12326

etj opened this issue Jun 10, 2024 · 2 comments · May be fixed by #12411
Assignees
Milestone

Comments

@etj
Copy link
Contributor

etj commented Jun 10, 2024

Continuation of #12124

Some old installations have the uploaded data into /data.
The recent importer stores the uploaded data into .../STATIC_ROOT/uploaded, and GeoServer publishes the geotiff from that directory
The final migration to Assets will store the files in .../STATIC_ROOT/assets, and GeoServer shall publish the files from there.
In order to clean up such obsolete setups, a migration script could be done that:

  • moves the files from uploaded/ to assets/ (first checks if all the files are into uploaded/)
  • updates the LocalAsset.location field
  • updates the GeoServer config to point to the new path

Such a migration script should only be run by hand since we do not know if such files are also used for other purposes outside GeoNode, so the admin can choose to run it or not.

@giohappy
Copy link
Contributor

@etj I guess that keeping the old files inside the previous location won't break GeoNode and Geoserer, right?

@etj
Copy link
Contributor Author

etj commented Jun 25, 2024

@giohappy
Keeping the old files where they are does not break anything, BUT:

  • Assets logic manages files completely ONLY when they are within the assets/ dir: for instance, when an Asset object is removed, the related files are removed only if they are in the assets/ dir. If the files are outside such directory, it is assumed that they were created with other means wrt the uploading procedures, so they are outside the "normal" geonode flow.
  • We may want to protect assets file, so that their directory is not exposed directory through NGINX;
  • We may want to serve assets files via X-Accel-Redirect, and having them into a brand new assets/ directory can make the work much cleaner.

@giohappy giohappy changed the title Assets: cleanup uploaded files Assets: implement migration for old uploaded files Jun 26, 2024
@giohappy giohappy added this to the 4.4.0 milestone Jun 26, 2024
@mattiagiupponi mattiagiupponi linked a pull request Jun 26, 2024 that will close this issue
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment