Softbody Volume Constraint - Recommended techniques for generation

[PhoenixIllusion]

I looked into one of the soft-body volume constraint, trying to find any soft-body logic I could add to my code other than just fixed-world bound cloth. I saw that with the SoftBodyCreator, it uses cube that seem to use tetrahedron based volume constraints. Porting these over, it appears the logic uses almost a voxel-based soft-body approach as it adds the edges and tetrahedron.

For aiming to use these soft-bodies, I am not yet clear on an "easy to use" way to use these. Is there a recommended way to auto-rig volumetric soft-bodies, or are some of the other soft-body features besides volume tetrahedron that are better casually adding arbitrary shaped soft-body volumes to a simulation? Outside of the volume constraint, it appears to me that the Sphere demos essentially rig things like blimps, acting as soft-shells held in form with internal pressure, although have not yet ported spheres tests.

I think I could probably code a way to decompose Mesh shapes using kind of an fixed-3d-grid, essentially converting their internals to the voxel cells like the jelly-cubes, or add internal points and apply Delaunay tetrahedralization.

[jrouwe]

TBH I haven't really used the volume constraints much myself yet, so I haven't done any experimentation on what the best way to generate these would be. Creating an outer shell and using 'pressure' instead of volume constraints is the easiest (and cheapest), but will deform the shape since it will 'balloon' out. To me the Delaunay tetrahedralization seems the most promising. Note that there has been some discussion about this before here: jrouwe/JoltPhysics#789

B.t.w. the jelly cubes look very tasty, looking forward to the PR!

[jrouwe]

It sounds like a lot of moving parts are needed here to get to a demo, and I'm not sure if its worth it. I'd rather not create a dependency on a WASM build of TetWild for a demo. I think the cube jellies you had before are a perfect demo.

My focus at the moment is to create a good working system for creating constraints for cloth setups from a mesh (SoftBodySharedSettings::CreateConstraints), making something to automatically produce volume constraints is also on my TODO list but is a bit further out, so eventually we shouldn't need something like TetWild anymore.

[PhoenixIllusion]

If I was to include model loading, it would only be if the data was easily baked into a text file with comments on how someone would approach generating their own, but not the actual Tet-Wild. I did get a compile of wasm fTetWild, but it was 11MB and quite a hassle, mainly assumptions in their lib-geom dependency and lots of makefile patching to tailor towards cross compiling instead of always assuming host compiling.

Likely for the PR, I will stick to soft Sphere and Cube that are in the original CPP library samples, but maybe checkout the cloth auto-constraint of the soft-creator Util that was demoed on YouTube.

For simple automating, I'd probably have better luck porting the logic from the demo 13 python code referenced above, and used in the blender tetrahedron code plugin. It just has a dependency of a BVH builder and ray tracing for inside/outside testing, which Jolt already has if you load into the physics engine, or is viable with some small JS NodeJS extras.

[LeXXik]

is also on my TODO list but is a bit further out

This made me curious if there is some sort of a road-map for this year, or a list of upcoming features to get hyped about :)

[jrouwe]

My TODO list contains roughly 200 items at the moment, but I'm deliberately not sharing a road-map as this is supposed to be a fun project for me and I don't want to create expectations. A lot of my time goes to support / fixing peoples issues, so it's very hard to predict what will happen next anyway.

[PhoenixIllusion]

@LeXXik - Not "upcoming", but randomly I see some new things to try like https://github.com/jrouwe/JoltPhysics/blob/master/Samples/Tests/Water/BoatTest.cpp

So easy to get side-tracked, and I probably need to force myself to at least hit v1.0.0 release of my actual app instead of getting side-tracked like I did for 2x weeks getting a wasm-Embree build working to use in Hi-Z cpu-driven culling

[PhoenixIllusion]

It kind of feels like cheating using physX as a tool for Jolt things, but the WASM physX library has features to pre-process models into (vertex, index, faces, tetras) -
https://github.com/fabmax/physx-js-webidl

https://github.com/NVIDIA-Omniverse/PhysX/tree/main/physx/source/physxextensions/src/tet
https://nvidia-omniverse.github.io/PhysX/physx/5.2.1/_build/physx/latest/class_px_tet_maker.html

By itself, their algorithm runs quite quick, although for quality it likely prefers running another pre-process on it (remesh & simplify) to get more uniform tetras. If you do remesh/simplify, UV-mapping difficulty spikes quite a bit, and you probably will want to skin your high-res model to the low-res soft-vertex, like by driving the model vertex using nearest soft-faces. I may look into that, treat the nearest Soft tri/verts as bones, and dump the SoftVertex position into a float-3 data texture every frame and send it to the WebGL2 context to act as bones.

Some workflows for conversions, although parts of the description are mapping to voxel-tetra instead of regular tetra though.
https://nvidia-omniverse.github.io/PhysX/physx/5.3.1/docs/SoftBodies.html
https://github.com/NVIDIA-Omniverse/PhysX/blob/439b4767167e1753d335b613b059140d4d1b4757/physx/source/geomutils/src/cooking/GuCookingTetrahedronMesh.cpp#L1865

I will probably see if I can make 2 or 3 models work well, possibly fully textured.

The actual Answer to this question is likely:

• Currently there are no built-in tools to solve this
• On the longer road-map, possibly there will be a built-in feature that can generate this data
• This thread has demonstrated that existing tools (physX, tetWild, etc) can take full models like OBJ and Jolt can make them "soft"
• For now, likely the best solution is to pre-process your models using one of these tools to target tetra-settings and compliance settings

[PhoenixIllusion]

I do have a direct JS port of the blender python exporter code, put in a gist . It works to some degree. None of them handle non-shell mesh, so some of my tests (like a man with a hat) break for the head itself due to it being 2x intersecting meshing and failing the inside/outside test.

https://gist.github.com/PhoenixIllusion/216bb93f16f2d55587f164394537f046

[PhoenixIllusion]

To anyone interested, here is some code that uses the PhysX simplification/remesh/ and tetrahedron creation, loaded into Jolt to create "soft mesh" based on OBJ files.

I'm still working towards skinning a full mesh or GLTF to create a complex, multi-texture soft mesh

https://gist.github.com/PhoenixIllusion/561e36d75471e2217a5b2a8d8f22683e

See Live Example (requires your own OBJ's to load via Open OBJ):
https://jsfiddle.net/phoenixillusion/zmvoy1au/

Meshes: https://kenney.nl/assets/graveyard-kit

[PhoenixIllusion]

As what I think is my final update for now, the following loads a textured GLTF, converts it to geometry (not clean for every GLTF, and it may require 'baking' the geometry if it is a compound geometry with instances, scales, and offsets for the child parts). It supports a scene with multiple children, assuming they all have a local transform of identity.

https://gist.github.com/PhoenixIllusion/c6b0507b64c3ca4f3634e2bb0693ca61

The steps performed are:

Prep the Mesh:

1. Load the original mesh (gltf)
2. Scale the mesh if needed (too small for the demo or tetra calculations)
3. Compile all geometry of the model into a single geometry

Create the Tetrahedron

1. Load the tetrahedron to PhysX
2. Simplify and Remesh
3. Create Tetraheadron

To demonstrate the situation more easily, I have a 2d version of the logic below. Triangles are used in place of tetrahedron.
At this point, we now have our original detailed mesh (black below) and the tetrahedron soft body data (red below).
You can see the detailed graphic has vertex that is either on the border of a tetra, inside it, or outside it.

At this point, I followed the logic that is done in the PhysX cooking flow. We take every vertex we want to use, and attempt to map it to a tetrahedron. I accelerated this using a hierarchal tree (AABB tree).

For each vertex P in the model:

1. Find the nearest tetrahedron
2. If P is inside a tetrahedron, save that tetra
3. If P is not inside the tetrahedron, for every face in the tetrahedron, calculate the distance and nearest Point on the tetrahedron.

This results in one nearest tetrahedron for every vertex, then I calculate the influence that tetrahedron has on the point. This is done Barycentric coordinate

  function barycentricTetragon(a,b,c,d,p) {
    const V = tetraVolume(a,b,c,d); // total volume of original tetrahedron
    const dA = tetraVolume(p,b,c,d); // volume of tetrahedron if you used P instead of A
    const dB = tetraVolume(p,a,c,d); // volume of tetrahedron if you used P instead of B
    const dC = tetraVolume(p,a,b,d);
    const dD = tetraVolume(p,a,b,c);

    return [ dA/V, dB/V, dC/V, dD/V ]
  }

The below shows that P outside a tetra are mapped to the closest point on the tetra itself.
The next image shows that any P inside a tetrahedron can be mapped based on it's relation to the vertex to any of the Tetra vertex.
If a point P is directly on top of a vertex, that specific vertex (A) has 100% influence on P and all other vertex are 0.
If a point P is as far as possible from a vertex, that specific vertex (A) has 0 influence on it.

The magic of barycentric coordinates is that they allow you to map any point located inside the tetrahedron using fixed weights per vertex of the 4 tetrahedron coordinates. With a barycentric coordinate for every detailed Mesh vertex, I can essentially toss out the original geometry coordinates. During runtime, when multiplied by the live-soft-vertex data of the simulation vertex of that Tetra, I resolve a location inside the tetra that should approximate my GLTF vertex location.

My render loop in the Jolt simulation is now:

1. For the soft body, pull all used Vertex (this is not every simulation vertex, just the vertex of Tetra that contain or touch a vertex I want to render
2. For each vertex in a model or child model, loop through the pair of [nearest Tetra Vertex, barycentric Tetra Weights]
3. Multiply (vA * weightA + vB * weightB + vC * weightC + vD * weightD)
4. Set this as the new model vertex for Position

This does fail on simulations where the origin detailed mesh has long stretches of triangles, like a lamp-post or column or the bench, since the detail mesh possesses no subdivided vertex in the middle of the long stretch of model to render the "bend". If you manually apply a subdivide to the models yourself in a 3d model editor, it should resolve that issue.

[PhoenixIllusion]

The barycentric tetra logic can be done in WebGL2 in a shader using a pair of attributes, 4xf32 and 4xU32 or U16. You could setup a transform feedback to recalculate the vertex data, with the soft vertex stored in a f32x4 data texture (rgba)

The challenging logic would be recalculation of normals. Possibly you could run this on the feedback-transform as well, but I do not have a good idea the best way to go about this, to preserve flat and smooth shading during that phase. Possibly you store the normal with regard to the tetra, and transform that as well based on which face the normal vector leaves during cooking.

Edit: the existing code can be cleaned up by using Matrix3 determinant based barycentric coordinate calculation, to produce coordinates correctly outside the tetra bounds. That prevents model distortions caused by the "rewrite position vertex to closest tetras" logic, and allows easy storing of world-space normals in barycentric space along side the positions using just another set of barycentric coordinate weights.

Store the world-space normal as calculated by "position + normal". During runtime, the normal could be calculated, subtracted from position, then normalize again.

[jrouwe]

Thanks for sharing! It looks really nice!