Create a MaterialX graph for the glTF BSDF

[fire]

I am investigating making a match for the glTF BSDF.

Ideally the development process should be overseen by the designers of the glTF BSDF at Khronos.

See AcademySoftwareFoundation/MaterialX#653

[proog128]

See #1949 for some information on MaterialX and how a glTF BSDF fits in (scroll down a bit, the original discussion went off-topic and arrived at MaterialX). In particular, the last comment contains a few hints on how to get started and how to validate results.

[fire]

Reposting for ease of review.

https://github.com/KhronosGroup/glTF/blob/0343ff13ba7f3a4b717a235aa75b45655ab512c7/extensions/README.md#user-content-pbr-material-extensions

In addition to the math described in the Enterprise PBR specification, we provide test scenes and ground truth renderings to check implementations. There is also a WebGL-based path tracer which implements the BSDF.

1. https://dassaultsystemes-technology.github.io/EnterprisePBRShadingModel/spec-2022x.md.html
2. https://github.com/DassaultSystemes-Technology/EnterprisePBRShadingModel/tree/master/validation
3. https://github.com/DassaultSystemes-Technology/dspbr-pt

[fire]

@proog128 Do you have a full materialx sample I can try creating an automated material to png github cicd sample for?

[fire]

From an gltf2 extension standards point of view the missing steps seem to be:

1. include materialx .mtlx
2. include attached images in .glb or mime type urls.
3. Draft a proposal similar to webp https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Vendor/EXT_texture_webp

[emackey]

@fire I think a glTF extension would only be needed if we were attempting to include all of MaterialX as a glTF material extension. This is a lot to ask from glTF implementations and viewers, some of which are lightweight and don't support MaterialX.

The current thinking is to go the other way around: Implement the glTF PBR material (including recent PBR extensions) as a new gltf_pbr MaterialX node. This would allow content authors to target glTF's PBR system directly from within a MaterialX-capable authoring system, by targeting that material node. Such materials could then be exported to classic glTF files, without any custom MaterialX extension on the file. It would also allow easy importing of classic glTF materials into a system using MaterialX.

Does that sound useful / reasonable?

We have a prototype of this going, see KhronosGroup/MaterialX#1.

[fire]

Proposed workflow:

I want to transmit a final character gltf2 that has MaterialX procedural materials embedded into it and the MaterialX inputs are changed via different MaterialX file.

1. Character model
2. MaterialX defined materials (only gltf2 brdf is fine)

For example: we have a woman wearing yoga pants and t-shirt. Every material including the design graphic of the t-shirt and on the pants is defined in MaterialX. Variations are provided as different MaterialX files.

Your proposed workflow is:

1. Character model
2. MaterialX is rasterized in Blender to png and jpg images.
3. glTf2 export as normal from Blender

This is the prototype today with the experimental MaterialX gltf2 brdf.

Criticism:

1. One example of a workflow is 15-20mb total size. Let's assume that the mesh is 2-5mb and the rest are the textures 5-15mb. Each variation difference could be an additional 5-15MB rather than one parameter change in the MaterialX. For example: the base colour of the clothing is changed from white to blue.
2. The workflow to manually match MaterialX to corresponding material is not feasible as the number of MaterialX defined mesh materials increases to a large amount.

[emackey]

I want to transmit a final character gltf2 that has MaterialX procedural materials embedded into it and the MaterialX inputs are changed via different MaterialX file.

Yeah. My concern is, this sounds like "artist interchange" territory, not runtime transmission/delivery. The goals of glTF lie wholeheartedly in the latter category, focused on delivering finished assets to thick and thin clients. The scope of glTF has expanded a bit as mobile clients today are far less "thin" than in the past, and online model exchange via glTF has become commonplace. End users are of course free to use glTF for any purpose they see fit, but official Khronos efforts are encouraged to remain focused on the core goals of the format.

Of course we can't rule out the possibility of a full MaterialX extension for glTF in the future, for example a possible future in which MaterialX rendering has become near-universal. If you want to get a jump start on that, the webp extension you mention above is certainly a great starting point, and EXT_ is probably the correct prefix for it. But I don't expect Khronos members will switch gears to jump on that bandwagon too quickly, as we're pushing towards a different use with different goals.

[fire]

If you want to get a jump start on that, the webp extension you mention above is certainly a great starting point, and EXT_ is probably the correct prefix for it.

That sounds good.

The missing piece for the above is a MaterialX BSDF baker, I appreciate the Khronos members making that work.

[robertlong]

While I agree that supporting all of MaterialX is a lot to ask of thin clients, there's no way in the current glTF workflow to create custom materials. This has resulted in a number of custom extensions for apps like Mozilla Hubs where there is need to implement materials like water or animate UVs or something more advanced.

In the past, glTF has had support for embedding custom shaders however this was WebGL specific and presents many challenges for asset portability.

MaterialX has standard nodes for time, frame, and up vector, which are basic, but provide a useful way to at least start on some sort of dynamic material. If there were a glTF compatible output node defined for MaterialX, I do think that runtime generation of a shader targeting that output node would be fairly powerful and cover most of the gaps left in the current material definition. The current static material is great, but artists working on games are currently left with a huge gap in their workflow.

If not MaterialX, what other alternatives have been suggested for standardizing custom materials for glTF?

[donmccurdy]

I'm not aware of any better alternative than (or even a comparable alternative to) MaterialX for portable shader graphs. I think the big questions here have more to do with viability – particularly for runtime loading. I'd venture to guess (and would be glad to be proven wrong!) that no one is actually using MaterialX for runtime loading in performance-sensitive applications today. That's not to say it isn't possible, but it does seem like a strong sign that it's too soon to have official support in glTF. A glTF-compatible output node for MaterialX, or a similar 'baking' target, would be a good step in the right direction I think.

[devshgraphicsprogramming]

I have an implementation of a Material Graph for raytracing (already in production) that could implement this:
https://docs.google.com/presentation/d/1jT97cDCiIu9_AiDxKjlqHHC2n0nlHAev7RVt5Ncb9uY/edit#slide=id.g8b5eec73d6_0_10

Sorry about the half finished presentation, you should get the picture.

My $0.02 you should standardize the AST/IR/IL of the material graphs [while looking to cover MaterialX/NvidiaMDL/Mitsuba], and if possible keep it binary.

Basically keep ye-good-olde compiler design of FrontEnd->IR->Backend

Leave it up to the consumer of glTF how to implement the backend.

[emackey]

Oops, this old issue never got closed. The MaterialX graph for glTF's PBR first shipped with MaterialX 1.38.4 about 6 months ago, and has been refined a bit since then.

Usage examples are here:

https://github.com/AcademySoftwareFoundation/MaterialX/tree/main/resources/Materials/Examples/GltfPbr