User Interface

InferenceSession

• Read inference_session.h to get familiar with the common interface functions,

Following are the important ones for training

- Load()
- Run()
- NewIOBinding()
- RegisterGraphTransformer()
- RegisterExecutionProvider()
- Advance: Initialize()

	- What happens under the hood when we call session.initalize()? 

• Understand the configs in SessionOptions (session_option.h)

Followings are important ones for training

- execution_order
- enable_mem_pattern
- use_deterministic_compute
- session_log_severity_level

IOBinding (IOBinding.h)

• Prerequisite: What is an ORTValue? (ml_value.h)

• BindInput()

  • How to create an ORTValue?

• BindOutput()

  • With preallocated buffer

  • Without preallocated buffer

    • Who allocates the output buffer? and How is this returned back to user?

• What should be the lifespan of an IOBinding? Can user reuse IOBinding accross multiple Session::Run()?

• How is binded inputs/outputs passed into ExecutionFrame?

• Advance: How is IOBinding different from dlpack's approach? What are their advantages?

ORTModule (ortmodule.py)

• Read training_agent.h to understand the available interface functions

• Understand how ORTModule uses TrainingAgent

• Read ORTModule forward() and backward() function

  • How does ORTModule get an ONNX graph from torch nn.module?
  • How is ORT doing the auto-diff without torch's autograd engine?
  • How is ORT hijacking torch's foward/backward call?

Advance: PyBind

• How's C++ InferenceSession used as python's onnxruntime.InferenceSession?
• Read onnxruntime_pybind_state.cc, onnxruntime_inference_collection.py for InferenceSession binding
• Read orttraining_pybind_state.cc for TrainingAgent binding