Execution Engine

InferenceSession::Run()

• Read RunOptions and understand the options (run_option.h)

SequentialExecutor::Execute()

• What's the purpose of ExecutionFrame? (execution_frame.h)

  • How is one nodes output passed in as another node's input?
  • What happens when we call context->Output() inside an op kernel?
  • How are feeds and fetches stored in ExecutionFrame?

• How is the execution order determined? (graph_viewer.cc)

  • Default execution order uses Graph::ReverseDFS() to generated topological sort
  • Priority-based execution order uses Graph::KahnsTopologicalSort with per-node priority

• How is each node's kernel invoked ?

• How does ORT guarantees all the cuda kernel is completed before session.run return?

Advance: GraphPartitioner

• How each node is determined to be place on which execution provider? (graph_partitioner.h)