Memory Management, Capture Safety, Graph Updates
Topics covered in this part:
- why memory addresses need to remain stable when capturing and replaying CUDA Graphs
- how input data can change between replays while the underlying memory addresses stay the same
- how static input/output buffers are used to make repeated graph replay possible
- how PyTorch manages captured workloads using graph-private memory pools and why memory sharing between graphs can matter
- what kinds of GPU operations can be safely captured inside a CUDA Graph
- why certain synchronization operations, CPU-side logic, and capture-incompatible operations can break or invalidate graph capture
- what happens when an already instantiated CUDA Graph needs to change
- how individual node parameters can be updated without rebuilding the entire executable graph
- how cudaGraphExecUpdate() can update an existing executable graph when the changes are compatible
- what happens when a graph has incompatible topology changes and needs to be re-instantiated