Example of Backend Dependency Tracking
To illustrate how a deep learning backend tracks dependencies in a computational graph, consider a toy example computing . When the Python frontend thread executes the assignment statements for , , and , it simply returns these tasks to the backend queue without performing the mathematical operations. The Python frontend thread will only pause and wait for the C++ backend thread to finish computing the result of the variable when that result explicitly needs to be printed. Because the Python frontend thread is freed from performing the actual computations, this decoupled design ensures there is little impact on the program's overall performance, regardless of Python's slower execution speed.
0
1
Tags
D2L
Dive into Deep Learning @ D2L