To understand the differences between automatic differentiation libraries, let�s talk about the engineering trade-offs that were made. I would personally say that none of these libraries are �better� than another, they simply all make engineering trade-offs based on the domains and use cases they were aiming to satisfy. The easiest way to describe these trade-offs is to follow the evolution and see how each new library tweaked the trade-offs made of the previous.

Early TensorFlow used a graph building system, i.e. it required users to essentially define variables in a specific graph language separate from the host language. You had to define �TensorFlow variables� and �TensorFlow ops�, and the AD would then be performed on this static graph. Control flow constructs were limited to the constructs that could be represented statically. For example, an `ifelse` function statement is very different from � READ MORE