In Fall 2020, I took MIT class 18.337 (Parallel Computing and Scientific Machine Learning) taught by Prof. Chris Rackauckas. For the final project, I implemented and analyzed several automatic differentiation techniques for nonlinear nodal analysis in Julia.
Due to their grid-like construction, nodal systems often have sparse Jacobian matrices, and we can take advantage of this sparsity to accelerate their computation. In my project, I implemented forward-mode and reverse-mode automatic differentiation to construct the Jacobian of a nodal system. I implemented a matrix coloring scheme to accelerate sparse forward-mode automatic differentiation of the Jacobian of nodal systems. In addition, I implement a combined sparse automatic differentiation technique to compute a Jacobian that is mostly sparse, with several dense rows.
The Jacobian matrix for a simple system and its compressed representation.