Meetup summary

2025-09-05 - Linear algebra basics - part 3 (diagonalization)

The following videos will give you a good intuition/understanding about the intention behind a change of basis and eigendecomposition. We only briefly touched on the change-of-basis interpretation of matrix multiplication in the last session, so this is a good refresher and backs it with nice visualizations.

Agenda:

This will be the last session on linear algebra basics (assuming we get through everything). The general theme is diagonalization and eigendecomposition. This is a very useful and widely applicable operation that helps with all of the following (not exhaustive):

  • Gain insight into the “core” impact of effect of left-multiplication by a given matrix.
  • Quantify numerical stability of multiplication by a given matrix or its inverse
  • Distinguish saddle points from local optima for high-dimensional differentiable functions.
  • Directly compute fixpoints of (finite) Markov chains.
  • Compute arbitrary complex functions of matrices (given an appropriate series expansion).
  • Analyze various optimization routines through the lens of linear algebra to understand what’s happening at a deeper level.

Note that we will not cover the above use cases above (but might touch on some of them). Instead, we will directly cover the following:

  • Eigendecomposition:
    • The eigenvector/eigenvalue conditions
    • How to compute them
    • Eigenvector renormalization
  • Linear forms
  • Quadratic forms
  • Symmetric/Hermitian matrices:
    • Review of definitions
    • Canonicalization of quadratic forms
    • As a special case of eigendecomposition
  • Spectral theorem (just the most salient results and their derivations):
    • The eigenvalues of a Hermitian are real.
    • Eigenvectors of distinct eigenvalues are orthogonal.
    • A Hermitian matrix can always be decomposed into a full basis of mutually-orthogonal eigenvectors (even with algebraic multiplicity).
  • Positive (negative) (semi)definite matrices:
    • Quadratic form definition
    • Eigenvalue definition
    • Equivalence between the above
    • Why is this useful?

tags: