Meetup summary

2025-09-26 - Intro to probability - part 4

None.

Agenda:

Picking up the probability intro series from last time. As per our running tradition, I expect to get through only a small subset of the agenda.

  • Derived RVs
    • Multivariate method of transformations (analogous to single-variable MOT but uses Jacobian)
    • Special case: sum of random variables (convolution). Works for both discrete and continuous RVs.
  • Foundational processes/distributions
    • Bernoulli process/RV, binomial RV, geometric RV, negative binomial RV, etc.
    • Multinomial (categorical) process, multinoulli.
    • Poisson process (limiting case of binomial), Poisson distribution, exponential distribution, Erlang/Gamma distribution
    • Gaussian distribution (different limiting case of binomial, but the derivation is long and we won’t get into it today; also arises from the CLT)
  • Moment generating functions
    • Equivalent to a two-sided Laplace transform, so it does not exist when RV doesn’t have finite moments.
  • Characteristic functions
    • Equivalent to Fourier transform, so it always exists but cannot be used to easily recovery moments from the series expansion.
  • Basic estimators
    • Definition
    • Estimator bias
    • Estimator variance
    • Estimator MSE
    • Sample mean
    • “Naive” sample variance
    • Unbiased sample variance
  • Foundational inequalities
    • Union bound
    • Markov inequalitiy
    • Chebychev inequality
    • Cauchy-Schwarz inquality for expectations (covered earlier)
    • Jensen’s inequality (I don’t know a general proof—this will just be an intuitive argument)
    • Gibbs’ inequalitiy (preview for information theory—won’t drill into entropy yet)
  • Inference preview (not planning to go deep here; will need to dedicate future sessions to particular areas)
    • Classical vs Bayesian perspective in a nutshell (raw MLE vs explicit priors, underlying parameter is an (unknown) constant vs RV)
    • Conjugate priors (e.g., beta-binomial)

tags: