Probability Theory

A probability model assigns probabilities to events and describes random variables through distributions. Joint distributions represent several variables together; marginalization removes variables, while conditioning describes uncertainty after information is observed. Independence is a substantive property that can simplify factorization, not a default consequence of having separate columns. Expectation summarizes a distribution, and variance and covariance describe spread and linear co-variation. Bayes' rule connects conditional probabilities in opposite directions. These ideas support likelihood-based estimation and model evaluation, but a probability assigned by a model is only as meaningful as the model's connection to the data-generating process.

Translate a question into events and variables before calculating. Identify what information is available at prediction time and which quantities are conditioned on it. Check probability normalization, support and dependence assumptions; use small enumerated examples to test a proposed factorization. When interpreting predictions, distinguish individual-event uncertainty from uncertainty about an estimated probability. The practical outcome is a model whose quantities and assumptions can be explained, with simulations or analytic checks showing that conditional calculations and aggregate summaries behave as intended.

Conditional probabilities can be reversed incorrectly, and rare-event reasoning is especially sensitive to base rates. Zero correlation does not generally establish independence, while conditional independence can disappear after marginalizing variables. Expected values can conceal extreme or asymmetric outcomes. A fitted probability is not automatically calibrated on future data. Probability theory provides internal consistency, but empirical assumptions still need checking. Always state the population and conditioning information before comparing predictions or combining event probabilities.