The Equivalence of Logistic Regression and Maximum Entropy Modeling

http://web.engr.illinois.edu/~mqian2/upload/research/notes/The%20Equivalence%20of%20Logistic%20Regression%20and%20Maximum%20Entropy%20Modeling.pdf

B. Dual Problem The dual problem of maximum entropy modeling is an unconstrained optimization problem

The Equivalence of Logistic Regression and Maximum Entropy Modeling

Posterior probability

From Wikipedia, the free encyclopedia

This article includes a list of references, but its sources remain unclear because it has insufficient inline citations. Please help to improve this article by introducing more precise citations. (November 2009)

Part of a series on Statistics
Bayesian statistics
Theory
Admissible decision rule Bayesian efficiency Bayesian probability Probability interpretations Bayes' theorem Bayes' rule Bayes factor Bayesian inference Bayesian network Prior Posterior Likelihood Conjugate prior Posterior predictive Hyperparameter Hyperprior Principle of indifference Principle of maximum entropy Empirical Bayes method Cromwell's rule Bernstein–von Mises theorem Bayesian information criterion Credible interval Maximum a posteriori estimation
Techniques
Bayesian linear regression Bayesian estimator Approximate Bayesian computation
Statistics portal
v t e

In Bayesian statistics, the posterior probability of a random event or an uncertain proposition is the conditional probability that is assigned after the relevant evidence or background is taken into account. Similarly, the posterior probability distribution is the probability distribution of an unknown quantity, treated as a random variable,conditional on the evidence obtained from an experiment or survey. "Posterior", in this context, means after taking into account the relevant evidence related to the particular case being examined.

Contents

  [hide] 

• 1Definition
• 2Example
• 3Calculation
• 4Classification
• 5See also
• 6References

Definition[edit]

The posterior probability is the probability of the parameters  given the evidence : .

It contrasts with the likelihood function, which is the probability of the evidence given the parameters: .

The two are related as follows:

Let us have a prior belief that the probability distribution function is  and observations  with the likelihood , then the posterior probability is defined as

^[1]

The posterior probability can be written in the memorable form as