View source for Bayesian inference

{{Bayesian statistics}}
'''Bayesian inference''' ({{IPAc-en|ˈ|b|eɪ|z|i|ən}} {{respell|BAY|zee|ən}} or {{IPAc-en|ˈ|b|eɪ|ʒ|ən}} {{respell|BAY|zhən}}){{refn|{{MerriamWebsterDictionary|=2023-08-12|Bayesian}}}} is a type of [[statistical inference]]. In Bayesian inference, [[evidence]] or [[information]] is available, [[Bayes' theorem]] is used to change (or update) the [[probability]] of a [[hypothesis]]. Bayesian inference uses a [[Prior probability|prior distribution]] to estimate [[Posterior probability|posterior probabilities]]. Bayesian inference is an important to [[statistics]], [[mathematical statistics]], [[decision theory]], and [[sequential analysis]]. Bayesian inference is used in [[science]], [[engineering]], [[philosophy]], [[medicine]], [[sport]], and [[law]].
== Bayes' rule ==
[[File:Bayes theorem visualisation.svg|thumb|upright=1.2|A geometric visualisation of Bayes' theorem. In the table, the values 2, 3, 6 and 9 give the relative weights of each corresponding condition and case. The figures denote the cells of the table involved in each metric, the probability being the fraction of each figure that is shaded. This shows that P(A<nowiki>|</nowiki>B) P(B) = P(B<nowiki>|</nowiki>A) P(A) i.e. P(A<nowiki>|</nowiki>B) = {{sfrac|P(B<nowiki>|</nowiki>A) P(A)|P(B)}} . Similar reasoning can be used to show that P(¬A<nowiki>|</nowiki>B) = {{sfrac|P(B<nowiki>|</nowiki>¬A) P(¬A)|P(B)}} etc.]]
{{Main|Bayes' theorem}}
{{See also|Bayesian probability}}
{| class="wikitable floatright" style="font-size:100%;"
|+ [[Contingency table]]
! {{diagonal split header|<br /><br />Evidence|Hypothesis}} !! Satisfies<br />hypothesis<br />H !! Violates<br />hypothesis<br />¬H !! rowspan="5" style="padding:0;"| !! <br />Total
|-
! Has evidence<br />E
| |'''P(H|E)·P(E)'''<br />= P(E|H)·P(H) || |'''P(¬H|E)·P(E)'''<br />= P(E|¬H)·P(¬H) || '''P(E)'''
|-
! No evidence<br />¬E
| nowrap|'''P(H|¬E)·P(¬E)'''<br />= P(¬E|H)·P(H) || nowrap|'''P(¬H|¬E)·P(¬E)'''<br />= P(¬E|¬H)·P(¬H) || nowrap|'''P(¬E) =<br />1−P(E)'''
|-
| colspan="5" style="padding:0;"|
|-
! Total
| &nbsp;&nbsp; P(H) || style="text-align:right;" nowrap|P(¬H) = 1−P(H) || style="text-align:center;"|1
|}
Bayesian inference figures out the [[posterior probability]] from [[prior probability]] and the "[[likelihood function]]". The [[likelihood function]] comes from a [[statistical model]] of the [[Experimental data|data]].
<math display="block">P(H \mid E) = \frac{P(E \mid H) \cdot P(H)}{P(E)},</math>
where
* <math>H</math> is a ''[[hypothesis]]'' that is changed by [[Experimental data|data]] (or ''evidence''). There are usually many hypotheses. The point of the test is to see which hypothesis is more [[Probability|likely]].
* <math>P(H)</math> is the ''[[prior probability]]''. It [[Estimation|estimates]] the [[probability]]  of a hypothesis before there is any evidence.
* <math>E</math> is the ''evidence'', or data. It is any new [[Experimental data|data]] that is found.
* <math>P(H \mid E)</math> is the ''[[posterior probability]].'' This is what we want to know.
* <math>P(E \mid H)</math> is the  ''[[likelihood function]]''. 
* <math>P(E)</math> is the [[marginal likelihood]]. It is the same for all possible [[Hypothesis|hypotheses]] that are being tested. <math>P(E)</math> has to be [[greater than]] 0. If <math>P(E)</math> is 0, then you [[Division by zero|divide by zero]].

== Bayesian Inference Media ==
<gallery widths='160px' heights='100%' mode='traditional' caption=''>
File:Bayesian inference event space.svg|Diagram illustrating event space \Omega in general formulation of Bayesian inference. Although this diagram shows discrete models and events, the continuous case may be visualized similarly using probability densities.

File:Bayesian inference archaeology example.jpg|Example results for archaeology example. This simulation was generated using c=15.2.

File:Ebits2c.png|How evidence in bits might (or might not) add up to the level of confidence that you require.
</gallery>
== Related pages ==
* [[Epistemology]]
* [[Free energy principle]]
== Further reading ==
* {{cite book|last=Vallverdu|first=Jordi|title=Bayesians Versus Frequentists A Philosophical Debate on Statistical Reasoning|publisher=Springer|year=2016|location=New York|isbn=978-3-662-48638-2}}
* {{Cite book |last=Clayton |first=Aubrey |author-link=Aubrey Clayton|url=https://cup.columbia.edu/book/bernoullis-fallacy/9780231199940 |title=Bernoulli's Fallacy: Statistical Illogic and the Crisis of Modern Science |date=August 2021 |publisher=Columbia University Press |isbn=978-0-231-55335-3}}

* Stone, JV (2013), "Bayes' Rule: A Tutorial Introduction to Bayesian Analysis",   [http://jim-stone.staff.shef.ac.uk/BookBayes2012/BayesRuleBookMain.html Download first  chapter here], Sebtel Press, England.
* {{Cite book| title=Understanding Uncertainty, Revised Edition| author=Dennis V. Lindley | publisher=John Wiley | year=2013| edition=2nd | isbn=978-1-118-65012-7| author-link=Dennis V. Lindley }}
* {{Cite book| title=Scientific Reasoning: The Bayesian Approach| author=Colin Howson |author2=Peter Urbach |name-list-style=amp | publisher=[[Open Court Publishing Company]]| year=2005| edition=3rd | isbn=978-0-8126-9578-6| author-link=Colin Howson }}
* {{Cite book|author=Berry, Donald A.|title=Statistics: A Bayesian Perspective|url=https://archive.org/details/ssmstatisticsbay0000dona|publisher=Duxbury| year=1996|isbn=978-0-534-23476-8}}
*{{Cite book|author=Morris H. DeGroot|author2=Mark J. Schervish|name-list-style=amp|title=Probability and Statistics|edition=third|isbn=978-0-201-52488-8|publisher=Addison-Wesley|year=2002|url-access=registration|url=https://archive.org/details/probabilitystati00degr_0|author-link=Morris H. DeGroot}}
* Bolstad, William M. (2007) ''Introduction to Bayesian Statistics'': Second Edition, John Wiley {{ISBN|0-471-27020-2}}
*{{cite book |author=Winkler, Robert L |title=Introduction to Bayesian Inference and Decision |url=https://archive.org/details/introductiontoba0000wink |publisher=Probabilistic |year=2003 |isbn=978-0-9647938-4-2 |edition=2nd  }} Updated classic textbook. Bayesian theory clearly presented.
* Lee, Peter M. ''Bayesian Statistics: An Introduction''. Fourth Edition (2012), John Wiley {{ISBN|978-1-1183-3257-3}}
* {{Cite book| title = Bayesian Methods for Data Analysis, Third Edition | url = https://archive.org/details/bayesianmethodsf0000carl | publisher = Boca Raton, FL: Chapman and Hall/CRC | year = 2008 | isbn = 978-1-58488-697-6|author1=Carlin, Bradley P.  |author2=Louis, Thomas A. |name-list-style=amp }}
* {{Cite book| title = Bayesian Data Analysis, Third Edition | publisher = Chapman and Hall/CRC | year = 2013 | isbn = 978-1-4398-4095-5|last1=Gelman|first1=Andrew|author-link1=Andrew Gelman|last2=Carlin|first2=John B.|last3=Stern|first3=Hal S.|last4=Dunson|first4=David B.|last5=Vehtari|first5=Aki|last6=Rubin|first6=Donald B.|author-link6=Donald Rubin}}

* {{Cite book|author=Berger, James O| title=Statistical Decision Theory and Bayesian Analysis| edition=Second|year=1985| publisher=Springer-Verlag|series=Springer Series in Statistics|isbn=978-0-387-96098-2| bibcode=1985sdtb.book.....B| author-link=James Berger (statistician)}}
*{{Cite book|title=Bayesian Theory|publisher=Wiley|year=1994|author-link1=José-Miguel Bernardo|last1=Bernardo|first1=José&nbsp;M.|author-link2=Adrian Smith (statistician)|last2=Smith|first2=Adrian&nbsp;F.&nbsp;M.}}
* [[Morris H. DeGroot|DeGroot, Morris H.]], ''Optimal Statistical Decisions''. Wiley Classics Library. 2004. (Originally published (1970) by McGraw-Hill.) {{ISBN|0-471-68029-X}}.
* {{cite book|title=Theory of statistics|first=Mark J.|last=Schervish|publisher=Springer-Verlag|year=1995|isbn=978-0-387-94546-0}}
* Jaynes, E. T. (1998). [http://www-biba.inrialpes.fr/Jaynes/prob.html ''Probability Theory: The Logic of Science''].
* O'Hagan, A. and Forster, J. (2003). ''Kendall's Advanced Theory of Statistics'', Volume 2B: ''Bayesian Inference''. Arnold, New York. {{ISBN|0-340-52922-9}}.
* {{Cite book|author=Robert, Christian P|title=The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation|publisher=Springer|year=2007|edition=paperback|isbn=978-0-387-71598-8}}
* [[Judea Pearl|Pearl, Judea]]. (1988). ''Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference'', San Mateo, CA: Morgan Kaufmann.
* Pierre Bessière et al. (2013). "[http://www.crcpress.com/product/isbn/9781439880326 Bayesian Programming]". CRC Press. {{ISBN|9781439880326}}
* Francisco J. Samaniego (2010). "A Comparison of the Bayesian and Frequentist Approaches to Estimation". Springer. New York, {{ISBN|978-1-4419-5940-9}}
== References ==
{{reflist}}
== Other websites ==
* [http://www.scholarpedia.org/article/Bayesian_statistics Bayesian Statistics] from Scholarpedia.
* [http://www.dcs.qmw.ac.uk/%7Enorman/BBNs/BBNs.htm Introduction to Bayesian probability] {{Webarchive|url=https://web.archive.org/web/20090504220301/http://www.dcs.qmw.ac.uk/%7Enorman/BBNs/BBNs.htm |date=2009-05-04 }} from Queen Mary University of London
* [http://webuser.bus.umich.edu/plenk/downloads.htm Mathematical Notes on Bayesian Statistics and Markov Chain Monte Carlo]
* [http://cocosci.berkeley.edu/tom/bayes.html Bayesian reading list] {{Webarchive|url=https://web.archive.org/web/20110625052506/http://cocosci.berkeley.edu/tom/bayes.html |date=2011-06-25 }}, categorized and annotated by [https://web.archive.org/web/20060711151352/http://psychology.berkeley.edu/faculty/profiles/tgriffiths.html Tom Griffiths]
* A. Hajek and S. Hartmann: [https://web.archive.org/web/20110728055439/http://stephanhartmann.org/HajekHartmann_BayesEpist.pdf Bayesian Epistemology], in: J. Dancy et al. (eds.), A Companion to Epistemology. Oxford: Blackwell 2010, 93–106.
* S. Hartmann and J. Sprenger: [https://web.archive.org/web/20110728055519/http://stephanhartmann.org/HartmannSprenger_BayesEpis.pdf Bayesian Epistemology], in: S. Bernecker and D. Pritchard (eds.), Routledge Companion to Epistemology. London: Routledge 2010, 609–620.
* [http://plato.stanford.edu/entries/logic-inductive/ ''Stanford Encyclopedia of Philosophy'': "Inductive Logic"]
*[https://web.archive.org/web/20150905093734/http://faculty-staff.ou.edu/H/James.A.Hawthorne-1/Hawthorne--Bayesian_Confirmation_Theory.pdf Bayesian Confirmation Theory] (PDF)
* [https://causascientia.org/math_stat/DataUnkInf.html ''Data, Uncertainty and Inference''] — Informal introduction with many examples, ebook (PDF) freely available at [https://causascientia.org causaScientia]
* [https://databasecamp.de/en/statistics/bayesian-statistics Broad Introduction to Bayesian Statistics for Machine Learning]
{{Commonscat|Bayesian inference}}
{{Statistics|inference}}

{{DEFAULTSORT:Bayesian Inference}}

[[Category:Bayesian statistics|interference]]