Probabilistic Reasoning
The real world is uncertain, so AI must reason with probability. Bayes' theorem and Bayesian networks let a machine update its beliefs as evidence arrives.
The ontologies of last week dealt in certainties: a dog definitely is a mammal. But most of the real world is uncertain. Will it rain? Does this patient have the disease? Is this email spam? An intelligent agent cannot wait for certainty; it must weigh the odds and act. Probabilistic reasoning is how AI handles uncertainty, and at its centre is one of the most useful and most misunderstood ideas in all of science: Bayes' theorem. This page builds it up, and then hands you a calculator that reveals a result almost nobody guesses correctly.
Reasoning under uncertainty
Classic logic deals in true and false. But an agent in the real world rarely has that luxury. Sensors are noisy, the future is unknown, and evidence is incomplete. Rather than pretend to be certain, a rational agent represents its uncertainty with probability: a number from 0 (impossible) to 1 (certain) for how likely something is.
Probability lets an agent do something logic cannot: update its beliefs as evidence arrives. Before a test, you have some belief a patient is ill. After a positive result, that belief should change, but by exactly how much? Getting that update right is the whole game, and it is where human intuition famously fails.
Conditional probability
The key concept is conditional probability: the probability of one thing given that we know another. It is written P(A | B), read "the probability of A given B".
P(rain)is the plain chance of rain.P(rain | dark clouds)is the chance of rain given that we can see dark clouds, which is higher.
Evidence changes probabilities, and conditional probability is how we express that. The crucial and slippery point is that the order matters: P(A | B) is generally not the same as P(B | A). The probability of a positive test given disease is not the probability of disease given a positive test. Confusing those two is the single most common mistake in reasoning about evidence, and Bayes' theorem is the tool that correctly connects them.
Bayes' theorem
Bayes' theorem is the rule for flipping a conditional probability, for turning "the chance of the evidence given the cause" into "the chance of the cause given the evidence", which is usually what we actually want to know.
Read it as a story of updating. You start with a prior belief. Evidence arrives. Bayes' theorem multiplies your prior by how well A explains the evidence (the likelihood) and rescales, giving your new, updated posterior belief. It is the mathematically correct way to change your mind, and it is used everywhere from spam filters to medical diagnosis to robot navigation.
The test that fools everyone
Here is the famous example. A disease affects 1 percent of people. A test correctly flags 99 percent of sick people, and wrongly flags only 5 percent of healthy people. You test positive. What is the chance you are actually sick? Most people, including many doctors, say around 95 percent. The real answer is far lower. Move the sliders and see for yourself.
Set the numbers, and the bars show, out of 10,000 people, who tests positive and who is truly sick.
Bayesian networks
Real problems involve many uncertain variables that influence each other. A Bayesian network is a graph that captures these relationships compactly: each node is a variable, and an arrow from one node to another means the first directly influences the second.
The classic example: Rain makes the Grass wet, but so does the Sprinkler. Each node carries a small table (a conditional probability table) giving its probability for each combination of its parents' values. From this compact structure, the network can answer questions like "given the grass is wet, how likely is it that it rained?" by propagating probabilities through the graph.
- Nodes are random variables (Rain, Sprinkler, Wet grass).
- Arrows show direct influence (Rain points to Wet grass).
- Conditional probability tables quantify each influence.
Naive Bayes and beyond
One of the simplest and most successful uses of Bayes' theorem is the Naive Bayes classifier. To decide whether an email is spam, it uses Bayes' theorem to compute the probability of spam given the words in the email. The "naive" part is a simplifying assumption that all the words are independent of each other, which is not really true but works remarkably well in practice, and made early spam filters possible.
Probabilistic reasoning runs quietly through modern AI: in speech recognition weighing likely word sequences, in robot localisation tracking where a robot probably is, in recommendation and diagnosis, and in the uncertainty estimates that make systems trustworthy. Wherever an agent must act without certainty, it reasons with probability.
Key points and practice
- AI handles uncertainty with probability, updating beliefs as evidence arrives.
- Conditional probability
P(A | B)is the chance of A given B; order matters,P(A | B)is notP(B | A). - Bayes' theorem flips a conditional using the prior, likelihood, and evidence to give the posterior.
- With a rare condition, a positive test can still mean a low chance of having it, because false alarms dominate.
- A Bayesian network is a graph of variables and influences with conditional probability tables.
- Naive Bayes classifies (for example, spam) by assuming features are independent.
Bayes' theorem states that P(A | B) = P(B | A) * P(A) / P(B). The prior P(A) is the belief in hypothesis A before seeing the evidence. The likelihood P(B | A) is how probable the evidence B is if A is true. The posterior P(A | B) is the updated belief in A after observing B. The theorem combines the prior belief with how well the hypothesis explains the new evidence to produce the revised belief.
Because the disease is rare, most people are healthy, so even a small false positive rate produces many false alarms. Out of 10,000 people, about 100 are sick and roughly 99 of them test positive, but of the 9,900 healthy people about 5 percent, nearly 495, also test positive. So around 594 test positive in total, of whom only about 99 are truly sick, giving a probability of being sick of roughly 99 / 594, about 17 percent. The 99 percent figure is the probability of testing positive given illness, not the probability of illness given a positive test.
A Bayesian network is a graphical model that represents a set of uncertain variables and the influences between them. Each node is a random variable, and an arrow from one node to another indicates that the first variable directly influences the second. Each node also has a conditional probability table giving its probability for each combination of its parents' values, and together these let the network compute the probability of any variable given evidence about others.