[IA Series 4/n] A Big Question: Why Study Logic in a World of Probabilistic AI?

Introduction

The purpose of this article is to help me answer the question “Why am I studying Logic?”. If it helps you, that’d be great, let me know!

The question comes from a nagging feeling of, why don’t I see logic used more in the ‘real world’. It could be a personal bias as I more easily see the utility of Rosenblatt’s work, where he looked at both Symbolic Logic and Probability Theory to help solve a problem and choose Probability Theory ([NN Series 1/n] From Neurons to Neural Networks: The Perceptron), with that we had the birth of the Artificial Neuron and the rest is history!

So, in an attempt to realign my bias, the short answer is that Logic is a key component in Knowledge Representation and Reasoning. That doesn’t address why we don’t see more of it though.

I have heard reports that Expert Systems (i.e. systems that hold knowledge and could be reasoned with) failed to deliver on the promise that people thought they had but it didn’t really click.

A significant part of why we don’t see more use of classical logic alone, is that it has proved insufficient for real-world knowledge representation. More expressive forms of logic have been developed, but they are (reportedly) computationally intractable. If anyone has tangible examples that show it being used, please let me know (as I’m certain that it does have a place).

This post covers the foundations of using Logic, Knowledge Representations, and Reasoning in AI. The next post will cover the key moments in the evolution of AI Knowledge Representation from purely logical approaches to probabilistic methods.

Building on previous articles of rational agents and search algorithms, logic provides the foundation for how agents can represent knowledge about their environment and reason about actions. A rational agent, as defined earlier ([IA Series 3/n] Intelligent Agents Term Sheet), selects actions expected to maximise its performance measure - logic gives us the formal machinery to make those selections systematic and verifiable. An example is search for the solution, either blindly via an uninformed search or with help via an informed search ([IA Series 2/n] Search Algorithms and Intelligent Agents).

Let’s take a step back - What is logic?

I’m keeping this short as I assume you have an idea, it’s part of our everyday speech and work culture. The key point that I need to remind myself of is that it isn’t just one logic… There are multiple logics, and I’ve found it useful to think of the noun “logic” in the same way as the noun “language”.

Whilst we’re defining things, let’s talk about what Knowledge Representation is…

What is Knowledge Representation?

This is a really important concept to understand, you could argue (and win) that building and using a representation of knowledge is the main aim of Artificial Intelligence!

One way of looking at it is that Knowledge Representation is the study of how knowledge can be represented symbolically and manipulated by reasoning systems so machines can understand and reason about the world.

Humans have been representing knowledge for millennia - through songs, stories, and books that capture how to handle situations or record historical events. AI faces a similar but more formal challenge: making human knowledge machine-readable.

McCarthy’s “Programs with Common Sense” (1958) was a key moment, he addressed the core challenge of converting human knowledge into forms computers can use for reasoning.

This paper will discuss programs to manipulate in a suitable formal language (most likely a part of the predicate calculus) common instrumental statements. The basic program will draw immediate conclusions from a list of premises. These conclusions will be either declarative or imperative sentences. When an imperative sentence is deduced the program takes a corresponding action. These actions may include printing sentences, moving sentences on lists, and reinitiating the basic deduction process on these lists.

I understand that four main approaches have evolved from the initial idea of formal logic including:

• Logical representations (formal logic systems)
• Structural representations (Frames, Semantic Networks, and Ontologies)
• Probabilistic representations (Bayesian networks and Markov Decision Processes)
• Distributed representations (Embeddings and Knowledge Graphs)

In this post, we look at the foundations for these. In the next post, we’ll look at the first (Logical representations) and third (Probabilistic representations), I hope to have more time to cover the other two…

What is Reasoning?

According to the Oxford Dictionary, reasoning is defined as:

the action of thinking about something in a logical, sensible way

Let’s focus on logical (note: sensible has “likely to benefit” in its definition), logical is defined as:

of or according to the rules of logic or formal argument

Above we confirmed that there is more than one logic, therefore there is more than one logical way to reason. That’s pretty important for me, when I think about the plurality (that’s a new word I learnt doing this work!) of logic, this part of AI makes a lot more sense.

I’ve heard it said, and I think this is a cornerstone of the Scientific Method, reasoning is the process of deriving new knowledge (conclusions) from existing knowledge (premises) using formal rules of inference.

Put another computer sciencey-way, it is the manipulation of symbolic representations according to well-defined rules to produce new representations that correspond to true statements about the world… or a logical process that enables machines to conclude, make predictions, and solve problems.

Types of Reasoning

There are different methods of reasoning, GeeksForGeeks have a nice article with examples on the first 8 from this list and I’ll cover more on Probabilistic reasoning in the next post.

1. Deductive Reasoning
2. Inductive Reasoning
3. Abductive Reasoning
4. Analogical Reasoning
5. Common Sense Reasoning
6. Monotonic Reasoning
7. Non-monotonic Reasoning
8. Fuzzy Reasoning
9. Probabilistic Reasoning
10. Heuristic Reasoning (like the search problem solving methods already covered)

Probabilistic Reasoning allows for handling uncertainty through mathematical probability theory. Probabilistic reasoning quantifies the likelihood of different possible outcomes or the degree of belief in various propositions. It’s my personal favourite when thinking about sport!

Before going into more detail, let’s bring it all together as it’ll help understand why Logic, specifically Rule-based expert systems, didn’t deliver on expectations.

Bring Logic, Knowledge, and Reasoning together

Key components of reasoning include:

• Facts/Assertions: Statements believed to be true
• Inference rules: Patterns for deriving new statements
• Sentences: Well-formed formulas in a logical language
• Knowledge base: Collection of sentences representing knowledge

The reasoning done on a knowledge base can be chosen depending on the problem at hand. Each reasoning type has environments where they are fit to use:

• Deductive, Inductive, and Abductive reasoning form the core approaches to logical inference
• Analogical reasoning helps transfer knowledge between domains
• Common sense reasoning attempts to capture the implicit knowledge humans take for granted
• Monotonic vs. Non-monotonic reasoning addresses how systems handle changing information
• Fuzzy reasoning deals with vagueness and uncertainty in the real world
• Probabilistic reasoning handles uncertainty by quantifying likelihoods and degrees of belief (using Bayes Theorem)
• Heuristic reasoning provides practical shortcuts for finding solutions

Some I find better than others, for example I’m not a fan of Common Sense reasoning (I think that’s impossible to define) and I’ll discuss Monotonic and Non-Monotonic reasoning in the next post - as a point of reference this was la very important learning for me to understand why we don’t have more Rule-based Expert Systems. In short Penguins can’t fly!

Reasoning gives life to knowledge representation and can make it useful; without reasoning capabilities, stored knowledge remains inert and of little use.

Next up, the problems with using classical logic for knowledge representation, why this approach struggled, and how probabilistic methods emerged as a solution to many of these challenges.