From the course: Data Science Foundations: Fundamentals

The enumeration of explicit rules

From the course: Data Science Foundations: Fundamentals

Start my 1-month free trial Buy for my team

The enumeration of explicit rules

“

- [Instructor] Let's say you meet someone at a party, and after talking for a while, you start to wonder if that person might be interested in you. This is apparently a question that is on a lot of people's minds. If Google's auto-complete is to be trusted, assessing attraction is a major research question of the top 10 statements to start how to tell. The first two are on this topic, shortly followed by how to tell an egg is bad. And in fact, men appear to be sufficiently difficult to read that they get to appear twice on the top 10 list. And so that lets you know, we need an answer to this question. How can you tell if somebody is interested in you? Well, maybe we can propose some rules. Maybe they're interested in you because they said so, or maybe they smiled and made eye contact or maybe they just swiped right. Then again, there are the insecure doubts that pop up and undermine your belief in these things. Maybe it's wonderful to meet you isn't diagnostic and they say that to everybody. Maybe they smile when they're bored or maybe they slipped on the ice and fell and accidentally swiped the wrong way. All these things that can undermine your faith in the rules that you have. But lest you think I'm just being silly in these examples, I want to point out that this is a legitimate data science problem. Dating apps are a multi-billion dollar business. And if you can help people find someone they love, then you've truly accomplished something worthwhile too. So, if we want to write a program to help people figure out if someone likes them, maybe we just need to include a little more detail and create a flow chart with explicit rules and qualifications before concluding with a definitive yes or no. This is an example of what's called an expert system. An expert system is an approach to machine decision-making in which algorithms are designed that mimic the decision-making process of a human domain expert. In this case, maybe Jane Austen, she wasn't incisive ane strategic decision maker when it came to matters of the heart. For example, here's a set of observations from Pride and Prejudice where we're pointed out that being wholly engrossed by one person and inattentive to others, offending people by not asking them to dance, having people speak to you and not answering. She says, "Could there be finer symptoms Could there be more diagnostic criteria for amorous inclinations?" She says, "Is not general incivility the very essence of love". And so maybe we could create an expert system based on these criteria. Now, I can imagine a lot of situations where this might not work well, but there are many other situations in which expert systems which model the decision-making of an expert have worked remarkably well. Those include things like flow charts to help you determine which data analysis method is most appropriate for answering the question you have given the data that's available to you. Or criteria for medical diagnoses, I'm in psychology, and so we use the DSM, which has the diagnostic and statistical manual of the American Psychiatric Association to say, if a person has these symptoms, but not these, then this is the likely diagnosis. Or even business strategies where you can give general checklists of approaches on what to do first, what to do next and how to reach the goals that you have. But it's important to remember that like any system, logic and the flow of expert based decisions has its limits. You're eventually going to hit the wall. You're going to meet situations that you just can't accurately predict or there'll be things that you never anticipated. And so it turns out that there are significant limits to enumerating explicit rules for decision-making. And in fact, we need more flexible and more powerful methods which is what I'm going to turn to next.

Contents