Deciding what to model

(This is part 4 of a series)

Deciding what to model and what not to model can be challenging. When presented with a new technology it is often difficult not to use it – we all enjoy new challenges and new toys!

One way is to first decide what it is we are trying to solve – what is the business challenge we need to fix or what is the opportunity we are presented with. In other words we are asking ourselves at this stage, “should we build a solution for a given opportunity or problem?” If the answer is “yes”, then we have the opportunity to create a solution.

Next we need to define our problem. What is the nature of the problem, can it be defined in terms of our outcomes, and how accurate should our predictions be? Let’s explore these:

1. What is the nature of the problem? Is the problem translatable into a set of steps or is it nebulous? We require, at this stage, a problem with a set of concrete goals. We want to predict the weight of people given their heights. We want to know which of our customers might be interested in our new product range. We want to know which transactions are possibly fraudulent. And so on. Our problem is not of the nature: “how do we solve world hunger?”
2. Once we have our prediction (weight, target customers, fraudulent transactions), how do we represent it? Is is a continuous value as with weights, or a discrete data point, as with yes/no for fraudulent transactions.
3. Lastly, what is the range of error we would be happy with? For weight we may be happy with +-5kgs, for target customers we may be happy with 60% (for every 100 people we select, 60 will buy the new product), and for fraudulent transactions we may be happy with a 10% error rate (rather double-check a few more transactions, than lose money to fraud)

The last part is vital – as we learned in the last post we cannot be 100% accurate all the time – with difficult prediction models we will have errors. We must just decide how large is acceptable and on what side we will err. Remember at this stage that we are using an organic machine learning model to predict values in an uncertain area, or based on uncertain input data. If we had a clear, unambiguous environment we would not need a learning algorithm!