From EP 5, here we go for the another classification model. It is …
Decision tree is a classification model in the same group as Logistic regression but it is likely to create a box over a group of data. The boxes can be resizing smaller to cover the group of data as much as possible. While Logistic regression is to draw a line to separate data into two groups straightforwardly.
It is, when there are outliers, Decision tree would be struggling to cover them and output pretty different result. On the other hand, Logistic regression is able to deal against them in a better way.
Let’s prepare the data
x as a DataFrame of “Pclass”, “is_male”, and “Age”.
y as classified “Fare” through our function using
Grow the tree
DecisionTreeClassifier with two main parameters:
we can choose one of two followings:
- “gini” from “gini impurity”. This is to measure how far and often the random results are incorrectly labels. It is for reducing misclassification.
- “entropy” is to measure how uncertainly the results are. This is for exploratory analysis.
it defines a number of layer of the tree. This time we define 3.
Eh, what are all labeled results? We can use
Show the tree
Use this to view our tree.
It’s hard to understand. Now we run this to generate an intuitive diagram.
try: from StringIO import StringIO except ImportError: from io import StringIO import sklearn.tree import IPython.display import pydot File_obj = StringIO() sklearn.tree.export_graphviz(tree, out_file=File_obj) Graph = pydot.graph_from_dot_data(File_obj.getvalue()) IPython.display.Image(Graph.create_png())
Ok. Let’s find the case inside. Just try the most-left branch.
xor “Pclass” is less than or equal 1.5 (actually Pclass is 1)
- And if
xor “is_male” is less than or equal 0.5 (actually 0 or female)
- And if
xor “Age” is less than 43.5
- Then we got [2, 11, 23, 26]
- Compare to the all labeled results, this person can be “Quite rich” with 26 points.
We can find how effective each column is with
.feature_importances_ where “feature” means the column we are interested in.
Try predict one. Let’s say this person is in “Pclass” 1, is female (“is_male” = False) and 40 years old. As a results, she was predicted as “Quite rich” in “Fare”.
Here we go to the end and may have a question, when we apply which classification between Decision tree and Logistic regression.
As far as I known, in case we ensure the data is able to be 2 groups separately by a straight line, we can use Logistic regression. Otherwise, use Decision tree.
Let’s see next time.