In this example, we build a machine learning application to predict whether customers will purchase a product within the next shopping period. This application is structured into three important steps:
Prediction Engineering
Feature Engineering
Machine Learning
In the first step, we generate new labels from the data by using Compose. In the second step, we generate features for the labels by using Featuretools. In the third step, we search for the best machine learning pipeline by using EvalML. After working through these steps, you will learn how to build machine learning applications for real-world problems like predicting consumer spending. Let’s get started.
[1]:
from demo.next_purchase import load_sample from matplotlib.pyplot import subplots import composeml as cp import featuretools as ft import evalml
We will use this historical data of online grocery orders provided by Instacart.
[2]:
df = load_sample() df.head()
Will customers purchase a product within the next shopping period?
In this prediction problem, we have two parameters:
The product that a customer can purchase.
The length of the shopping period.
We can change these parameters to create different prediction problems. For example, will a customer purchase a banana within the next 3 days or an avocado within the next three weeks? These variations can be done by simply tweaking the parameters. This helps us explore different scenarios which is crucial for making better decisions.
Let’s start by defining a labeling function that checks if a customer bought a given product. We will make the product a parameter of the function. Our labeling function will be used by a label maker to extract the training examples.
[3]:
def bought_product(ds, product_name): return ds.product_name.str.contains(product_name).any()
Then, let’s represent the prediction problem by creating a label maker with the following parameters:
target_entity as the columns for the customer ID, since we want to process orders for each customer.
target_entity
labeling_function as the function we defined previously.
labeling_function
time_index as the column for the order time. The shoppings periods are based on this time index.
time_index
window_size as the length of a shopping period. We can easily change this parameter to create variations of the prediction problem.
window_size
[4]:
lm = cp.LabelMaker( target_entity='user_id', time_index='order_time', labeling_function=bought_product, window_size='3d', )
Now, let’s run a search to get the training examples by using the following parameters:
The grocery orders sorted by the order time, since the search will expect the orders to be sorted chronologically, otherwise an error will be raised.
num_examples_per_instance to find the number of training examples per customer. In this case, the search will return all existing examples.
num_examples_per_instance
product_name as the product to check for purchases. This parameter gets passed directly to the our labeling function.
product_name
minimum_data as the amount of data that will be used to make features for the first training example.
minimum_data
[5]:
lt = lm.search( df.sort_values('order_time'), num_examples_per_instance=-1, product_name='Banana', minimum_data='3d', verbose=False, ) lt.head()
The output from the search is a label times table with three columns:
The customer ID associated to the orders. There can be many training examples generated from each customer.
The start time of the shopping period. This is also the cutoff time for building features. Only data that existed beforehand is valid to use for predictions.
Whether the product was purchased during the shopping period window. This is calculated by our labeling function.
As a helpul reference, we can print out the search settings that were used to generate these labels. The description also shows us the label distribution which we can check for imbalanced labels.
[6]:
lt.describe()
Label Distribution ------------------ False 7 True 6 Total: 13 Settings -------- gap None minimum_data 3d num_examples_per_instance -1 target_column bought_product target_entity user_id target_type discrete window_size 3d Transforms ---------- No transforms applied
We can also get a better look at the labels by plotting the distribution and cumulative count across time.
[7]:
%matplotlib inline fig, ax = subplots(nrows=2, ncols=1, figsize=(6, 8)) lt.plot.distribution(ax=ax[0]) lt.plot.count_by_time(ax=ax[1]) fig.tight_layout(pad=2)
In the previous step, we generated the labels. The next step is to generate the features.
Let’s start by representing the data with an entity set. This way, we can generate features based on the relational structure of the dataset. We currently have a single table of orders where one customer can have many orders. This one-to-many relationship can be represented by normalizing a customer entity. The same can be done for other one-to-many relationships like aisle-to-products. We want to make predictions based on the customer, so we will use this customer entity as the target entity for generating features.
[8]:
es = ft.EntitySet('instacart') es.entity_from_dataframe( dataframe=df.reset_index(), entity_id='order_products', time_index='order_time', index='id', ) es.normalize_entity( base_entity_id='order_products', new_entity_id='orders', index='order_id', additional_variables=['user_id'], make_time_index=False, ) es.normalize_entity( base_entity_id='orders', new_entity_id='customers', index='user_id', make_time_index=False, ) es.normalize_entity( base_entity_id='order_products', new_entity_id='products', index='product_id', additional_variables=['aisle_id', 'department_id'], make_time_index=False, ) es.normalize_entity( base_entity_id='products', new_entity_id='aisles', index='aisle_id', additional_variables=['department_id'], make_time_index=False, ) es.normalize_entity( base_entity_id='aisles', new_entity_id='departments', index='department_id', make_time_index=False, ) es["order_products"]["department"].interesting_values = ['produce'] es["order_products"]["product_name"].interesting_values = ['Banana'] es.plot()
Now, we can generate features by using a method called Deep Feature Synthesis (DFS). This will automatically build features by stacking and applying mathematical operations called primitives across relationships in an entity set. The more structured an entity set is, the better DFS can leverage the relationships to generate better features. Let’s run DFS using the following parameters:
entity_set as the entity set we structured previously.
entity_set
target_entity as the customer entity.
cutoff_time as the label times that we generated previously. The label values are appended to the feature matrix.
cutoff_time
[9]:
fm, fd = ft.dfs( entityset=es, target_entity='customers', cutoff_time=lt, cutoff_time_in_index=True, include_cutoff_time=False, verbose=False, ) fm.head()
5 rows × 116 columns
There are two outputs from DFS: a feature matrix and feature definitions. The feature matrix is a table that contains the feature values with the corresponding labels based on the cutoff times. Feature definitions are features in a list that can be stored and reused later to calculate the same set of features on future data.
In the previous steps, we generated the labels and features. The final step is to build the machine learning pipeline.
Let’s start by extracting the labels from the feature matrix and splitting the data into a training set and a holdout set.
[10]:
y = fm.pop('bought_product') splits = evalml.preprocessing.split_data( X=fm, y=y, test_size=0.2, random_state=0, ) X_train, X_holdout, y_train, y_holdout = splits
Then, let’s run a search on the training set for the best machine learning model. During the search process, predictions from several different pipelines are evaluated to find the best pipeline.
[11]:
automl = evalml.AutoMLSearch( problem_type='binary', objective='f1', random_state=0, allowed_model_families=['catboost', 'random_forest'], max_pipelines=3, ) automl.search( X=X_train, y=y_train, data_checks='disabled', show_iteration_plot=False, )
Generating pipelines to search over... ***************************** * Beginning pipeline search * ***************************** Optimizing for F1. Greater score is better. Searching up to 3 pipelines. Allowed model families: random_forest, catboost (1/3) Mode Baseline Binary Classification P... Elapsed:00:00 Starting cross validation Finished cross validation - mean F1: 0.167 (2/3) CatBoost Classifier w/ Imputer Elapsed:00:00 Starting cross validation Finished cross validation - mean F1: 0.489 (3/3) Random Forest Classifier w/ Imputer +... Elapsed:00:00 Starting cross validation Finished cross validation - mean F1: 0.489 Search finished after 00:03 Best pipeline: CatBoost Classifier w/ Imputer Best pipeline F1: 0.488889
Once the search is complete, we can print out information about the best pipeline found, such as the parameters in each component.
[12]:
automl.best_pipeline.describe() automl.best_pipeline.graph()
********************************** * CatBoost Classifier w/ Imputer * ********************************** Problem Type: Binary Classification Model Family: CatBoost Pipeline Steps ============== 1. Imputer * categorical_impute_strategy : most_frequent * numeric_impute_strategy : mean * categorical_fill_value : None * numeric_fill_value : None 2. CatBoost Classifier * n_estimators : 10 * eta : 0.03 * max_depth : 6 * bootstrap_type : None * silent : True * allow_writing_files : False
Let’s score the model performance by evaluating predictions on the holdout set.
[13]:
best_pipeline = automl.best_pipeline.fit(X_train, y_train) score = best_pipeline.score( X=X_holdout, y=y_holdout, objectives=['f1'], ) dict(score)
{'F1': 0.6666666666666666}
From the pipeline, we can see which features are most important for predictions.
[14]:
feature_importance = best_pipeline.feature_importance feature_importance = feature_importance.set_index('feature')['importance'] top_k = feature_importance.abs().sort_values().tail(20).index feature_importance[top_k].plot.barh(figsize=(8, 8), fontsize=14, width=.7);
<AxesSubplot:ylabel='feature'>
We are ready to make predictions with our trained model. First, let’s calculate the same set of features by using the feature definitions. We will use a cutoff time based on the latest information available in the dataset.
[15]:
fm = ft.calculate_feature_matrix( features=fd, entityset=es, cutoff_time=ft.pd.Timestamp('2015-03-02'), cutoff_time_in_index=True, verbose=False, ) fm.head()
5 rows × 115 columns
Now, let’s predict whether customers will purchase bananas within the next 3 days.
[16]:
values = best_pipeline.predict(fm).values prediction = fm[[]] prediction['bought_product (estimate)'] = values prediction.head()
At this point, we have completed the machine learning application. We can revisit each step to explore and fine-tune with different parameters until the model is ready for deployment. For more information on how to work with the features produced by Featuretools, take a look at the Featuretools documentation. For more information on how to work with the models produced by EvalML, take a look at the EvalML documentation.