Data Slice Generator

The data slice generator is the underlying function used to generate data slices for the labeling function. If the label maker raises an error during the search or the output labels doesn’t seem right, then you will need to check the logic in the labeling function or inspect the data for any inherent errors. This is where the data slice generator can help us do both. Ideally, you also want to use the generator during the development of your labeling function for best practice. However, it is an optional step and not required to generate labels.

In this guide, we will use the data slice generator to inspect data slices and apply our labeling function. To get started, let’s load a mock dataset of transactions and sample the data to see how the transactions look.

[1]:
import composeml as cp
[2]:
df = cp.demos.load_transactions()
df = df[df.columns[:7]]
df.sample(n=5, random_state=0)
[2]:
transaction_id session_id transaction_time product_id amount customer_id device
26 94 24 2014-01-01 05:55:20 5 100.42 5 tablet
86 274 7 2014-01-01 01:46:10 5 14.45 3 tablet
2 495 1 2014-01-01 00:14:05 5 69.45 2 desktop
55 275 4 2014-01-01 00:45:30 5 108.11 1 mobile
75 368 27 2014-01-01 06:36:30 5 139.43 1 mobile

Labeling Function

Let’s define a labeling function that will return how much a customer spent given a slice of transactions.

[3]:
def total_spent(df):
    total = df['amount'].sum()
    return total

Data Slices

The LabelMaker.slice() method will create the data slice generator. The parameters of this method can be passed directly to LabelMaker.search() to generate the labels. In the following sections, we will see how to use the data slice generator to make data slices consecutive, overlap, or spread out.

See also

For a conceptual explanation of the process, see Main Concepts.

Consecutive

When the the gap size is equal to the window size, the data slices are consecutive. In other words, the data slices do not overlap and are not spread out (e.g. don’t skip any data). This is the default value for the gap size. To demonstrate this example, let’s generate data slices using these parameters.

We create a label maker with the 2-hour window size.

[4]:
lm = cp.LabelMaker(
    target_entity="customer_id",
    time_index="transaction_time",
    labeling_function=total_spent,
    window_size="2h",
)

Then, we create a data slice generator with the 2-hour gap size. The default value for the gap size is the window size.

Tip

You can directly set minimum_data as the first cutoff time.

[5]:
slices = lm.slice(
    df.sort_values('transaction_time'),
    num_examples_per_instance=-1,
    minimum_data='2014-01-01',
)

Consecutive - Data Slice #1

By printing this data slice, we can see that it’s the first slice of transactions (denoted by the slice_number) for customer 1. This data slice contains all of the customer’s transactions that occurred within the 2-hour window between 2014-01-01 00:00:00 and 2014-01-01 02:00:00. We can also see that the 2-hour gap aligns the cutoff times to the window. So, the next data slice will start at the end of this data slice.

[6]:
ds = next(slices)
print(ds)
ds
slice_number                                             1
customer_id                                              1
window          [2014-01-01 00:00:00, 2014-01-01 02:00:00)
gap             [2014-01-01 00:00:00, 2014-01-01 02:00:00)
[6]:
transaction_id session_id product_id amount customer_id device
transaction_time
2014-01-01 00:45:30 275 4 5 108.11 1 mobile
2014-01-01 00:46:35 101 4 5 112.53 1 mobile
2014-01-01 00:47:40 80 4 5 6.29 1 mobile
2014-01-01 00:52:00 163 4 5 31.37 1 mobile
2014-01-01 00:53:05 293 4 5 82.88 1 mobile
2014-01-01 00:57:25 103 4 5 20.79 1 mobile
2014-01-01 01:03:55 488 4 5 129.00 1 mobile
2014-01-01 01:05:00 413 4 5 119.98 1 mobile
2014-01-01 01:31:00 191 6 5 139.23 1 tablet
2014-01-01 01:37:30 372 6 5 114.84 1 tablet
2014-01-01 01:38:35 387 6 5 49.71 1 tablet

Let’s apply our labeling function for the total spent on this data slice.

[7]:
total_spent(ds)
[7]:
914.7300000000001

Consecutive - Data Slice #2

In the second data slice, we can see the next 2 consecutive hours of transactions between 2014-01-01 02:00:00 and 2014-01-01 04:00:00. This is useful for generating labels that will consecutively process the data only once.

[8]:
ds = next(slices)
print(ds)
ds
slice_number                                             2
customer_id                                              1
window          [2014-01-01 02:00:00, 2014-01-01 04:00:00)
gap             [2014-01-01 02:00:00, 2014-01-01 04:00:00)
[8]:
transaction_id session_id product_id amount customer_id device
transaction_time
2014-01-01 02:28:25 287 9 5 50.94 1 desktop
2014-01-01 03:29:05 190 14 5 110.52 1 tablet
2014-01-01 03:39:55 7 14 5 107.42 1 tablet

Let’s apply our labeling function for the total spent on this data slice.

[9]:
total_spent(ds)
[9]:
268.88

Overlap

When the the gap size is less than the window size, the data slices will overlap. We can use this for rolling window based labeling processes. The amount of overlap is the difference between the window and gap size. For example, if the window size is 3 hours and the gap size is 1 hour, then 2 hours will overlap on each data slice. To demonstrate this example, let’s generate data slices using these parameters.

We create a label maker with the 3-hour window size.

[10]:
lm = cp.LabelMaker(
    target_entity="customer_id",
    time_index="transaction_time",
    labeling_function=total_spent,
    window_size="3h",
)

Then, we create a data slice generator with the 1-hour gap size.

[11]:
slices = lm.slice(
    df.sort_values('transaction_time'),
    num_examples_per_instance=-1,
    minimum_data='2014-01-01',
    gap="1h",
)

Overlap - Data Slice #1

The first data slice contains all of the customer’s transactions that occurred within the 3-hour window between 2014-01-01 00:00:00 and 2014-01-01 03:00:00. We can also see that the 1-hour gap spaces apart the cutoff time of this data slice at 2014-01-01 00:00:00 from the cutoff time of the next data slice at 2014-01-01 01:00:00.

[12]:
ds = next(slices)
print(ds)
ds
slice_number                                             1
customer_id                                              1
window          [2014-01-01 00:00:00, 2014-01-01 03:00:00)
gap             [2014-01-01 00:00:00, 2014-01-01 01:00:00)
[12]:
transaction_id session_id product_id amount customer_id device
transaction_time
2014-01-01 00:45:30 275 4 5 108.11 1 mobile
2014-01-01 00:46:35 101 4 5 112.53 1 mobile
2014-01-01 00:47:40 80 4 5 6.29 1 mobile
2014-01-01 00:52:00 163 4 5 31.37 1 mobile
2014-01-01 00:53:05 293 4 5 82.88 1 mobile
2014-01-01 00:57:25 103 4 5 20.79 1 mobile
2014-01-01 01:03:55 488 4 5 129.00 1 mobile
2014-01-01 01:05:00 413 4 5 119.98 1 mobile
2014-01-01 01:31:00 191 6 5 139.23 1 tablet
2014-01-01 01:37:30 372 6 5 114.84 1 tablet
2014-01-01 01:38:35 387 6 5 49.71 1 tablet
2014-01-01 02:28:25 287 9 5 50.94 1 desktop

Let’s apply our labeling function for the total spent on this data slice.

[13]:
total_spent(ds)
[13]:
965.6700000000001

Overlap - Data Slice #2

In the second data slice, we can see that there is a 2-hour overlap on the transactions that occurred between 2014-01-01 01:00:00 and 2014-01-01 03:00:00. By adjusting the gap size, we can set the precise amount of overlap in the data slices. This is useful for generating labels with specific overlap.

[14]:
ds = next(slices)
print(ds)
ds
slice_number                                             2
customer_id                                              1
window          [2014-01-01 01:00:00, 2014-01-01 04:00:00)
gap             [2014-01-01 01:00:00, 2014-01-01 02:00:00)
[14]:
transaction_id session_id product_id amount customer_id device
transaction_time
2014-01-01 01:03:55 488 4 5 129.00 1 mobile
2014-01-01 01:05:00 413 4 5 119.98 1 mobile
2014-01-01 01:31:00 191 6 5 139.23 1 tablet
2014-01-01 01:37:30 372 6 5 114.84 1 tablet
2014-01-01 01:38:35 387 6 5 49.71 1 tablet
2014-01-01 02:28:25 287 9 5 50.94 1 desktop
2014-01-01 03:29:05 190 14 5 110.52 1 tablet
2014-01-01 03:39:55 7 14 5 107.42 1 tablet

Let’s apply our labeling function for the total spent on this data slice.

[15]:
total_spent(ds)
[15]:
821.6400000000001

Spread Out

When the the gap size is greater than the window size, then there is data in-between data slices that will be skipped. We can use this for labeling data at specific intervals of time. The amount of data skipped is the difference between the gap and window size. For example, if the gap size is 3 hours and the window size is 1 hour, then 2 hours of data will be skipped in-between data slices. To demonstrate this example, let’s generate data slices using these parameters.

We create a label maker with the 1-hour window size.

[16]:
lm = cp.LabelMaker(
    target_entity="customer_id",
    time_index="transaction_time",
    labeling_function=total_spent,
    window_size="1h",
)

Then, we create a data slice generator with the 3-hour gap size.

[17]:
slices = lm.slice(
    df.sort_values('transaction_time'),
    num_examples_per_instance=-1,
    minimum_data='2014-01-01',
    gap="3h",
)

Spread Out - Data Slice #1

The first data slice contains all of the customer’s transactions that occurred within the 1-hour window between 2014-01-01 00:00:00 and 2014-01-01 01:00:00. We can also see that the 3-hour gap spaces apart the cutoff time of this data slice at 2014-01-01 00:00:00 from the cutoff time of the next data slice at 2014-01-01 03:00:00.

[18]:
ds = next(slices)
print(ds)
ds
slice_number                                             1
customer_id                                              1
window          [2014-01-01 00:00:00, 2014-01-01 01:00:00)
gap             [2014-01-01 00:00:00, 2014-01-01 03:00:00)
[18]:
transaction_id session_id product_id amount customer_id device
transaction_time
2014-01-01 00:45:30 275 4 5 108.11 1 mobile
2014-01-01 00:46:35 101 4 5 112.53 1 mobile
2014-01-01 00:47:40 80 4 5 6.29 1 mobile
2014-01-01 00:52:00 163 4 5 31.37 1 mobile
2014-01-01 00:53:05 293 4 5 82.88 1 mobile
2014-01-01 00:57:25 103 4 5 20.79 1 mobile

Let’s apply our labeling function for the total spent on this data slice.

[19]:
total_spent(ds)
[19]:
361.96999999999997

Spread Out - Data Slice #2

In the second data slice, we can see that 2 hours of transactions were skipped between 2014-01-01 01:00:00 and 2014-01-01 03:00:00. By adjusting the gap size, we can set the precise amount of data to skip in-between data slices. This is useful for generating labels that target specific portions of a dataset.

[20]:
ds = next(slices)
print(ds)
ds
slice_number                                             2
customer_id                                              1
window          [2014-01-01 03:00:00, 2014-01-01 04:00:00)
gap             [2014-01-01 03:00:00, 2014-01-01 06:00:00)
[20]:
transaction_id session_id product_id amount customer_id device
transaction_time
2014-01-01 03:29:05 190 14 5 110.52 1 tablet
2014-01-01 03:39:55 7 14 5 107.42 1 tablet

Let’s apply our labeling function for the total spent on this data slice.

[21]:
total_spent(ds)
[21]:
217.94

Data Slice Context

Each data slice has a context attribute to access its metadata. This is useful for integrating the context with the logic in the labeling function.

[22]:
vars(ds.context)
[22]:
{'gap': (Timestamp('2014-01-01 03:00:00'), Timestamp('2014-01-01 06:00:00')),
 'window': (Timestamp('2014-01-01 03:00:00'),
  Timestamp('2014-01-01 04:00:00')),
 'slice_number': 2,
 'target_entity': 'customer_id',
 'target_instance': 1}

From this guide, hopefully you have a better understanding on how to use the data slice generator to develop your labeling function.