[BUG] Error is thrown in model.py when we send a query for a specific model

[oyilmaz-nvidia]

To reproduce the error, please check with @rnyak to get the notebook.

When we send the query, _transform_tensors functions gives the following error;

Traceback (most recent call last):
  File "/model/nvt-examples/models/yoochoose_nvt/1/model.py", line 295, in _transform_tensors
    tensors = column_group.op.transform(column_group.columns, tensors)
  File "/nvtabular/nvtabular/ops/rename.py", line 52, in transform
    df.columns = self.output_column_names(columns)
  File "/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py", line 664, in __setattr__
    super().__setattr__(key, col)
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py", line 2707, in columns
    raise ValueError(
ValueError: Length mismatch: expected 1 elements ,got 7 elements
0804 16:02:51.763651 88 pb_stub.cc:555] ValueError: Length mismatch: expected 1 elements ,got 7 elements

At:
  /usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py(2707): columns
  /usr/lib/python3.8/contextlib.py(75): inner
  /usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py(677): __setattr__
  /nvtabular/nvtabular/ops/rename.py(52): transform
  /model/nvt-examples/models/yoochoose_nvt/1/model.py(299): _transform_tensors
  /model/nvt-examples/models/yoochoose_nvt/1/model.py(263): _transform_tensors
  /model/nvt-examples/models/yoochoose_nvt/1/model.py(263): _transform_tensors
  /model/nvt-examples/models/yoochoose_nvt/1/model.py(143): execute

When I print out the column_group.columns and tensors in the _transform_tensors function, I see the following;

['item_id', 'category']

{'item_id': array([214714927, 214517450, 214714929], dtype=int64), 'category': array([0, 0, 0], dtype=int64)}
----------------------------

----------------------------
['timestamp']

             timestamp
0  1396774534190000000
1  1396774556013000000
2  1396774617109000000
----------------------------

----------------------------
['ts']

       timestamp
0  1396774534190
1  1396774556013
2  1396774617109
----------------------------

----------------------------
['timestamp']

             timestamp
0  1396774534190000000
1  1396774556013000000
2  1396774617109000000
----------------------------

----------------------------
['timestamp_hour']

   timestamp
0         11
1          1
2          4
----------------------------

----------------------------
['timestamp']

             timestamp
0  1396774534190000000
1  1396774556013000000
2  1396774617109000000
----------------------------

----------------------------
['timestamp_day']

   timestamp
0          1
1         11
2         19
----------------------------

----------------------------
['ts-first', 'item_id-list', 'ts-list', 'session_id', 'category-list', 'ts-last', 'item_id-count']

   item_id  category  session_id             ts            timestamp  timestamp_hour_hour  timestamp_day_day
0      720         1         549  1396774534190  1396774534190000000                   11                  1
1      805         1         549  1396774556013  1396774556013000000                    1                 11
2     1324         1         549  1396774617109  1396774617109000000                    4                 19
----------------------------

----------------------------
['ts-first', 'item_id-list', 'ts-list', 'session_id', 'category-list', 'ts-last', 'session_size']

   s-e-s-s-i-o-n-_-i-d
0                  549
----------------------------

And, the error is thrown in the last step where the column_group.columns doesn't match with the numbers of columns in the tensor. Seems like a simple error though.

[oyilmaz-nvidia]

The error reported above is thrown when the rename op with lambda function is used in the workflow. @rnyak updated the notebook and removed those rename ops and I got another error with this updated notebook. Posting here in case it helps;

Failed to transform operator <nvtabular.ops.list_slice.ListSlice object at 0x7f4af5c768e0>
Traceback (most recent call last):
  File "/model/nvt-examples/models/yoochoose_nvt/1/model.py", line 289, in _transform_tensors
    tensors = column_group.op.transform(column_group.columns, tensors)
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/nvtabular/nvtabular/ops/list_slice.py", line 68, in transform
    c = df[col]._column
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py", line 723, in __getitem__
    return self._get_columns_by_label(arg, downcast=True)
  File "/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py", line 1449, in _get_columns_by_label
    new_data = super()._get_columns_by_label(labels, downcast)
  File "/usr/local/lib/python3.8/dist-packages/cudf/core/frame.py", line 630, in _get_columns_by_label
    return self._data.select_by_label(labels)
  File "/usr/local/lib/python3.8/dist-packages/cudf/core/column_accessor.py", line 346, in select_by_label
    return self._select_by_label_grouped(key)
  File "/usr/local/lib/python3.8/dist-packages/cudf/core/column_accessor.py", line 408, in _select_by_label_grouped
    result = self._grouped_data[key]
KeyError: 'category-list'
0804 20:52:35.319187 110 pb_stub.cc:555] KeyError: 'category-list'

At:
  /usr/local/lib/python3.8/dist-packages/cudf/core/column_accessor.py(408): _select_by_label_grouped
  /usr/local/lib/python3.8/dist-packages/cudf/core/column_accessor.py(346): select_by_label
  /usr/local/lib/python3.8/dist-packages/cudf/core/frame.py(630): _get_columns_by_label
  /usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py(1449): _get_columns_by_label
  /usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py(723): __getitem__
  /usr/lib/python3.8/contextlib.py(75): inner
  /nvtabular/nvtabular/ops/list_slice.py(68): transform
  /usr/lib/python3.8/contextlib.py(75): inner
  /model/nvt-examples/models/yoochoose_nvt/1/model.py(293): _transform_tensors
  /model/nvt-examples/models/yoochoose_nvt/1/model.py(263): _transform_tensors
  /model/nvt-examples/models/yoochoose_nvt/1/model.py(263): _transform_tensors
  /model/nvt-examples/models/yoochoose_nvt/1/model.py(143): execute