Implement concatenate, lists.explode, merge, sorting, and stream compaction in pylibcudf

[vyasr]

Description

Contributes to #13921

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[wence-]

Looks great, thanks! Some minor suggestions

[vyasr]

/merge

[cwharris]

Is there a reason get_column_names was removed? Is it replaced by anything? We were using it in Morpheus, but we can probably work around it being missing now. Just curious if there's another API we should use instead.

[vyasr]

It was removed because it was a completely unused function. It is not replaced by anything. If you need it, though, feel free to copy out the source. It was previously a cdef function, but there's really not much reason for it to be one because it was interacting with purely Python objects (DataFrame/Series/Index) except for the return value. Here's the old code:

cdef vector[string] get_column_names(object tbl, object index):
    cdef vector[string] column_names
    if index is not False:
        if isinstance(tbl._index, cudf.core.multiindex.MultiIndex):
            for idx_name in tbl._index.names:
                column_names.push_back(str.encode(idx_name))
        else:
            if tbl._index.name is not None:
                column_names.push_back(str.encode(tbl._index.name))

    for col_name in tbl._column_names:
        column_names.push_back(str.encode(col_name))

    return column_names

That should be replaceable with something like this (up to how you want to handle the truthiness of the index parameter)

def get_column_names(tbl, index):
    column_names = []
    if index:
        idx = tbl._index
        if isinstance(idx, MultiIndex):
            column_names = idx.names
        elif idx.name is not None:
            column_names = [idx.name]
    column_names.extend(tbl._column_names)
    return [c.encode() for c in column_names]

Then you can use that in a Cython context relying on Cython's automatic list->vector conversion

cdef vector[string] names = get_column_names(tbl, index)