Implement streamer for text-generation and add context arg to Pipeline.engine_forward

[mgoin]

Allows usage of the Transformers TextStreamer and TextIteratorStreamer classes with DeepSparse

from deepsparse import Pipeline
from transformers import TextStreamer

pipe = Pipeline.create(
    task="codegen",
    model_path="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base-none",
    max_generated_tokens=64,
    prompt_processing_sequence_length=1,
    use_deepsparse_cache=False,
)

streamer = TextStreamer(pipe.tokenizer)

# While also returning the usual output, the streamer will also print the generated text to stdout
_ = pipe(sequences="def fib():", streamer=streamer)

Screen.Recording.2023-07-23.at.4.52.34.PM.mov

===

This also extends beyond printing to stdout by using TextIteratorStreamer

from deepsparse import Pipeline
from transformers import TextIteratorStreamer
from threading import Thread

pipe = Pipeline.create(
    task="codegen",
    model_path="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base-none",
    max_generated_tokens=64,
    prompt_processing_sequence_length=1,
    use_deepsparse_cache=False,
)

streamer = TextIteratorStreamer(pipe.tokenizer)

generation_kwargs = dict(sequences="def fib():", streamer=streamer)
thread = Thread(target=pipe, kwargs=generation_kwargs)
thread.start()
generated_text = ""
for new_text in streamer:
    generated_text += new_text
print(generated_text)

"""
def fib():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

def fib2(n):
    a, b = 0, 1
    while a < n:
        yield a
        a, b
"""

[bfineran]

LGTM let's also update the PR title to include that context is added to engine forward - in terms of what's added there I think it looks good as a first step, we might want to think through mechanism/rules for updating the context dictionary in engine forward, if we want to allow that at all