fix(dspy): Fix template.extract to be more robust

[dat-boris]

While experimenting with GPT4/Gemini, noticed that sometimes the completion will contain the input.

In that case, the previous template.extract will fail to extract (see test_single_output_with_noise test). This provides a fix for such a situation.

[okhat]

OK this may be a great thing to think about including but it's also a breaking change in some sense. It changes behavior for exiting code quite a bit. So we can't directly merge to main. I'll keep this open while we're exploring better parsing in general in the next few days and will update here

[dat-boris]

Thanks @okhat - appreciate the feedback, and thanks for creating + maintaining DSpy!

it's also a breaking change in some sense. It changes behavior for exiting code quite a bit.

Yeah agree that will change the existing behaviour - some of which is thankfully is documented with some of the existing test cases (and new ones). IMHO the change is a net improvement to the parsing behaviour since we always want to remove those un-necessary repeat of input in the output, and this doesnt change the behaviour if such input is not found. But totally understand that there's lots of edge cases to be considered!

I'll keep this open while we're exploring better parsing in general in the next few days and will update here

Sounds great! Yeah it sounds like that existing parsing can be improved (e.g. if we can shift to using structured output) - looking forward to it, as it will help with my current use case :-)

p.s. big fan of the MLOps session you did a few months ago to talk about DSpy!

[oldcai]

@okhat With OpenAI, only 'gpt-3.5-turbo-instruct' works well for completion. Other models, even 'gpt-4-1106-preview', tend to repeat the input.

@dat-boris I had some tests, and it works great. Thank you very much.