Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

big difference between setting title and not setting title #103

Open
Jeffwan opened this issue Jun 1, 2023 · 1 comment
Open

big difference between setting title and not setting title #103

Jeffwan opened this issue Jun 1, 2023 · 1 comment

Comments

@Jeffwan
Copy link

Jeffwan commented Jun 1, 2023

We are using cross decoder to rerank the results. for some QA pairs. We use following format to get the ranking scores.

<query, para1, title1>
<query, para2, title2>
<query, para3, title3>
model.matching(query: List[str], para: List[str], title: List[str])

There's one observation I notice is, for the same question, setting title give me much better results, top1 or top2. However, If I do not set title, it is really bad (~top10 for 20 records).

I am curious why title makes a big difference here? In my case, for query, we have many tiles with same name. I felt adding title is not a big deal but it is

@Jeffwan
Copy link
Author

Jeffwan commented Jun 1, 2023

if len(title) != 0:
assert len(para) == len(title)
for q, t, p in zip(query, title, para):
data.append(q + '\t' + t + '\t' + p)
else:
for q, p in zip(query, para):
data.append(q + '\t\t' + p)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant