Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check flair models for static embeddings #151

Closed
ktagowski opened this issue Dec 30, 2021 · 1 comment
Closed

Check flair models for static embeddings #151

ktagowski opened this issue Dec 30, 2021 · 1 comment
Assignees
Labels

Comments

@ktagowski
Copy link
Collaborator

Test flair models for static embeddings. Compare our pipeline results with results in literature or other leaderboards

@djaniak
Copy link
Collaborator

djaniak commented Jan 11, 2022

The static embeddings work fine with the flair pipelines, achieving satisfactory results when evaluated on polemo2 dataset (text classification).
The HPS pipeline was run for 100 iterations with the following outcome:

HPS results

image

These results were compared with the ones reported in two papers:

Results were obtained with hotels and medicine domains.

Name Score Metric Pooling Embeddings
our (flair) ~0.83 F1-macro CNN fastText KGR10
KLEJ ~0.83 Accuracy LSTM fastText
original paper ~0.75 F1-macro biLSTM fastText KGR10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants