News_Category_Dataset_v2

[davidniki02]

I'm trying magpie on the News_Category_Dataset_v2.json which contains 200,000 news from Huffpost with their categories. Though I got an 81% accuracy during training, the actual predictions are very off, i.e. it barely ever reaches above 50% on any of its predictions, and most of the time it predicts the "politics" category even though the text is purely "technical".

Do we need a bigger sample to train on, or is there something wrong?

[bigredbug47]

Hi @davidniki02 ,

Many aspects that can affect the accuracy, I think 200,000 news is enough for training the model, maybe your problem is come from your dataset.

Did you analyze the dataset's distribution - does the dataset balanced or not? What is the top frequency represent words in each categories? etc.