HDP improvements: Topic Hierarchy and Tutorial

[bhargavvader]

As mentioned in #941, and previously in #262, HDP isn't very consistent with LdaModel. Also #901, #937 are other documented issues to do with HDP. We could use a complete overhaul in the way these Topic Models are organised (ideally with a parent topic-model class), but for the time-being, HDP needs improvements in it's documentation and structure.

Particularly, it would be useful to have a comprehensive tutorial of what HDP is and what it does.

I would like to take this up in the future, but if someone else has time on their hand and would like to go ahead right away, we could use this issue to discuss the same!

Edit: #938 has already talked about a topic model class.

[markroxor]

I am up for it. What more changes can we incorporate? For a start I guess editing the tutorial should do like the one here. as pointed out in #941
hdp = HdpModel(corpus, id2word)
hdp.print_topics(show_topics=20, num_words=10)

show_topics should be changed to num_topics without altering the API.

[tmylk]

@markroxor Creating a tutorial in the tutorial folder would be a great place to start.

Agree about the change to print_topics

[bhargavvader]

Sounds good! About the tutorial, it will be nice to have it like a 'story' so it is interesting. Would look even better after we have a Topic Hierarchy functionality.
@markroxor could you look at the ldaseqmodel tutorial, and the news-classification to get an idea of how to make it more read-friendly? Even the doc2vec and topic coherence tutorials are good examples.

[bhargavvader]

@tmylk could you add appropriate labels to this please?

[markroxor]

@bhargavvader It will take some time but be assured it will be done. Will catch up with you on this.

[markroxor]

I am available to take this now. If I am right, it is required to create a hdpmodel tutorial in the tutorial folder?

[tmylk]

The folder is correct. The notebook content is in the main description of this issue.

[tmylk]

Ping @markroxor

[bhargavvader]

@tmylk , I was looking up how to maybe do the topic hierarchy, but couldn't find anything.
This post, this, and this ask for the same as well. Is there any resource, paper, video or anything which maybe hints at how to do this?

[menshikh-iv]

Resolved in #1055.

[zeyd31]

The selection of the number of topics at the end of the process is still unclear.