Dynamic modelling in Twitter

aileenzhou

How to capture behavior change over time in Twitter, for example, perception towards vaccination during and post pandemic.
Hello, has any one done dynamic topic modelling that can reflect topic change over a certain period? Thank you. 

rfuentealba

Hello @aileenzhou,

That's work for NLP, and RapidMiner does that very well (you could also use Python NLTK together with RapidMiner). However, extracting data from Twitter is proven to be complicated because they put limits on how many tweets you get (max. 3200 per profile). If you can solve that, the rest is matter of: sorting information, tokenizing, getting parts of speech, lemmatization, stop words...

If you are interested in the second part, ping us; the other one, I am pretty sure someone here did such a task but I don't remember who.

All the best,

Rod.

aileenzhou

Thank you, Rod. To get Twitter sorted as suggested, ie, tokenising, lemmatisation, stop words ... is part of a standardised process. However, how to extract topic and the topic evolution over the time is where I stuck.  

rfuentealba

Hello @aileenzhou

There are two operators that can help you:

• Extract topics from documents
• Extract topics from data

Those are the same, the input varies. Have you tried these already?

aileenzhou

Thanks a lot. It has been awhile since I tried the database last time. I will try again and let you know later. 

rfuentealba

Sure! If you can share a sample process for us to see what you're doing, that would be marvelous!