question about groupby windowing(?)
Hello, everyone
How are you?
I have one question.
My data look like below
city / date / amount
NY / 20210401 / 100
NY / 20210402 / 150
NY / 20210403 / 50
NY / 20210404 / 30
LA / 20210401 / 40
LA / 20210402 / 20
LA / 20210403 / 50
Chicago / 20210401 / 30
Chicago / 20210402 / 40
Huston / 20210401 / 30
Huston / 20210402 / 20
Huston / 20210403 / 40
....
....
....
In this data, there are over 1 thousand cities (hugh number of cities)
I want to create a new variable (previous amount) in the data above, like this
city / date / amount / previous amount
NY / 20210401 / 100 / NA
NY / 20210402 / 150 / 100
NY / 20210403 / 50 / 150
NY / 20210404 / 30 / 50
LA / 20210401 / 40 / NA
LA / 20210402 / 20 / 40
LA / 20210403 / 50 / 20
Chicago / 20210401 / 30 / NA
Chicago / 20210402 / 40 / 30
Huston / 20210401 / 30 / NA
Huston / 20210402 / 20 / 30
Huston / 20210403 / 40 / 20
That is, I want to generate a lagged variable for each city
So I used operator "Loop by value" and "Filter example" and "Windowing"
but it requires hugh amount of memory and it was very slow
So could you please help me with this task?
How can I do this task using rapidminer operator?
Thank you in advance
Answers
-
Hi,please try: Group Into Collection and then Loop Collection with Windowing inside. That can be more efficient. But keep in mind that Windowing just multiplies your data by window_size. So it just can become big.~Martin0
-
Hello, mschmitz:
Thank you so much for your help and care.
It helped me a lot.
Have a nice day and see you again.
1 -
Try the "lag" operator as well. You can lag by 1 or by -1 unit for each city.
0 -
Hello, yyhuang.
Thank you for your comment.
Right, there is a "lag" operator. I can remember.
Have a nice day and see you, yyhuang.
0
Altair Employee