Optimize Selection (FORWARDS): what is it that fills up the memory?
I'm running a forwards with a linear regression, 300/300 (T/F) cases with 800 attributes.. X-Val 3 folded (Parallel) running on 8 threads.
Within a couple hundered tryouts with just 1 attribute, memory gets maxed at about 10GB and execution time increases exponentially.. maybe these pictures help understand:
I was wondering why this is happening, as there seems to be some kind of garbage data inescapably, exponentially filling my memory.
I thought maybe the X-Val subsets where left in memory and that was the problem, but it wasn't, no Val and still same issue.
Tried a Free Memory after X-Val (inside optimize selection) but it seems to be worse.
I can't figure out why this behaviour in memory consumption is normal... there's something wrong, right? There must be something I can do about this.
Thanks a lot for your insight.
Regards.
Within a couple hundered tryouts with just 1 attribute, memory gets maxed at about 10GB and execution time increases exponentially.. maybe these pictures help understand:
I was wondering why this is happening, as there seems to be some kind of garbage data inescapably, exponentially filling my memory.
I thought maybe the X-Val subsets where left in memory and that was the problem, but it wasn't, no Val and still same issue.
Tried a Free Memory after X-Val (inside optimize selection) but it seems to be worse.
I can't figure out why this behaviour in memory consumption is normal... there's something wrong, right? There must be something I can do about this.
Thanks a lot for your insight.
Regards.