Slow iterations? Try MaterializeDataInMemory !!
I have a parameter iteration which includes an iterating operator chain, in effect to perform multiple label sliding window validations. In order to keep things clean I've been careful to consume IO objects and clear memory after each loop. Nevertheless the process took three hours to complete, until I put in the magic ingredient. Now it takes 25 minutes!
If anyone has a spare moment could they explain what "MaterializeDataInMemory" actually does? I stumbled across it in the documentation for memory cleanup, and am as a consequence none the wiser. What is certain is that for me the following proved to be quite some understatement...
If anyone has a spare moment could they explain what "MaterializeDataInMemory" actually does? I stumbled across it in the documentation for memory cleanup, and am as a consequence none the wiser. What is certain is that for me the following proved to be quite some understatement...
Might be very useful in combination with the MemoryCleanUp (see section 5.2.9)
operator after large preprocessing trees using lot of views or data copies.