What file size limitations exist?

Nico Chart_21517
Nico Chart_21517
Altair Employee
edited October 15 in Altair RapidMiner

File size restrictions are influenced by both the computer operating system and the bit-ness of the application being used. 32-bit Altair SLC has a much smaller address space than 64-bit Altair SLC.

The total number of observations generally allowed in a WPD format dataset is controlled by two system options OBS and WPDHUGE. The WPDHUGE option affects any WPD-format datasets created whilst it is in force. A dataset created under NOWPDHUGE will probably be limited to (2^32)-1 observations. You can check your current settings using:

proc options option=obs; run;
proc options option=wpdhuge; run;

WPDHUGE is not set by default due to the effect it has on the size of index files because 64-bit record pointers are required.

You can either test a particular job locally using:

OPTIONS WPDHUGE;

Or add –WPDHUGE to a altairslc.cfg file for a more permanent setting.

Log files and CSV files are sequential output files and can be as large as the operating system allows.