SLC stability and performance tuning
Altair EmployeeThe performance of Altair SLC depends on a number of factors including the number of users sharing the service, the types of applications being run, the types of disk access to fast local storage, where the data is and how it is retrieved. On a shared server there may be other memory or CPU constraints needed to enable concurrent users to execute similar programs on the same machine, for example by adjusting the MEMSIZE/SORTSIZE system options.
We would be happy to review your current system options and offer our comments. System options can be listed by executing the following SAS language program:
PROC OPTIONS; RUN;
Send us the output from the Log window.
LOCATING RESOURCE BOTTLENECKS
If you have a program written in the language of SAS that you believe has unacceptable performance then we would be happy to review the full job log - preferably with the FULLSTIMER option turned on at the top of the source file - to identify which datastep or procedure is taking the most time and where possible improvements could be made.
If you are running Altair SLC on a Windows operating system then it might be useful to familiarize yourself with a windows performance monitoring tool. This should indicate whether your processes are cpu-bound or I/O bound. There will inevitably be slight differences in performance between virtual servers and physical boxes because of the inherent overhead in mapping I/O structures.
USE A FAST LOCAL DISK FOR USER WORK LIBRARIES
The WORK library is used to store temporary files that may be read/written repeatedly by sas language programs. Ensure that this library is located on a SSD or other fast disk, local to the computer that is running SLC, and make sure that it is not being auto-scanned, auto-backed-up, or auto-compressed.
SET SUITABLE VALUES FOR MEMSIZE, SORTSIZE and SUMSIZE
On a shared server with other users also running Altair SLC there can be resource conflicts trying to access a finite amount of RAM. For example, Altair SLC will attempt to grab as much memory as it can when sorting large datasets and using MEMSIZE/SORTSIZE/SUMSIZE system options can help to constrain what is used before secondary storage completes the task. The tradeoff here is that the more you constrain Altair SLC with MEMSIZE/SORTSIZE/SUMSIZE the more secondary storage you will need in your WORK location. Altair SLC will also utilise the cores available unless constrained by CPUCOUNT.
Say you have 6 users accessing the same machine and potentially running SAS language programs that have to process up to 100Gb of data. The more in-memory processing that can be performed and the faster the disk access the better the performance and the subsequent increase in stability.
How much RAM does your server have?
What current system options have you configured? (PROC OPTIONS; RUN; at the top of a source file will tell you).
Is your data stored locally or remotely via network drives?
What is your application doing with that data, sorting needs different resources to searching, filtering and reporting?
Is your WORK location for the storage of temporary datasets on a local drive or a network drive (remote)?
It is also possible that for the amount of data you are processing and the analysis you are trying to do a single server is insufficient to deal with the total workload. Perhaps a second server would be appropriate to distribute the load and the applications across more than a single machine. Or that some individuals use a workstation version of Altair SLC using a PC with a large amount of RAM and do some local processing rather than the shared approach.
Consider the following fictitious scenario:
A Windows Server 2019, quad-core, 64Gb RAM, fast disc access and at least 10x available free space compared to largest dataset, WORK location on local C: drive, Altair SLC installation on local C: drive, databases on shared network drive, 6x users, applications loading and sorting large datasets: We might suggest that you should constrain Altair SLC to allow everyone access to part of the total memory space and trade-off performance against completion of each users job by setting these restrictions:
-MEMSIZE 8G -SORTSIZE 7G -SUMSIZE 7G
This would allow 6 people to use 8G of memory to run Altair SLC but after that secondary storage is used to page/swap information. Of that 8G, sorting can utilize 7G (it is always better that SORTSIZE be smaller than MEMSIZE as Altair SLC still needs to have some space to keep processing) before the data is allocated to secondary storage to complete. This would also allow other applications to co-exist in the remaining available memory.
BUFFER SIZES
The system options BUFNO, BUFSIZE, IBUFNO and IBUFSIZE can also affect performance.
BUFSIZE - Specifies the size of the page used to create WPD datasets. This option only affects the size of the page used to create new WPD datasets; it does not affect already existing datasets.
Default: 0 (i.e. use the hard-coded internal BUFSIZE 4096).
Suggestion: Increase to 64K
IBUFSIZE - Specifies the size of an index page for a WPD dataset.
Default: 0
Suggestion: Increase to 32767
BUFNO - Specifies the number of buffers to use when reading a dataset. Increasing the number of buffers used to read data from a dataset can increase the speed with which data can be read; however, this reduces the amount of memory available for other operations, which might slow overall performance.
Default: 1
Suggestion: Increase to 10
IBUFNO - Specifies the number of index file buffers used by a library engine for a dataset.
Default: 0
Suggestion: Increase to 10
(n.b. another system option, VBUFSIZE, is used when viewing/scrolling the display of large datasets.)
WPD LIBRARY LOCKING
On some systems, adding this setting to the altairslc.cfg config file can improve performance:
-wpdlockingpolicy PERMANENT_LOCATIONS
Try setting -WPDLOCKINGPOLICY PERMANENT_LOCATIONS in the config file. The default in SLC 4 versions is “ALL_LOCATIONS” but in future versions of SLC it will always be “PERMANENT_LOCATIONS”.
MACRO CACHE
Macro cache: Prior to SLC 2023 the default MCACHE setting was 0 but in 2023 the default value is set to MCACHE=32. For programs that use a lot of macros, you can increase the MCACHE value to keep more macros in memory, for faster recall.
REDUCE ODS OUTPUT
If individual users are finding performance unsatisfactory, ask them to check their use of live ODS outputs in the Workbench 'Output Explorer' subwindow. Log and Listing are relative lightweight outputs but HTML, RTF, and PDF are increasingly demanding and can slow down the overall execution time for programs. Visit the Preferences settings (on the 'Window' menu of Workbench) and under Altair / Results you can set whether HTML, PDF, RTF, and/or LISTING outputs are automatically started when SLC is launched.