Creating Datasets from Results Files for Altair romAI
Altair EmployeeAltair romAI is an AI-powered technology that creates reduced order models (ROMs) to accelerate complex simulations like DEM, CFD, etc. ROMs created with romAI capture the dynamic response of a system accurately and yield almost instant results.
One of the most time-consuming steps when working with romAI is preparing your data in the correct format. Engineers often need to consolidate numerous individual results files into a single CSV dataset that meets romAI's specific requirements. This data preparation step, while critical, can be tedious and error-prone when done manually.
This guide presents an automated solution using an OML script I made in Altair Compose that streamlines the dataset creation process.
Let’s jump into the script:
First, we need a function that finds and returns all results files in the results directory, even if they are placed in different subdirectories:
It’s common to use the runs from a HyperStudy study to build a romAI dataset, this function would find all results files by just declaring file extension and base directory to look in.
Afterwards, the actual extension and results directory are defined (in this case I’m searching for PSIM .smv results files) and the output folder and name for each dataset are defined (both are CSV files). Two datasets are created:
- Train/test dataset, for training the ROM
- Validation dataset, for testing how the ROM performs in the post-processor tab of romAI Director
A fraction of the results is selected randomly (in this case 20%) for the validation dataset. The rest goes into the train/test dataset:
Once we have the results that will go into each dataset, we can loop over both files and add all results into the files. This part of the script should be adapted to your needs:
- At line 62, where the headers are defined, you should update for your own headers, keeping in mind that the first column should be labeled “time” (in lowercase). Also, leaving "\n" at the end of the string is needed to start a new line afterwards.
- Lines 70-75 do the actual reading of the results and should be changed depending on the files you are reading. I’m using the in-built PSIM reading commands in the example, but you could also use the OML CAE reader functions, CSV readers, etc.
Once read, the data is written into the CSV using csvwrite function with the “-append” flag so each set of results gets stacked one after the other (as required by romAI):
Now, you are ready to go! You can leverage this script and make this tedious process a lot faster, as well as create several different datasets if needed. Both the script and some files for testing are attached here.