5 ways to reduce EDEM file size

Stephen Cole · May 23

A large simulation for EDEM can be 10’s or 100’s of Gb in size. Larger simulations can have a file size of multiple Terabytes. Reducing file sizes or understanding what is causing large files can help improve productivity on EDEM.

Firstly, it’s important to understand the EDEM File types:

The .dem file is the main index file of a simulation and what you open however typically this will be very small as it doesn’t contain any data.
The .dfg contains all the Simulator and Analyst settings, again this will typically be a small file.
The .efd contains the Field Data information, if you import an external flow field to EDEM this is stored here. This file can be quite large but is directly related to the size of the field imported.
The .ess is also related to post-processing as it contains the bin group setup.
The .pdf (Particle Template File) is also a small file as this contains any CAD templates for the imported particle shapes.

The file types are also detailed in the EDEM elearning courses

Looking at the _data folder, this contains all the simulation files and results in .h5 format. This is where the large simulation files will be located. Inside this folder we will see sequentially numbered .h5 files:

The first file in this case the 0.h5 is quite small as it only contains the setup information and the imported CAD. As the simulation progresses you can see an increase in size of the files as the particles and contacts are all stored in the .h5 files and we are increasing the number of particles in the system.

By 100.h5 the number of particles has remained fairly constant to typically around 115 MB per file with the last file been larger as it isn’t compressed.

Below are 3 points which can help save simulation space by using the Simulator Settings .

Use ‘Compress Data’ in the EDEM Simulator

The compress data option compresses the simulation .h5 files on writing them, it does have a slight computational overhead but reduces the file size of the .h5 files. The compression amount depends on the numbers of particles and geometry elements however you can see up to 25% space saving compared to uncompressed files. There is also a command line option --compress-data

Use ‘selective save’

The second option is to selectively save data, defined in the EDEM Simulator.

For selective save this is one of the best methods to only choose to save a set amount of data. In this example we have chosen to save data every 0.01 s but every full save every 0.1 s. You can only restart the simulation on a full save but there is a prediction of 68% reduction in particle save data by not saving the angular velocity, forces and torque and other properties in this case.

Export the required data during simulation using Simulator Queries

Simulator queries is a feature first implemented in EDEM 2022.3 . Simulator queries can be used instead of selective save as you can just select the queries you want to save. We can see in this case there are 5 queries each set to export different measures at different frequencies. The full EDEM saves are taken every 0.25 s, during the simulation processing, and between the 0.25 s save interval, the selected data is exported during the simulation run. This does also have a slight computational overhead however overall this is a faster method than waiting to the end of the simulation to export the data.

Already run your simulation and have a huge set of files to manage? We can use the EDEM Analyst settings to truncate the simulation files.

4. Export a Simulation Deck

This option creates a new EDEM simulation file from the time-step that you choose.

In the Analyst, first move to the required time-step, then go to the Export > Simulation Deck option. This gives you the option to create a new EDEM file:

Typically, this option is used if the initial setup of a simulation isn’t required. For example when a simulation requires a bed of material , we only want to save the bed of material to use as an input to a new simulation. This way we can export the final time-step, reset the time to 0 s and delete the original files.

5. Truncate the simulation

We can Truncate the Simulation Deck to reduce the overall Simulation Deck size. This option is used to create a new, smaller simulation or reduce the overall size of an existing simulation. To use go to the Analyst > Truncate File option. Selecting a Step Factor of integer n will reduce the simulation file size by removing any intermediary files, just saving every n^th file.

This can be applied to the existing Simulation Deck or Create New Simulation Deck can be selected to duplicate the existing simulation, only with reducing the saved .h5 results files.

There are a number of tools to help you get started with EDEM, for example:

If you are looking for further information on EDEM or other Altair products we have plenty more on Altair Community:

Altair Community EDEM

5 ways to reduce EDEM file size

Categories