How to model millions of particles in reasonable time?
Hello,
Im new in EDEM simulation and would be very happy if I can get some advice.
My project is to model planetary ball mill but first just to see how metallic powder is being mixed in the vial. The powder is spherical with very narrow particle size distribution around 50 micrometers. There is several grams of powder in the vial which gives nuber of particles in the level of millions. The vial itself is a cylinder several centimeters in diameter and it rotates about 400rpm as in the model im attaching. I don’t want to introduce any destriction criteria, just to see how powder behaves in the vial. I have read a lot about EDEM and I could see that with GPU it should be possible to model millions of particles in reasonable time. For my project we just bought a reasonably good PC , precisely speaking: Asus PRO WS WRX80E-SAGE SE WIFI . 5965 WX threadripper, 256 GB DDR4 RAM, 3 x Nvidia RTX 3090 24GB , 14 TB of 7GB/s NVME SSD not much more I can imagine… the PC is worth >10kEUR. So I thought that with 3 GPU I should be able to proceed reasonably fast with calculations but it seems its not the case. If I put large balls (milimeters) , tens, hundreds, thousands , MY 24 core CPU is much much faster than 3 GPU but still its onot very fast, GPU is jst slow. When trying to model millions , GPU is really slow …CPU sometimes even don’t start and Im getting the solver error. Automatic caluclation of integration step gives 1*10-11 s sep which is crazy. 1*10-6 results in balls to “escape “ from vial. Can anyone please help me in configuration of the simulation in order to get some reasonable results of the powder being mixed. I assume that all the tutorials etc that we can find are not made with such powerful PC so I would be really happy to find out what Im doing wrong. Im attaching the files. Thank you!
Answers
-
Hi Iwona,
When verified your simulation file, I guess you are not calibrated bulk material.
1) Shear modulus chosen is very high which we won't recommend until otherwise the material behavior is too much sensitive with shear modulus magnitude.
Shear modulus given in the simulation is 8.77e10 Pa, this is one of the culprits to make your simulation slower.
Below graph explains computational time with respect to shear modulus, it is recommended to use shear modulus value anything between 1e7 to 1e8 Pa so that computational time can be reasonable. please remember you should do material calibration against physical behavior.
2) Regarding hardware i.e GPU card generally Number of particles to be created depends on the type of GPU card and its memory, larger thee memory in general GPU card has the capability in handling large number of particles.
in your case, GPU card is ok, but memory is less, so I feel this is limiting number of particles. However, you're using Multi GPU it should handle large number of particles.
Regarding computation time is large due to explained in point 1.
3) Coming to optimizing the problem, if I were you might try a small slice with periodic boundary condition.
As shown in below picture marked with red line is the small slice to be considered.
The width should be considered at least 6 times, mean particle size.
in your case particle size is 50 Micron, so you can choose width of slice not less than 300 microns with periodic boundary condition.
Periodic boundary condition option is available in Creator--> Environment--> boundary Condition
You can go through below link to understand cylindrical periodic boundary condition
Tutorials, How To, Troubleshooting - EDEM Tutorial 15: Cylindrical Periodic Boundaries (altair.com)
Thanks,
Prasad A
1 -
Hi Iwona,
As Prasad says calibrating the material and looking at optimising the shear modulus will help with the setup. Just on the interactions a value of 2 for the Standard Model of rolling friction is likely to be too high. I'd consider using Type C rolling or reducing down to a maximum of around 0.1 if using standard.
However I think one main issue is the particle size, it's set to 50 nanometres not 50 micrometres. Updating that automatically speeds up by 1000x and helps with the memory usage which maybe causing issues on the GPU.
I notice the factory is set to max attempts to place particles is too high at 99,999 (default 20) the factory can cause simulation slowdown -
https://www.youtube.com/watch?v=T9RJ_3uf0us&list=PLGNemB0NFb0Cf__QxGmnC_CODrPZYmWkAIt's good to consider periodic conditions however I think the motion in this case doesn't allow for it as this motion doesn't allow for periodic boundaries.
Regards
Stephen
1 -
Dear Prasad and Stephen,
thank you very much for your suggestions and spotted issues regarding my model. I really appreciate your time and effort and that you really checked the model etc. It rarely happens that such constructive comments can be found in any other community. I feel im in good place???? Thank you very much.
Your suggestions already helped a lot. Now I will try to implement them and I must also learn how to calibrate the material etc. I will try to do this by searching over the community and likely come back here for more help in next days. I think you are the experts experienced in a way that can give me a lot of solutions but I also understand I must learn by myself. My project is a PhD project for next 4 years and to solve my problems I need handle the software really well and im just at the beginning of the learning curve.
Im just wondering about the shear and young moduli of my material. The material Im mixing is bacically steel or titanium alloy, in form of almost perfect spheres, that is why I used so high values – just took from table. But I see that shear modulus may be not exactly the shear modulus from materials science point of view. Before we bought this PC I testet Ryzen 5 with preffesional Radeon W6800 (32 GB) GPU and I readt that CUDA works much better , so I expected significant increase in calculation speed having the current configuration, but it seem not to be the case. Do you think I should try 1x Radeon W6800 and OPENCL instead of RTX3090 nvidias? Or this is just a feeling that they work “slow”? How would you judge the choice of GPUS we made. 3x24GB or GPU Ram seems to be a lot and the power is also not bad , for gaming, but for this application do you think its more or less ok?
Again thank you so much!
0 -
Hi Iwona,
For the GPU I'd recommend the RTX3090 and the CUDA solver in EDEM 2022.1, this should be the fastest option so long as there are enough particles in the simulation to run efficiently. Anything over 100,000 particles should run well on the GPU and it is still typically quite efficient even at 10,000 particles.
For calibration I'd start by reviewing this webinar - https://www.altair.com/resource/edem-material-model-calibration
We have tutorials such as https://community.altair.com/community?id=kb_article_view&sysparm_article=KB0037196 for calibration but more recently have been using the Hyperstudy link for this - https://community.altair.com/csm?id=kb_article_view&sysparm_article=KB0119776
Regards
Stephen
0
Altair Employee