RANDOM FAIL IN HYPERSTUDY SIMULATION WITH FLUX MODEL

Mattia Pinna

Hi,

i use hyperstudy 2020.1 in order to optimize a 2d flux model 2018.1 ("TESTCASE_INI_TRAN_ALU.F2G.FLU") using a post processing script ("Postprocessing_Python_File.PY"). In this script i simply create a lot of scenarios until reach the final simulation.

During the hyperstudy DOE or OPT, sometimes and randomly, during the post process of the script, the run fail during the calculation of the scenario. With randomly i mean that if i simulate the same model with hyperstudy, sometimes there is no fail, sometimes the fail comes in a different scenario (sometimes in the first scenario, sometimes in the ninth exc.) The strange thing is that if i run the post processing script in the same flux model, the simulation always proceeeds until the end without any problem (see the Flux2D_bak.report files for the two cases). The same problem occured also with different release of hyperstudy (2017,2019) and flux (2018 and 2019).

Note: line 13755 for the fail simulation.

What kind of problem is this? How can i fix this?

 

Thank you for your time.

 

Mattia  

Diana M._20503

Hello Mattia,

Would it be possible to share the *.F2G, *.F2G.FLU and the *.py for check?

Regards,

Mattia Pinna

Hi,

 

i'm sorry but i can't for company policy reason. Can you help without these file, maybe with some advice?

Thank you.

Diana M._20503

Ok we'll try. Let's first clarify this "create a lot of scenarios until reach the final simulation". Can you detail a bit more this? Do you select also a scenario through the F2G generation?

Mattia Pinna

The study is composed as follows:

-We have one file *F2G.FLU (TESTCASE_INI_TRAN_ALU.F2G.FLU)

-We have one post processing file (Postprocessing_Python_File.PY)

-Hyperstudy run a scenario and after that start with the post processing file.

-The post processing file is a for cycle where we create in the same file different scenario:

for iai in range(0,20) :
                              FLU_FILE="the path of the hyperstudy.FLU file"                Scenario(name='SCENARIOOcurr_%d' % iai)            startMacroTransaction()            Scenario['SCENARIOOcurr_%d'% iai].addPilot(pilot=MultiValues(parameter=VariationParameter['TIME'],                                                                       intervals=[IntervalStepNumber(minValue=0.0,                                                                                                            maxValue=LIMITMIN,                                                                                                            stepNumber=2)]))            Scenario['SCENARIOOcurr_%d'% iai].addPilot(pilot=MonoValue(parameter=VariationParameter['IPH'],                                                                                                  value=float(ccc)))            Scenario['SCENARIOOcurr_%d'% iai].addPilot(pilot=MonoValue(parameter=VariationParameter['ANGOLO_NOMINALE'],                                                    value=asas))            endMacroTransaction()            Scenario['SCENARIOOcurr_%d' % iai].solve(projectName=str(FLU_file))  

 

-Only in hyperstudy the run fail during the simulation (not during the modification of parameter), infact we have:

Scenario[SCENARIOOCURR_11] created
Scenario[12]
addPilot executed
addPilot executed
addPilot executed
Begin of physical check ...
End of physical check.
15:40:45 187 sec. Start save ...
15:40:45 188 sec. ... end save
Active formulation(s) :
MT2VEC : 2D transient magnetic vector potential formulation
CTX2VITRES : Composant bipolaire resistif (potentiel integre dans le temps)
CTX2VITIND : Composant bipolaire inductif (potentiel integre dans le temps)
CTX2VITBOB : Bobine filaire bipolaire (potentiel integre dans le temps)
CTX2VITMAS composant massif 2D pot integre dans le temps
Matrix system : non linear, real, symmetrical

15:40:46 188 sec. Start solving problem ...

Step number 1
Value of parameter ANGOLO_NOMINALE : 67.5700000000003
Value of parameter ANGPOS_MOBILE : 67.5700000000003
Value of parameter ANGVT_MOBILE : 10800
Value of parameter B1_ANG : 0
Value of parameter B1_BR : 1
Value of parameter B1_R : 0
Value of parameter B1_RAD : 18.82
Value of parameter B1_SB : 1.92
Value of parameter B1_SD : 0
Value of parameter B1_SO : 1.68
Value of parameter B1_W1 : 4.08
Value of parameter B1_W2 : 4.44
Value of parameter B1_W3 : 4.44
Value of parameter B1_X : 0.7
Value of parameter B1_Y : -0.14
Value of parameter B2_ANG : 0
Value of parameter B2_BR : 1
Value of parameter B2_R : 0

[62578|1612276846811|1|INFO ][ErrorManager|ErrorManager.createFortranException] : Type: 500 Message: FICDET => Error detected by operating system ( 28 )
[62579|1612276846811|1|INFO ][Fortran.Error|ErrorManager.createFortranException] : Type: 500 Message: FICDET => Error detected by operating system ( 28 )
[62580|1612276846814|1|INFO ][Fortran.Error|ErrorManager.addMessageToException] : Type: 1 Message: FICDET => Input/output error
[62581|1612276846815|1|INFO ][Fortran.Error|ErrorManager.addMessageToException] : Type: 1 Message: RENDIR => Failed
[62582|1612276846815|1|INFO ][Fortran.Error|ErrorManager.addMessageToException] : Type: 1 Message: GP3PBE => Failed to save the Flux project
[62583|1612276846815|1|INFO ][Fortran.Error|ErrorManager.addMessageToException] : Type: 1 Message: GP3ENR_2 => No recording of project has been made.
[62584|1612276846816|1|INFO ][Fortran.Error|ErrorManager.addMessageToException] : Type: 1 Message: ROESAM => Failed
[62585|1612276846816|1|INFO ][Fortran.Error|ErrorManager.addMessageToException] : Type: 1 Message: PBEPVD_4 => Failed
[62586|1612276846816|1|INFO ][Fortran.Error|ErrorManager.addMessageToException] : Type: 1 Message: Non managed error which has stopped solving process

 

Thank you

Diana M._20503

It looks strange indeed, especially if it occurs randomnly for different runs and/or for different scenarios within the list of 20 scenarios.

I provide below some directions to start the inverstigation:

1) I'd suggest to post the Flux messages below in the Flux forum for advise. Indeed the error is retrieved in HyperStudy, but something went wrong in the solver (even though you didn't reproduce it when trying directly). The idea is to check if it is something known, what would be the cause and is it fixed, today we have Flux 2021 (you have tried with 2018 and 2019 correct?). 

FICDET   => Error detected by operating system ( 28 )
FICDET   => Error detected by operating system ( 28 )
FICDET   => Input/output error
RENDIR   => Failed
GP3PBE   => Failed to save the Flux project
GP3ENR_2 => No recording of project has been made.
ROESAM   => Failed
Message: PBEPVD_4 => Failed
Non managed error which has stopped solving process

2) can you also check if there are other messages in the task_exe_stderr file under one of the failed runs folder. normally you'll the same ones as in the Flux report file.

3) the message Failed to save the Flux project makes me think that the file is "taken" by another program. You confirm that the same solved Flux project (for instance the one under run_00004/m_1) was not opened in Flux to test the postpro.py?

 

 

Diana M._20503

For your information I thought it can be of interest for you for further studies:

1) today it is possible to specify also a preprocessing python file along with the postprocessing one within the Coupling component generation for Hst allowing to perform some changes (e.g. material assignment to regions) before the solving.

2) there is a feature request to implement the multiple scenarios selection through the coupling component generation, and so multiple scenario solving through the Hst process

Diana M._20503

Mattia,

something else to check:

Would it be possible that you encounter memory issue on your machine ?

Are there other tools running at the same time as the Hst-Flux calculation? Also, I see that Flux2D is launched with 10329 MiB for the numerical memory. It looks high for a 2D project. Is it really needed?

Mattia Pinna

Hi Diana,

 

thank you for your quick response. In the task_exe_stderr is reported this message:

 

[62313|1612276658787|1|WARNING ][UseCase|UseCaseManager.changeUseCase] : No use case transaction found
[62598|1612276848732|1|WARNING ][ActionManager|UIActionManager.postAction] : Error during action execution: Action: Scenario_resoudre Exception: (SoftwareFortranException, executeAlgo_c : Execution failed. Failed execution)
[62600|1612276848735|1|WARNING ][ActionManager|UIActionManager.postAction] : Error during action execution: Action: executeSpy Exception: (JavaException, Error occured while executing the command file : Postprocessing_Python_File.PY. Line : ?. executeAlgo_c : Execution failed. Failed execution)
[62602|1612276848743|1|WARNING ][ExecuteBatchSpyAction|ExecuteBatchSpyAndExitAction.doAction] : Execution of C:/PROGRA~1/Altair/2020/HWDESK~1/common/python/python3.5/win64/Lib/SITE-P~1/alt/hst/eac/cmd/HSTUPD~1.PY failed: Error occured while executing the command file : C:/PROGRA~1/Altair/2020/HWDESK~1/common/python/python3.5/win64/Lib/SITE-P~1/alt/hst/eac/cmd/HSTUPD~1.PY. Line : 585. Error occured while executing the command file : Postprocessing_Python_File.PY. Line : ?. executeAlgo_c : Execution failed. Failed execution

 

The model is open only in hyperstudy and no other model flux is open during the hyperstudy iteration.

Now i reduce until default value the numerical memory, but the error is still persistent. Furthermore, there are cases where the error is not reported but the simulation of the run simply stop, as shown in file attached:

 

PS: the problem is persistent in different flux model with different post porcessing script and in different computer.

 

This is a big issue for us because we are spendig a huge amount of time in order to find this fails and restart the optimizations.

Thank you for your help.

 

Diana M._20503

Hi Mattia,

Thanks for following the suggestions. I understand your concern of the time spent. Let's see if some suggestions on Flux side and we'll continue investigating.

However, I believe that I will be needed to share the study so that we can try to reproduce the issue on our side and fine the cause.

Would it be possible to submit a support ticket and share there the study (not on the forum)?

Concerning the optimization restart, I guess that you use Inclusion matrix in order to recover already solved runs? I mean runs that have been solved properly are not lost. 

When you restart, does the same run that failed before is solved correctly?

 

Regards,

Diana

Diana M._20503

Mattia,

Something else: Could you check on your side if enough free space on the disk?

 

Diana M._20503

Hello Mattia,

I come back to you to know if you still run to this issue? Did you get input about the error message in Flux?

Regards,

Diana

Mattia Pinna

Hi Diana,

 

The problem is currently unsolved.
There is enough free space on the disk, so there is no problem of memery. Futhermore, when i restart the optimization, sometimes the previous failed simulations (runs) are solved correctly and sometimes an other one that solved correctly fails the execution , so we have a random duty of the solver.

How can i open a private ticket for Altair support?

 

Thank you for your time,

 

 

Mattia

Diana M._20503

Mattia Pinna said:

Hi Diana,

 

The problem is currently unsolved.
There is enough free space on the disk, so there is no problem of memery. Futhermore, when i restart the optimization, sometimes the previous failed simulations (runs) are solved correctly and sometimes an other one that solved correctly fails the execution , so we have a random duty of the solver.

How can i open a private ticket for Altair support?

 

Thank you for your time,

 

 

Mattia

Hi Mattia,

I am sorry to hear this.

Yes please do open a ticket in the Altair support. Probably it would be needed to organize a live support session on your machine.

In the meantime, it would be really helpful if you can upgrade the tools version and run your study with the latest release.

Regards,

Diana