Error 37389: DGFM

Eliz
Eliz Altair Community Member
edited October 2020 in Community Q&A

Error 37389: DGFM is only supported when solving the linear equation system in main memory

I keep getting this error when trying to use DGFM to solve a Vivaldi array. The array has only 2 elements but it seems as thought it is very resource intensive. What exactly does this error mean? What can I do to fix it? Does it have anything to do with the other problem of trying to use DGFM in conjunction with SEP for dielectric regions or is it a separate problem altogether? Is there a better way to simulate a finite array?

Tagged:

Answers

  • Mel
    Mel Altair Community Member
    edited August 2019

    Hi Eliz


    The DGFM does not support out-of-core solving like a default Method of Moments solution would.

     

    Please attach an image of your model, or if you can, the cfx for further assistance.

  • Eliz
    Eliz Altair Community Member
    edited August 2019

    Good day Mel

     

    Thank you for your reply. I have attached my model - it is a 2-element Vivaldi array with weighted ports so as to achieve a specific polarization angle. I would appreciate any further suggestions to make this simulation more memory and time efficient. 

    Unable to find an attachment - read this blog

  • Mel
    Mel Altair Community Member
    edited August 2019

    Hi Eliz

     

    With 2019 using the DGFM it will take 133 GByte. With a full solution (DGFM off) this will take 330 GByte.
    The default solver for dielectrics, Surface Equivalence, is computationally expensive when such fine meshing is used.

     

    A FEM solution would be much more efficient. The concept is demonstrated in the 2 videos on altairuniversity.com at:
    https://altairuniversity.com/learning-library/?type=learninglibraryitem&filter_resource_type=Training+Videos&filter_discipline=Electromagnetic&pg=3&search&filter_language&filter_source
    Note there are 2 parts, as shown in the image. (Not sure why the 2nd part is being listed twice).

    <?xml version="1.0" encoding="UTF-8"?>videos.png

     

    Instead of edge ports you would use a FEM line port with a FEM current source.

     

    Also, since the meshing is quite fine, the basis functions for the FEM should be set to first order:

    fem_first_order.png

    I estimate a few Gigabytes for this solution.

  • Eliz
    Eliz Altair Community Member
    edited August 2019

    Thank you so much for you help! I will investigate your solution and come back to you. I tried using a FEM once before, but granted, I still used some form of an edge port, but got an error saying FEM cannot be used with finite arrays? Won't this be a problem?

  • Mel
    Mel Altair Community Member
    edited August 2019

    For a few elements, you don't have to use a finite array and it would still be quite cheap to solve. Just use a normal copy/translate and modify operation.

    But yes, finite arrays and Fem are not supported.

  • Eliz
    Eliz Altair Community Member
    edited August 2019

    Good afternoon. 

     

    I have several questions regarding the FEM implementation. I was able to follow the video's instructions for a single antenna element (just to check) and got similar results to when I used other methods - and it is so much faster - thank you! I do have some questions:

    1. If I want to use FEM for a two element array - do I have to place the air dielectric around both of them together (in other words they wont each have their own air dielectric box, but share one)?

    2. Will this method take mutual coupling into account? 

     

    Furthermore, after some consideration I wondered whether I could simulate the Vivaldi without the feed line, thereby eliminating much of the fine mesh required to mesh the feed line. In other words, could I feed the Vivaldi with a modal port at the end of the taper (as with the waveguide port in CST) or could I feed the Vivaldi with a line port spanning the start of the taper. I attempted the former in the attached file called Viv_pol_test2_3_FEM_v2 and the latter in  Viv_pol_test2_3_FEM_v3. However, I got errors for both cases. For the modal port I received and error that stated 'A modal port must include dielectric areas in order to support wave tranmission' - which I thought I was doing, since it is on a face of the dielectric region. For the line port I received 'Error 19088: Geometry port: Portx - The edge Feedx.Wire2595 must be on the boundary of a dielectric or embedded in a dielectric region' (screenshot attached). Which once, again I thought I was doing anyway? 

     

    I would appreciate any advice!

    <?xml version="1.0" encoding="UTF-8"?>FEM_line_port_error.PNG

    Unable to find an attachment - read this blog

  • Mel
    Mel Altair Community Member
    edited August 2019

    Hi Eliz

     

    1. The idea is to add as little as possible 'air' tetrahedra to the model while still 'absorbing' the fine elements inside the added tetrahedra. It will always depend on the model. With the original model and spacing required I would create two identical elements each with it's own 'air' dielectric. For example:

    2elem_vivaldi_array_air_concept.png

    2. FEM/MoM and FEM/MLFMM is essentially full wave solutions, so all coupling effects are taken into account.

     

    In the v2 and v3 models you attached I don't see the feed lines you had in the original models. I cannot comment on the design/thinking behind the antenna. The modal port is not suitable for 'microstrip' type feeds. Attached an example from the original model where I added one line port only. No air has been added.

    Unable to find an attachment - read this blog

  • Eliz
    Eliz Altair Community Member
    edited August 2019

    Good afternoon

     

    Thank you for all your advice! My simulations are so much faster and use a lot less memory!

     

    I wanted to obtain the S-parameters for this array, but got an error that stated 'Error 37374: S-parameter calculation currently not allowed for DGFM'. Obviously if I switch off DGFM, it will once again take quite long and use a lot of memory and the FEM line port is not as accurate as the edge ports for S-parameter calculations. Do you perhaps have any advice on how to efficiently obtain the S-parameters for this array? Also, when using a finite array, how do you specify that you want the S-parameters for the other element's ports as well - it only give an option for port 1 and port 2?

  • Eliz
    Eliz Altair Community Member
    edited August 2019

    Good afternoon 

     

    I tried switching off the DGFM and switch on the MLFMM solver and then tried to obtain the S-parameters. However, after a while it failed as well with 'Error 36772: Not enough memory available for dynamic allocation'. I have attached the cfx model file as well as the out file. I would appreciate any advice. 

    Unable to find an attachment - read this blog

  • Mel
    Mel Altair Community Member
    edited August 2019

    Hi Eliz

     

    >> FEM line port is not as accurate as the edge ports for S-parameter calculations

    Did you solve the S-parameters with edge ports?

     

    If you really want to use an edge port, it has to be embedded inside a free space region (which in turn is embedded inside the FEM). See attached example of a finite size patch with edge port and solved with FEM. Again, the surrounding air is only really necessary (to save resources) if fine meshing exists on the outer surfaces of the model.

    <?xml version="1.0" encoding="UTF-8"?>finite_patch.png

     

    Finite arrays don't give access to the copied elements' ports for S-parameter calculations.

     

    MLFMM is inefficient if the mesh contains areas with fine meshing.

    The best solution for S-parameters for your model is FEM with line ports (or edge ports if you really prefer) and without using DGFM or finite arrays, shown in the image I posted on the 16th.

     

     

    Unable to find an attachment - read this blog

  • Eliz
    Eliz Altair Community Member
    edited August 2019

    Good morning Mel

     

    I read about the accuracy of the FEM line port in the user manual. Under current sources it says 'Note: An intrinsic limitation of the impressed current source is that no radius is considered.
    The field is singular in the vicinity of the filament affecting the accuracy of the computed input impedance of the source.' I managed to simulate the s-parameter of a 2 element finite array with MLFMM (I ran the simulation before you mentioned that MLFMM might not be useful if you have a fine surface mesh) (this is Viv_2el_sparam_v2) and I simulated the s-parameters with FEM (please see attached - Viv_2el_sparam_v4) and the S-parameters differed quite a bit.

     

    I also tried your method with the SEP surrounding an edge port, but I got an error message that stated: 'Same solution method must be applied to all dielectric regions in the model', which I find confusing (this is Viv_2el_sparam_v5)? Do you perhaps know why I would get this error message?

    Unable to find an attachment - read this blog

  • Eliz
    Eliz Altair Community Member
    edited August 2019

    Good morning Mel

     

    I found my error in Viv_2el_sparam_v5 - the SEP region around the edge port was supposed to be free space and not air. However, the S-parameter results between the 3 different methods differ considerably. I would've thought Viv_2el_sparam_v2 (without FEM, with SEP) would be more similar to Viv_2el_sparam_v5 (with FEM, except for small SEP box around edge port)? How do I know which ones are more correct? Should I maybe apply local meshing to the edge port in V5? Please see attacehed the V5 output file with S-parameters at 3 GHz.

    Unable to find an attachment - read this blog

  • Mel
    Mel Altair Community Member
    edited August 2019

    Hi Eliz

     

    Regarding, 'Note: An intrinsic limitation of the impressed current source is that no radius is considered. The field is singular in the vicinity of the filament affecting the accuracy of the computed input impedance of the source'

    The effect of this limitation will depend on the model. If the line port is meant to represent a feed pin of specific thickness (e.g. the center pin of an SMA connector of 1.3 mm thickness) and the substrate is quite thick, then the impedance modeled will be different from using a wire port with the radius set correctly, or rather an exactly constructed connector for example (such as constructed in the video).

    An edge port could be made to mimic the feed pin thickness by making the edge port width twice the diameter of the feed pin. 

     

    Also note that a mesh setting that works OK for one method would not necessarily be sufficient for another. (FEM vs SEP). It is also good practice to use fine meshing around the port.

     

    You could consider to reduce your model to just a single vivaldi and only modelling a section of the feed line with two ports to get the feed right.

    The attached files shows the edge port, line port and FEM modal port construction for a microstrip line.

    <?xml version="1.0" encoding="UTF-8"?>sparameters_50Ohm_line.png

     

    The FEM modal port is intended for use in shielded boundaries, such as connectors. A workaround is to use a very large boundary in the unshielded directions of the port face, but this adds additional mesh elements to the model increasing the required computational resources.

     

     

     

     

     

     

     

     

     

    Unable to find an attachment - read this blog

  • Eliz
    Eliz Altair Community Member
    edited August 2019

    Good afternoon Mel

     

    I have tried to obtain the S-parameters for a single Vivaldi antenna over 2-6 GHz (which is the main range I am looking at). I have two questions:

     

    1. Whenever I simulate the antenna and try the FEM on a region, I get warnings and or errors - all of these warnings and or errors deal with the convergence of the solution. I have noticed that the warnings or errors tend to happen when higher frequencies (above 5GHz) are being processed. Some of the errors and warnings I have received are: ' WARNING   830: Maximum number of iterations reached without convergence, using in the following the solution with the smallest residuum'; ' ERROR   33498: Maximum number of iterations reached without convergence'; ' NOTE     4973: Iterative solution of the system of linear equations not sufficiently converged at the stopping residuum, continuing with the iterations'; ' ERROR    4673: Iterative solution of the system of linear equations failed, maybe try another preconditioner (solution settings)'; ' WARNING 36002: No convergence achieved during the iterative solution (residuum diverges, larger than maximum value)'. I have attached three of my attempts for which I have all received one or more of the beforementioned errors, notes or warnings. I think I might have to change the preconditioner, but am not sure what that does and what the best choice would be, or maybe change the mesh? In one of my experiments, I increased the size of my airbox around my antenna and some of the warnings went away (when I simulated from 2-4GHz), but the warnings returned when I tried to obtain the parameters for 2-6 GHz. Any advice would be appreciated. 

     

    2. You mentioned that the modal port has to be 'very large'. I know in CST the waveguide port should be between 5-10 times the thickness of the dielectric wider on each side and higher than the dielectric. Is this rule applicable in FEKO as well? How do I know it is large enough?

     

    Thank you for all your help thus far! It is very much appreciated!

    Unable to find an attachment - read this blog

  • Mel
    Mel Altair Community Member
    edited August 2019

    Hi Eliz

     

    Please use first order, mentioned previously.

  • Eliz
    Eliz Altair Community Member
    edited September 2019

    Good morning Mel!

     

    Thank you for all your help so far. When I used the first order FEM, the simulation results varied quite a bit from the measure results, so I ended up switching on double precision, that also helped. However, I tested this with a single Vivaldi. When I return to 2 antennas to form a dual-polarized vivaldi, I am once again facing convergence errors. I would like to avoid the first order if possible, since it is less accurate and does not agree well with the measured S-parameters. Which is why I now want to decrease the number of mesh elements, to see whether I can still have an accurate simulation without the convergence issues. Which leads to my meshing and the SEP edge port with the FEM model ( please see file attached). Viv_FEM_air_SEP_edge_V26 has an accurate result in comparison with the manufactured antenna's S-parameters, however, when I try and decrease the meshing in any way (as in Viv_FEM_air_SEP_edge_V37 for example increasing the growth rate), a spike randomly appears around 4.125 GHz in my S-parameter results. So now I am wondering whether I am doing something wrong with my meshing? Or whether there is an alternative approach that will avoid this spike from appearing while using less mesh elements? Or even better,  whether I can in some away avoid the convergence error (see Viv_FEM_air_SEP_edge_2el_sp_v1) but still obtain accurate results (therefore avoiding first order)? 

     

    Your help will be much appreciated.

     

    Eliz

    Unable to find an attachment - read this blog

  • Mel
    Mel Altair Community Member
    edited September 2019

    Please attach the result files with saved POSTFEKO session file (pfs) that will show the 'spike' in the results.


    Remember if you change the order to first, you must create the mesh again, save, and then run Feko.

  • Eliz
    Eliz Altair Community Member
    edited September 2019

    Good morning Mel

     

    I have attached the pfs file. Thank you. 

     

    *Edit: I have included version 42 in the pfs as well - v42 is just v26 but with first order FEM. To illustrate the great difference in results. 

     

    Unable to find an attachment - read this blog

  • Mel
    Mel Altair Community Member
    edited September 2019

    For 'Viv_FEM_air_SEP_edge_2el_sp_v1' I have enabled MLFMM so that this is now a MLFMM/FEM solution instead of FEM/MoM and there were no convergence errors with 2019.1

    It was 14 GByte and a few hours.

     

    Using the direct sparse solver (for the FEM/MoM) will avoid any convergence problems. This is more memory hungry though and takes about 25 GByte but it depends on the number of processes.

  • Eliz
    Eliz Altair Community Member
    edited September 2019

    Good afternoon Mel 

     

    Thank you very much for your advice. I will investigate it. I have a question about the number of processes you mentioned though - I have always assumed that the more processes the better. For example, if I run a model with 12 processes or 6 processes, the former will run faster? Was I incorrect to assume this? Or how does the number of processes relate to memory and speed of the solution? 

  • Eliz
    Eliz Altair Community Member
    edited September 2019

    Good afternoon

     

    In addition to my previous question about processes, I ran the Viv_FEM_air_SEP_edge_2el_sp_v1 with MLFMM. I did not get any errors and the results correspond well to the measured results, but the .out file stated that a total of 35 GByte was required in the end, but you mentioned 14 Gb (which is significantly less). I have attached the model and the .out file. I would ideally like to determine the S-parameters of a four or nine element array of dual-polarized Vivaldi elements. Which is why the memory requirements are so important to me (I have a computer with 128 Gb memory and 16 cores to my disposal).

    Unable to find an attachment - read this blog

  • Mel
    Mel Altair Community Member
    edited September 2019

    The memory is dependent on the number of processes since not all aspects of the solution can be shared between processes.

    More cores will be faster, but it will use more memory.

     

    If the 4/9 array does not fit in to memory, reduce the number of processes. 

     

    Else, for more computational resources, I believe your institution has licenses/agreements with CHPC. Contact your study leader regarding this option.

     

  • Eliz
    Eliz Altair Community Member
    edited September 2019

    Good morning Mel

     

    Thank you for the reply and all the advice!

  • Mel
    Mel Altair Community Member
    edited September 2019

    Another option for computational resources is the Hyperworks Unlimited Virtual Appliance.

     

    A free trial is available for 48 hours. The trial starts when you log in with the password emailed to you after registering and clicking the 'Activate' button.

    Two compute nodes of 220 GByte and 16 processes each are available for trial users, so a total of 32 processes can be run using then 440 GByte in total.

    hwul.png

     

     

  • Eliz
    Eliz Altair Community Member
    edited September 2019

    Excellent! Thank you for this information!

  • Eliz
    Eliz Altair Community Member
    edited September 2019

    Good evening Mel

     

    Is there a way to determine how much memory my simulation will require without actually running the simulation and checking the .out file? I read about --estimate-resource-requirements-only that seems to be what I can use, however I am unclear of how to set up the machines file and what is meant by nodes? For example I wanted to run a Vivaldi array of 13 elements but got an error that stated that there was not enough memory, but not how much memory will be required? Please see the file I am referring to attached. I would appreciate your assistance.

    Unable to find an attachment - read this blog

  • Mel
    Mel Altair Community Member
    edited September 2019

    Unfortunately estimates for FEM/MoM is not yet available.

    You could try to half the number of vivaldi elements and see how much memory that takes. Then doubling them would roughly more than quadruple the required memory.

     

    Alternately enable MLFMM to use a FEM/MLFMM solution.