Export CSV create scientific notation for very small reals

f_laperna
f_laperna New Altair Community Member
edited November 5 in Community Q&A

Hi, I'm having this issue with RM when trying to export a dataset to a csv file. I have two columns with real numbers which can be very small and I noticed in the .csv file the scientific notation is used.

How can I turn it off and write the full number with all the decimal places? I checked the "Write CSV" node's documentation but didn't find anything about it. 

 

Thanks in advance.

Tagged:

Answers

  • SGolbert
    SGolbert New Altair Community Member

    Hi,

     

    I don't think it is possible with the Write CSV Operator. I think it is a rare use case.

     

    If you absolutely need to have this format, you will have to use one of the scripting opereators (R, Python, Groovy). As far as I know, it is not so simple in R either, you would have to change the default options for numerical conversion (with options(scipen = 15), meaning that it will write with fixed point notation up to 15 zeros).

     

    Regards,

    Sebastian

  • lionelderkrikor
    lionelderkrikor New Altair Community Member

    Hi @f_laperna,

     

    Here you can find an example of process (to set according your data), using Python, to write the full number with all the decimal places in a .csv file.

    <?xml version="1.0" encoding="UTF-8"?><process version="8.1.000">
      <context>
        <input/>
        <output/>
        <macros/>
      </context>
      <operator activated="true" class="process" compatibility="8.1.000" expanded="true" name="Process">
        <process expanded="true">
          <operator activated="true" class="generate_data_user_specification" compatibility="8.1.000" expanded="true" height="68" name="Generate Data by User Specification" width="90" x="112" y="85">
            <list key="attribute_values">
              <parameter key="a" value="0.000000000015"/>
              <parameter key="b" value="0.00000000000000000016"/>
            </list>
            <list key="set_additional_roles"/>
          </operator>
          <operator activated="true" class="python_scripting:execute_python" compatibility="7.4.000" expanded="true" height="82" name="Execute Python" width="90" x="313" y="85">
            <parameter key="script" value="import pandas as pd&#10;from decimal import Decimal&#10;import csv&#10;&#10;# rm_main is a mandatory function, &#10;# the number of arguments has to be the number of input ports (can be none)&#10;def rm_main(data):&#10;&#10;  file = 'C:/Users/Lionel/Documents/Formations_DataScience/Rapidminer/Tests_Rapidminer/Precision_Python/test_precision_Python.csv'&#10;  precision = '%.20f'&#10;  &#10;  pd.set_option('precision',20) &#10;  data = pd.DataFrame(data)&#10;  &#10;  data.to_csv(path_or_buf =file ,float_format = precision)&#10;   &#10;  &#10;&#10;    &#10;&#10;    # connect 2 output ports to see the results&#10;  return data"/>
          </operator>
          <connect from_op="Generate Data by User Specification" from_port="output" to_op="Execute Python" to_port="input 1"/>
          <connect from_op="Execute Python" from_port="output 1" to_port="result 1"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="sink_result 1" spacing="0"/>
          <portSpacing port="sink_result 2" spacing="0"/>
        </process>
      </operator>
    </process>

    I hope it helps,

     

    Best regards,

     

    Lionel

     

Categories