Enrich Data by Webservice debugging

sgenzer
sgenzer
Altair Employee
edited November 2024 in Community Q&A
hi...is there any way to debug the Enrich Data by Webservice operator when it fails?  I would to see the JSONpath that it is sending out in a POST request, and the actually reply from the server with the error code.  I can do it in curl but not via the RapidMiner interface.

In a similar vein, is there a way to send a curl request directly from RM?

And by the way, when it fails on a GET request, the salmon-colored error box is enormous (due to the large URL) and runs off the left side of the screen.  It is almost impossible to click the "Got it" button.

Scott

Answers

  • BalazsBaranyRM
    BalazsBaranyRM New Altair Community Member
    Hi,
    sgenzer wrote:

    hi...is there any way to debug the Enrich Data by Webservice operator when it fails?  I would to see the JSONpath that it is sending out in a POST request, and the actually reply from the server with the error code.  I can do it in curl but not via the RapidMiner interface.
    If your web service is accessible over HTTP, you can easily dump the entire communication with Wireshark.
    If it needs HTTPS, you could try something like Fiddler <http://www.telerik.com/fiddler>.

    In a similar vein, is there a way to send a curl request directly from RM?
    You can use the Execute Program operator.
  • sgenzer
    sgenzer
    Altair Employee
    hmm thanks.  I was hoping to tap into the logging features in RM rather than something outside of it.

    ah Execute Program.  :)  I should have thought of that!  Duh. Thanks.  How do I pass the information into Execute Program so that I can use curl?  macros?

    Scott
  • JEdward
    JEdward New Altair Community Member
    It comes with handy output ports so you can use the result in RapidMiner. 

    See this example:
    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <process version="7.1.000-BETA">
      <context>
        <input/>
        <output/>
        <macros/>
      </context>
      <operator activated="true" class="process" compatibility="6.0.002" expanded="true" name="Process">
        <parameter key="logverbosity" value="init"/>
        <parameter key="random_seed" value="2001"/>
        <parameter key="send_mail" value="never"/>
        <parameter key="notification_email" value=""/>
        <parameter key="process_duration_for_mail" value="30"/>
        <parameter key="encoding" value="SYSTEM"/>
        <parameter key="parallelize_main_process" value="false"/>
        <process expanded="true">
          <operator activated="true" class="productivity:execute_program" compatibility="7.1.000-BETA" expanded="true" height="103" name="Execute Program" width="90" x="112" y="85">
            <parameter key="command" value="cmd /c java -version"/>
            <parameter key="log_stdout" value="true"/>
            <parameter key="log_stderr" value="true"/>
            <list key="env_variables"/>
          </operator>
          <operator activated="true" class="text:read_document" compatibility="7.0.000" expanded="true" height="68" name="Read Document (2)" width="90" x="380" y="136">
            <parameter key="extract_text_only" value="true"/>
            <parameter key="use_file_extension_as_type" value="true"/>
            <parameter key="content_type" value="txt"/>
            <parameter key="encoding" value="SYSTEM"/>
          </operator>
          <operator activated="true" class="text:read_document" compatibility="7.0.000" expanded="true" height="68" name="Read Document" width="90" x="380" y="34">
            <parameter key="extract_text_only" value="true"/>
            <parameter key="use_file_extension_as_type" value="true"/>
            <parameter key="content_type" value="txt"/>
            <parameter key="encoding" value="SYSTEM"/>
          </operator>
          <connect from_op="Execute Program" from_port="out" to_op="Read Document" to_port="file"/>
          <connect from_op="Execute Program" from_port="err" to_op="Read Document (2)" to_port="file"/>
          <connect from_op="Read Document (2)" from_port="output" to_port="result 2"/>
          <connect from_op="Read Document" from_port="output" to_port="result 1"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="sink_result 1" spacing="0"/>
          <portSpacing port="sink_result 2" spacing="0"/>
          <portSpacing port="sink_result 3" spacing="0"/>
        </process>
      </operator>
    </process>
  • sgenzer
    sgenzer
    Altair Employee
    ok yes that works.  Thank you.  It's still not solving my problem.  I am running this command:

    curl -X GET --header 'Accept: text/plain' --header 'accept: text/plain' 'https://watson-api-explorer.mybluemix.net/language-translation/api/v2/identify?text=je suis faim'

    if I run it in terminal, I get the correct response ("fr"), but in "Execute Program", I get an error message and it will not run.

    Scott

  • BalazsBaranyRM
    BalazsBaranyRM New Altair Community Member
    Hi!

    You can use "Read Document" to convert the standard output and standard error of your process into readable text.

    Example process:

    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <process version="7.0.001">
     <context>
       <input/>
       <output/>
       <macros/>
     </context>
     <operator activated="true" class="process" compatibility="7.0.001" expanded="true" name="Process">
       <process expanded="true">
         <operator activated="true" class="productivity:execute_program" compatibility="7.0.001" expanded="true" height="103" name="Execute Program" width="90" x="112" y="34">
           <parameter key="command" value="ls /tmp"/>
           <list key="env_variables"/>
         </operator>
         <operator activated="true" class="text:read_document" compatibility="6.5.000" expanded="true" height="68" name="Read stdout" width="90" x="380" y="34"/>
         <operator activated="true" class="text:read_document" compatibility="6.5.000" expanded="true" height="68" name="Read stderr" width="90" x="514" y="85"/>
         <connect from_op="Execute Program" from_port="out" to_op="Read stdout" to_port="file"/>
         <connect from_op="Execute Program" from_port="err" to_op="Read stderr" to_port="file"/>
         <connect from_op="Read stdout" from_port="output" to_port="result 1"/>
         <connect from_op="Read stderr" from_port="output" to_port="result 2"/>
         <portSpacing port="source_input 1" spacing="0"/>
         <portSpacing port="sink_result 1" spacing="0"/>
         <portSpacing port="sink_result 2" spacing="0"/>
         <portSpacing port="sink_result 3" spacing="0"/>
       </process>
     </operator>
    </process>
    Do you get the error message from RapidMiner or on the error output from curl? Do you need authentication for accessing the web service?

    With curl -v you can get verbose output that shows the details of the communication with the server.
  • sgenzer
    sgenzer
    Altair Employee
    thanks.  No I do not get any errors when I use curl in the Unix terminal shell.  Only in "Execute Program".  In fact I cannot even get that to work when using your example, finding the java version.