Enrich Data by Web Service - how to parse the response

oleksandr_dykle

Hi,

 

I'm trying to perform following task in RapidMiner - request the data from a web-service, and then parse the response as a data set.

the response from the server is returned as XML and i can do it successfully.

This i was able to successfully do with Enrich Data by Webservice operation.

As a result, i have a table of form

Row No.    InputParameter   Response

1               1111111111          <here is my response in XML>

 

How can i parse now the response from this column and convert it to a data set?

I tried different operations, e.g. Process Documents from Data but it compains about "The example set must contain at least one text attribute".

So, how and where do i set this text attribute?

I tried to Google for some minimal example, but did not find.

Thomas_Ott

does your Xpath statement contain /text() ?  that will extrac the text and put it into an attribute column.  You can check out my example with Open Street Map data here: http://www.neuralmarkettrends.com/Extracting-OpenStreetMap-Data-In-RapidMiner/

oleksandr_dykle

Hi Thomas,

 

thank you for replying me! But, my problem is a bit different. In your example, you call web service for each input row, and then just append returned value, queried by XPath.

 

In my case, i have to make one call, retrieve XML which contains collection of objects, about 20. Now, i need to apply XPath to get these 20 rows, and i cannot do it. It just selects one element from collection and that's it.

My XML looks like:

<?xml version='1.0' encoding='UTF-8'?>
<ArrayOfCreditBureauData xmlns="http://microsoft.com/webservices/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <CreditBureauData>
        <id>1</id>
        <INCOME>70000</INCOME>
        <ts>2013-02-22T07:37:05</ts>
    </CreditBureauData>
    <CreditBureauData>
        <id>2</id>
        <INCOME>50000</INCOME>
        <ts>2014-02-22T07:37:05</ts>
    </CreditBureauData>
    <CreditBureauData>
        <id>3</id>
        <INCOME>60000</INCOME>
        <ts>2015-02-22T07:37:05</ts>
    </CreditBureauData>
</ArrayOfCreditBureauData>

So, i need to turn all the entries for CreditBureauData into data rows.

Thomas_Ott

Ok,

 

In that case try an Open File operator, set to the URL you want and connect it to a Read XML operator. You'll ping the XML file and then download it all and parse it with the Read XML opreator into an exampleset.

 

<?xml version="1.0" encoding="UTF-8"?><process version="7.3.001">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="7.3.001" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="open_file" compatibility="7.3.001" expanded="true" height="68" name="Open File" width="90" x="45" y="34">
        <parameter key="resource_type" value="URL"/>
        <parameter key="url" value="http://myURL.com"/>
      </operator>
      <operator activated="true" class="advanced_file_connectors:read_xml" compatibility="7.3.001" expanded="true" height="68" name="Read XML" width="90" x="246" y="34">
        <parameter key="file" value="C:\Users\ThomasOtt\Desktop\Credit.xml"/>
        <parameter key="xpath_for_examples" value="//default:ArrayOfCreditBureauData/default:CreditBureauData"/>
        <enumeration key="xpaths_for_attributes">
          <parameter key="xpath_for_attribute" value="default:id[1]/text()"/>
          <parameter key="xpath_for_attribute" value="default:INCOME[1]/text()"/>
          <parameter key="xpath_for_attribute" value="default:ts[1]/text()"/>
        </enumeration>
        <list key="namespaces">
          <parameter key="default" value="http://microsoft.com/webservices/"/>
          <parameter key="xsd" value="http://www.w3.org/2001/XMLSchema"/>
          <parameter key="xsi" value="http://www.w3.org/2001/XMLSchema-instance"/>
        </list>
        <parameter key="default_namespace" value="http://microsoft.com/webservices/"/>
        <list key="annotations"/>
        <list key="data_set_meta_data_information"/>
      </operator>
      <connect from_op="Open File" from_port="file" to_op="Read XML" to_port="file"/>
      <connect from_op="Read XML" from_port="output" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

oleksandr_dykle

But, i need to POST some parameters in a body to that URL in order to get the data.

This XML is a response from a web-service.

Open File does not support POSTing, does it?

Thomas_Ott

Get Page will work, it has a a POST parameter.