🎉Community Raffle - Win $25

An exclusive raffle opportunity for active members like you! Complete your profile, answer questions and get your first accepted badge to enter the raffle.

Join and Win

"Web mining operator"

17900713r

Hello ,

I am new to rapidminer and I am trying to use the cut document in the web mining operator.I would like to know how to construct a regular expression that will one select online reviews.You help will be much appreciated.I have been trying to follow the tutorials rapidminer suggested but they arent helping much.I am trying to scrap reviews from Amazon .

Find more posts tagged with

AI Studio

Web Mining

Getting Started

Accepted answers

All comments

sgenzer

hello @17900713r welcome to the community. I don't believe we actually have a go-to resource for how to create RegEx expressions, but there are many others online. I personally use https://regexr.com/.

Scott

17900713r

Thank you for your quick reply.May I please know where to locate my whole process code.I would like to paste it here.

17900713r

Below is my XML process.

<?xml version="1.0" encoding="UTF-8"?><process version="8.1.003">
  <operator activated="true" class="concurrency:loop" compatibility="8.1.003" expanded="true" height="82" name="Loop" width="90" x="45" y="34">
    <parameter key="number_of_iterations" value="5"/>
    <parameter key="iteration_macro" value="iteration"/>
    <parameter key="reuse_results" value="false"/>
    <parameter key="enable_parallel_execution" value="true"/>
    <process expanded="true">
      <operator activated="true" class="web:process_web_modern" compatibility="7.3.000" expanded="true" height="68" name="Process Documents from Web" width="90" x="112" y="85">
        <parameter key="url" value="https://www.rottentomatoes.com/m/the_dark_knight/reviews/?page=%{iteration}"/>
        <list key="crawling_rules"/>
        <parameter key="max_crawl_depth" value="1"/>
        <parameter key="retrieve_as_html" value="false"/>
        <parameter key="enable_basic_auth" value="false"/>
        <parameter key="add_content_as_attribute" value="false"/>
        <parameter key="max_pages" value="4"/>
        <parameter key="max_page_size" value="1000"/>
        <parameter key="delay" value="200"/>
        <parameter key="max_concurrent_connections" value="100"/>
        <parameter key="max_connections_per_host" value="100"/>
        <parameter key="user_agent" value="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"/>
        <parameter key="ignore_robot_exclusion" value="true"/>
        <process expanded="true">
          <operator activated="true" class="text:cut_document" compatibility="8.1.000" expanded="true" height="68" name="Cut Document" width="90" x="179" y="34">
            <parameter key="query_type" value="XPath"/>
            <list key="string_machting_queries"/>
            <parameter key="attribute_type" value="Nominal"/>
            <list key="regular_expression_queries"/>
            <list key="regular_region_queries"/>
            <list key="xpath_queries">
              <parameter key="seg" value="//h:table[@class='table table-striped']/h:tr"/>
            </list>
            <list key="namespaces"/>
            <parameter key="ignore_CDATA" value="true"/>
            <parameter key="assume_html" value="true"/>
            <list key="index_queries"/>
            <list key="jsonpath_queries"/>
            <process expanded="true">
              <connect from_port="segment" to_port="document 1"/>
              <portSpacing port="source_segment" spacing="0"/>
              <portSpacing port="sink_document 1" spacing="0"/>
              <portSpacing port="sink_document 2" spacing="0"/>
            </process>
          </operator>
          <operator activated="true" class="text:extract_information" compatibility="8.1.000" expanded="true" height="68" name="Extract Information" width="90" x="313" y="34">
            <parameter key="query_type" value="XPath"/>
            <list key="string_machting_queries"/>
            <parameter key="attribute_type" value="Nominal"/>
            <list key="regular_expression_queries"/>
            <list key="regular_region_queries"/>
            <list key="xpath_queries">
              <parameter key="text" value="//h:p/text"/>
            </list>
            <list key="namespaces"/>
            <parameter key="ignore_CDATA" value="true"/>
            <parameter key="assume_html" value="true"/>
            <list key="index_queries"/>
            <list key="jsonpath_queries"/>
          </operator>
          <connect from_port="document" to_op="Cut Document" to_port="document"/>
          <connect from_op="Cut Document" from_port="documents" to_op="Extract Information" to_port="document"/>
          <connect from_op="Extract Information" from_port="document" to_port="document 1"/>
          <portSpacing port="source_document" spacing="0"/>
          <portSpacing port="sink_document 1" spacing="0"/>
          <portSpacing port="sink_document 2" spacing="0"/>
        </process>
      </operator>
      <connect from_op="Process Documents from Web" from_port="example set" to_port="output 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_output 1" spacing="0"/>
      <portSpacing port="sink_output 2" spacing="0"/>
    </process>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="8.1.003">
  <operator activated="true" class="append" compatibility="8.1.003" expanded="true" height="82" name="Append" width="90" x="179" y="34">
    <parameter key="datamanagement" value="double_array"/>
    <parameter key="data_management" value="auto"/>
    <parameter key="merge_type" value="all"/>
  </operator>
</process>

sgenzer

hello @17900713r - so in RapidMiner your XML is your process "code".

Scott