🎉Community Raffle - Win $25

An exclusive raffle opportunity for active members like you! Complete your profile, answer questions and get your first accepted badge to enter the raffle.
Join and Win

Crawl web not fetching any data from IMDB

User: "kartikss7"
New Altair Community Member
Updated by Jocelyn
Hello All,
I am using crawl web operation to fetch reviews of a specific movie in rapid miner. For this I am using a Loop Operator in the main process and inside loop am using macro and crawl web.

Loop -
Iterations: 74 (as 74 pages of reviews are there for http://www.imdb.com/title/tt0454876/reviews?start=1)
Limit Time: Checked
Timeout: 60

Inside loop operatior, I have generate macro and log from input to output and in parallel crawl web connected to output.

Generate Macro -
Function Description: macro name - pagePos and functions expression - %{pagePos} + 10 (I have initialized this macro)

Crawl Web -
URL: http://www.imdb.com/title/tt0454876/reviews?start=%{pagePos}
Crawling rules -
follow_link_with_matching_url - .+reviews.+
store with matching_url - .+reviews.+
write pages into file: checked
add pages as attribute: checked


image


When I run this process I get no observations in output. Please help me.

Find more posts tagged with

Sort by:
1 - 1 of 11