Yahoo Historical Stock Price Operator is not returing data

pradeep · April 2017

Yahoo Historical Data is returning empty values in rapidminer, is there any bug ?

I was able to download rawdat from yahoo finance website, but this operator is unable to do it.

Thomas_Ott · April 2017

Hi Pardeep.

Just do a search for how to do it on the Community. Here's a great response from Ingo: http://community.rapidminer.com/t5/RapidMiner-Studio-Forum/Import-XML-code-to-process/m-p/32606

qwertz2 · April 2017

Dear all,

In the current version you have to click on menu view -> show panel -> XML. Then you can paste the XML code in the new XML windows and confirm with the green checkmark.

Cheers

Sachs

keith_drake · April 2017

Yes, I noticed the same thing. And another app I use to download data from Yahoo (YLoader) won't work either -- it throws a "HTTP response error 301"

Anyone else?

keith_drake · April 2017

Yes, I noticed the same thing. And another app I use to download data from Yahoo (YLoader) won't work either -- it throws a "HTTP response error 301"

Anyone else?

Vayao · April 2017

I'm suffering this malfunction also. Please can someone help?

Thanks a lot

keith_drake · April 2017

I've confirmed with another software vendor the problem is with Yahoo changing their protocol: "The main fix in version 4.10 is for error 301 when downloading Yahoo data, which was due to a change in the Yahoo protocol."

The Yahoo Historical Stock Data operator is in the Finance and Economic Data extension provided by BroadReachAnalytics. Visiting their URL provided in the RM documentation (http://www.broadreachanalytics.com/) it appears they are no longer active.

RapidMiner: Is there any opportunity to update this operator? Thanks.

gglsbmt · April 2017

Hello my friend thankyou so much

Thomas_Ott · April 2017

Yes, I can confirm that this is no longer working and BroadReachAnalytics appears to be gone.

With RapidMiner, you can always find a work around and here it is.

<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="7.4.000" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="open_file" compatibility="7.4.000" expanded="true" height="68" name="Open File" width="90" x="45" y="34">
        <parameter key="resource_type" value="URL"/>
        <parameter key="url" value="http://www.google.com/finance/historical?q=NASDAQ:GOOGL&amp;amp;ei=u5z8WJGtC8ejeeqWgbgJ&amp;amp;output=csv"/>
      </operator>
      <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV" width="90" x="179" y="34">
        <parameter key="column_separators" value=","/>
        <list key="annotations"/>
        <list key="data_set_meta_data_information"/>
      </operator>
      <connect from_op="Open File" from_port="file" to_op="Read CSV" to_port="file"/>
      <connect from_op="Read CSV" from_port="output" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

sgenzer · April 2017

Hi...I have moved over to "openexchangerates.com" which has a nice API. I built a simple app for an accounting client using the API on RM server. Send me a PM if you want to see it.

Scott

qwertz2 · April 2017

Hi Thomas,

Thanks for that work around. I can confirm that it works pretty well with a single symbol. However, is there also a way to load multiple symbols at a time?

From your tutorials I know that it is possible to append several single symbols but in this case I want to download about 20. Searching the internet I found a site (LINK) which describes a select query but I couldn't manage to get this working in Rapidminer...

Kind regards

Sachs

pradeep · April 2017

Hello Sachs,

I am new for Rapidminer,

could you please help me out in using 'xml code' for Yahoo Historical Stock price operator, which has been provided Thomas,

using it in Rapidminer?

Thanks

pradeep · April 2017

Hello Thomas,

Thank you for providing the work around solution,

I am new for Rapidminer,

could you please help me out in using 'xml code' for Yahoo Historical Stock price operator, using it in Rapidminer?

Thanks

Pradeep.

keith_drake · April 2017

Hello Thomas ... thanks for the solution. How do I implement the XML script? I copied it into the Execute Script operator and got an error (see attached). Thanks.

Thomas_Ott · April 2017

Hi Sachs,

Piece of cake. Just make a text file with the header Ticker and then a new symbol on each line. Then use this process below. You should save them seperately in a repository by appending %{symbol}_stock

<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="7.4.000" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="open_file" compatibility="7.4.000" expanded="true" height="68" name="Open File" width="90" x="45" y="34">
        <parameter key="filename" value="C:\Users\ThomasOtt\Desktop\Ticker File.txt"/>
      </operator>
      <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV" width="90" x="179" y="34">
        <list key="annotations"/>
        <list key="data_set_meta_data_information"/>
      </operator>
      <operator activated="true" class="extract_macro" compatibility="7.4.000" expanded="true" height="68" name="Extract Macro (2)" width="90" x="313" y="34">
        <parameter key="macro" value="num"/>
        <list key="additional_macros"/>
      </operator>
      <operator activated="true" class="concurrency:loop" compatibility="7.4.000" expanded="true" height="82" name="Loop" width="90" x="514" y="34">
        <parameter key="number_of_iterations" value="%{num}"/>
        <process expanded="true">
          <operator activated="true" class="extract_macro" compatibility="7.4.000" expanded="true" height="68" name="Extract Macro" width="90" x="112" y="34">
            <parameter key="macro" value="symbol"/>
            <parameter key="macro_type" value="data_value"/>
            <parameter key="attribute_name" value="Ticker"/>
            <parameter key="example_index" value="%{iteration}"/>
            <list key="additional_macros"/>
          </operator>
          <operator activated="true" class="open_file" compatibility="7.4.000" expanded="true" height="68" name="Open File (2)" width="90" x="112" y="136">
            <parameter key="resource_type" value="URL"/>
            <parameter key="filename" value="http://www.google.com/finance/historical?q=NASDAQ:%{symbol}&amp;amp;ei=LRD-WJmhHIOrmgH_4KToAg&amp;amp;output=csv"/>
            <parameter key="url" value="http://www.google.com/finance/historical?q=NASDAQ:%{symbol}&amp;amp;ei=LRD-WJmhHIOrmgH_4KToAg&amp;amp;output=csv"/>
          </operator>
          <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV (2)" width="90" x="447" y="85">
            <parameter key="csv_file" value="http://www.google.com/finance/historical?q=NASDAQ:%{symbol}&amp;amp;ei=LRD-WJmhHIOrmgH_4KToAg&amp;amp;output=csv"/>
            <parameter key="column_separators" value=","/>
            <list key="annotations"/>
            <list key="data_set_meta_data_information"/>
          </operator>
          <connect from_port="input 1" to_op="Extract Macro" to_port="example set"/>
          <connect from_op="Open File (2)" from_port="file" to_op="Read CSV (2)" to_port="file"/>
          <connect from_op="Read CSV (2)" from_port="output" to_port="output 1"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="source_input 2" spacing="0"/>
          <portSpacing port="sink_output 1" spacing="0"/>
          <portSpacing port="sink_output 2" spacing="0"/>
        </process>
      </operator>
      <connect from_op="Open File" from_port="file" to_op="Read CSV" to_port="file"/>
      <connect from_op="Read CSV" from_port="output" to_op="Extract Macro (2)" to_port="example set"/>
      <connect from_op="Extract Macro (2)" from_port="example set" to_op="Loop" to_port="input 1"/>
      <connect from_op="Loop" from_port="output 1" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

Thomas_Ott · April 2017

Hi Pardeep.

Just do a search for how to do it on the Community. Here's a great response from Ingo: http://community.rapidminer.com/t5/RapidMiner-Studio-Forum/Import-XML-code-to-process/m-p/32606

qwertz2 · April 2017

Dear all,

In the current version you have to click on menu view -> show panel -> XML. Then you can paste the XML code in the new XML windows and confirm with the green checkmark.

Cheers

Sachs

pradeep · April 2017

Thank you Sachs.

pradeep · April 2017

Thank you @Thomas_Ott.

qwertz2 · April 2017

Hi Thomas,

This already looks very promising - and brings the advantage to control stock symbols in separate external file. Pretty cool!

Now I am struggling with two more problems:

1) The process provides a collection but I rather want to have each symbol in its own column in a single example set / table.

2) I tried to access the yahoo web api with the open file operator:

http://ichart.finance.yahoo.com/table.csv?s=AAPL&d=4&e=10&f=2016&g=d&a=11&b=12&c=2015&ignore=table.csv

But I couldn't manage to make the yahoo links work though they would do fine a web browser.

Lookling forward to getting some advice...

Cheers

Sachs

Thomas_Ott · April 2017

Yahoo did something to bork the whole thing recently, so I would default to Google right now.

With respect to putting it all in a table, I'd have to think about it but it would include using a Rename by Replacing so you can do AAPL_Close and XOM_Close and then something to merge them all together, maybe with a combination of a Remember and Recall operator but I need go AFK know.

Yahoo Historical Stock Price Operator is not returing data

Best Answers

Answers

Categories