Yahoo Historical Stock Price Operator is not returing data

User: "pradeep"
pradeep
New Altair Community Member
Updated by Jocelyn

Yahoo Historical Data is returning empty values in rapidminer, is there any bug ?

I was able to download rawdat from yahoo finance website, but this operator is unable to do it.

Find more posts tagged with

AI Studio
Finance
Sort by:
1 - 18 of 181
    User: "keith_drake"
    keith_drake
    New Altair Community Member

    Yes, I noticed the same thing. And another app I use to download data from Yahoo (YLoader) won't work either -- it throws a "HTTP response error 301"

    Anyone else?

    User: "keith_drake"
    keith_drake
    New Altair Community Member

    Yes, I noticed the same thing. And another app I use to download data from Yahoo (YLoader) won't work either -- it throws a "HTTP response error 301"

    Anyone else?

    User: "Vayao"
    Vayao
    New Altair Community Member

    I'm suffering this malfunction also. Please can someone help?

    Thanks a lot

     

    User: "keith_drake"
    keith_drake
    New Altair Community Member
    I've confirmed with another software vendor the problem is with Yahoo changing their protocol: "The main fix in version 4.10 is for error 301 when downloading Yahoo data, which was due to a change in the Yahoo protocol."

    The Yahoo Historical Stock Data operator is in the Finance and Economic Data extension provided by BroadReachAnalytics. Visiting their URL provided in the RM documentation (http://www.broadreachanalytics.com/) it appears they are no longer active.

    RapidMiner: Is there any opportunity to update this operator? Thanks.
    User: "gglsbmt"
    gglsbmt
    New Altair Community Member

    Hello my friend thankyou so much

    User: "Thomas_Ott"
    Thomas_Ott
    New Altair Community Member

    Yes, I can confirm that this is no longer working and BroadReachAnalytics appears to be gone.

     

    With RapidMiner, you can always find a work around and here it is. 

     

    <?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
      <context>
        <input/>
        <output/>
        <macros/>
      </context>
      <operator activated="true" class="process" compatibility="7.4.000" expanded="true" name="Process">
        <process expanded="true">
          <operator activated="true" class="open_file" compatibility="7.4.000" expanded="true" height="68" name="Open File" width="90" x="45" y="34">
            <parameter key="resource_type" value="URL"/>
            <parameter key="url" value="http://www.google.com/finance/historical?q=NASDAQ:GOOGL&amp;amp;ei=u5z8WJGtC8ejeeqWgbgJ&amp;amp;output=csv"/>
          </operator>
          <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV" width="90" x="179" y="34">
            <parameter key="column_separators" value=","/>
            <list key="annotations"/>
            <list key="data_set_meta_data_information"/>
          </operator>
          <connect from_op="Open File" from_port="file" to_op="Read CSV" to_port="file"/>
          <connect from_op="Read CSV" from_port="output" to_port="result 1"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="sink_result 1" spacing="0"/>
          <portSpacing port="sink_result 2" spacing="0"/>
        </process>
      </operator>
    </process>
    User: "sgenzer"
    sgenzer
    Altair Employee
    Hi...I have moved over to "openexchangerates.com" which has a nice API. I built a simple app for an accounting client using the API on RM server. Send me a PM if you want to see it.

    Scott
    User: "qwertz2"
    qwertz2
    New Altair Community Member

     

    Hi Thomas,

     

    Thanks for that work around. I can confirm that it works pretty well with a single symbol. However, is there also a way to load multiple symbols at a time?

     

    From your tutorials I know that it is possible to append several single symbols but in this case I want to download about 20. Searching the internet I found a site (LINK) which describes a select query but I couldn't manage to get this working in Rapidminer... :(

     

     

    Kind regards

    Sachs

    User: "pradeep"
    pradeep
    New Altair Community Member
    OP

    Hello Sachs,

     

     

    I am new for Rapidminer,

    could you please help me out in using 'xml code' for Yahoo Historical Stock price operator, which has been provided Thomas,

    using it in Rapidminer?

     

    Thanks

    User: "pradeep"
    pradeep
    New Altair Community Member
    OP

    Hello Thomas,

     

     

    Thank you for providing the work around solution,

     

     

    I am new for Rapidminer,

    could you please help me out in using 'xml code' for Yahoo Historical Stock price operator, using it in Rapidminer?

     

    Thanks

    Pradeep.

    User: "keith_drake"
    keith_drake
    New Altair Community Member
    Hello Thomas ... thanks for the solution. How do I implement the XML script? I copied it into the Execute Script operator and got an error (see attached). Thanks.
    Google stock data download script 20170423.JPG
    User: "Thomas_Ott"
    Thomas_Ott
    New Altair Community Member

    Hi Sachs,

     

    Piece of cake. Just make a text file with the header Ticker and then a new symbol on each line. Then use this process below. You should save them seperately in a repository by appending %{symbol}_stock

     

    <?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
      <context>
        <input/>
        <output/>
        <macros/>
      </context>
      <operator activated="true" class="process" compatibility="7.4.000" expanded="true" name="Process">
        <process expanded="true">
          <operator activated="true" class="open_file" compatibility="7.4.000" expanded="true" height="68" name="Open File" width="90" x="45" y="34">
            <parameter key="filename" value="C:\Users\ThomasOtt\Desktop\Ticker File.txt"/>
          </operator>
          <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV" width="90" x="179" y="34">
            <list key="annotations"/>
            <list key="data_set_meta_data_information"/>
          </operator>
          <operator activated="true" class="extract_macro" compatibility="7.4.000" expanded="true" height="68" name="Extract Macro (2)" width="90" x="313" y="34">
            <parameter key="macro" value="num"/>
            <list key="additional_macros"/>
          </operator>
          <operator activated="true" class="concurrency:loop" compatibility="7.4.000" expanded="true" height="82" name="Loop" width="90" x="514" y="34">
            <parameter key="number_of_iterations" value="%{num}"/>
            <process expanded="true">
              <operator activated="true" class="extract_macro" compatibility="7.4.000" expanded="true" height="68" name="Extract Macro" width="90" x="112" y="34">
                <parameter key="macro" value="symbol"/>
                <parameter key="macro_type" value="data_value"/>
                <parameter key="attribute_name" value="Ticker"/>
                <parameter key="example_index" value="%{iteration}"/>
                <list key="additional_macros"/>
              </operator>
              <operator activated="true" class="open_file" compatibility="7.4.000" expanded="true" height="68" name="Open File (2)" width="90" x="112" y="136">
                <parameter key="resource_type" value="URL"/>
                <parameter key="filename" value="http://www.google.com/finance/historical?q=NASDAQ:%{symbol}&amp;amp;ei=LRD-WJmhHIOrmgH_4KToAg&amp;amp;output=csv"/>
                <parameter key="url" value="http://www.google.com/finance/historical?q=NASDAQ:%{symbol}&amp;amp;ei=LRD-WJmhHIOrmgH_4KToAg&amp;amp;output=csv"/>
              </operator>
              <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV (2)" width="90" x="447" y="85">
                <parameter key="csv_file" value="http://www.google.com/finance/historical?q=NASDAQ:%{symbol}&amp;amp;ei=LRD-WJmhHIOrmgH_4KToAg&amp;amp;output=csv"/>
                <parameter key="column_separators" value=","/>
                <list key="annotations"/>
                <list key="data_set_meta_data_information"/>
              </operator>
              <connect from_port="input 1" to_op="Extract Macro" to_port="example set"/>
              <connect from_op="Open File (2)" from_port="file" to_op="Read CSV (2)" to_port="file"/>
              <connect from_op="Read CSV (2)" from_port="output" to_port="output 1"/>
              <portSpacing port="source_input 1" spacing="0"/>
              <portSpacing port="source_input 2" spacing="0"/>
              <portSpacing port="sink_output 1" spacing="0"/>
              <portSpacing port="sink_output 2" spacing="0"/>
            </process>
          </operator>
          <connect from_op="Open File" from_port="file" to_op="Read CSV" to_port="file"/>
          <connect from_op="Read CSV" from_port="output" to_op="Extract Macro (2)" to_port="example set"/>
          <connect from_op="Extract Macro (2)" from_port="example set" to_op="Loop" to_port="input 1"/>
          <connect from_op="Loop" from_port="output 1" to_port="result 1"/>
          <portSpacing port="source_input 1" spacing="0"/>
          <portSpacing port="sink_result 1" spacing="0"/>
          <portSpacing port="sink_result 2" spacing="0"/>
        </process>
      </operator>
    </process>
    User: "Thomas_Ott"
    Thomas_Ott
    New Altair Community Member
    Accepted Answer

    Hi Pardeep. 

     

    Just do a search for how to do it on the Community. Here's a great response from Ingo: http://community.rapidminer.com/t5/RapidMiner-Studio-Forum/Import-XML-code-to-process/m-p/32606

    User: "qwertz2"
    qwertz2
    New Altair Community Member
    Accepted Answer

     

    Dear all,

     

    In the current version you have to click on menu view -> show panel -> XML. Then you can paste the XML code in the new XML windows and confirm with the green checkmark.

     

    Cheers

    Sachs

    User: "pradeep"
    pradeep
    New Altair Community Member
    OP

    Thank you Sachs.

    User: "pradeep"
    pradeep
    New Altair Community Member
    OP

    Thank you @Thomas_Ott.

    User: "qwertz2"
    qwertz2
    New Altair Community Member

     

    Hi Thomas,

     

    This already looks very promising - and brings the advantage to control stock symbols in separate external file. Pretty cool!

     

    Now I am struggling with two more problems:

    1) The process provides a collection but I rather want to have each symbol in its own column in a single example set / table.

    2) I tried to access the yahoo web api with the open file operator:

    http://ichart.finance.yahoo.com/table.csv?s=AAPL&d=4&e=10&f=2016&g=d&a=11&b=12&c=2015&ignore=table.csv

    But I couldn't manage to make the yahoo links work though they would do fine a web browser.

     

    Lookling forward to getting some advice...

     

     

    Cheers

    Sachs

     

     

    User: "Thomas_Ott"
    Thomas_Ott
    New Altair Community Member

    Yahoo did something to bork the whole thing recently, so I would default to Google right now. 


    With respect to putting it all in a table, I'd have to think about it but it would include using a Rename by Replacing so you can do AAPL_Close and XOM_Close and then something to merge them all together, maybe with a combination of a Remember and Recall operator but I need go AFK know. 

Quick Links