Rapidminer studio detects pandas module but is unable to load it
VI_DS
New Altair Community Member
Hello everyone,
I am trying to set up the python scripting operator and installed Anaconda 3.7. I used the specific python binary to enter the path to python.exe in the base environment.
When I test the path, I get an error message that pandas module is detected but is unable to load.I already reinstalled Anaconda and separately reinstalled the pandas module, still no change in the error message.
The error log set to "Finest" gives me these messages:
I am trying to set up the python scripting operator and installed Anaconda 3.7. I used the specific python binary to enter the path to python.exe in the base environment.
When I test the path, I get an error message that pandas module is detected but is unable to load.I already reinstalled Anaconda and separately reinstalled the pandas module, still no change in the error message.
The error log set to "Finest" gives me these messages:
Jun 28, 2019 10:15:41 PM com.rapidminer.extension.pythonscripting.operator.scripting.python.PythonProcessBuilder getPathPrefix
FINEST: Python executable C:\Users\****\AppData\Local\Continuum\anaconda3\python.exe detected and it is not a Conda installation.
Jun 28, 2019 10:15:41 PM com.rapidminer.extension.pythonscripting.operator.scripting.python.PythonProcessBuilder start
FINEST: Starting Python process C:\Users\****\AppData\Local\Continuum\anaconda3\python.exe using PATH=C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Users\****\AppData\Local\Microsoft\WindowsApps;
Jun 28, 2019 10:15:42 PM com.rapidminer.extension.pythonscripting.operator.scripting.python.PythonSetupTester scriptingSetupTest
WARNING: Pandas module detected, but the extension is unable to load it. Check your installation.
Jun 28, 2019 10:15:42 PM com.rapidminer.extension.pythonscripting.operator.scripting.python.PythonProcessBuilder getPathPrefix
FINEST: Python executable C:\Users\****\AppData\Local\Continuum\anaconda3\python.exe detected and it is not a Conda installation.
Jun 28, 2019 10:15:42 PM com.rapidminer.extension.pythonscripting.operator.scripting.python.PythonProcessBuilder start
FINEST: Starting Python process C:\Users\****\AppData\Local\Continuum\anaconda3\python.exe using PATH=C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Users\****\AppData\Local\Microsoft\WindowsApps;
Jun 28, 2019 10:15:42 PM com.rapidminer.extension.pythonscripting.operator.scripting.python.PythonSetupTester scriptingSetupTest
FINEST: Tested Python executable: C:\Users\****\AppData\Local\Continuum\anaconda3\python.exe
@varunm1
@varunm1
Tagged:
2
Best Answer
-
I have this problem on one machine but not another. The quick fix is to create an environment with Python 3.6.8 and install Pandas and then set Rapidminer to use that. If I try and use my main 3.7 environment which I use all the time with Pycharm, I get the error. I am not sure why the 3.6.8 env works and the 3.7 doesn't. This is most likely a path problem on this particular machine as I do have a working Rapidminer and Anaconda Python 3.7 running in a different office.1
Answers
-
Hi @VI_DS - this is MarlaBot. I found these great videos on our RapidMiner Academy that you may find helpful:
MarlaBot
1 -
I have this problem on one machine but not another. The quick fix is to create an environment with Python 3.6.8 and install Pandas and then set Rapidminer to use that. If I try and use my main 3.7 environment which I use all the time with Pycharm, I get the error. I am not sure why the 3.6.8 env works and the 3.7 doesn't. This is most likely a path problem on this particular machine as I do have a working Rapidminer and Anaconda Python 3.7 running in a different office.1
-
@hughesfleming68 thanks a lot for the help, that solved the problem. Still would be interesting to know why it works on some machines and not on others3
-
Hi @VI_DS, I am glad that this got you up and running. I will look into it this weekend. There is a thread about the problem already but I couldn't get the suggested path changes to work on my first try. I will see if I can get it to work and report back.2
-
Hi @VI_DS ,
if you want to use Anaconda, you can choose in Settings >> Preferences... >> Python Scripting the conda (anaconda) option as a package manager, then select your desired environment (the base, in your case, which is the default). (If Anaconda is not found, add the main Anaconda and the Anaconda\bin directory to the search paths in the same settings tab first.) This should solve your problem.
Let me know, if you have still problems with Anaconda 3.7.
Bence Tamás1 -
Hi @btamas, unfortunately this does not work in my case. I have to select specific python binaries and only 3.6.8 works. The conda environment option remains empty on this particular machine that I am using right now even if I add the the correct paths in the 'Search Paths'. I know it can work as I mentioned above but this is a problem that does affect many users. It is still not completely solved and seems specific to Anaconda and Python 3.7.
https://community.rapidminer.com/discussion/54673/pandas-module-was-not-found
1 -
Hi All,
The root cause for this is usually that Conda does not find its DLLs when RapidMiner calls it. Recent Conda versions need a different path setup.
Python Scripting Extension 9.3.1 brings all kinds of heuristics to avoid any similar issues for Windows Anaconda installations.
Please let us know if you experience any issue with that version.
Best,
Peter2 -
Upgrading Rapidminer to 9.4 might solve the problem!2
-
@joeanalytica
this did work for me. thank you1 -
hi @hsayhumza . You're welcome. Great to hear that it worked for you!1
-
@phellinger, I actually just got the same thing after installing the rapidminer python package. Seems it was adding an additional numpy package and I had to remove 2 versions of it. After reinstalling numpy pandas got loaded correctly again.
2 -
I am stuck at an error message when running the Keras Model operator.
***
Unsupported Pandas version
The current version 1.0.3 of the pandas module for python is not supported.
Please make sure that the pandas module for Python has at least version 0.12.0.
***
How do I resolve this? How do I get pandas 0.12.0 and install it, if it is still available?
Are there other workarounds to this problem? I have downgraded my Anaconda from 3.7 to 3.6 in an attempt to fix this. Nonetheless, the problem still exists ...
0 -
Hi @Goh,
Have you tried to update RapidMiner to the last official version (RapidMiner 9.6) ?
Regards,
Lionel0 -
Yes, I am already using RapidMiner 9.6, downloaded from the official website.0
-
@Goh,
In this case, uninstall Pandas 1.0.3 and install Pandas 1.0.0.
If the error still persists, uninstall Pandas 1.0.0 and install Pandas 0.25.3
Regards,
Lionel0 -
I have uninstalled Pandas 1.0.3 and installed 0.25.3.
The operator before this, Execute Python, worked ok. It could detect the presence of 0.25.3 when I clicked the test button.
However, the Keras Model operator now displays the error message as:
The Pandas module for Python was not found.
How do I solve this?0 -
@Goh,
It seems to me that the RapidMiner's extension "Keras" is no longer maintained. That's why there is some bugs like that from my point of view.
Can you instead :
- use the new Deep Learning extension (to install from the MarketPlace)
OR
- Build your Keras model manually in a Execute Python operator using Python code.
Regards,
Lionel0 -
I am not using the Keras extension anymore but I did have it all working with python 3.6.8. You might try creating an environment for that.0
-
@Goh
yes, unfortunately Keras is not maintained the same way as Python Scripting extension is maintained. The latter keeps adapting to the changes the rapidly evolving Python ecosystem dictates.
My guess is that the cause for the error is the same described above in my comment:
"The root cause for this is usually that Conda does not find its DLLs when RapidMiner calls it. Recent Conda versions need a different path setup. Python Scripting Extension 9.3.1 brings all kinds of heuristics to avoid any similar issues for Windows Anaconda installations."
Keras extension calls out to Python, but it does not sets the PATH environment variable the way how Conda requires it to find its dependencies.
I suggest to follow @lionelderkrikor's recommendations.
(If you still want to give a last try to the Keras extension and you are using Windows, you may potentially solve it this way: try to add these subfolders into the PATH environment variable before starting RapidMiner: <conda_home>/Library/mingw-w64/bin, <conda_home>/Library/usr/bin, <conda_home>/Library/bin, <conda_home>/Scripts, <conda_home>/bin, where you use the conda install path instead of <conda_home>.)
Best,
Peter0
