-
How to convert numerical values in result file back to original nominal values of input
Hello everyone! I am just a beginner whom have just started to study RM for a few months. I am having a group problem to detect the outliers of Bank Marketing Dataset. This is my process (image below). The dataset has more than 40.000 examples and Outlier Detection Operator seems too slow for both Nominal and Numerical…
-
How to do Anomaly Detection with AutoEncoder?
I am unable to do anomaly detection with AutoEncoder operator. I have gone through the tutorial process of autoencoder but unable to understand it. Please help.
-
Help interpreting outliers/anomalies when using Isolation Forest operator
Hi. I'm really liking the Isolation Forest operator under the Anomaly Detection Extension. Trees =100, Leaf Size =2, and average path as the score calculation gives me a result where the first 5 outliers match exactly with an R script using the Mahalanobis Distance function. That is great for comparisons. But is there a…
-
How to calculate ROC and AUC and compare unsupervised algorithms?
I am working on a unlabeled dataset and trying to find out anomalies. I would like to compare unsupervised algorithm performance but I couldn't find out how to do it with rapid miner? Could anyone help me to do it?
-
Analysis of results of anomaly detection algorithm
Hello there. I have built a process in Rapid Miner using k-nn Global Anomaly Score operator in order to detect outliers in my dataset. The problem is, that I choose attributes in appropriate operators and just in dead-end to describe what is going on vizualization tab. I need help in understanding, what is going on…
-
Anomaly detection in Rapidminer with one label column Yes/No
Hello All, I have a dataset with 1000 rows which has one column contains Yes/No to identify as Anomaly. I want to use this dataset to train the model. Which model i should use in supervised techniques and how can I design my job which has 2 inputs one as training set with label and another one without label. Any sample…
-
Set Role messes with outlier detection, but setting role later ruins predictions
Hi, I've run into an interesting issue. If I assign a label to my target before standardization and outlier detection, my outliers are wrong. In general however the model performs OK. I do however have a few very significant outliers in the dataset that I would like to detect and remove. These are only found when…
-
How to solve
Dear Community Members, I am trying to execute a process for a dataset with CMGOS anomaly detection technique. However, every time I try to execute with different parameters, the process is failed with "Matrix is Singular" error. From the discussion, I tried with a covariance matrix before the example set is passed to the…
-
How to Detect Anomalies in Multivariant Time Series?
Hi, I found some tutorials and examples in RM of process which can detect anomalies in univariant time series (1 variable) and I wanted to know if someone tried using similar process with multivariant timeseries (a series which consists of several variables). Any help will be greatly appreciated, Thanks!
-
How to extract a model from Robust PCA
In general, Robust PCA (rPCA) is an excellent algorithm capable of doing both PCA in the presence of outliers and identifying two classes of outliers, i.e. those that fall within the subset of principal components and those outside (which otherwise would not be detected after dimensionality reduction). Anomaly Detection…
-
In Extensions top downloads or top rated, I cannot locate anomaly detection. I'm using Rapid Miner 9
In Extensions top downloads or top rated, where is anomaly detection? I'm using Rapid Miner 9.6 with Windows 10.
-
Anomaly detection method
Hi I’m currently in my final year for computer science for cyber security. My FYP module is in regards to anomaly detection for DDoS attacks, I am using MATLAB to create this anomaly detection system. Where my issue falls is the fact that I am having a hard time finalising a method I have currently selected isolation…
-
Anomaly extention Generate ROC seems to mirror FP/FN rate
I am using the anomaly extention against a artificial dataset. I use three algoritms to assign an anomaly score. These are K-NN Global, uCBLOF and LOF. My dataset contains a label of the anomalies that are supposed to show up. I use the Generate ROC to measure performance. What GenerateROC does first is choose a treshold…
-
Cannot genereate ROC from HBOS example set
Hello, I have a model using HBOS to calculate an outlier score. I want to feed that into an "GenerateROC" component to evauluate the scores, like with other outlier detection components. However, the GenerateROC component gives an design time error that the exampleset coming from the HBOS in facts is not an example set. I…
-
HBOS memory issue in rapidminer studio 9.3.001
Hi all, There was a previous thread about this issue, but that did not solve my problem. I have a dataset with 13 features. I use HBOS from the analmaly extension version 2.4.001. If i sample my dataset down to 100 items and then apply HBOS, studio will still run out of memory. It uses up to the max of 30Gb and then stops…
-
Outlier detection taking lot of time and not giving any results
I am using Automodel feature in Rapidminer educational license version. My processor is 6 core, i7-8750h with 16 GB ram.The dataset have more than 30000 rows and 12 columns . It have numeric and text data both. I tried running using the best features(as detected by Auto model) and am using the distance based outlier…
-
How to process categorical type data using unsupervised algorithm in anomaly detection?
I encounter a problem in anomaly detection. We know that distance is measured between different instances. Now my dataset contains categorical data. I have 3 choices. First, I remove the categorical features, however, I think that there are useful messages in categorical features. Second, I transform the categorical data…
-
Outlier detection algorithms comparison
Hello, I'm new to RapidMiner and I'm kinda having a little bit of troubles here. I'm trying to compare outlier detection algorithms such as LOF LoOP in terms of performance... and I have no clue how to do it.
-
Anomaly detection
Hi, I use histogram-based outlier score for anomaly detection but it dose not consider the dependencies between features. Which anomaly detection algorithm should i use to consider the dependencies between features? Thanks
-
Anomaly detection
Hi. I’m new to RapidMiner and started using it. I would like to seek for advice on what suitable data mining technique that can be used in the energy time
series data for anomaly detection. The
anomaly is based on abnormal and irregularities energy consumption (kWh)
behavior. Thanks
-
Attribute-wise Learning (ALSO)
Hello Community, I have a hard time trying to get outlier scores from the ALSO operator. Whatever my tweaking, the dataset, the internal regression model… the result is a useless long list of zero outlier scores. Thanks in advance for your help!
-
Java error on using k-NN Global Anomaly Detection with missing values
Hi, I am using RM Studio 8.2 and I have a process in which I am using k-NN Global Anomaly Detection to find anomaly scores for a set of data instances that have missing values in some of the attributes. In doing so, I get a Java error message "Process failed. Comparison method violates its general contract!". When I…
-
Anomaly Detection Extension Generate ROC Operator
Hi all; I'm newbie at Rapidminer. I'm using anomaly detection extension. But I cannot use generate ROC operator. I don't know what is wrong with me? I couldn't find any document or video. Is there anyone who can help me about this operator? Thanks.
-
Credit Card Fraud Detection using Kaggle Data Set and Anomaly Detection
Anomaly Detection using Rapidminer and Python I have always felt that anomaly detection could be a very interesting application of machine learning. I can think of several scenarios where such techniques could be used. In this article I shall describe some experiments I carried out with the Credit Card Fraud Detection…
-
"Anomaly detection extension - evaluation"
Hi I'm new here. I want to ask how to do an evaluation (ROC and AUC) of anomaly detection extension ( nearest neighbour based, clustering based and statistical based). I want to compare each model. Thank you.