-
Reshaping Data from Wide to Long Format in RapidMiner
Hello RapidMiner Community, I am currently working on a project where I need to compare fossil fuel subsidies and environmental taxes by country over time. I have two panel data tables: both has data on 124 countries (cross section). One table contains total fossil fuel subsidies, with data collected from 1995-2021. The…
-
ERROR al hacer PIVOT de tabla y luego agregar campos calculados
Por favor, alguien sabe si es una limitación de Rapidminer o estoy haciendo algo mal? Tengo una tabla, hice pivot para dejar en las columnas los años y a partir de ahí quiero calcular la diferencia entre una columna y otra pero Rapidminer no reconoce esos campos como válidos para hacer los cálculos. Adjunto pantallas Tabla…
-
join the result of a depivot operation warning
Hello, I'm new to RapidMiner so bear with me :) I'm trying to do a rather simple process in which I use de-pivot (aka "melt") an example set and then use the result example set in a join operation. The de-pivot works exactly as I'd expected but the follow up join does not find the newly created column which I'd like to use…
-
Pivoting/grouping multiple attributes after time-series feature extraction
Hello, I extracted features of multiple time series batches (IDx) with multiple signal columns (signalx) using the extract aggregates block on these multiple signals. I receive data in the form of: ID1 - "signal1" - feat1 - feat2 - ... ID1 - "signal2" - feat1 - feat2 - ... ID2 - "signal1" - feat1 - feat2 - ... ID2 -…
-
Create/ Calculate new example
Hi,I have a data set like this:Value | Nr.X | 1Y | 2 And like to have this kind of data set:Value | Nr.X | 1Y | 1Y | 1 Question:How can I generate new examples from the existing one? Do I need to use transpose to generate new Attributes? Or should I use De-Pivot or a Macro? What is the most effective way? Thanks for…
-
how to make a football ranking?
Hi guys, so i'm stuck to make a football ranking with my data. i did the data processing but now i don't have any idea how to group each team with the points. So if home team won + 3 Home team lose + 0 scoreless + 1 away team + 3 away team lose + 0 it would help me a lot :)
-
Quartile
Hello, I have a process in which I aggregate the scores as mean by year and week, using pivot operator. I then forecast the weekly mean(score). I want also to try to forecast the first quartile, but I'm not finding a way to extract it. Neither in the pivot operator options nor in the other operators. I thought about…
-
Pivot log data by dates
Hi all, I'm trying to use RM in a certain task in which I have (kind of) log data in the following (abstract) schema: * ID | Date | Result I wish to transform this table to vectors from the following structure: * id1: [date1: result1, date2: result2, ... ] * id2: [date11: result11, date22: result22, ... ] [notice that the…
-
JSON to data and de-pivot with missing attributes and values
Hi everyone, after all i could not find out how to get this problem fixed: i have a JSON file that is inconsistent with its attribute and values, some are missing. Like this:…
-
JSON to data and de-pivot for exampleset
I'm having a 'mare trying to get some data to de-pivot from json... The below process gets data from Poloniex in json format. The first row works fine and the exampleset is generated as I'd like, the other 2 (the orderbook and historical trades) I just can't get my head around. The Poloniex public api info is…
-
Replicating Excel Pivot Table features
Hello, Often I use excel to make a deeper inspection of my process results such as inspecting cluster composition based on the attributes, or inspecting GLM predictions on a scoring set. E.g. What is the distribution in each cluster for different cities of City attribute, or how are the predicted leads distributed across…
-
Pivot with multiple group attributes
Hi everyone, I'm new to Rapidminer, so my question might be super easy. I have a data set with the following Attributes: * Model Code (identifies a group of cars) * VIN (identifies cars) * Part (identifies part of the car: Fender or door) * Measuring Point (identifies measuring points at the part) * Station (IFC or FQC ->…
-
De-pivot?
Imagine that I have an example set (after using clustering) that looks like this: id,cluster,product1,product2 1,clust0,10.5,20.3 2,clust1,15.75,30.12 3,clust0,5.25,27.2 4,clust1,12.75,8.25 5,clust0,18.05,22.3 6,clust1,1.25,3.75 And I need it in the following format: id,cluster,value,type 1,clust0,10.5,product1…
-
Pivot Operator - "potential problem detected: attribute missing" after Select Attributes
Hi all, according to the pivot operator in my data preparation process RapidMiner is experiencing a 'potential problem': I have to stream a big amount of data. In Select Attributes I chose three attributes. One of them named TBLUNIQUELRU_ID is being missed by the pivot operator although it is contained in the Select…
-
"Pivot/table - association rules"
Hello all, I have a table like this name cluster frank cluster_0 frank cluster_0 frank cluster_4 thomas cluster_0 thomas cluster_3 betty cluster_8 betty cluster_10 And for association rules and sequential patterns I would like (and I think I need) that table look like this name cluster_0 cluster_3 cluster_4 cluster_8…
-
"De-Pivot Memory issue"
Hello Everyone, I am running a process that needs to use 2 De-pivot operators. The dataset is large (aprox 900k examples). The first De-pivot runs fine but the second one crashes out of memory, even though I have a machine with 16GB of RAM. I know there should be some workaround by using the Loop Batches (inserting inside…
-
Missing id attribute after pivot operator
Hi everyone, I am having the problem that I can no longer access the Id-attribute after using the pivot operator. Although the ID-attribute is shown under the pivot operator info, trying to get hold of it, for example using the set role or select attribute operator would result in an "Attribute not found " type of error.…
-
[SOLVED]How to transpose/de-pivot a Utility Matrix
How do I turn a matrix such as this one:Customer IDProduct1 Product2 Product3 Product4 Product5 110301201201310004 into a new transposed one that looks like…
-
[SOLVED] Pivot nominal values
Hi, This is the format of my data ID DOB device_mac 1 1993-02-08 2c:5a:05:a4:0O:c8 1 1993-02-08 94:eb:cd:17:22:9c I would like to to be in the following format ID DOB device_mac_1 device_mac_2 1 1993-02-08 2c:5a:05:a4:0O:c8 94:eb:cd:17:22:9c is it possible to do this in RapidMiner? The pivot operator does not seem to be…
-
"[SOLVED] Pivot operation: RapidMiner vs Knime"
Hello, By using the surveys I thoutght that RapidMiner is better than Knime. However, when I tryed to pivot a large amount of data (1.2 million lines that have the shape:transaction id, user id, quantity) knime did it in about 15 minutes and RapidMiner didn't manage to do it. I specify that yes I used the last version of…
-
De-pivot excel file [SOLVED]
I am trying to de-pivot a excel file like below, the "NA" are null values, I have Field3 and field4 set as integers I have: attribute name >> edit list >> attribute name = field3 & atrributes = .* index attribute = myNewFieldName checked create nominal index checked keep missings I keep getting the error:Process failed…
-
[SOLVED] De-Pivot question
Hello, I'm a newcomer to RapidMiner And I have a question about the de-pivot operator: This is my input data:Date/TimeSolMac 10MT3 (121) SolMac 15MT3 (125) SolMac 15MT3 (136) 01/07/2013 106.8 103.4 94.3 02/07/2013119.7 116.5 106.1 03/07/2013 122.6 119.3 108.7 And this is the result that I…
-
[SOLVED] Is here someone can help me about operator pivot
1.my origin data like this: FORMULA_ID HERB_ID 15 65 15 152 16 96 16 185 17 96 17 44 18 29 19 59 19 92 20 33 20 44 ... ... 2.i want to use operator "pivot" to trans data("FORMULA_ID" as group attr,"HERB_ID" as index arrt) for next Association data mind.The attr "HERB_ID" has >50 different values,then the operator "pivot"…
-
"Pivot operator"
Hello. We are trying to use the pivot operator. We want to collapse the rows that have the same date with summing of the values of the variables. So that a table like this Date Variable1 Variable2 Variable3 Variable4 2012/04/20 5 2 2 6 2012/04/20 6 5 2 6 2012/04/20 5 8 6 2 2012/04/21 5 7 2 2 2012/04/21 8 6 2 6…
-
"Memory problem during Pivot operator with large file for market basket analysis"
Hey! I spent the whole day trying to solve my problem but to no avail. So I thought I ask for some help here. Perhaps someone has an idea. I want to do a market basket analysis on my data. It basically is a .csv file of this format: SomePlaceA, 2005-01-01, SomeNameA SomePlaceA, 2005-01-02, SomeNameB SomePlaceA, 2005-01-02,…