LSTM in Deeplearning errors
Repletion
New Altair Community Member
Hello, Rapidminer community!
I recently started using Rapidminer and drawing inspiration from an LSTM network from Kaggle I have set out to try and make my own. Whenever I run the model it throws me the error "java.lang.IllegalArgumentException: J: Index [0] must not be >= shape[0]=1." at the apply model process.
The second problem I have (which might be the root to the evil) is the fact that my model keeps saying "Couldn't update network in epoch 1", "Couldn't update network in epoch n". This shows up in the logs view of the process. I have tried adjusting the biases and weights but I haven't been able to remove the said error.
Any help would be greatly appreciated.
I recently started using Rapidminer and drawing inspiration from an LSTM network from Kaggle I have set out to try and make my own. Whenever I run the model it throws me the error "java.lang.IllegalArgumentException: J: Index [0] must not be >= shape[0]=1." at the apply model process.
The second problem I have (which might be the root to the evil) is the fact that my model keeps saying "Couldn't update network in epoch 1", "Couldn't update network in epoch n". This shows up in the logs view of the process. I have tried adjusting the biases and weights but I haven't been able to remove the said error.
Any help would be greatly appreciated.
<div><?xml version="1.0" encoding="UTF-8"?><process version="9.6.000"></div><div> <context></div><div> <input/></div><div> <output/></div><div> <macros/></div><div> </context></div><div> <operator activated="true" class="process" compatibility="9.6.000" expanded="true" name="Process"></div><div> <parameter key="logverbosity" value="init"/></div><div> <parameter key="random_seed" value="2001"/></div><div> <parameter key="send_mail" value="never"/></div><div> <parameter key="notification_email" value=""/></div><div> <parameter key="process_duration_for_mail" value="30"/></div><div> <parameter key="encoding" value="SYSTEM"/></div><div> <process expanded="true"></div><div> <operator activated="true" class="retrieve" compatibility="9.6.000" expanded="true" height="68" name="Retrieve DIA Basics" width="90" x="45" y="34"></div><div> <parameter key="repository_entry" value="//Local Repository/Stock data/DIA Basics"/></div><div> </operator></div><div> <operator activated="true" class="replace_missing_values" compatibility="9.6.000" expanded="true" height="103" name="Replace Missing Values" width="90" x="112" y="187"></div><div> <parameter key="return_preprocessing_model" value="false"/></div><div> <parameter key="create_view" value="false"/></div><div> <parameter key="attribute_filter_type" value="all"/></div><div> <parameter key="attribute" value=""/></div><div> <parameter key="attributes" value="ACCBL_20|ACCBM_20|ACCBU_20|AD|ADOSC_3_10|ADX_14|AMAT_LR_2|AMAT_SR_2|AO_5_34|AOBV_LR_2|AOBV_SR_2|APO_12_26|AROOND_14|AROONU_14|ATR_14|BBL_20|BBM_20|BBU_20|BOP|CCI_20_0.015|CG_10|close|CMF_20|CMO_14|COPC_11_14_10|DCL_10_20|DCM_10_20|DCU_10_20|DEC_1|DEMA_10|DMN_14|DMP_14|DPO_1|EFI_13|EMA_10|EOM_14_100000000|FISHERT_5|FWMA_10|high|HL2|HLC3|HMA_10|INC_1|KAMA_10_2_30|KCB_20|KCL_20|KCU_20|KST_10_15_20_30_10_10_10_15|KSTS_9|KURT_30|LDECAY_5|LOGRET_1|low|LR_14|MACD_12_26_9|MACDH_12_26_9|MACDS_12_26_9|MAD_30|MASSI_9_25|MEDIAN_30|MFI_14|MIDPOINT_2|MIDPRICE_2|MOM_10|NATR_14|NVI_1|OBV|OBV_EMA_2|OBV_EMA_4|OBV_max_2|OBV_min_2|OHLC4|open|PCTRET_1|PPO_12_26_9|PPOH_12_26_9|PPOS_12_26_9|PVI_1|PVOL|PVT|PWMA_10|QS_10|QTL_30_0.5|RMA_10|ROC_10|RSI_14|RVI_14_4|RVIS_14_4|SINWMA_14|SKEW_30|SLOPE_1|SMA_10|STDEV_30|STOCH_3|STOCH_5|STOCHF_3|STOCHF_14|SWMA_10|TEMA_10|TRIMA_10|TRUERANGE_1|TSI_13_25|UO_7_14_28|VAR_30|volume|VTXM_14|VTXP_14|VWAP|VWMA_10|WILLR_14|WMA_10|Z_30|ZLEMA_10"/></div><div> <parameter key="use_except_expression" value="false"/></div><div> <parameter key="value_type" value="attribute_value"/></div><div> <parameter key="use_value_type_exception" value="false"/></div><div> <parameter key="except_value_type" value="time"/></div><div> <parameter key="block_type" value="attribute_block"/></div><div> <parameter key="use_block_type_exception" value="false"/></div><div> <parameter key="except_block_type" value="value_matrix_row_start"/></div><div> <parameter key="invert_selection" value="false"/></div><div> <parameter key="include_special_attributes" value="false"/></div><div> <parameter key="default" value="average"/></div><div> <list key="columns"/></div><div> </operator></div><div> <operator activated="true" class="set_role" compatibility="9.6.000" expanded="true" height="82" name="Set Role" width="90" x="246" y="187"></div><div> <parameter key="attribute_name" value="close"/></div><div> <parameter key="target_role" value="label"/></div><div> <list key="set_additional_roles"/></div><div> </operator></div><div> <operator activated="true" class="time_series:windowing" compatibility="9.6.000" expanded="true" height="82" name="Windowing" width="90" x="313" y="340"></div><div> <parameter key="attribute_filter_type" value="subset"/></div><div> <parameter key="attribute" value=""/></div><div> <parameter key="attributes" value="close"/></div><div> <parameter key="use_except_expression" value="false"/></div><div> <parameter key="value_type" value="attribute_value"/></div><div> <parameter key="use_value_type_exception" value="false"/></div><div> <parameter key="except_value_type" value="time"/></div><div> <parameter key="block_type" value="attribute_block"/></div><div> <parameter key="use_block_type_exception" value="false"/></div><div> <parameter key="except_block_type" value="value_matrix_row_start"/></div><div> <parameter key="invert_selection" value="false"/></div><div> <parameter key="include_special_attributes" value="true"/></div><div> <parameter key="has_indices" value="true"/></div><div> <parameter key="indices_attribute" value="timestamp"/></div><div> <parameter key="window_size" value="30"/></div><div> <parameter key="no_overlapping_windows" value="false"/></div><div> <parameter key="step_size" value="1"/></div><div> <parameter key="create_horizon_(labels)" value="true"/></div><div> <parameter key="horizon_attribute" value="close"/></div><div> <parameter key="horizon_size" value="1"/></div><div> <parameter key="horizon_offset" value="0"/></div><div> </operator></div><div> <operator activated="true" class="set_role" compatibility="9.6.000" expanded="true" height="82" name="Set Role (3)" width="90" x="447" y="340"></div><div> <parameter key="attribute_name" value="timestamp"/></div><div> <parameter key="target_role" value="id"/></div><div> <list key="set_additional_roles"/></div><div> </operator></div><div> <operator activated="true" class="concurrency:join" compatibility="9.6.000" expanded="true" height="82" name="Join" width="90" x="447" y="238"></div><div> <parameter key="remove_double_attributes" value="true"/></div><div> <parameter key="join_type" value="outer"/></div><div> <parameter key="use_id_attribute_as_key" value="true"/></div><div> <list key="key_attributes"/></div><div> <parameter key="keep_both_join_attributes" value="false"/></div><div> </operator></div><div> <operator activated="true" class="split_data" compatibility="9.6.000" expanded="true" height="124" name="Split Data" width="90" x="514" y="34"></div><div> <enumeration key="partitions"></div><div> <parameter key="ratio" value="0.7"/></div><div> <parameter key="ratio" value="0.2"/></div><div> <parameter key="ratio" value="0.1"/></div><div> </enumeration></div><div> <parameter key="sampling_type" value="linear sampling"/></div><div> <parameter key="use_local_random_seed" value="false"/></div><div> <parameter key="local_random_seed" value="1992"/></div><div> </operator></div><div> <operator activated="true" class="set_role" compatibility="9.6.000" expanded="true" height="82" name="Set Role (2)" width="90" x="648" y="187"></div><div> <parameter key="attribute_name" value="close + 1 (horizon)"/></div><div> <parameter key="target_role" value="regular"/></div><div> <list key="set_additional_roles"/></div><div> </operator></div><div> <operator activated="true" class="deeplearning:dl4j_sequential_neural_network" compatibility="0.9.003" expanded="true" height="103" name="Deep Learning" width="90" x="648" y="34"></div><div> <parameter key="loss_function" value="Mean Squared Error (Linear Regression)"/></div><div> <parameter key="epochs" value="50"/></div><div> <parameter key="use_miniBatch" value="false"/></div><div> <parameter key="batch_size" value="32"/></div><div> <parameter key="updater" value="Adam"/></div><div> <parameter key="learning_rate" value="0.01"/></div><div> <parameter key="momentum" value="0.9"/></div><div> <parameter key="rho" value="0.95"/></div><div> <parameter key="epsilon" value="1.0E-6"/></div><div> <parameter key="beta1" value="0.9"/></div><div> <parameter key="beta2" value="0.999"/></div><div> <parameter key="RMSdecay" value="0.95"/></div><div> <parameter key="weight_initialization" value="Xavier"/></div><div> <parameter key="bias_initialization" value="143.0"/></div><div> <parameter key="use_regularization" value="false"/></div><div> <parameter key="l1_strength" value="0.1"/></div><div> <parameter key="l2_strength" value="0.1"/></div><div> <parameter key="optimization_method" value="Conjugate Gradient Line Search"/></div><div> <parameter key="backpropagation" value="Standard"/></div><div> <parameter key="backpropagation_length" value="50"/></div><div> <parameter key="infer_input_shape" value="true"/></div><div> <parameter key="network_type" value="Simple Neural Network"/></div><div> <parameter key="log_each_epoch" value="true"/></div><div> <parameter key="epochs_per_log" value="10"/></div><div> <parameter key="use_local_random_seed" value="false"/></div><div> <parameter key="local_random_seed" value="1992"/></div><div> <process expanded="true"></div><div> <operator activated="true" class="deeplearning:dl4j_lstm_layer" compatibility="0.9.003" expanded="true" height="68" name="Add LSTM Layer" width="90" x="179" y="85"></div><div> <parameter key="neurons" value="142"/></div><div> <parameter key="gate_activation" value="ReLU (Rectified Linear Unit)"/></div><div> <parameter key="forget_gate_bias_initialization" value="1.0"/></div><div> </operator></div><div> <operator activated="true" class="deeplearning:dl4j_dense_layer" compatibility="0.9.003" expanded="true" height="68" name="Add Fully-Connected Layer (2)" width="90" x="648" y="85"></div><div> <parameter key="number_of_neurons" value="2"/></div><div> <parameter key="activation_function" value="Softmax"/></div><div> <parameter key="use_dropout" value="false"/></div><div> <parameter key="dropout_rate" value="0.25"/></div><div> <parameter key="overwrite_networks_weight_initialization" value="false"/></div><div> <parameter key="weight_initialization" value="Normal"/></div><div> <parameter key="overwrite_networks_bias_initialization" value="false"/></div><div> <parameter key="bias_initialization" value="0.0"/></div><div> </operator></div><div> <connect from_port="layerArchitecture" to_op="Add LSTM Layer" to_port="layerArchitecture"/></div><div> <connect from_op="Add LSTM Layer" from_port="layerArchitecture" to_op="Add Fully-Connected Layer (2)" to_port="layerArchitecture"/></div><div> <connect from_op="Add Fully-Connected Layer (2)" from_port="layerArchitecture" to_port="layerArchitecture"/></div><div> <portSpacing port="source_layerArchitecture" spacing="0"/></div><div> <portSpacing port="sink_layerArchitecture" spacing="0"/></div><div> </process></div><div> </operator></div><div> <operator activated="true" class="apply_model" compatibility="9.6.000" expanded="true" height="82" name="Apply Model" width="90" x="782" y="136"></div><div> <list key="application_parameters"/></div><div> <parameter key="create_view" value="false"/></div><div> </operator></div><div> <operator activated="true" class="performance_regression" compatibility="9.6.000" expanded="true" height="82" name="Performance" width="90" x="916" y="136"></div><div> <parameter key="main_criterion" value="first"/></div><div> <parameter key="root_mean_squared_error" value="false"/></div><div> <parameter key="absolute_error" value="false"/></div><div> <parameter key="relative_error" value="true"/></div><div> <parameter key="relative_error_lenient" value="false"/></div><div> <parameter key="relative_error_strict" value="false"/></div><div> <parameter key="normalized_absolute_error" value="false"/></div><div> <parameter key="root_relative_squared_error" value="false"/></div><div> <parameter key="squared_error" value="false"/></div><div> <parameter key="correlation" value="false"/></div><div> <parameter key="squared_correlation" value="false"/></div><div> <parameter key="prediction_average" value="false"/></div><div> <parameter key="spearman_rho" value="false"/></div><div> <parameter key="kendall_tau" value="false"/></div><div> <parameter key="skip_undefined_labels" value="true"/></div><div> <parameter key="use_example_weights" value="true"/></div><div> </operator></div><div> <connect from_op="Retrieve DIA Basics" from_port="output" to_op="Replace Missing Values" to_port="example set input"/></div><div> <connect from_op="Replace Missing Values" from_port="example set output" to_op="Set Role" to_port="example set input"/></div><div> <connect from_op="Set Role" from_port="example set output" to_op="Windowing" to_port="example set"/></div><div> <connect from_op="Windowing" from_port="windowed example set" to_op="Join" to_port="left"/></div><div> <connect from_op="Windowing" from_port="original" to_op="Set Role (3)" to_port="example set input"/></div><div> <connect from_op="Set Role (3)" from_port="example set output" to_op="Join" to_port="right"/></div><div> <connect from_op="Join" from_port="join" to_op="Split Data" to_port="example set"/></div><div> <connect from_op="Split Data" from_port="partition 1" to_op="Deep Learning" to_port="training set"/></div><div> <connect from_op="Split Data" from_port="partition 2" to_op="Deep Learning" to_port="test set"/></div><div> <connect from_op="Split Data" from_port="partition 3" to_op="Set Role (2)" to_port="example set input"/></div><div> <connect from_op="Set Role (2)" from_port="example set output" to_op="Apply Model" to_port="unlabelled data"/></div><div> <connect from_op="Deep Learning" from_port="model" to_op="Apply Model" to_port="model"/></div><div> <connect from_op="Deep Learning" from_port="history" to_port="result 1"/></div><div> <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/></div><div> <connect from_op="Performance" from_port="performance" to_port="result 2"/></div><div> <connect from_op="Performance" from_port="example set" to_port="result 3"/></div><div> <portSpacing port="source_input 1" spacing="0"/></div><div> <portSpacing port="sink_result 1" spacing="0"/></div><div> <portSpacing port="sink_result 2" spacing="0"/></div><div> <portSpacing port="sink_result 3" spacing="0"/></div><div> <portSpacing port="sink_result 4" spacing="0"/></div><div> </process></div><div> </operator></div><div></process></div><div></div>
Tagged:
0
Best Answer
-
This one!. I had switched off some attributes to save time. The site is unusable. Lot of problems.5
Answers
-
With LSTM, you need to use the TimeSeries to Tensor operator. I suggest that you look at the Airline passenger with regression LSTM example. It is a bit more involved to setup. Also, I wouldn't split your data like that. You will get quite different results depending on your split points. Take a look at sliding windows.1
-
Thank you for the help, however I after modelling my model after the airline passenger example I get a problem in the select process. "Process failed: Collection size is 0, requested element index is 1.".
Everything is made as in the in the sample however it still throws that error
It also gives the following output when training which is concerning (and possible root to the select process problem): "Epoch: 1, training score: NaN, testing score: NaN"<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000"> <context> <input/> <output/> <macros/> </context> <operator activated="true" class="process" compatibility="9.6.000" expanded="true" name="Process"> <parameter key="logverbosity" value="init"/> <parameter key="random_seed" value="2001"/> <parameter key="send_mail" value="never"/> <parameter key="notification_email" value=""/> <parameter key="process_duration_for_mail" value="30"/> <parameter key="encoding" value="SYSTEM"/> <process expanded="true"> <operator activated="true" class="retrieve" compatibility="9.6.000" expanded="true" height="68" name="Retrieve DIA Basics" width="90" x="45" y="34"> <parameter key="repository_entry" value="//Local Repository/Stock data/DIA Basics"/> </operator> <operator activated="true" class="replace_missing_values" compatibility="9.6.000" expanded="true" height="103" name="Replace Missing Values" width="90" x="45" y="187"> <parameter key="return_preprocessing_model" value="false"/> <parameter key="create_view" value="false"/> <parameter key="attribute_filter_type" value="all"/> <parameter key="attribute" value=""/> <parameter key="attributes" value="ACCBL_20|ACCBM_20|ACCBU_20|AD|ADOSC_3_10|ADX_14|AMAT_LR_2|AMAT_SR_2|AO_5_34|AOBV_LR_2|AOBV_SR_2|APO_12_26|AROOND_14|AROONU_14|ATR_14|BBL_20|BBM_20|BBU_20|BOP|CCI_20_0.015|CG_10|close|CMF_20|CMO_14|COPC_11_14_10|DCL_10_20|DCM_10_20|DCU_10_20|DEC_1|DEMA_10|DMN_14|DMP_14|DPO_1|EFI_13|EMA_10|EOM_14_100000000|FISHERT_5|FWMA_10|high|HL2|HLC3|HMA_10|INC_1|KAMA_10_2_30|KCB_20|KCL_20|KCU_20|KST_10_15_20_30_10_10_10_15|KSTS_9|KURT_30|LDECAY_5|LOGRET_1|low|LR_14|MACD_12_26_9|MACDH_12_26_9|MACDS_12_26_9|MAD_30|MASSI_9_25|MEDIAN_30|MFI_14|MIDPOINT_2|MIDPRICE_2|MOM_10|NATR_14|NVI_1|OBV|OBV_EMA_2|OBV_EMA_4|OBV_max_2|OBV_min_2|OHLC4|open|PCTRET_1|PPO_12_26_9|PPOH_12_26_9|PPOS_12_26_9|PVI_1|PVOL|PVT|PWMA_10|QS_10|QTL_30_0.5|RMA_10|ROC_10|RSI_14|RVI_14_4|RVIS_14_4|SINWMA_14|SKEW_30|SLOPE_1|SMA_10|STDEV_30|STOCH_3|STOCH_5|STOCHF_3|STOCHF_14|SWMA_10|TEMA_10|TRIMA_10|TRUERANGE_1|TSI_13_25|UO_7_14_28|VAR_30|volume|VTXM_14|VTXP_14|VWAP|VWMA_10|WILLR_14|WMA_10|Z_30|ZLEMA_10"/> <parameter key="use_except_expression" value="false"/> <parameter key="value_type" value="attribute_value"/> <parameter key="use_value_type_exception" value="false"/> <parameter key="except_value_type" value="time"/> <parameter key="block_type" value="attribute_block"/> <parameter key="use_block_type_exception" value="false"/> <parameter key="except_block_type" value="value_matrix_row_start"/> <parameter key="invert_selection" value="false"/> <parameter key="include_special_attributes" value="false"/> <parameter key="default" value="average"/> <list key="columns"/> </operator> <operator activated="true" class="set_role" compatibility="9.6.000" expanded="true" height="82" name="Set Role" width="90" x="45" y="340"> <parameter key="attribute_name" value="close"/> <parameter key="target_role" value="label"/> <list key="set_additional_roles"/> </operator> <operator activated="true" class="time_series:windowing" compatibility="9.6.000" expanded="true" height="82" name="Windowing" width="90" x="179" y="340"> <parameter key="attribute_filter_type" value="subset"/> <parameter key="attribute" value=""/> <parameter key="attributes" value="close"/> <parameter key="use_except_expression" value="false"/> <parameter key="value_type" value="attribute_value"/> <parameter key="use_value_type_exception" value="false"/> <parameter key="except_value_type" value="time"/> <parameter key="block_type" value="attribute_block"/> <parameter key="use_block_type_exception" value="false"/> <parameter key="except_block_type" value="value_matrix_row_start"/> <parameter key="invert_selection" value="false"/> <parameter key="include_special_attributes" value="true"/> <parameter key="has_indices" value="true"/> <parameter key="indices_attribute" value="timestamp"/> <parameter key="window_size" value="30"/> <parameter key="no_overlapping_windows" value="false"/> <parameter key="step_size" value="1"/> <parameter key="create_horizon_(labels)" value="true"/> <parameter key="horizon_attribute" value="close"/> <parameter key="horizon_size" value="1"/> <parameter key="horizon_offset" value="0"/> </operator> <operator activated="true" class="set_role" compatibility="9.6.000" expanded="true" height="82" name="Set Role (3)" width="90" x="447" y="340"> <parameter key="attribute_name" value="timestamp"/> <parameter key="target_role" value="id"/> <list key="set_additional_roles"/> </operator> <operator activated="true" class="concurrency:join" compatibility="9.6.000" expanded="true" height="82" name="Join" width="90" x="313" y="238"> <parameter key="remove_double_attributes" value="true"/> <parameter key="join_type" value="outer"/> <parameter key="use_id_attribute_as_key" value="true"/> <list key="key_attributes"/> <parameter key="keep_both_join_attributes" value="false"/> </operator> <operator activated="true" class="split_data" compatibility="9.6.000" expanded="true" height="124" name="Split Data" width="90" x="313" y="34"> <enumeration key="partitions"> <parameter key="ratio" value="0.7"/> <parameter key="ratio" value="0.2"/> <parameter key="ratio" value="0.1"/> </enumeration> <parameter key="sampling_type" value="linear sampling"/> <parameter key="use_local_random_seed" value="false"/> <parameter key="local_random_seed" value="1992"/> </operator> <operator activated="true" class="set_role" compatibility="9.6.000" expanded="true" height="82" name="Set Role (2)" width="90" x="648" y="289"> <parameter key="attribute_name" value="close + 1 (horizon)"/> <parameter key="target_role" value="regular"/> <list key="set_additional_roles"/> </operator> <operator activated="true" class="collect" compatibility="9.6.000" expanded="true" height="82" name="Collect" width="90" x="849" y="34"> <parameter key="unfold" value="false"/> </operator> <operator activated="true" class="deeplearning:dl4j_timeseries_converter" compatibility="0.9.003" expanded="true" height="82" name="TimeSeries to Tensor" width="90" x="983" y="34"/> <operator activated="true" class="collect" compatibility="9.6.000" expanded="true" height="82" name="Collect (2)" width="90" x="849" y="187"> <parameter key="unfold" value="false"/> </operator> <operator activated="true" class="deeplearning:dl4j_timeseries_converter" compatibility="0.9.003" expanded="true" height="82" name="TimeSeries to Tensor (2)" width="90" x="983" y="187"/> <operator activated="true" class="deeplearning:dl4j_tensor_sequential_neural_network" compatibility="0.9.003" expanded="true" height="103" name="Deep Learning (Tensor)" width="90" x="1251" y="34"> <parameter key="loss_function" value="Mean Squared Error (Linear Regression)"/> <parameter key="epochs" value="50"/> <parameter key="use_miniBatch" value="false"/> <parameter key="batch_size" value="32"/> <parameter key="updater" value="Adam"/> <parameter key="learning_rate" value="0.01"/> <parameter key="momentum" value="0.9"/> <parameter key="rho" value="0.95"/> <parameter key="epsilon" value="1.0E-6"/> <parameter key="beta1" value="0.9"/> <parameter key="beta2" value="0.999"/> <parameter key="RMSdecay" value="0.95"/> <parameter key="weight_initialization" value="Xavier"/> <parameter key="bias_initialization" value="0.0"/> <parameter key="use_regularization" value="false"/> <parameter key="l1_strength" value="0.1"/> <parameter key="l2_strength" value="0.1"/> <parameter key="optimization_method" value="Stochastic Gradient Descent"/> <parameter key="backpropagation" value="Standard"/> <parameter key="backpropagation_length" value="50"/> <parameter key="infer_input_shape" value="true"/> <parameter key="network_type" value="Simple Neural Network"/> <parameter key="log_each_epoch" value="true"/> <parameter key="epochs_per_log" value="10"/> <parameter key="use_local_random_seed" value="false"/> <parameter key="local_random_seed" value="1992"/> <process expanded="true"> <operator activated="true" class="deeplearning:dl4j_lstm_layer" compatibility="0.9.003" expanded="true" height="68" name="Add LSTM Layer" width="90" x="112" y="136"> <parameter key="neurons" value="142"/> <parameter key="gate_activation" value="ReLU (Rectified Linear Unit)"/> <parameter key="forget_gate_bias_initialization" value="1.0"/> </operator> <operator activated="true" class="deeplearning:dl4j_dense_layer" compatibility="0.9.003" expanded="true" height="68" name="Add Fully-Connected Layer" width="90" x="380" y="136"> <parameter key="number_of_neurons" value="1"/> <parameter key="activation_function" value="None (identity)"/> <parameter key="use_dropout" value="false"/> <parameter key="dropout_rate" value="0.25"/> <parameter key="overwrite_networks_weight_initialization" value="false"/> <parameter key="weight_initialization" value="Normal"/> <parameter key="overwrite_networks_bias_initialization" value="false"/> <parameter key="bias_initialization" value="0.0"/> </operator> <connect from_port="layerArchitecture" to_op="Add LSTM Layer" to_port="layerArchitecture"/> <connect from_op="Add LSTM Layer" from_port="layerArchitecture" to_op="Add Fully-Connected Layer" to_port="layerArchitecture"/> <connect from_op="Add Fully-Connected Layer" from_port="layerArchitecture" to_port="layerArchitecture"/> <portSpacing port="source_layerArchitecture" spacing="0"/> <portSpacing port="sink_layerArchitecture" spacing="0"/> </process> </operator> <operator activated="true" class="collect" compatibility="9.6.000" expanded="true" height="82" name="Collect (3)" width="90" x="916" y="289"> <parameter key="unfold" value="false"/> </operator> <operator activated="true" class="deeplearning:dl4j_timeseries_converter" compatibility="0.9.003" expanded="true" height="82" name="TimeSeries to Tensor (3)" width="90" x="1050" y="289"/> <operator activated="true" class="deeplearning:dl4j_apply_tensor_model" compatibility="0.9.003" expanded="true" height="82" name="Apply Model (Tensor)" width="90" x="1385" y="136"/> <operator activated="true" class="select" compatibility="9.6.000" expanded="true" height="68" name="Select" width="90" x="1519" y="136"> <parameter key="index" value="1"/> <parameter key="unfold" value="false"/> </operator> <operator activated="true" class="performance_regression" compatibility="9.6.000" expanded="true" height="82" name="Performance" width="90" x="1653" y="136"> <parameter key="main_criterion" value="first"/> <parameter key="root_mean_squared_error" value="true"/> <parameter key="absolute_error" value="false"/> <parameter key="relative_error" value="true"/> <parameter key="relative_error_lenient" value="false"/> <parameter key="relative_error_strict" value="false"/> <parameter key="normalized_absolute_error" value="false"/> <parameter key="root_relative_squared_error" value="false"/> <parameter key="squared_error" value="true"/> <parameter key="correlation" value="true"/> <parameter key="squared_correlation" value="false"/> <parameter key="prediction_average" value="true"/> <parameter key="spearman_rho" value="false"/> <parameter key="kendall_tau" value="false"/> <parameter key="skip_undefined_labels" value="true"/> <parameter key="use_example_weights" value="true"/> </operator> <connect from_op="Retrieve DIA Basics" from_port="output" to_op="Replace Missing Values" to_port="example set input"/> <connect from_op="Replace Missing Values" from_port="example set output" to_op="Set Role" to_port="example set input"/> <connect from_op="Set Role" from_port="example set output" to_op="Windowing" to_port="example set"/> <connect from_op="Windowing" from_port="windowed example set" to_op="Join" to_port="left"/> <connect from_op="Windowing" from_port="original" to_op="Set Role (3)" to_port="example set input"/> <connect from_op="Set Role (3)" from_port="example set output" to_op="Join" to_port="right"/> <connect from_op="Join" from_port="join" to_op="Split Data" to_port="example set"/> <connect from_op="Split Data" from_port="partition 1" to_op="Collect" to_port="input 1"/> <connect from_op="Split Data" from_port="partition 2" to_op="Collect (2)" to_port="input 1"/> <connect from_op="Split Data" from_port="partition 3" to_op="Set Role (2)" to_port="example set input"/> <connect from_op="Set Role (2)" from_port="example set output" to_op="Collect (3)" to_port="input 1"/> <connect from_op="Collect" from_port="collection" to_op="TimeSeries to Tensor" to_port="collection"/> <connect from_op="TimeSeries to Tensor" from_port="tensor" to_op="Deep Learning (Tensor)" to_port="training set"/> <connect from_op="Collect (2)" from_port="collection" to_op="TimeSeries to Tensor (2)" to_port="collection"/> <connect from_op="TimeSeries to Tensor (2)" from_port="tensor" to_op="Deep Learning (Tensor)" to_port="test set"/> <connect from_op="Deep Learning (Tensor)" from_port="model" to_op="Apply Model (Tensor)" to_port="model"/> <connect from_op="Collect (3)" from_port="collection" to_op="TimeSeries to Tensor (3)" to_port="collection"/> <connect from_op="TimeSeries to Tensor (3)" from_port="tensor" to_op="Apply Model (Tensor)" to_port="unlabelled tensor"/> <connect from_op="Apply Model (Tensor)" from_port="labeled data" to_op="Select" to_port="collection"/> <connect from_op="Select" from_port="selected" to_op="Performance" to_port="labelled data"/> <connect from_op="Performance" from_port="performance" to_port="result 1"/> <portSpacing port="source_input 1" spacing="0"/> <portSpacing port="sink_result 1" spacing="0"/> <portSpacing port="sink_result 2" spacing="0"/> </process> </operator> </process>0 -
@hughesfleming68 any input on my comment? Not to be rude or anything, but its been a few days. I appreciate the help0
-
@Repletion. I will check your process when I get to the office. Keep in mind that many people provide support to other users on a voluntary basis. We can't always be available. If you are going to post a process that you would like help with, please post some data. It is very hard to debug without it.0
-
@hughesfleming68 yea I have alot of respect for you and the others who are helping here on the forums. Regarding the data, I forgot to post it in the comment I made. I have attached the csv file in this comment now.1
-
Got it. I will take a look.0
-
-
-
This one!. I had switched off some attributes to save time. The site is unusable. Lot of problems.5
-
@hughesfleming68 thank you so much! You are god sent. 2 questions though (theoritical)
What is the optimal amount of tensors to use? 1 tensor for every variable?
What is most effective to use in a deep learning model, minibatches, or no minibatches? If no, then when is it a good idea to use minibatches?
Again thank you for the help, much appriciated!1 -
@hughesfleming68 I see that part of the solution was changing the activation function of the LSTM layer to sigmoid instead of Relu, why cant I use Relu? It just throws the Nan epochs. I want to use Relu since its "superior" to the other activation functions.0
-
Yes, I noticed that too. Before you can determine which activation function is better you will need sort out your data. Your data scaling is all over the place because of the lack of normalization. Once the site is working again I will help you with your other questions. Right now it is very difficult to use.1