Can anyone help with CNN later input error in RM's Deep Learning extension?

MickMcWilliams

Hi... I'm new to RapidMiner's Deep Learning extension and have encountered an issue. I'm trying to incorporate an Add Convolutional Layer operator within a Deep Learning (Tensor) operator, placing it after an Add Fully-Connected Layer operator. I'm getting an error message saying, "java.lang.IllegalStateException: Invalid input for Convolution layer (layer name='Convo Layer'): Expected CNN input, got InputTypeFeedForward(12)". I get a similar error if I make the Add Convolutional Layer operator the first add layer operator within the Deep Learning (Tensor) operator. Apparently, the Add Convolutional Layer operator expects a certain type of input but the documentation just says, "Connect this port to the layer Architecture output port of another add layer operator or to the layer port of the "Deep Learning" operator if this layer is the first one." There's no reference to the Add Convolutional Layer operator requiring a "CNN input," how to create a CNN input or even what a CNN input is. Can anyone help me out here? Thanks very much!

hughesfleming68

Starting with the simplest network you should try CovNet-Fully connected-Output Layer and then follow up to see the effect of adding a Pooling layer. CovNet-Pooling- Fully Connected- Output layer. You can continue to build your network complexity in the form CovNet-Pooling-Covnet-Pooling-Fully Connected-Output layer. Take a look at the LSTM example for the proper usage of the tensor operator. It is not needed for standard convolution networks. It is well worth studying simple architectures as well as common complex ones such as Alex-net. This is a wide subject all on its own. While the DL tools in Rapidminer can give you a good introduction, at some point you are going to want to go straight to code. You will have a lot more granularity with how you setup your networks.

hughesfleming68

Starting with the simplest network you should try CovNet-Fully connected-Output Layer and then follow up to see the effect of adding a Pooling layer. CovNet-Pooling- Fully Connected- Output layer. You can continue to build your network complexity in the form CovNet-Pooling-Covnet-Pooling-Fully Connected-Output layer. Take a look at the LSTM example for the proper usage of the tensor operator. It is not needed for standard convolution networks. It is well worth studying simple architectures as well as common complex ones such as Alex-net. This is a wide subject all on its own. While the DL tools in Rapidminer can give you a good introduction, at some point you are going to want to go straight to code. You will have a lot more granularity with how you setup your networks.

MickMcWilliams

Thanks @hughesfleming68! First, you've helped me to realize that my "CNN input" problem was apparently just because you can't connect an Add Convolutional Layer operator to a "tensor-ized" data source. Second, your suggestions on how to develop the DL architecture are also quite helpful. Very much appreciated... :-)

hughesfleming68

Your're welcome. Happy to help.For many tasks more complexity does not give better results so take an evolutionary approach to your networks and just add layers when you know they are helping. Training times can be long so keeping things tight will be an advantage.

MickMcWilliams

Thanks @Hamne1r. I'm still somewhat new to Deep Learning and so was not previously familiar with dilated convolutional neural networks (DCNN). However, I'm starting to research them based on your remark and can already see how the concept would be relevant here. Are you aware of a way the DCNN approach can somehow be used to coax the RapidMiner DL extension into applying the Add Convolutional Layer operator to tensor data? Or is it that you think a "dilation" approach could be used in some other way to get around the tensor data issue to apply CNN layers to series data? Thanks again!