how to generate a neural net weight-matrix?

eldenoso

Hello altogether,

as I am currently trying to get more insight into what neural nets are doing, I wanted to know, whether there is a way of exporting the weight-matrices of a neural net so that you can visualize it (e.g. heat map). By then changing the training set a couple of times, you may see differences in the "heat-map-weight-matrix" and can conclude which attributes are the ones that matter.

I know that there are real scientists out there, searching for answers to the black-box-problem of neural nets. But my curiosity just drives me towards this way and I would be glad to try this out :-).

Thank you for your answers

 

Philipp

MartinLiebig

Hi Philipp,

 

in fact you can use Groovy Script to do this. I've tried this, but it wasn't as easy as i thought. Attached is a process with the script. It always extracts the first Layer. i've not tested it in various layouts though

 

Best,

Martin

 

<?xml version="1.0" encoding="UTF-8"?><process version="7.3.001">
 <context>
 <input/>
 <output/>
 <macros/>
 </context>
 <operator activated="true" class="process" compatibility="7.3.001" expanded="true" name="Process">
 <process expanded="true">
 <operator activated="true" class="retrieve" compatibility="7.3.001" expanded="true" height="68" name="Retrieve Sonar" width="90" x="112" y="34">
 <parameter key="repository_entry" value="//Samples/data/Sonar"/>
 </operator>
 <operator activated="true" class="neural_net" compatibility="7.3.001" expanded="true" height="82" name="Neural Net" width="90" x="313" y="34">
 <list key="hidden_layers">
 <parameter key="1" value="10"/>
 </list>
 </operator>
 <operator activated="true" class="execute_script" compatibility="7.3.001" expanded="true" height="82" name="Execute Script" width="90" x="581" y="34">
 <parameter key="script" value=" import java.util.logging.Level&#10; import com.rapidminer.tools.LogService;&#10; import com.rapidminer.tools.Ontology;&#10; import com.rapidminer.example.utils.ExampleSetBuilder;&#10; import com.rapidminer.example.utils.ExampleSets;&#10;// we assume only one layer&#10;model = input[0]&#10;&#10;int NumberOfNodes = model.innerNodes.size()&#10;int weightLength = model.innerNodes[10].getWeights().size()&#10;&#10;for(int k = 0; k &lt; NumberOfNodes; ++k){&#10;&#9;if(model.innerNodes[k].layerIndex == 0)&#10;&#9;&#9;weightLength = model.innerNodes[k].getWeights().size();&#10;&#10;}&#10;attributes= new Attribute[weightLength];&#10;for(int n = 0; n &lt; weightLength; n++){&#10; &#9;attributes[n] = AttributeFactory.createAttribute(&quot;Weight_&quot;+Integer.toString(n), Ontology.REAL);&#10;}&#10;ExampleSetBuilder builder = ExampleSets.from(attributes)&#10;double[] row = new double[weightLength]&#10;&#10;for(int n = 0; n &lt; NumberOfNodes; n++){&#10;&#9;innerNodes = model.innerNodes[n]&#10;&#9;&#10;&#9;weights = innerNodes.getWeights()&#10;&#9;LogService.root.log(Level.SEVERE,Integer.toString(weights.size()))&#10;&#9;LogService.root.log(Level.SEVERE,Integer.toString(weightLength))&#10;&#9;for(int i = 0; i &lt; weights.size(); ++i){&#10;&#9;&#9;if(innerNodes.layerIndex == 0){&#10;&#9;&#9;&#9;row[i] = weights[i]&#10;&#9;&#9;}&#10;&#9;&#9;//LogService.root.log(Level.INFO,Double.toString(weights[i]))&#10;&#9;}&#10;&#9;if(innerNodes.layerIndex==0)&#10;&#9;&#9;builder.addRow(row)&#10;}&#10;&#10;return builder.build()"/>
 </operator>
 <connect from_op="Retrieve Sonar" from_port="output" to_op="Neural Net" to_port="training set"/>
 <connect from_op="Neural Net" from_port="model" to_op="Execute Script" to_port="input 1"/>
 <connect from_op="Execute Script" from_port="output 1" to_port="result 1"/>
 <portSpacing port="source_input 1" spacing="0"/>
 <portSpacing port="sink_result 1" spacing="0"/>
 <portSpacing port="sink_result 2" spacing="0"/>
 </process>
 </operator>
</process>