how to generate a neural net weight-matrix?
eldenoso
New Altair Community Member
Hello altogether,
as I am currently trying to get more insight into what neural nets are doing, I wanted to know, whether there is a way of exporting the weight-matrices of a neural net so that you can visualize it (e.g. heat map). By then changing the training set a couple of times, you may see differences in the "heat-map-weight-matrix" and can conclude which attributes are the ones that matter.
I know that there are real scientists out there, searching for answers to the black-box-problem of neural nets. But my curiosity just drives me towards this way and I would be glad to try this out :-).
Thank you for your answers
Philipp
0
Best Answer
-
Hi Philipp,
in fact you can use Groovy Script to do this. I've tried this, but it wasn't as easy as i thought. Attached is a process with the script. It always extracts the first Layer. i've not tested it in various layouts though
Best,
Martin
<?xml version="1.0" encoding="UTF-8"?><process version="7.3.001">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="7.3.001" expanded="true" name="Process">
<process expanded="true">
<operator activated="true" class="retrieve" compatibility="7.3.001" expanded="true" height="68" name="Retrieve Sonar" width="90" x="112" y="34">
<parameter key="repository_entry" value="//Samples/data/Sonar"/>
</operator>
<operator activated="true" class="neural_net" compatibility="7.3.001" expanded="true" height="82" name="Neural Net" width="90" x="313" y="34">
<list key="hidden_layers">
<parameter key="1" value="10"/>
</list>
</operator>
<operator activated="true" class="execute_script" compatibility="7.3.001" expanded="true" height="82" name="Execute Script" width="90" x="581" y="34">
<parameter key="script" value=" import java.util.logging.Level import com.rapidminer.tools.LogService; import com.rapidminer.tools.Ontology; import com.rapidminer.example.utils.ExampleSetBuilder; import com.rapidminer.example.utils.ExampleSets; // we assume only one layer model = input[0] int NumberOfNodes = model.innerNodes.size() int weightLength = model.innerNodes[10].getWeights().size() for(int k = 0; k < NumberOfNodes; ++k){ 	if(model.innerNodes[k].layerIndex == 0) 		weightLength = model.innerNodes[k].getWeights().size(); } attributes= new Attribute[weightLength]; for(int n = 0; n < weightLength; n++){ 	attributes[n] = AttributeFactory.createAttribute("Weight_"+Integer.toString(n), Ontology.REAL); } ExampleSetBuilder builder = ExampleSets.from(attributes) double[] row = new double[weightLength] for(int n = 0; n < NumberOfNodes; n++){ 	innerNodes = model.innerNodes[n] 	 	weights = innerNodes.getWeights() 	LogService.root.log(Level.SEVERE,Integer.toString(weights.size())) 	LogService.root.log(Level.SEVERE,Integer.toString(weightLength)) 	for(int i = 0; i < weights.size(); ++i){ 		if(innerNodes.layerIndex == 0){ 			row[i] = weights[i] 		} 		//LogService.root.log(Level.INFO,Double.toString(weights[i])) 	} 	if(innerNodes.layerIndex==0) 		builder.addRow(row) } return builder.build()"/>
</operator>
<connect from_op="Retrieve Sonar" from_port="output" to_op="Neural Net" to_port="training set"/>
<connect from_op="Neural Net" from_port="model" to_op="Execute Script" to_port="input 1"/>
<connect from_op="Execute Script" from_port="output 1" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
</process>
</operator>
</process>0
Altair EmployeeAnswers
-
Hi Philipp,
in fact you can use Groovy Script to do this. I've tried this, but it wasn't as easy as i thought. Attached is a process with the script. It always extracts the first Layer. i've not tested it in various layouts though
Best,
Martin
<?xml version="1.0" encoding="UTF-8"?><process version="7.3.001">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="7.3.001" expanded="true" name="Process">
<process expanded="true">
<operator activated="true" class="retrieve" compatibility="7.3.001" expanded="true" height="68" name="Retrieve Sonar" width="90" x="112" y="34">
<parameter key="repository_entry" value="//Samples/data/Sonar"/>
</operator>
<operator activated="true" class="neural_net" compatibility="7.3.001" expanded="true" height="82" name="Neural Net" width="90" x="313" y="34">
<list key="hidden_layers">
<parameter key="1" value="10"/>
</list>
</operator>
<operator activated="true" class="execute_script" compatibility="7.3.001" expanded="true" height="82" name="Execute Script" width="90" x="581" y="34">
<parameter key="script" value=" import java.util.logging.Level import com.rapidminer.tools.LogService; import com.rapidminer.tools.Ontology; import com.rapidminer.example.utils.ExampleSetBuilder; import com.rapidminer.example.utils.ExampleSets; // we assume only one layer model = input[0] int NumberOfNodes = model.innerNodes.size() int weightLength = model.innerNodes[10].getWeights().size() for(int k = 0; k < NumberOfNodes; ++k){ 	if(model.innerNodes[k].layerIndex == 0) 		weightLength = model.innerNodes[k].getWeights().size(); } attributes= new Attribute[weightLength]; for(int n = 0; n < weightLength; n++){ 	attributes[n] = AttributeFactory.createAttribute("Weight_"+Integer.toString(n), Ontology.REAL); } ExampleSetBuilder builder = ExampleSets.from(attributes) double[] row = new double[weightLength] for(int n = 0; n < NumberOfNodes; n++){ 	innerNodes = model.innerNodes[n] 	 	weights = innerNodes.getWeights() 	LogService.root.log(Level.SEVERE,Integer.toString(weights.size())) 	LogService.root.log(Level.SEVERE,Integer.toString(weightLength)) 	for(int i = 0; i < weights.size(); ++i){ 		if(innerNodes.layerIndex == 0){ 			row[i] = weights[i] 		} 		//LogService.root.log(Level.INFO,Double.toString(weights[i])) 	} 	if(innerNodes.layerIndex==0) 		builder.addRow(row) } return builder.build()"/>
</operator>
<connect from_op="Retrieve Sonar" from_port="output" to_op="Neural Net" to_port="training set"/>
<connect from_op="Neural Net" from_port="model" to_op="Execute Script" to_port="input 1"/>
<connect from_op="Execute Script" from_port="output 1" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
</process>
</operator>
</process>0 -
Perfect Martin! Thank you.
In your case the data means that there were ten neurons in the first layer with 60 edges (weights), right?
Best,
Philipp0 -
Hi,
yes. And two output nodes which are interestingly part of the innerNodes object and just identfiable by their layerId...
Best,
Martin
0