How to change the value of the attribute 'label' in text classification?

mdc

Hi,

I am doing text classification using modelapplier. The output contains the "prediction(label)" attribute and all the "confidence(x1)", "confidence(x2)", etc. attributes. The prediction(label) depends on whichever has the maximum confidence value.

What I want to do is to change the "label" attribute equal to the "prediction(label) attribute but only on those with confidence value greater than let say 0.5.

I'm sure it can be done in RM but I couldn't figure out which operator to use, or if there is other way of achieving my purpose. ???

thanks
Matthew

land

Hi,
this should be very easy using the AttributeConstruction operator. You might define an if condition there, depending on the confidence and take the value of the prediction or label attribute depending on the outcome.

Greetings,
  Sebastian

mdc

Hi,

I looked at the AttributeConstruction operator. I think the 'if' function is what I need but I am still missing something. Since I have many "confidence(xxx)" attributes,  I need to create another attribute that is equal to the maximum of all confidence attributes and use this for the 'if' condition. I see that the AttributeConstruction operator has the 'max(x,y,x...) operator but I don't want to list all the confidence attributes since these can vary. Is there a way to get the maximum value from all "confidence*" attributes?

thanks,
Matthew

land

Hi,
it would be easy to get the maximum value of these attributes, but in fact, you don't want to have this value alone, but additionally you need the information, with with class it was associated. Otherwise, you could use the aggregation operator, I guess...

Greetings,
  Sebastian

mdc

Thanks  again for the reply. I am going to use the maximum value together with the 'prediction(label)' attribute.

I tried the aggregation operator but it doesn't seem to work on special attributes. Is this correct? I am now trying to figure out how to convert all the special attributes to regular attributes without using the ChangeAttribueRole operator. The reason is that I have many special attributes and their number can vary. Can you suggest a way to do this?

thanks,
Matthew

land

Hi,
you can use the AttributeSubsetPreprocessing operator. If configured using all attributes and with a checked "processe special attributes" this operator will remove the special attribut roles for its inner operators and restore them later on.

Greetings,
  Sebastian

mdc

I used the AttributeSubsetPreprocessing operator and it works fine now.

Thanks,
Matthew