"Best practice in dealing with malformed XML"

robin

I am dealing with malformed xml that is using illegal characters as well as quatations all over the place. The first file from the client I went to the effort of correcting the XML issues after importing the file and before sending it through to the Read XML operator.

 

The next file from the client will be impossible to apply the same fixes as the file is real broken. I was wondering what the best practise was in terms of reading the file into a MySQL database. If I should be using RapidMiner or if I should be calling an external program to fix the issue before processing the data through the operator. 

 

If you recommend staying inside of RapidMiner what would your approach be in terms of tackeling the problem?

 

Kind regards

 

Robin

land

Hi,

I cannot agree more with Balazs, however, sometimes you cannot simply hit your customer if he delivers bad quality data. He might not be happy about that

Hence I would like to add two more things:
If these are random, probably manually created problems, than nor RapidMiner nor any external program can safely import that data. Then Balazs answer applies.

First check whether you are importing the file in the right encoding. This can create really funny effects if you don't!
If that's not the problem, check if there's a common rule behind all the problems. If there is, you can use the Text Processing extension to load the file as blank document and apply search and replace, possibly with regular expressions or even more to compensate that.

 

I would not recommend to treat the file as html as this probably will remove a lot of the content. May be valid XML then, but probably will contain only a blank html page and loose all information

 

Greetings,

Sebastian