Closest encoding to utf8mb4
I am working with social media data and all those emojis are driving me crazy, when I import them they are getting changed to system encoding and are a bunch of squiggles. What encoding is closets to utf8mb4 so that I can preserve the encoding when reading from a CSV?
Find more posts tagged with
Sort by:
1 - 4 of
41
kayman
New Altair Community Member
If your csv is utf8 you should just be able to read it when you import it as utf8. Never had any issues with this, as long as the source was properly encoded.
New Altair Community Member
OPNope, all emojis and non ascii characters are turning into their base encoding. Any idea why encoding is closest to UTF8MB4?
Altair EmployeeHi,
UTF8MB4 is a workaround for the broken UTF8 type in mysql which only supports up to 3 byte character.
In the csv export it should be just regular utf-8.
Maybe the selected RapidMiner Studio font doesn't contain all the smileys and is displaying squares instead?
Greetings,
Jonas
UTF8MB4 is a workaround for the broken UTF8 type in mysql which only supports up to 3 byte character.
In the csv export it should be just regular utf-8.
Maybe the selected RapidMiner Studio font doesn't contain all the smileys and is displaying squares instead?
Greetings,
Jonas
New Altair Community Member
OPAccepted Answer
@jwpfau when I am importing into the DB it is failing saying the character is not UTF8 with error message: Incorrect string value: '\xE2 \x94 \x82....'
This is basically all of the emojis that were being rejected. I was under the impression that I needed to set the encoding inside of Rapid Miner, however it was a change that was needed on the DB. Changing the free text field to TEXT and making the encoding UTF8mb4 sorted the issue out.
This is basically all of the emojis that were being rejected. I was under the impression that I needed to set the encoding inside of Rapid Miner, however it was a change that was needed on the DB. Changing the free text field to TEXT and making the encoding UTF8mb4 sorted the issue out.
Sort by:
1 - 1 of
11
New Altair Community Member
OPAccepted Answer
@jwpfau when I am importing into the DB it is failing saying the character is not UTF8 with error message: Incorrect string value: '\xE2 \x94 \x82....'
This is basically all of the emojis that were being rejected. I was under the impression that I needed to set the encoding inside of Rapid Miner, however it was a change that was needed on the DB. Changing the free text field to TEXT and making the encoding UTF8mb4 sorted the issue out.
This is basically all of the emojis that were being rejected. I was under the impression that I needed to set the encoding inside of Rapid Miner, however it was a change that was needed on the DB. Changing the free text field to TEXT and making the encoding UTF8mb4 sorted the issue out.