nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Siemens Community Catalyst Program

The Siemens Community Catalyst program was co-created with our community to acknowledge technology leaders who consistently contribute to the Siemens Community. Nominations are accepted on a rolling basis.

Nominate Now

preprocessing: remove email signature

Joos

Hi
I am trying to apply LDA to emails. I have the mails in an excel file. My model works, but I have to find a way to remove the email signature. Does anyone have experience?

Thanks

Find more posts tagged with

AI Studio

Accepted answers

rfuentealba

Hi @Joos,

The problem is that messages have two parts: a header with a number of directions and a body containing text. You want to parse the "body".

The first answer can be demonstrated with tje following two "e-mails":

<b>From: Rodrigo <rodrigo@example.com>
To: Joos <joos@example.org></b>

Hello Joos,

This is an example message.

--
Rodrigo Fuentealba
Chile

<b>From: Joos <joos@example.org>
To: Rodrigo <rodrigo@example.com></b>

Hi Rodrigo,

I see. Even though the footers are different, there is something many users do, which is putting a -- before their signature. Not everyone follows this but a big part do.

--
Joos
Netherlands

Now, finding the last -- wouldn't work on all e-mails, because it is a convention but not a fact that people use -- on their e-mails to separate the signature from the rest. Let's think of another solution Let's say you have 1000 e-mails from me. If 300 of these e-mails end up with the classic "Sent from my iPhone" as the last line, you can identify that parameter and delete the e-mail. But what about all the e-mails that I sent with my own signature? You may be able to identify that 600 e-mails from rodrigo@example.com always have the "Rodrigo Fuentealba / Chile" signature, thus it can be removed.

Answering your other questions:

Yes, you can use Python code inside RapidMiner with the Python Scripting extension. However, the mail parser extension probably won't help you, this is a natural language processing (or pattern recognition) issue.
Yes, you can do the pattern recognition in Dutch. I don't speak it, but have done similar stuff in German.

All the best,

Rodrigo.

All comments

rfuentealba

Hello @Joos,

I can only recommend two ways. The first one is to remove everything from the last -- signs together to the end. Or, if you have the recipient of the e-mail, trim the message and check the last line on each e-mail until no last lines are the same.

Both aren't battle tested, and involve some processing that I wouldn't have done with RapidMiner but much earlier, while retrieving the e-mails, so you are better of trying your luck with loading your data with Python to remove the e-mail signatures, I'm afraid.

All the best,

Rodrigo.

Joos

Thank you for your answer. Not sure I understand your first option, because all the footers are different, so I do not know how to recognize them. I did find python code on github (mailparser). Is it possible to include this as python script in the code? I can include it in my loop going over the different mails and pass it on the python parser as a document? Probably the python code would need adjustment to get this working? Moreover, it would have to do the parsing in Dutch? Do you have experience in this? Your

rfuentealba

<b>From: Rodrigo <rodrigo@example.com>
To: Joos <joos@example.org></b>

Hello Joos,

This is an example message.

--
Rodrigo Fuentealba
Chile

<b>From: Joos <joos@example.org>
To: Rodrigo <rodrigo@example.com></b>

Hi Rodrigo,

I see. Even though the footers are different, there is something many users do, which is putting a -- before their signature. Not everyone follows this but a big part do.

--
Joos
Netherlands

Yes, you can use Python code inside RapidMiner with the Python Scripting extension. However, the mail parser extension probably won't help you, this is a natural language processing (or pattern recognition) issue.
Yes, you can do the pattern recognition in Dutch. I don't speak it, but have done similar stuff in German.

All the best,

Rodrigo.

Joos

Thanks Rodrigo...I kind of fixed the issue in excel with formulas

Quick Links

Upcoming Events

Advancing power electronics design with Simcenter PSIM and High-Performance Computing
Join us for a community conversation on how High-Performance Computing (HPC) and Simcenter PSIM work together to meet the increasing demands of modern power electronics simulation. During this one-hour technical session, you'll discover how leveraging HPC can speed up simulations, improve scalability, and empower engineers…
Getting Started with Faster Motor Building in Simcenter Simlab
Join us for a community conversation on accelerating your motor simulation workflow with Simcenter Simlab. In this beginner-friendly session, we'll introduce the platform's automated model-building capabilities and show how easy it is to set up your first electric motor analysis. You'll see the basic steps of geometry…
Practical hybrid solver workflows in Simcenter Feko
Join us for a Community Conversation where we'll discuss practical approaches to analyzing complex electromagnetic models using hybrid solver techniques. This session will provide a general overview of how different computational methods can be combined within a single simulation to address models containing varied…

View all community events