Combining tokenized n-grammed name variations

ngaved91
ngaved91 New Altair Community Member
edited November 2024 in Community Q&A

Hello :)

 

I have to count Name occurences in a text file. The problem is that the format of the names vary. For example:

 

a) Bratt (Forename) Pitt (Lastname)

b) Pitt Bratt

c) B. Pitt

d) Pitt B.

e) Pitt

 

The inclusion of the forename is also important in order to distinguish actors with the same lastname.

 

My Process looks like this:

1) Main Process: Process Documents from files -> Data to documents -> Process documents

2) Process documents sub process:
    Tokenize -> generate 2-grams -> replace tokens (to replace the "_" of the 2-grams with a space") -> Filter tokens using an example set (with an example set of actor names contain varations of name constellations as described above.)

 

The problem is that all variations are counted seperately. Is there a way to aggregate all results so that the final results shows "Bratt Pitt  Total occurences: xx."

 

@sgenzer 

Tagged:

Answers

  • Telcontar120
    Telcontar120 New Altair Community Member

    You can of course use "replace tokens" operator and enter the variations yourself, but this is very manual.

    The Aylien extension also does entity extraction for names, and that will handle common misspellings, etc.

    You might want to look into the extension from Namsor, they might have something for name grouping. 

    But I have a feeling you are not going to be able to find a solution that is 100% reliable.

     

Categories