Mastering Development

How to remove duplicate named entities from Azure Cognitive Search EntityRecognition skill output?

I am using an Azure Cognitive Search skillset that includes the EntityRecognition skill to find all people, locations, and organizations from within blobs in Azure Storage.

When I run the skill, with varying minimumPrecision values, it always returns a list with duplicate values.

Is there a way to tell the skill to remove duplicates? or do I need to create a custom skill that processes the results of the EntityRecognition skill to remove said duplicates?

enter image description here

Leave a Reply

Your email address will not be published. Required fields are marked *