DNN.png

Credits: Leveraging Deep Learning for Multilingual Sentiment Analysis - blog.aylien.com

Deep Learning Email Classification, Entity Extraction & Sentiment Analysis

Why?

User Problem

While engaging candidates users need to go each candidate reply to segregate them into actionable buckets - to be rejected, to be interviewed, needs more information to make a decision, etc. This leads to a lot of manual work which could be dedicated to other things such as finding the right candidate, engaging them, running recruitment activities, etc. Also, since this is a lot of work, the processing rates are usually low leading to only a hand few candidates being processed while rest remain unprocessed leading to pool recruitment funnel ratios and bad candidate experience.

Business Objective

If recruiters can only be made to focus on the candidates which are highly interested and relevant recruitment process will move faster and have better efficiency. This increased efficiency and reduced time can be a great selling point for all of our products. Metrics this feature can impact:

  • Time to Process candidates (in days)

  • Coverage of Processed Candidates

My Role

My role in this was:

  1. Defining user problems & business goals

  2. Conducting user research and tech research into possible classification technologies

  3. Defining the product scope, requirements and algorithm quality standards

  4. Working with data science team and engineering teams to successfully test and analyze the model’s precision & recall, and defining the error rates via confusion metrics so the product UX could counter for errors in the model

  5. Launching, measuring, and tweaking

Solutioning Process

We used the below process to define to arrive at the product we should be building. User research, competitive understanding, feedback, constant iteration, etc played a very important role in defining and building the right product.

design process.png

Algorithm

We worked on a deep learning-based classification, entity extraction and sentiment analysis model, that would understand natural language and make sense out of it. We needed a dataset of about 11,000 email responses which were manually tagged and fed to the algorithm. The model's accuracy, precision and recall were tested & measured using a "confusion matrix" against a set of 1,000 mails which served as a test case (learning set was 10,000).

confusionMatrxiUpdated.jpg

Product

Product was a classification, entity extraction and sentiment analysis model that could read a candidate reply and then automate certain actions and extract a quick summary to help recruiters focus on only relevant and interested candidates.

The model could:

1. classify the response as "Interested", "Not Interested" and "Not Sure"

2. Extract entities such as resume, greeting, date, etc.

3. Understand the sentiment as positive or negative.

Classification.jpg
classifier.jpg

Metrics & Impact

Chart Classfiier.png
Classifier Chart2.png