Generative AI in Action: Boosting TalentNet’s Accuracy by 20%
Background
The talent acquisition and recruitment industry is essential for building a company’s workforce, focusing on efficiently and accurately matching candidates to roles. This process is time-sensitive, as delays can increase costs and lead to missed opportunities. Accurate matching is crucial to enhance productivity and reduce turnover, making it a key strategic function in competitive markets.
The Challenge
TalentNet, a 50-person private software company in Toronto, focused on providing a web-based SaaS talent acquisition platform to solve this problem by creating private talent pools for hiring fleets of contractors and providing a direct sourcing experience where candidates can view jobs and directly apply for jobs and talent acquisition specialists can communicate with candidates and manage the hiring processes seamlessly. However, as the volume of resumes continues to increase and role descriptions become more diverse, new solutions need to be explored to deliver a timely, seamless experience.
TalentNet is interested in leveraging language processing capabilities to address the problem. Working with OpsGuru, TalentNet identified three use cases where GenAI can significantly enhance performance. They are:
using GenAI to enhance the clarity, conciseness and compliance of job descriptions,
summarizing results from the data instantly and
ingesting job applicants’ resumes into vectors to more efficiently identify the fit of candidates and job postings.
Our Solution
As with any new technology, OpsGuru first started with a rapid proof of concept (POC) to validate the technology stack. After validating the POC, which consisted of a vector database and a few agents, TalentNet rapidly committed to productizing the solution due to its apparent value. This led to a complete redesign of the workflow and resources to ensure the performance and validity of the job-matching process and the security of sensitive personal information.
For the production system, multiple agents were configured to decompose queries and route the relevant parts to the respective resume, job posting, and company intelligent agents, enabling indexing into summary, embedding, and question indices to improve the overall performance and accuracy of semantic search. The end-to-end workflow leverages Claude on Amazon Bedrock as the foundation model and Weaviate as the vector database.
The Result
AWS best practices for data protection, including Amazon KMS for encryption, Amazon S3 for raw data storage, and AWS IAM for access control, have been adopted. At the same time, special care has been taken to ensure that ethical guidelines and governance are incorporated: benchmark tests using frameworks such as BLEU and ROUGE are used to measure the performance of the natural language processing workflows. A growing set of reference data has been incorporated into the testing process to regularly assess the fairness and basis of the matching and summarization. Human-in-the-loop feedback has also been used to provide feedback on the accuracy of the process–it has been found in the early rounds of testing that the Gen AI solution has introduced at least a 20% improvement of accuracy compared with the previous workflows, not to mention the improvement in the time taken to analyze resumes, job descriptions and find matching candidates to the job posts.
At the end of the implementation, OpsGuru also conducted knowledge transfer sessions to ensure TalentNet had full ownership of the solution. As a result, TalentNet is confident it can shepherd the productized solution and expand it to include additional use cases, alongside rapidly advancing GenAI technologies.
