TalentNet Readies Generative AI Features at Enterprise Scale with OpsGuru
Talk to a Cloud Expert
Background
New Ambitions
In 2023, companies around the world rushed to understand how new generative artificial intelligence (AI) models like OpenAI’s ChatGPT could fuel innovation in their businesses.
TalentNet was quick to recognize the opportunity. As a nimble 50-person private software company headquartered in Toronto, Canada, TalentNet serves major brands with its web-based software-as-a-service (SaaS) Talent Acquisition platform for Direct Sourcing, a concept it originated.
Organizations establish private Talent Pools for hiring fleets of contractors, an online destination where candidates can view jobs and directly apply, providing a more seamless recruitment experience. Talent Acquisition specialists use the platform to cultivate and communicate with candidates, as well as more efficiently manage hiring processes.
Although TalentNet already uses AI to match and score candidates against job descriptions, the Development Team recognized the potential of generative AI could provide in significantly elevating the platform’s capabilities and set it apart in the market. It began by using some of ChatGPT’s functionality to allow Curators, at the click of a button, to improve job descriptions for stronger matches. “Realistically, that feature is not technically difficult,” says Shawn Duggan, VP, App Development & Platform Engineering, “but generative AI was something new for us. This was a way for us to dip our toes in.”
What TalentNet wanted to do next, however, was more ambitious
The Challenge
Scaling to Production with TalentNavigator
The development team created a tool called TalentNavigator, with the aim of radically improving the efficiency and workflow of how the platform is used: instead of pointing and clicking to execute searches and filter results, Curators could open a chatbot and use natural language prompts to generate results from their data almost instantly.
As a proof of concept, TalentNavigator worked well. The tool was built with third-party frameworks — reusable pieces of code for applications — relying on a standard embeddings flow. OpenAI was selected for both the embeddings and the large language model (LLM), with a containerized deployment of Weaviate open-source vector database. The tool ingested job applicants’ resumes into the vector database, which assigned numerical representation to textual information. Duggan and his team could provide the job description to the LLM and ask it to search for application resumes that matched in the vector database, which would return accurate answers.
“It was a cool demo, but not ultimately useful,” says Duggan.
“It was limited in scope to only resumes, so it did not use the full breadth of our data, and we weren’t able to tie that to other actions or do much with it.”
The team started to struggle with moving from the demo environment to the live production system at scale for enterprise customers, but envisioned more ambitious enhancements as well, like making the LLM aware of different types of data in their systems that could help them better match to a potential new role. “We wanted to understand how to link embeds in the vector database to other embeds,” says Duggan. “That was something new for us, and we needed to save time and avoid pitfalls.”
Our Solution
A Clear Path Forward for Generative AI
TalentNet participated in an AWS program to incentivize migration to AWS Cloud using trusted AWS partners. AWS recommended OpsGuru and its Clear Path Forward program, a systematic approach to identifying opportunities to adopt, improve or optimize Generative AI tools and services. Starting with a technical deep dive assessment, OpsGuru conducts a detailed review of their client’s existing data, analytics, and AI/ML capabilities and then produces a comprehensive report with an actionable roadmap, all tailored to an organization’s technical or business goals.
In TalentNet’s case, it wanted to focus solely on the technical requirements of taking its Generative AI tool to production. Duggan and his team laid out multiple requirements:
- Enhanced search capability, including Boolean queries and multiple data sources
- Multi-tenant support
- Availability, scalability, and load management plan
- Integrated data sources
- Data security and privacy framework
- Bias prevention mechanism for ethical AI considerations
- Compliance with audit and regulations
- Advanced resume processing
- Ability to create workflows or take actions
Cost was an additional factor, given that LLMs charge based on the volume of interactions, including the amount of text it needs to parse, such as with a resume or job description. “As a small company, cost-efficiency is a huge consideration because costs can quickly escalate,” says Duggan. “While we want to offer leading-edge features, we must do so sustainably.”
Closing Knowledge Gaps
Over the course of five weeks, OpsGuru met with TalentNet in a series of workshops to gather inputs, outline their initial architecture suggestions and collect feedback, an iterative process that ensures the final recommendations meet their clients’ needs.
Throughout the meetings, TalentNet requested extensive detail. “By the end, they had an understanding of where their knowledge gaps are, and what they need to address,” says Alex Talesnik, Director of Data and AI at OpsGuru, who adds, this is quite common given how relatively new Generative AI is and the almost daily advances in the field. “There are very few experts in the industry right now that have best practices around deploying these kinds of stacks.”
Duggan says the exercise was very useful, “We learned a lot working with OpsGuru, just from how they approached the problem and the kinds of questions that they were asking.”
A New Architecture
The Clear Path Forward report provided recommendations for how to re-architect the tool, as well as a roadmap for how to get it into production. “We wanted to find any avenues for improvement, whether performance, cost, or scalability,” says Talesnik. The report also provided detailed explanations and additional context behind the guidance.
OpsGuru highlighted a few key areas. One was best practices for LLM flows, introducing TalentNet to Retrieval-Augmented Generation (RAG): this approach improves the performance of natural language processing tasks by first filtering and selecting text passages from the vector database that are likely to contain relevant information (retrieval), then using them to provide additional context to the prompt (augment) and help the LLM produce more fluent and coherent responses (generation).
Another was an agent-based approach to problem solving. Because LLMs are simply input-output mechanisms that don’t inherently know how to interact with any information outside of the text it is provided, OpsGuru introduced TalentNet to third-party frameworks. These offer LLMs a way to trigger functions, such as an API call to gather additional relevant information from the Internet. “That’s a big piece of the modernized architecture we provided for them,” says Talesnik.
The proposed architecture introduced to TalentNet “a breadth of new technologies,” according to Duggan, many of which are native to the AWS platform:
- Amazon Kendra - an intelligent search service powered by machine learning
- Amazon Lex - for building conversational interfaces into an application using voice and text
- OpenSearch - to ingest, search, visualize, and analyze data
- Amazon Bedrock - a managed service that offers a choice of high-performing foundation models (FMs) via a single API, along with a broad set of capabilities for security, privacy, and responsible AI
In Duggan’s view, Bedrock was an especially important recommendation for TalentNet because it would minimize technology risk. “It’s sort of an abstraction layer over the model,” he says, “so we could swap in other models and control what LLM we want to use for the particular task.”
Breaking out the application architecture into microservices was a strategic choice for OpsGuru, which cautions against taking monolithic approaches. “We always recommend an open approach to interacting with models, so you’re not relying on a single provider,” says Talesnik. “That way, if there’s a great new LLM that comes out in a week’s time, you don’t need to refactor all of your base code.
The Result
Making Informed Decisions
The Clear Path Forward report also provided a detailed comparison of alternative vector databases, with an explanation of different performance and service-level agreements they could expect to deliver to users and highlighted how to handle data security and governance when interacting with LLMs and what guardrails to build.
Finally, OpsGuru provided a costing model for AWS workloads with multiple scenarios so TalentNet could make an informed decision about how to move forward. The report modelled the current and future state of 5x and 10x growth in the application and outlined various options open to them. “Something that we had not really considered was the cost of ingestion,” says Duggan. “When a candidate uploads their resume into our platform, the platform has to parse that resume. What tool do you use? What are the trade-offs? OpsGuru brought a whole breadth of knowledge, asked great questions, and offered many different options, possibilities, and price points.”
Adds Duggan, “They took the time to document and explain all our questions and concerns. Their guidance and feedback are quite valuable and will inform how we move forward in formally launching TalentNavigator.”
A Transformational Opportunity
Duggan and his team are now prepared to take TalentNavigator to the next level. Having presented the initial proof of concept at a customer conference, TalentNet is convinced customers are ready.
The demo of TalentNavigator generated a lot of excitement. If our Generative AI interactive tools enables Curators to fill jobs faster by improving and adding efficiencies to the workflow while maintaining privacy, security and auditing for their data, our customers will be all for it.”
- Shawn Duggan, VP, App Development & Platform Engineering
But Duggan also knows that developing Generative AI tools requires strategic foresight as much as coding skill and that the proposed architecture devised by OpsGuru in its Clear Path Forward report is just a starting point. “It’s very much a shifting landscape. A design from even a month ago may not be the best solution now. It is a fast-moving technology playing field, and we’re just at the beginning.” With newfound insights and a robust roadmap, TalentNet stands ready to continue to pioneer innovation in the era of Generative AI.