Role

I acted as the Product Manager and Developer, I was responsible for:

Why Are We Building This?

The CX team, at a wealth-tech platform, faces challenges in quickly answering product-centric questions from customers due to a lack of in-depth product understanding. Custom product comparisons are time-consuming as the team needs to navigate through multiple documents to compare products. This leads to:

What Is the Solution?

To address these challenges, I proposed building a Gen AI RAG (Retrieval Augmented Generation) system that follow below architecture.

Key building blocks

How it was implemented

Tech Stack

Coding

Import the core libraries such as Langchain, Pandas and OpenAI.

import pandas as pd
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain.agents import Tool, AgentExecutor
# ... (other imports)

Configuration and Environment

The chatbot's environment is carefully configured to ensure optimal performance. We set up the OpenAI API key, initialize the language model and embeddings, and configure the Pinecone vector store for efficient information retrieval.

llm_model = "gpt-4o"
os.environ["OPENAI_API_KEY"] = "--"
llm = ChatOpenAI(model=llm_model, temperature=0)
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

Specialized Agent Functions

The heart of chatbot lies in its specialized agent functions. These functions are designed to handle specific types of financial queries with precision:

Each agent function is tailored to its specific domain, ensuring accurate and relevant responses.

Tool Definitions

The chatbot's capabilities are extended through a set of custom tools:

SubsCutoffTool = Tool(
    name='Subscription Cutoffs',
    func=subs_cutoffs_agent,
    description="Answers questions about kristal cutoff dates and applicable month-end NAV dates for subscriptions"
)
# ... (other tool definitions)

These tools encompass various financial operations, from handling cutoff dates to performing complex calculations and querying JSON data.

The Brain: Agent Setup

The chatbot's intelligence is powered by a sophisticated agent setup:

agent = create_react_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools,
                               memory=conversational_memory,
                               handle_parsing_errors=True,
                               verbose=True)

This setup includes a react agent with access to all defined tools, conversational memory to maintain context, and error handling for robustness.

Query Enhancement

One of the most important things to finetune the response of the LLM.

We have implemented several prompting techniques:

User Interface: Streamlit App

The chatbot is wrapped in a sleek Streamlit web application, providing an intuitive interface for users to interact with the system:

def main():
    st.title("Kristal Chatbot")
    user_input = st.text_input("User: ")
    if st.button("Ask", use_container_width=True):
        # Process query and display response

This interface allows users to input their financial queries and receive instant, detailed responses.

Evaluation & Feedback

I am developing a robust evaluation mechanism to benchmark its performance against industry standards. The approach involves comparing our custom model with leading market-standard language models using a single, carefully selected financial document as input. This method allows us to assess our chatbot's ability to comprehend and generate accurate responses based on limited but diverse financial information. The scales on which model will be evaluated is - Helpfulness, Honesty and Harmlessness (HHH).

References