How FinTech Startup Streamlined Customer Support with Custom AI

Executive Summary

FastFinance, a rapidly growing fintech startup offering SMB lending and financial management tools, faced significant challenges scaling their customer support operations. As their user base expanded, their small support team struggled to maintain response quality and speed while managing an increasingly complex knowledge base.

Divinci AI worked with FastFinance to develop a custom AI support system that combined advanced RAG (Retrieval-Augmented Generation) technology with domain-specific fine-tuning and human-in-the-loop workflows. The results were transformative:

This case study details how Divinci AI and FastFinance collaborated to achieve these results, the specific challenges encountered, and the technical approach that made it possible.

Client Background

Founded in 2021, FastFinance offers a suite of financial tools designed specifically for small and medium-sized businesses, including:

• Working capital loans with rapid approval
• Cash flow forecasting and management
• Automated bookkeeping and reconciliation
• Tax optimization and compliance tools
• Integration with popular accounting and payment platforms

By 2024, FastFinance had grown to serve over 15,000 businesses across the United States and Canada, with a team of 85 employees. Their support team consisted of just 12 specialists handling approximately 450 support tickets daily.

Technical Solution

After evaluating FastFinance's needs, we designed a comprehensive AI customer support system with several integrated components:

1. Unified Knowledge System

The first challenge was consolidating FastFinance's fragmented knowledge sources. We developed an automated knowledge ingestion pipeline that:

• Extracted information from product documentation, internal wikis, compliance guidelines, and past support tickets
• Processed and normalized content using domain-specific NLP techniques
• Created a hierarchical knowledge structure organized by product, feature, and topic
• Established automated update mechanisms to capture new information

2. Advanced RAG Implementation

At the core of the solution was our FinRAG system—a specialized implementation of Retrieval-Augmented Generation tailored for financial services:

• Domain-Specific Embeddings: We fine-tuned embedding models on financial terminology to improve semantic search accuracy
• Multi-stage Retrieval: Queries were decomposed and processed through a three-stage retrieval pipeline:
  • Initial broad retrieval to identify relevant knowledge areas
  • Focused retrieval within those areas for specific information
  • Contextual retrieval to gather supporting details
• Dynamic Context Management: The system intelligently prioritized and organized retrieved information based on relevance and importance
• Compliance Guardrails: Built-in verification against regulatory requirements and company policies

3. Human-in-the-Loop Workflows

We designed integrated workflows that balanced automation with human expertise:

• Confidence-Based Routing: The system automatically handled high-confidence responses while routing complex or uncertain cases to specialists
• Suggestion Mode: For specialist-handled tickets, the AI provided draft responses and relevant information snippets
• Feedback Loop: Every human edit or override was captured to improve the system's future responses
• Continuous Learning: The system regularly retrained on successful support interactions and new knowledge

4. Omnichannel Integration

The solution integrated with FastFinance's existing support channels:

• Web chat widget with real-time responses
• Email integration with Zendesk
• Voice support system providing talking points for phone agents
• Self-service knowledge base with AI-powered search

5. Analytics and Monitoring

We implemented comprehensive monitoring to ensure quality and identify improvement opportunities:

• Real-time accuracy monitoring with random human verification
• Customer satisfaction tracking through post-interaction surveys
• Knowledge gap identification to flag areas needing documentation
• Performance dashboards for support team managers

Implementation Process

Results and Impact

The impact of the AI customer support system was immediately apparent and continued to improve over time:

Operational Improvements

• Response Time: Average first response time decreased from 8 hours to just 1.75 hours (78% reduction)
• Resolution Rate: 65% of tickets resolved without human intervention, rising to 72% by the third month
• Support Capacity: The same team of 12 specialists now handles over 1,400 tickets daily (3.2x increase)
• Knowledge Base Quality: 85% reduction in knowledge inconsistencies through automated verification

Customer Experience Improvements

• Satisfaction Scores: CSAT scores increased from 72% to 92% (42% improvement)
• Self-Service Rate: 38% increase in customers resolving issues through the knowledge base
• Repeat Interactions: 45% reduction in customers needing multiple contacts to resolve an issue
• Support NPS: Improved from +15 to +62

Additionally, the system created several unexpected benefits:

• Identified product friction points through analysis of common support queries
• Provided valuable training examples for new support specialists
• Generated insights for product development team
• Reduced onboarding time for new support team members by 60%

Key Learnings and Best Practices

Through our work with FastFinance, we identified several critical success factors for implementing AI customer support systems in financial services:

1. Domain-Specific Training Is Essential

General-purpose language models, even with RAG capabilities, were insufficient for financial support. Domain-specific fine-tuning with financial terminology and use cases improved accuracy by over 30% compared to off-the-shelf solutions.

2. Thoughtful Human-AI Collaboration

The most successful approach was not fully automated or human-only, but a carefully designed collaboration where:

• AI handled routine queries and information gathering
• Specialists focused on complex cases, relationship building, and quality assurance
• The system learned from human interventions to continuously improve

3. Multi-Stage Retrieval for Complex Domains

Single-stage retrieval wasn't sufficient for complex financial questions that often required information from multiple knowledge areas. The multi-stage approach improved relevant information retrieval by 45%.

4. Verification Mechanisms for High-Stakes Content

For financial services, we implemented additional verification layers to ensure responses were:

• Factually accurate based on source documentation
• Compliant with financial regulations
• Consistent with FastFinance's policies
• Appropriately cautious when uncertainty existed

5. Analytics-Driven Improvement

The system's performance improved over time through a structured analytics approach:

• Tracking which knowledge sources led to successful resolutions
• Identifying patterns in human-edited responses
• Analyzing customer follow-up questions to detect gaps
• Regular retraining cycles with verified high-quality interactions

Conclusion

The FastFinance implementation demonstrates the transformative potential of custom AI solutions for specialized support functions. By combining advanced RAG technology with domain expertise and thoughtful human-AI workflows, we were able to help FastFinance dramatically scale their support operations while improving quality.

The key to success wasn't just sophisticated technology, but a careful understanding of the financial domain, support workflows, and customer needs. This holistic approach enabled us to design a solution that balanced automation with human expertise, creating a system that continuously improves over time.

For financial services organizations facing similar support challenges, this case study provides a proven framework for implementing AI-powered support that enhances both operational efficiency and customer experience.