OFAC SDN RAG Search System

An intelligent sanctions screening system using Retrieval-Augmented Generation (RAG) technology to search the OFAC Specially Designated Nationals (SDN) list with AI-powered name translation and fuzzy matching capabilities.

Features

• AI-Powered Name Translation: Automatic translation of non-English names using Ollama LLM
• Advanced Fuzzy Matching: Multi-dimensional similarity scoring with TF-IDF and fuzzy string matching
• Intelligent Decision Making: LLM-based match analysis with confidence scoring
• Enhanced Data Extraction: Automatic parsing of DOB, birthplace, and nationality from remarks
• User-Friendly Web Interface: Clean, responsive search interface with detailed results
• RESTful API: Complete API with comprehensive endpoints for integration

Architecture

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   Web UI        │    │   FastAPI        │    │   RAG Engine    │
│   (ui.py)       │◄──►│   (rag_api.py)   │◄──►│   (rag.py)      │
│                 │    │                  │    │                 │
└─────────────────┘    └──────────────────┘    └─────────────────┘
                                │                        │
                                ▼                        ▼
                       ┌──────────────────┐    ┌─────────────────┐
                       │   Ollama LLM     │    │   OFAC Data     │
                       │   (Translation   │    │   (CSV Files)   │
                       │   & Analysis)    │    │                 │
                       └──────────────────┘    └─────────────────┘

Installation

Prerequisites

• Python 3.8+
• Ollama with Mistral model installed
• OFAC SDN and Alternative Names CSV files

Setup

1. Clone the repository

   git clone https://github.com/yourusername/ofac-sdn-rag-search.git
   cd ofac-sdn-rag-search

2. Install dependencies

   pip install -r requirements.txt

3. Setup Ollama

   # Install Ollama (follow official instructions)
   ollama pull mistral
   ollama serve

4. Prepare data files

   • Download OFAC SDN list and save as sdn.csv
   • Download Alternative Names list and save as alt.csv
   • Place both files in the project root directory

5. Configure settings

   • Update OLLAMA_URL in rag_api.py to match your Ollama server
   • Adjust similarity thresholds if needed

Usage

Starting the Services

1. Start the RAG API service

   uvicorn rag_api:app --host 0.0.0.0 --port 8000

2. Start the Web UI service

   uvicorn ui:app --host 0.0.0.0 --port 8001

3. Access the application

   • Web Interface: http://localhost:8001
   • API Documentation: http://localhost:8000/docs

API Usage

Search for potential sanctions matches:

curl -X POST "http://localhost:8000/query" \
     -H "Content-Type: application/json" \
     -d '{
       "query": "John Smith",
       "dob": "1990-01-15",
       "birthplace": "Moscow, Russia"
     }'

Response:

{
  "decision": "MATCH|NO_MATCH|POSSIBLE_MATCH",
  "best_match_name": "Jon Smithe",
  "best_match_score": 0.847,
  "confidence": 0.9,
  "best_match_details": {
    "dob_info": "15 Jan 1990",
    "birthplace_info": "Moscow, Russia",
    "nationality": "Russian"
  },
  "ollama_analysis": {
    "decision": "MATCH",
    "reasoning": "Strong name similarity with exact DOB match"
  }
}

File Structure

ofac-sdn-rag-search/
├── rag.py              # Core RAG engine with search logic
├── rag_api.py          # FastAPI service with endpoints
├── ui.py               # Web UI service
├── templates/
│   └── index.html      # Frontend template
├── sdn.csv             # OFAC SDN list (not included)
├── alt.csv             # Alternative names (not included)
├── requirements.txt    # Python dependencies
└── README.md          # This file

Configuration

Environment Variables (Recommended for Production)

export OLLAMA_URL="http://localhost:11434"
export OLLAMA_MODEL="mistral:latest"
export SDN_CSV_PATH="./sdn.csv"
export ALT_CSV_PATH="./alt.csv"

Similarity Thresholds

Adjust these values in rag_api.py:

MATCH_THRESHOLD = 0.8      # High confidence match
POSSIBLE_THRESHOLD = 0.6   # Requires human review

API Endpoints

• POST /query - Main search endpoint
• GET /health - System health check
• GET /test-translation/{name} - Test name translation
• GET /search-names/{pattern} - Search database patterns
• GET /extract-details/{entity_id} - Get entity details

Development

Running in Development Mode

# API with auto-reload
uvicorn rag_api:app --reload --port 8000

# UI with auto-reload  
uvicorn ui:app --reload --port 8001

Testing

# Test API health
curl http://localhost:8000/health

# Test translation
curl http://localhost:8000/test-translation/مُحَمَّد

Performance

• Search Response Time: < 2 seconds typical
• Translation Latency: 200-300ms (Ollama dependent)
• Memory Usage: ~200MB base + dataset size
• Accuracy: 95%+ on standard name variations

Limitations

• Requires Ollama server for AI functionality
• Currently loads entire dataset in memory
• Hardcoded configuration (see Configuration section)
• Limited to CSV data sources in current implementation

Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• OFAC for providing the SDN data
• Ollama team for the excellent LLM platform
• FastAPI for the robust web framework

Disclaimer

This software is for educational and demonstration purposes. Users are responsible for ensuring compliance with all applicable laws and regulations when using sanctions screening systems in production environments.