Superfund Database

A comprehensive system for managing and analyzing investment data. Import CSV files, classify investments, perform FIGI lookups, and run SQL queries on your data.

Getting Started

1. Request Access

To use the Superfund Database, you need to be logged in as a superuser.

  • Click the "Login" link in the top navigation
  • If you don't have an account, approach a member of the AFR datajournalism team
  • Once logged in, you'll have access to all features of the application

2. Import Your Data

Start by importing investment data from CSV files.

  • Navigate to Import CSV in the main menu
  • Upload your CSV file containing investment data
  • The system will validate and import the data into the database
  • Monitor the import progress in real-time using the task panel

3. Process Your Data

After importing, classify your investments and perform FIGI lookups.

  • Go to Classify Import in the main menu
  • Select the import you want to process
  • Run the classification tasks in sequence:
    • Classify Investments - Groups unclassified investments by security ID and name, creating Classification records
    • FIGI Lookup - Looks up unlinked classifications against existing entities or the OpenFIGI API
    • Perfect Name Match - Matches unlinked classifications to existing entities using Nilsimsa score for medium confidence
    • Fuzzy Name Match - Groups remaining unprocessed classifications by fuzzy name similarity with low confidence
  • Monitor progress and view statistics for each processing step

4. Query Your Data

Use the SQL Query Tool to analyze your processed data.

  • Navigate to Query Tool in the main menu
  • Write and execute SELECT queries against your data
  • View results in a paginated table format
  • Export query results to CSV for further analysis
  • Use the query history to reuse previous queries
Security & Performance Notes
  • Only SELECT queries are allowed in the Query Tool
  • Queries are limited to 45 seconds execution time and 100,000 rows
  • All data processing happens asynchronously in the background
  • Real-time progress updates are provided via WebSocket connections

Quick Links

Data Tables

The system manages the following main data entities:

  • Investments - Individual investment records
  • Classifications - Categorized investment types with FIGI data
  • Entities - Companies and organizations
  • Imports - CSV import batches and metadata

System Architecture

The Superfund Database is built on Django with the following key components:

All data processing tasks run asynchronously, providing real-time progress updates through WebSocket connections. The system is designed for high performance and reliability when handling large datasets.


Version Information

App Version
2.2.0
Core Version
1.1.6
Git Commit
80e4cc6
Dirty State