Building the Company Intelligence Platform

When a supplier wins a government tender in South Africa, the award record often contains little more than a company name and an amount. The company might be listed as "Thabo's Construction PTY LTD" on one award and "Thabos Construction (Pty) Ltd" on another — or even as a different trading name entirely. Connecting these records to a single, verified business entity is the core challenge of company intelligence.

Tenders-SA's Company Intelligence platform solves this by combining four data layers: CIPC (Companies and Intellectual Property Commission) registry data, public tender award history, multi-source enrichment, and company claims from verified business owners. This article explains how we built each layer and the normalization infrastructure that ties them together.

Layer 1: Company Name Normalization

Everything starts with name normalization. Award data comes from 50+ government sources, each with its own formatting conventions. A single company might appear as "ACME Cleaning Services (Pty) Ltd", "Acme Cleaning Services Pty Ltd", "ACME CLEANING SERVICES", or "Acme Cleaning CC". We needed a deterministic normalizer that could collapse all variations into a canonical form.

Copy
1export function normalizeCompanyName(name: string | null | undefined): string {
2    if (!name) return '';
3    return name
4        .trim().toUpperCase().replace(/\s+/g, ' ')
5        .replace(/^(THE|T\/A|TA)\s+/i, '')           // Strip legal prefixes
6        .replace(/\.?\s*PTY\s*LTD\.?$/gi, ' PTY LTD') // Standardize suffixes
7        .replace(/\.?\s*LTD\.?$/gi, ' LTD')
8        .replace(/\.?\s*CC\.?$/gi, ' CC')
9        .replace(/\.?\s*NPC\.?$/gi, ' NPC')
10        .replace(/[.,;:"]+/g, '')                       // Strip punctuation
11        .trim();
12}
TYPESCRIPT

This normalizer is applied at every ingestion point: when tenders arrive from the scraper pipeline, when CIPC data is imported, and when company profiles are matched against award records. It's the foundational building block for entity resolution.

Layer 2: Canonical Organization Name Resolution

Beyond individual company names, we needed to resolve the names of buying organizations (departments, municipalities, SOEs) to canonical forms. The organization resolution engine uses a three-step process:

Copy
1export function resolveCanonicalOrganizationName(rawName: string): {
2    canonicalName: string;
3    displayName: string;
4    type: CanonicalEntityType;
5    source: CanonicalSource;
6    confidence: number;
7} {
8    const normalized = normalizeCompanyName(rawName);
9
10    // Step 1: Exact match against canonical names
11    const exactMatch = ORGANIZATION_REGISTRY.find(o => o.canonicalName === normalized);
12    if (exactMatch) return { ...exactMatch, source: 'exact', confidence: 1.0 };
13
14    // Step 2: Alias match against ORGANIZATION_NAME_VARIATIONS
15    const aliasMatch = findAliasMatch(normalized);
16// ... (truncated)
Show More (5 more lines)
TYPESCRIPT

The alias dictionary maps hundreds of known variations: "Eskom Holdings SOC Ltd" → "Eskom", "City of Johannesburg Metropolitan Municipality" → "CoJ", "Department of Public Works and Infrastructure" → "DPWI". This matters because the same buying organization might be listed differently across sources, and we need to aggregate award data accurately.

Layer 3: Location Normalization

Tenders are published with location information that ranges from precise ("123 Main Street, Pretoria, Gauteng") to vague ("Northern Cape") to nonexistent. We built a province resolution system that handles this spectrum:

Copy
1export const PROVINCE_REGISTRY: readonly ProvinceEntry[] = [
2    {
3        id: 'gauteng',
4        canonicalName: 'Gauteng',
5        slug: 'gauteng',
6        keywords: ['gauteng', 'gp', 'pretoria', 'johannesburg', 'tshwane', 'jhb', 'pta'],
7        neighborNames: ['Limpopo', 'Mpumalanga', 'North West', 'Free State'],
8        postalCodePrefixes: ['0001', '0002', /* ... ~300 ranges */]
9    },
10    // ... all 9 provinces + national (10th entry)
11];
12
13export function findProvinceByKeyword(text: string): ProvinceEntry | null {
14    const lower = text.toLowerCase().trim();
15    for (const province of PROVINCE_REGISTRY) {
16// ... (truncated)
Show More (4 more lines)
TYPESCRIPT

The province registry includes keywords (major city names, abbreviations), neighbor names (for context disambiguation), and postal code prefixes (for precise lookups). This allows us to resolve "Midrand, 1685" to Gauteng even when the province isn't explicitly mentioned.

Layer 4: CIPC Data Integration

CIPC is South Africa's companies registry — the authoritative source for registered company information. We ingest CIPC data through a batch process that imports: company registration details (name, registration number, incorporation date, status), director information (name, ID number, appointment date, resignation date), and change events (director changes, address changes, name changes).

The CIPC data is stored in the CipcEnrichment, CipcDirector, CipcDirectorLink, and CipcChangeEvent models. When we display a company profile, we join this data with the tender award history to show a complete picture: what contracts the company has won, who its directors are, and whether any directors have been associated with restricted suppliers or deregistered entities.

Layer 5: Text Normalization

Tender titles and descriptions from government sources are often in inconsistent case — sometimes ALL CAPS, sometimes mixed case, sometimes with irregular punctuation. The text normalizer handles this:

Copy
1const ABBREVIATIONS = new Set([
2    'RFQ', 'RFP', 'RFI', 'BBBEE', 'CIDB', 'SASSA', 'SANRAL',
3    'GP', 'KZN', 'MP', 'LP', 'NW', 'NC', 'EC', 'WC', 'FS'
4]);
5
6export function normalizeText(text: string | null): string | null {
7    if (!text) return null;
8    // Tokenize, classify each token (abbreviation/reference/word/punctuation/whitespace)
9    // Abbreviations → always uppercase
10    // References → preserve as-is
11    // Regular words → sentence case
12    // Capitalize after . ! ?
13}
TYPESCRIPT

The Enrichment Pipeline

Beyond CIPC, we enrich company profiles from multiple sources: Wikidata (company descriptions, logos, Wikipedia summaries), Google Knowledge Graph (company type, industry classification, founding year), LinkedIn (company size, industry, website), South African Government sources (SALGA, CSD, GOV.ZA), and the World Bank debarment list (restricted supplier status).

Each enrichment source is fetched asynchronously and merged into the company profile. The enrichment pipeline includes automatic fallback — if one source fails (e.g., the Google API rate-limits us), we degrade gracefully and try again on the next refresh cycle. Data confidence scores tell users which fields came from authoritative sources vs. algorithmic extraction.

The Platform That Emerged

Today, Company Intelligence powers several platform features: company profile pages at /organization/[orgRef]/ show comprehensive supplier profiles with award history and director data; the Company Intelligence tool at /tools/company-intelligence/[slug]/ provides competitive analysis, market positioning, and comparison tools; forensic analysis uses the entity resolution layer to detect related companies, address clusters, and cross-directorships; and the JV Suite uses the same data to suggest potential JV partners.

The normalization layer is the quiet workhorse of the entire platform. Every time a user searches for a company, browses tenders by province, or reads an AI-generated summary, they're benefiting from the normalization pipeline that runs silently in the background, stitching together data from 50+ sources into a coherent whole.