01 / The Challenge

What problem does the HIES AI Platform solve?

The Household Income and Expenditure Survey is one of the most important statistical instruments any nation operates. It powers poverty measurement, the Consumer Price Index, cost-of-living analysis, social policy targeting, and the economic indicators ministries rely on to set direction. Conventionally, HIES is produced through long, expensive, periodic surveys — instruments that are accurate but slow, manual, and increasingly out of step with the speed of policy decisions in modern economies.

NCSI's HIES, like its peers across the GCC and beyond, faced a structural tension: the data is precious and the methodology is sound, but the delivery cadence cannot keep up with the speed of policy. To meet this brief, NCSI needed a partner that could combine deep AI/ML expertise with rigorous data-governance discipline and direct, on-the-ground implementation in Oman.

AiSPRY, in partnership with Otech Oman, was scoped to deliver the world's first national-level transition of an entire HIES from traditional survey-based analysis to AI/ML-driven estimation — while preserving statistical rigour, explainability, and the legal data-protection framework required for sovereign national statistics.

Slow survey-to-statistics cycle

Traditional HIES production cycles run for years between waves, leaving long gaps in the evidence base for policy decisions.

High labour and field-cost intensity

Manual surveys require large enumerator teams and long fieldwork windows, making continuous updates economically infeasible.

Limited microdata footprint

The most recent HIES microdata covers approximately 5,660 households — statistically rich, but a constraint on data volume available for direct ML training.

Strict data-protection regime

The solution must operate inside Oman's Personal Data Protection Law (PDPL) and MTCIT cloud localisation regulations, with data residency and PII safeguards as non-negotiable.

Trust and transparency requirement

A black-box model is a non-starter for a national statistics office. Every prediction must be explainable to economists, auditors, and policymakers.

Bilingual user base

Outputs must serve Arabic and English audiences across NCSI and partner ministries with equal fidelity — across dashboard, API, and reports.

02 / The Solution

How does the HIES AI Platform work?

AiSPRY designed and is delivering the HIES AI Platform — a five-layer, cloud-native AI/ML system built around four foundational pillars and a CRISP-ML(Q)-aligned methodology. The platform ingests historical HIES microdata, macroeconomic indicators, utility usage, and administrative feeds; cleans and harmonises them to international classifications (COICOP); engineers household-level features; and trains an ensemble of forecasting and regression models.

Every output runs through an explainability and bias-audit layer before being surfaced through a bilingual dashboard, REST API, and scenario-simulation tools for NCSI and partner ministries. The four pillars: AI-Driven Estimation (continuous ML-driven income and expenditure estimation, targeting 70–80% reduction in time to produce statistics); Predictive Analytics (forecasting across multiple horizons for evidence-based planning); Proactive Insights (real-time bilingual cross-ministry intelligence); and Explainable AI (100% of predictions explained via SHAP, with audit trails, bias and fairness checks, and back-testing against the 2010 HIES).

The modelling stack combines ARIMA, Prophet, and LSTM for macro-level time-series, XGBoost and LightGBM for household-level expenditure and income, Random Forest and logistic regression for segmentation, and K-means for household clustering. A synthetic-data generator expands the prototyping volume to 50,000 statistically faithful households, preserving the privacy of the underlying ~5,660 real records.

ARIMAProphetLSTMXGBoostLightGBMSHAP ExplainabilityFastAPID3 · ReactPDPL compliantSovereign cloud
03 / Project Demo

See the platform in action

A walkthrough of the HIES AI Platform — from multi-source ingestion of HIES microdata, macroeconomic indicators, utility usage, and administrative feeds, through the ensemble modelling stack, to the bilingual NCSI dashboard surfacing income and expenditure estimates with SHAP-based explanations and scenario simulation.

Bilingual (Arabic / English) dashboard
SHAP-based feature attribution per prediction
Scenario simulation and what-if modelling
Back-testing against the 2010 HIES
04 / Architecture

A five-stage pipeline — sovereign by design

From data sources, through governance and feature engineering, into the AI/ML core, layered with an explainability and trust framework, and surfaced through stakeholder applications. Every layer respects PDPL, MTCIT cloud localisation, and the security expectations of a national statistics office.

01 ▸ DATA
HIES + Admin + Macro Feeds
~5,660 real HIES households, macroeconomic indicators, utility usage, and administrative feeds. 50,000 synthetic households for privacy-preserving prototyping.
02 ▸ GOVERNANCE
PDPL-Compliant Lakehouse
Data Lakehouse with raw, processed, and curated zones. Encryption at rest and in transit, role-based access, anonymisation pipelines. COICOP-aligned harmonisation.
03 ▸ MODELS
AI/ML Core
Time-series (ARIMA, Prophet, LSTM) · Regression (XGBoost, LightGBM) · Classification (logistic, Random Forest) · K-means clustering for household segmentation.
04 ▸ TRUST
Explainability & Audit
100% of predictions explained via SHAP. Audit trails, bias and fairness checks, back-testing on 2010 HIES, and hierarchical reconciliation against national accounts.
05 ▸ SURFACES
Bilingual Dashboard & API
Bilingual (Arabic / English) D3 + React dashboard, FastAPI REST with OAuth2, scenario simulation, role-based access for NCSI and partner ministries.
05 / Model Families

A model stack matched to the structure of HIES data

No single model can carry household-level expenditure prediction, macro-level forecasting, and segmentation at once. The HIES AI Platform combines time-series, regression, classification, and clustering families — each chosen for the part of the HIES signal it captures best.

Time-Series

ARIMA · Prophet · LSTM

Macro-level forecasting of income distribution and expenditure patterns across multiple horizons. ARIMA and Prophet capture seasonality and trend baselines; LSTM picks up long-horizon, non-linear shifts in the underlying economy.

Regression

XGBoost · LightGBM

Household-level expenditure and income estimation against the MAPE < 10% target. Robust to noisy tabular features and strong on the nonlinear interactions between income per capita, dependency ratios, and COICOP-aligned expenditure fractions.

Classification & Clustering

Logistic · Random Forest · K-means

Logistic regression and Random Forest for segmentation tasks; K-means for household clustering. Used alongside synthetic-data generation (50,000 statistically faithful households) for privacy-preserving prototyping while the underlying ~5,660 real records stay protected.

06 / Results

What measurable outcomes does the HIES AI Platform deliver?

The platform was scoped against three aligned dimensions of success — business, ML, and economic — with quality assurance integrated throughout to make sure the platform deserves the trust of a national statistics office.

Speed & operational efficiency

  • Targeted 70–80% reduction in time required to produce HIES statistics
  • Continuous, automated estimation cycles instead of multi-year survey waves
  • Lower labour and field cost per statistical update
  • Faster availability of evidence for ministry-level decision-making
  • MVP delivered mid-December 2025; full go-live by March 2026
  • Six-phase delivery (Oct 2025 → Mar 2026) aligned to CRISP-ML(Q)

ML accuracy & rigour

  • Targeted MAPE < 10% on household expenditure prediction
  • Cross-validation across demographic segments
  • Back-testing on 2010 HIES data and against linear-regression baselines
  • Hierarchical reconciliation against national accounts for consistency
  • Synthetic-data generation (50,000 households) preserves statistical fidelity
  • Continuous learning as new survey waves and administrative data arrive

Transparency, trust & governance

  • 100% of predictions explained at the feature level via SHAP
  • Audit trails for every prediction and decision
  • Bias and fairness checks across demographic segments
  • Domain-expert plausibility reviews built into validation
  • PDPL-compliant by design, with data residency inside Oman
  • ISO 27001:2013 and CMMI Level 3 certified delivery organisation
07 / Frequently Asked

Questions about the platform

What is the HIES AI Platform?
The HIES AI Platform is a first-of-its-kind AI/ML engine for a national Household Income and Expenditure Survey, built by AiSPRY in partnership with Otech Oman for the National Centre for Statistics and Information (NCSI), Sultanate of Oman. It transitions Oman's national HIES from traditional surveys to intelligent, explainable, PDPL-compliant estimation — using time-series, regression, and classification models with SHAP-based explainability, deployed on sovereign Oman cloud infrastructure.
Why is this engagement described as the first of its kind?
It is the first national-level engagement, anywhere in the world, to transition an entire HIES from traditional survey-based analysis to AI/ML-driven estimation — while preserving statistical rigour, explainability, and the legal data-protection framework required for sovereign national statistics.
What are the four pillars of the platform?
AI-Driven Estimation (transition from periodic surveys to ML-driven estimation, targeting 70–80% time reduction); Predictive Analytics (income and expenditure forecasting across multiple horizons); Proactive Insights (forward-looking signals via a bilingual dashboard); and Explainable AI (100% of predictions explained via SHAP, with audit trails and bias checks).
What success metrics is the platform targeting?
Three aligned dimensions: business — 70–80% reduction in time to produce statistics; ML — MAPE < 10% on household expenditure prediction; explainability — 100% of predictions explained at the feature level via SHAP. The platform is back-tested on 2010 HIES data and hierarchically reconciled against national accounts for consistency.
How does the platform comply with Oman's data protection laws?
The platform is PDPL-compliant by design — anonymisation, encryption at rest and in transit, role-based access controls, multi-layer security across raw / anonymised / processing / validation zones, deployment on Omani / compliant cloud infrastructure, and explicit governance on cross-border data movement. AiSPRY is ISO 27001:2013 and CMMI Level 3 certified.
How is the platform delivered, and over what timeline?
Six phases from October 2025 to March 2026, aligned to CRISP-ML(Q): M1 Business & Data Understanding · M2 Architecture & Data Preparation · M3 AI/ML Modelling (MVP mid-December 2025) · M4 Integration (REST API + bilingual dashboard) · M5 Model Evaluation & UAT (incl. back-testing and bias detection) · M6 Deployment & Maintenance with knowledge transfer and warranty-period support. Full go-live by March 2026.
— Modernise national statistics with sovereign AI

Explainable, PDPL-compliant AI built for the speed of policy.

Talk to the AiSPRY public-sector AI team to learn how the HIES AI Platform methodology — built with Otech Oman for NCSI, Sultanate of Oman — can be adapted for your national or regional statistics office.