Technical Architect for Data Platforms & Scalable Backends

10+ years designing and operating high-throughput data pipelines, analytics infrastructure, and cost-efficient storage for products at scale. I build pragmatic, reliable systems with ClickHouse, Kafka, Airflow, MySQL, Redis, S3/ECS, Node.js & Python.

📄 Download Resume 🔗 LinkedIn ✉️ Email
10M+/day Personalized notifications pipeline
12h → ~1h ETL runtime reduction (notifications)
50+ TB S3 → Dell ECS migration
Snapshot

What I Do

  • Design fault-tolerant data architectures for analytics & growth
  • Build Kafka/Airflow pipelines with observability & SLAs
  • Operate ClickHouse warehouses with TTL-based hot/cold storage
  • Optimize costs via storage tiering & query performance
  • Mentor teams, lead KTs, align stakeholders across product/BI

Case Studies

Proof of impact with real-world scale
Data Platform

JobHai Analytics on ClickHouse

Conceived and operated a ClickHouse-based platform ingesting clickstream + job/application data to power BI dashboards and product decisions.

  • Near-real-time ingestion via Kafka; Airflow DAGs for batch/curation
  • TTL tiers → hot/cold storage; controlled retention & cost
  • Served multi-team dashboards with strict SLAs
ClickHouse Kafka Airflow S3/ECS
Performance

High-Volume Notifications Pipeline — 12h → ~1h

Re-architected high-volume notifications (10M+/day) with parallelized ETL, indexing, and smarter batching.

  • Introduced data partitioning & backpressure-aware consumers
  • Cut infra costs while improving latency and stability
  • Clear runbooks, on-call guides, and observability
MySQL Redis Node.js Python
Cost Optimization

Object Storage Migration (S3 → Dell ECS)

Designed dual-upload and staged migration for large document stores to internal object storage while maintaining availability.

  • Traffic cutover with presigned URL + CDN compatibility
  • Backup/restore strategy; integrity verification
  • ~70% estimated storage cost reduction
S3/ECS Presigned URLs CDN Security

Skills & Tools

Senior-level capabilities — not percentage bars

Data Systems

ClickHouse · Kafka · Airflow · Spark · Hadoop

Backend

Node.js · Python · FastAPI · REST · Microservices

Datastores

MySQL · MongoDB · PostgreSQL · Redis · S3/ECS

DevOps/Infra

Linux · Docker · CI/CD · Observability · TTL storage design

Analytics/BI

Superset · Metabase · Dashboard design & governance

Leadership

KT & mentoring · Cross-functional alignment · Runbooks & incident response

Experience

10+ years across startups & product orgs
Technical Architect — InfoEdge (JobHai)
Sep 2019 — Present · NCR, India
  • Lead data platform on ClickHouse; Kafka/Airflow pipelines for clickstream & jobs
  • Optimized MySQL master/slave; reduced master QPS 9k → 6k by routing reads
  • Designed S3 → Dell ECS migration with dual-upload & staged cutover
  • Defined SOPs for credit workflows, dormant-user lifecycle, and partner mapping
QA → Data Analyst → Backend Engineer — TrulyMadly
Mar 2014 — Aug 2019 · New Delhi
  • Built socket.io chat, reporting system, and recommendation engine
  • Implemented dynamic pricing models driven by usage analytics
  • Shipped multiple growth & engagement experiments end-to-end

Awards & Recognition

Gold Award for Excellence

InfoEdge (2023) — Outstanding contribution building vernacular PAN-India app & data infra.

Employee of the Year

TrulyMadly (2018–2019) — Back-to-back winner for engineering impact.

Mentorship & Leadership

Led KTs, guided engineers through PIPs with measurable improvement plans.

Let’s Build Something Reliable

Open to roles as Technical Architect, Lead/Principal Data Engineer, or Staff Backend Engineer focused on data platforms & high-scale systems.

✉️ Email Me 📄 Download Resume 🔗 View LinkedIn

Also on X: @thedeceptio (social & engineering notes).