AI-Powered Marketplace Content & Search Platform
Owning a two-service platform — 602K+ searchable listings across 36 marketplaces — with multi-modal semantic search (text and photo, EN + AR) built for how creators actually search.
System architecture
Problem & industry context
Creator-commerce platforms sit between influencers and marketplace catalogs spanning dozens of regional and international storefronts. Product data arrives inconsistent — mismatched categories, duplicate listings, multilingual titles — and creators need to find relevant items by intent, not just keyword match. Systems that can't reconcile catalog scale with search relevance either drown in stale data or return irrelevant results.
Insight
At this scale, the hard problem isn't scraping — it's reconciliation and retrieval. A category taxonomy that's 80% automated still needs the remaining 20% to be right, because a wrong mapping breaks discovery for every product under it. Vector search earns its complexity only when keyword search demonstrably fails on real creator queries — semantic search on top of a shaky data foundation just returns confidently wrong results faster. The two services stay deliberately decoupled — search reads a database snapshot, never live Postgres or the scrapers directly — so acquisition-side slowness or failure never shows up as a slow search response.
What I built
Own an AI-powered marketplace content and search platform end-to-end, split into two deliberately decoupled services sharing only a database. The acquisition side extracts and categorizes products from 36 marketplaces (EN + AR) into PostgreSQL, processes imagery through Cloudflare R2, and writes a clean catalog to MongoDB — 602K+ searchable listings today, run via a FastAPI orchestrator and ops dashboard. The discovery side reads that MongoDB snapshot and embeds every product across 5 Pinecone vector indexes — English text, Arabic text, product photos, and colour/style intent — so a creator's query, typed or an uploaded photo, resolves to real, in-stock, regionally-correct results (a product with no presence in a region correctly returns nothing there, rather than something unbuyable).
Technical approach
Stack and tooling for this work: Python, FastAPI, PostgreSQL, MongoDB, Pinecone, gRPC, Cloudflare R2. Topics covered: Vector Search, Data Engineering, gRPC, E-commerce.
Topics
Related case studies
Microservice Mesh for a Fitness & Wellness Platform
Digiflux Technologies · 2026 – Present
Designing a gRPC microservice mesh and a standalone mobile app for a fitness brand built on wearable data.
FinTech Data Pipeline at Scale
Digiflux Technologies · 2024 – Present
How 50+ serverless scrapers keep market and compliance data flowing for millions of users.
Conversational AI for Meal Subscriptions
Digiflux Technologies · 2024 – Present
LangChain + LLMs powering subscription support for 125K+ customers with 90%+ intent accuracy.