Systems in various states of evolution.

Finding good deals for Canada's Chinese-speaking community meant manually searching fragmented sources, rewriting product descriptions, and posting across multiple platforms by hand — every single day. The real problem wasn't just volume — it was that every step of the pipeline required a human decision.

Designed and built the entire platform from architecture to deployment. The pipeline runs across multiple stages, each built to handle failure gracefully and operate without manual intervention.

Crawling: Distributed Python + Playwright crawler workers bypass Amazon bot-detection via Chrome DevTools Protocol. Workers run across two machines with distributed locking mechanisms (Redis-backed) to prevent concurrent data conflicts and duplicate processing. Scheduling, monitoring, and logging are built in — failures surface immediately rather than silently corrupting data.

Data pipeline: Raw posts are always preserved before any processing — if the frontend surfaces a bad record, I can back-trace to the original crawl and correct it without data loss. A two-layer normalization pass cleans and standardises incoming data before it enters the main store. Deduplication handles both exact matches and the harder case: the same deal arriving from different channels with different titles, prices, or metadata — the system automatically reconciles these rather than creating duplicates.

Verification: A separate automated Amazon crawler re-checks live prices after ingestion, catching deals that expired between crawl and publish.

AI layer: A pluggable AI-provider layer (OpenAI / Claude / Deepseek, hot-swappable via one env variable) auto-generates Traditional Chinese titles, categories, and feature summaries. Prompt engineering was iterative — tuning for minimum token cost, maximum throughput, and output quality simultaneously, since all three pull in different directions.

Infrastructure: FastAPI backend + Next.js 14 frontend, containerized with Docker Compose, running on a self-managed Linux server. Redis caches hot queries to reduce database load and improve frontend response times. Meilisearch replaced Elasticsearch for full-text search — lower resource usage on the server, no credential management overhead. Facebook Graph, Instagram Stories, and Threads APIs handle scheduled auto-posting. Monitoring and logging added later to make system health observable at a glance.

Manual deal hunting, manual description rewriting, manual translation, manual deduplication, manual social posting — the entire content pipeline now runs on its own across two machines.

Distributed systems fail in ways single-machine systems don't — distributed locking was not an afterthought, it was necessary. Preserving raw data before processing turned out to be the right call more than once; the ability to back-trace and reprocess saved data that would otherwise have been lost. Prompt engineering for production is a different problem than prompt engineering for demos — cost, speed, and quality all degrade each other, and finding the right balance requires real measurement. Proactive security work matters: I audited the system myself and patched 3 critical vulnerabilities including hardcoded DB credentials, an X-Forwarded-For rate-limit bypass, and an XSS-readable admin key.

Grocery pricing has almost no transparency. ~12,500 T&T products across 2 stores and 3 languages with no way to track how prices actually move over time.

A Node.js crawler on a daily crontab + Docker Compose schedule. Two-phase crawl pipeline (bulk category listing, then detail fetch only for items with options) reduces unnecessary requests by ~95%. Polite rate-limiting with p-limit, random jitter, and exponential back-off keeps full runs under 5 minutes per combination. MongoDB price-history sub-documents append only on price change, cutting write volume by ~80%. Replaced Elasticsearch with Meilisearch to eliminate hardcoded-credential risk while keeping multi-language full-text search.

Manual price-checking and the guesswork of "is this actually a good price?" — replaced with a history that accumulates on its own, every day.

A race condition was serving empty API responses because of a delete-then-insert pattern; fixed with idempotent bulk upserts. Even a fairly simple system becomes powerful once data accumulates continuously — the value isn't in any single reading, it's in the history.

Sharing affiliate links at scale means managing custom aliases, tracking clicks, injecting affiliate tags consistently, and splitting traffic for A/B tests — none of which basic link shorteners support.

Full-stack FastAPI + MongoDB platform with custom alias support, QR code generation, and deep link routing. A/B testing for redirect targets with configurable weight distribution, scoped by subscription plan tier (Free / Pro / Business). Affiliate tag injection with per-domain rules and an ethical no-overwrite policy — existing affiliate parameters are never replaced. Click analytics pipeline tracking device type, geo-location, referrer, and time-series data with EPC metrics via REST API. JWT authentication with role-based access control.

Manual link management, manual affiliate tag insertion, and guesswork about which redirect targets actually convert.

The ethical no-overwrite policy on affiliate tags was a deliberate design decision, not an afterthought. Building plan-gated feature enforcement consistently across every endpoint is the kind of unglamorous work that makes a real product feel solid.

Flashfood has real discounted-grocery opportunities, but discovery is fragmented across cities, stores, and categories — with no good way to filter or track what's available.

Live deal feed using Firestore onSnapshot (no polling) with composable region/store/category/keyword filters and multiple sort modes. Infinite scroll via IntersectionObserver in 32-item batches. Per-city filter persistence in sessionStorage. Multi-language UI (EN / ZH / FR / KR / JP). Dynamic Open Graph images generated at the Edge for rich social-sharing previews.

The manual hunt-and-sort of finding and tracking discounts across a noisy, constantly changing source.

A filter-restore race condition on city switch required a ref-based guard to fix cleanly. Deriving context-aware category pills through a two-pass filtering strategy turned out to be the right call — it eliminated a whole class of inconsistency.

Tailoring a resume to a specific job description is repetitive, time-consuming, and easy to get wrong — especially when you have a large knowledge base of skills, projects, and experience to draw from.

Full-stack resume builder: Next.js 15 (App Router, TypeScript, TailwindCSS) frontend + FastAPI backend, deployed on Vercel + Docker. OpenAI API analyzes job descriptions and recommends the most relevant skills, projects, and experience entries from a structured knowledge base, with heuristic fallback for offline usage. Personal profile database with full CRUD for skills, work experience, and projects — supporting both manual entry and AI-assisted bulk import from raw text. Resume export in PDF, DOCX, Markdown, and plain text. A privacy-conscious AI rewrite feature polishes bullet points without fabricating facts, inflating responsibilities, or inventing metrics.

The manual process of reading a job description, cross-referencing your own experience, and deciding what to include — replaced with a system that does the matching and drafting automatically.

JWT authentication with per-user OpenAI API key management — masking stored keys on retrieval — is the kind of detail that separates a real product from a demo. The privacy constraint on the AI rewrite (no fabrication, no inflation) was a harder design problem than the AI integration itself.