API Ingestion at Scale: Lessons from Unifying 15 Inverter Vendor APIs
Practical lessons from building a Go ETL pipeline that ingests solar telemetry from 15 inverter vendor APIs across 24,350 plants: rate limits, data quality, idempotency.
Read moreAbout me - Transforming Data into Value
I convert data into value. For over 16 years, I’ve looked at consumers, businesses, and markets through a scientific lens. By deconstructing patterns in data, I have unlocked new business performance for clients across various industries.
My career has evolved from economic research through data science to production software engineering: shipping data solutions as real software, from systems-level code to full-stack platforms.
Education and Qualifications - A Foundation of Continuous Learning
My academic background and ongoing professional development form the bedrock of my expertise in data science and analytics.
- University Education. Studied Econometrics, Economics and Actuarial Science at theUniversity of Witwatersrand, South Africa.
- Professional Certifications. Professional Data Scientist certification fromIBM, and Julia for Scientific Programming from theUniversity of Cape Town.
- Specialized Training. Completed courses in Strategic Management and InnovationCopenhagen Business School, EconometricsErasmus University Rotterdam, and AWS Cloud certification.
Skills and Expertise - From Systems Code to Data Science
My technical skills span systems programming, backend engineering, modern web, and data science, so data solutions ship as production software instead of throwaway notebooks.
- Systems Programming. Zig, C++17 and Rust for performance-critical work, from a cross-platform invoicing core with its own PDF engine to solar-irradiance polygon processing.
- Backend Engineering. Go for geospatial ETL and partner-API integrations, Elixir with Phoenix LiveView for real-time platforms, Node.js with GraphQL, and Python for APIs and scraping.
- Web Development. TypeScript end-to-end: React and Next.js, SvelteKit, Astro and HTMX, styled with Tailwind CSS, as the surfacing layer of data products.
- Data & Machine Learning. Pandas, NumPy, scikit-learn and Keras in Python; DataFrames.jl, Flux.jl and Clustering.jl in Julia, for forecasting, clustering and NLP in production.
- Data Visualization. deck.gl for large-scale geospatial maps, plus Plotly Dash, Recharts, D3.js and Power BI for insightful, interactive dashboards.
- Databases & Infrastructure. PostgreSQL, MongoDB, MariaDB, SQLite (WAL mode) and DuckDB. Deployed with Docker on AWS, Google Cloud, Netlify and Vercel, with Linux as a daily driver.
My Approach - Collaborative, Data-Driven Solutions
I believe in close collaboration with clients, focusing on clear communication and measurable results. My process involves four key stages, guided by core values.
- Strategy. Work closely with clients to identify and prioritize challenges and opportunities, combining their industry knowledge with my data expertise.
- Build. Employ rapid prototyping to develop minimum viable products (MVPs) that address key business needs identified in the planning stage.
- Scale. Create robust, enterprise-grade solutions that can handle growing data volumes and integrate cleanly with existing infrastructure.
- Continuous Improvement. Regularly assess system performance against key business metrics and user feedback, ensuring solutions evolve with changing needs.
Core Values - Principles Guiding My Work
My approach is underpinned by six core values that drive me to create impactful data science solutions.
- Innovation. Exploring new techniques to keep clients ahead in their industries.
- Integrity. Upholding the highest standards of honesty and ethical conduct.
- Impact. Creating solutions that deliver measurable, meaningful results.
- Collective Intelligence. Leveraging diverse perspectives for robust solutions.
- Fairness. Treating all clients with respect and ensuring transparent practices.
- Purpose-led. Harnessing data science for positive societal impact.
Personal Interests - Beyond the Data
Outside of work, I’m a family man with diverse interests that complement my professional life.
- Family. Married with children, balancing professional pursuits with family life.
- Horse Riding. Passionate about horse riding, finding balance and perspective outside the world of data.
- Culinary Adventures. An enthusiast of spicy food, always ready to try new flavors and cuisines.
From the blog
Insights and thoughts on data, AI, analytics, and industry trends.
Do You Need a Data Lake? DuckDB and MotherDuck as the Pragmatic Alternative
Most businesses told they need a data lake or lakehouse don't. When Parquet + DuckDB or MotherDuck covers your analytics, and when Postgres alone is enough.
Read more