Andras Radics

SUMMARY

Lean, fast, high throughput, low latency microservices architecture and development: low latency high volume data pipelines, large datasets, storage schema design and query optimization, multithreaded and distributed computation, message queues, async and evented programming, efficiency, server capacity planning.

Expert Node.js/typescript, SQL, MongoDB, Git, Jira, Scrum/Kanban, unit testing, REST Apis, Linux/C/Unix. Oral description into completed, tested, deployed product, I get it done.

Enjoy thinking up ways to make code run 10x faster; node.js collaborator. If you're using setImmediate(), ioredis or mysql2, you're using some of my code.

EDUCATION

• BS Electrical Engineering and Computer Science (OS/compilers/graphics - UC Berkeley)
• MS Computer Science (architectures/parallelism - Stony Brook University)

EXPERIENCE

Staff Engineer, Shopify / 6 River Systems, Waltham, 2019-2023

Key member of squad working on the performance-critical microservice selecting the warehouse orders to pick next. Helped shape and redesign the internal architecture, code reviewed, mentored. Go-to resource for nodejs and system performance questions. Node.js, TypeScript, Jira, Github, Kubernetes, GCP, Jenkins, Circle-CI.

• Multithreaded the nodejs microservice, did in-depth code reviews, mentored junior developers, refactored and sped up code.

• Ported app with minimal source code changes from LoopBack2 to an application framework built around Nest.js/express, wrote LB2-workalike Postgres ORM and fast Bunyan-workalike logger.

• Improved google pubsub subsystem to more efficiently filter and extract relevant messages, sped up job scanning and job selection, unified and normalized job selection filters for simplicity and improved metrics gathering, revamped test scaffolding to allow more production-like tests, implemented new features.

Open Source Developer, AndrasQ, 2013-current

Authored and maintain open source packages, including best-of-breed or one-of-a-kind utilities. Faster setImmediate PR merged into node-v6.3.0. Node.js, Github actions. github.com/andrasq

• The 3x faster qtimers speedups were accepted as a pull request and appeared in node.js 6.3.0, and were backported to older versions.

• Best of breed: qtimers (now part of nodejs), qlist (used as denque in ioredis, mysql2 and mariadb), qheap, qlogger, qprintf, phpdate-js, uuid-quick, microreq, google-custom-metrics, prom-pushgateway, microrest, qtimeit, q-then, qrusage, tzinfo, ieee-float, qrpc.

• Larger projects: extensive test mocks library qmock, fast RPC library qrpc, full middleware framework microrest, job runner service kqueue.

• Open sourced the Quicklib toolkit, a PHP library for moving and processing large volumes of data, and for building very fast web services. Components for fast data transport, pub/sub messaging and job queue, REST applications, php co-processes, db access, logging, caching. github.com/andrasq/quicklib

Principal Software Engineer, Kinvey, Boston, 2014-2019

Expert node.js architect responsible for the database and analytics microservices of a BaaS (Backend-as-a-Service) product, connecting to MongoDB, Redis and MySQL. Refactored and multithreaded service for major performance improvements, became the in-house MongoDB performance expert. Node.js, Nginx, MongoDB, MySQL, Jira, Github, AWS, Prometheus, Stackdriver, Docker, Kubernetes.

• Refactored data shipping to eliminate event loop blockage, implemented custom db connection handling to not block queries, added first-come first-available fair share multi-tenant query scheduling.

• Reinvented the db data to JSON conversion resulting in 5x faster data delivery with 4x less cpu used compared to JSON.stringify(BSON.deserialize(raw)).

• Multithreaded the data microservice to utilize all cpu cores, with proper start/stop and signal delivery semantics, then spearheaded the effort to halve the number of assigned AWS servers.

• Analyzed data flow and optimized end-user data delivery for a 65% cpu savings.

• Transitioned code repositories from SVN to Git.

• Go-to person for investigating server glitches, mongodb performance anomalies, and app performance bottlenecks. Devised appropriate indexes to help slow mongo queries, consulted for server capacity planning.

Principal Software Engineer, Netprospex, Waltham, 2013-2014

Tech lead responsible for the external-facing SaaS REST API built on an OO MVC framework. Designed, ticketed and led team in building new microservice, and successfully deployed to production. Built on CentOS Linux, PHP/Apache and MySQL/Solr.

• 80/20 development/management. Supervised the project team (2-3 people): wrote and assigned tickets, pointed work, led daily scrum, monitored progress, and code reviewed changes. Helped project manager with triage.

• Built a data transfer service to interface to Marketing Automation APIs (SOAP and REST), and read and write data in a uniform, platform-neutral format. The service presents a very simple file-based interface, is highly modular, transfers data asynchronously, notifies via callbacks, and makes progress status available as it goes. Wrote adapter modules for Eloqua, Marketo, Silverpop, Salesforce, NetSuite, and a high troughput adapter to the in-house database.

• Rewrote the API boostrap code and transitioned it off PHP Zend onto a light weight MVC micro-framework for a 3x speedup in call times (.15 sec down to .05)

• Built an external facing micro service to re-deliver results from internal services. OAuth authentication, REST calls and responses, JSON data.

• Derived scalability profiles, both analytical and empirical, for server and capacity planning.

Senior Performance Engineer, Blue State Digital, Boston, 2008-2013

Part of Agile dev team coding O-O PHP with PHPUnit unit tests on a home-grown framework. Wrote and enhanced the queueing system used to run millions of off-line tasks daily for hundreds of client sites. Was the go-to guy for optimization, getting 10x-25x speedups in several cases. Helped coders with mysql schema design and query speed, php performance, space efficiency.

Consulted with sysadmins on server capacity and application impact. Coded custom PHP extensions in C for performance boost. Technology included Apache, Memcache, MySQL, ORM, SOA, Cassandra, Redis, RabbitMQ, AMQP, Chef, Logstash, Capistrano, StatsD, Graphite, Nagios, Ganglia, Cacti, git, Smarty, XML, SOAP, JSON, AJAX, PHP, PHPUnit, MySQL, InnoDB.

• Wrote a queueing system for executing asynchronous procedure calls, each in the calling client site context. Devised a fair scheduling algorithm that ensures progress by all task types from all client sites even during large backlogs. Designed for unlimited scaling across multiple servers, multiple daemons and multiple databases, the implementation easily sustained production rates of millions of tasks per day. The queue was still running 10 years later.

• Added batched running to the queueing system, making it suitable for high-throughput message processing. Like RabbitMQ, but without rabbitmq's limitations: with backlog tolerance, automatic load balancing, built-in load throttling, graceful degradation, fast restarts, loss-free message delivery, fair scheduling, limitless scaling, autonomous servers, unlimited message types.

• Devised numerous sql query speedups by reformulating the query, or changing or adding indexes, or by using a different approach. 4x speedup saving mysql results to file with a custom PHP C extension. 5x speedup inserting objects into mysql by automatically combining the data into a single query.

• 20x speedup to all-points closest pairs geospatial search by implementing matching with a K-D tree written in C instead of using the database.

• 25x speedup to matching records search by moving set-intersection test out of the mysql query into a C utility.

• 10x speedup to db record insertion loop by 25x speedup to the data preprocess portion and rewriting the ORM insertion code for another 4.5x.

• Developed the timezone conversion functions that needed precise mathematical correctness.

• Developed classes to parallelize php code and support cross-thread method calls, effectively making php multi-threaded. Used the classes to speed up nightly batch scripts.

• Wrote a small shared caching class 10x faster than localhost memcache.

• Sped up ORM object selection 2.5x without altering semantics (and identified another potential 2x with minor semantics changes) by tracing the code and rewriting inefficient constructs.

Contract Software Engineer, Freelance, Boston, 1994-2008

Freelance contract programmer, primarily projects for one long-term client, with occasional small projects on the side. Wrote the code that runs a chain of photo kiosks in malls, restaurants and amusement parks across the country, including the Cambridge Galleria and the Museum of Science. Most projects involved image processing, human-computer interaction, image presentation and image input/output. Persuaded company to pick Linux when transitioning away from DOS, and took the lead in porting the code to Linux. Tools used included C/DOS, C/Linux, Berkeley make (pmake), Perl, RCS, X11 X Windows, PHP, MySQL, ClearCase, C++/Solaris.

• Refactored the monolithic application into reusable modules, organized the software infrastructure, and recombined the resulting components into two new products.

• Wrote a face morphing special effect and a utility to parametrically morph portraits into caricatures. The morph was done on a triangle-by-triangle basis on a triangulated polygon mesh, with the per-pixel color value interpolation reusing the principles of the image rotation utility written earlier.

• Led the port of the photo kiosk application from DOS to Linux. Identified areas of incompatibility and developed DOS work-alike functionality to ease the transition (X11, getkey(), extended memory, frame buffer, chvt, and porting the MS nmake makefile to Berkeley make.)

• Interfaced Linux to various video capture cards, currency validators and custom i/o cards using proprietary, OS-supported or home-grown drivers.

• Wrote a Linux kernel device driver for a USB video camera. The driver had to read the raw YUYV samples, recover the adjusted RGB data for each pixel, and return the RGB image data within 1/30th of a second.

WHITEPAPERS

A sampler of my writing, more at github.com/andrasq/node-docs:

• Fair Job Queue Scheduling - fair-queue-scheduler.md
• Promises -- A Quick Reference Guide - promises-guide.md
• Fast Remote Logging - fast-remote-logging
• Same BaaS, 60% Less CPU - baas-cpu-usage.md
• Fast Unique Id Generation - unique_ids.md
• The Art of Code Layout - code-layout.md
• The Art of Careful Code - code-structure.md

ACRONYMS

I've worked with node.js npm Typescript JavaScript Jenkins Travis-CI Coveralls.io Codecov github OO OOP OOD TDD AWS Kubernetes Docker express mocha mocks unit tests C C++ Unix LAMP Linux Apache Nginx MySQL MongoDB Redis ORM PHP PHPUnit Memcache Capistrano Logstash PubSub AMQP Chef Nagios Ganglia Cacti Prometheus Stackdriver StatsD Graphite Jira git CVS SVN Mercurial Perl sh make Smarty XML SOAP JSON AJAX pmake ctags gdb awk sed bash DOS RCS JPEG ppm/pnm TIFF GIF HTML X11 XFree86 FreeBSD DSP LZW 80x86 i860 SPARC SunOS Solaris AIX VAX/VMS Alpha TeX nroff

PERSONAL

I've been into computers since forever; you could find me in 8th grade dragging around a book on Boolean algebra. I build my computers from parts, ride my motorcycle all year round, play serious badminton, and have bicycled from San Francisco to New York by way of Seattle.

Bumper sticker: Linux is Unix. GNU's Not Unix.

This document can be found on-line at aradics.com/andras/resume.html.
Copyright (C) 2006-2023 Andras Radics.
Last updated 6/22/2023.