Backend Engineer
Skills used: Amazon Web Services, Python, Postgres, Jinja, Bash
Role overview: I was responsible for designing, architecting, and implementing an eBook publishing pipeline. The pipeline was constructed out of a series of interconnected AWS Lambdas. The input was raw, transcribed chess game data. It would be imported into the database, enriched using native and third-party sources, and rendered into an EPUB format using Jinja templates—ready to be sold online. The Lambdas were packaged into an AWS Step Function in order to automate the passing of data and were exposed via AWS API Gateway. I was responsible for designing the database schema, performing Quality Assurance on the generated EPUBs, and generating thorough and intuitive documentation for future developers.
Interesting challenges: I needed to find a secure way to run migrations and manually administer the production database (AWS RDS). However it was inside of a VPC without access to the outside internet. I decided to create a bastion server (single EC2 instance) with a highly-restricted Security Group (only allowing ingress to my current IP address) which I would spin up on demand long enough to perform migrations and then would stop once the work was done. I created a simple Makefile target that would create a tunnel to the bastion server via `ssh`, allowing me to connect directly from my local machine using the `psql` command.