I've developed with Python in a professional context for many years. Projects have included natural language processing for which I was awarded patents. I've also completed considerable work on big data engineering challenges requiring very fast stream processing analytics. I also have experience with API development using Flask, SqlAlchemy and AWS API Gateway with Slack API definitions.

I've developed with Java in a professional context for over a decade. Projects have included high throughput multi-threaded email consumers. I've also completed considerable work on big data engineering projects requiring customised Kafka streaming modules that I've written in Java and Scala.

I've worked on numerous projects using SQL Server. The largest was during my time as CTO for a social media monitoring startup. The database held hundreds of millions of records and I optimised SQL Server for maximum performance. I also migrated billions of records from a live system into a No-Sql solution (MongoDB / Solr) to eliminate locking problems by using a platform with eventual consistency.

I've used Elastic Search on multiple big data projects. I've applied it to large document indexes and also used the percolator filter to register hundreds of thousands of query's to match against a stream of documents. I've optimised queries, index structure and language tokenisation.

I love Docker and I've used it on every project that I can! I've used Docker compose to represent entire production systems on my laptop, including mocked out AWS services like SQS, S3 etc. I've also used Docker in CI pipelines so that on check-in Jenkins will create the Docker image, run the unit tests within it, then deploy the image to Portus (a self hosted Docker registry). Orchestration platforms like Kubernetes and Rancher can then pull the images from Portus as part of deployment or auto system scale-out. I've also created Kafka and Spark clusters with Docker and Docker-compose.

Personal details

Brett A.

Summary

Technical skills