As Chief Technology Officer I am responsible for the whole tech stack for a music distribution platform, managing 6 squads, front-end, UX-UI, Back-end, including data squad, Testing, Blockchain, DevOps. I have created a fast, scalable, reliable, and secure architecture for all the processes inside the platform using an event-driven architecture with microservices and Kafka, exposing rest APIs to interact with the front-end everything hosted into AWS.

Interim Chief Technology Officer

As tech lead I was responsible of decide and design the data architecture to process the big data files. I decided to use parallelism and validate, clean, and transform the data with pandas, finally loading the processed data to a PG DB supporting scalability, high performance, and 0 downtime.

Tech Lead

As Data Engineer Lead, I was responsible for designing, implementing, and maintaining a whole big-data environment resilient, reliable, and scalable for this last-mile delivery company, providing solution for more than 36 companies, like Oxxo, Walmart, Exito, and others. The company offered 5 different services that each single company could choose or not. Each single service produced a huge amount of data that had to be consumed, normalized, cleaned, and loaded into one data mart for each company, and a principal datalake (Big Query). The stack used was Python, pandas, concurrence, and event-driven architecture with micro-services and docker, running over cluster, services, and tasks hosted in AWS, connected to a RabbitMQ instance, and managed using Terraform. This ETL on average could consume, process, and load around 9.000.000 registers every 3 minutes. Also, I used Airflow for no complex tasks and specific report extraction. As a BI tool, I selected a modified version of the open-source Re-dash.

Data Engineer Lead

As Data Engineer / Data Scientist, I was responsible for the ETL which scraped whole world data of web pages like Airbnb, Expedia, and Booking, to provide a big data solution for real state short rentals investors. I used, Python, pandas, Node JS, for scrapping, and managing scrapping jobs, terraform, Jenkins, Postgres, mongo, S3, and SnowFlake as datalake, respectively, this ETL pipeline, was over AWS, using Event Driven Architecture.

Personal details

Sebastian C.

Technical skills

About

Work Experience

Education

Certifications & Awards