Agustin M.

As Sr Data Engineer, Design and Build Agility Data Pipeline Service for Network Operators. I used the following tech: Docker Kubernetes, Helm, Python, MinIO.

Define and establish the infrastructure for the Cold Store component of our Anomaly Detection Product.

• Design and construct a Monitoring Dashboard for the core services of the Anomaly Detection Product, employing the RED (Rate, Errors, Duration) approach.
• Develop and implement the Anomaly Detection Product specifically tailored for Network Operators.
• Utilize a wide range of technologies, including Docker, Kubernetes, Helm, Python, PySpark, TrinoDB, Hive, MinIO, and Vertica DB

• Improved the data mining timeline of MVision EDR Hunting and Security Analytics from 2 months to 1 year by reducing data storage costs by 60% monthly through the implementation of a Big Data pattern architecture.
• Enhanced data quality by applying software engineering practices to the data pipeline.
• Led the tech team and defined the InfoSec Data Analytics strategy for the MVision EDR Content Team.
• Built the analytics data lake for feature engineering and applied software engineering practices to data and ML experiments.
• Utilized the following technologies: AWS EMR, AWS S3, AWS Glue, AWS Athena, AWS QuickSight, PySpark, Parquet, and Airflow.