Yuriy M.

Multiple project in the field of data warehousing and big data enginnering. Business development, team augmentation, mentoring, pre- and post-sales solution architecture, engineering and support

2 contracts (via Toptal). Both clients – major pharmaceutical companies. Subcontracted by BCG GAMMA Advance Analytics and Data Science division to provide engineering support for BCG’s data scientists on DMP and personalization projects. Mostly Feature Engineering and ETL but also devops tasks: Python utilities, Airflow installations and Airflow administration Python scripting, Spark to Excel Python scripts, other devops tasks. Designed and build dynamic S3-to-S3 RDS-driven (metadata in Postgres) ETL system in Spark/Hive. AWS Glue is used for Hive metastore, Athena for querying and Airflow for scheduling. ETL system build based on modern Data Warehousing best practices. Documented the system and provided training. Designed and build Feature Engineering S3 Data Mart and multi-layered S3 Customer-360 Data Lake. Proposed and enforced development standards, provided documentation, data validation procedures, operational support and maintenance guidelines.