Software Engineer

Job Summary

The Data Engineer will be responsible for developing ETL and data pipeline using AWS, Snowflake & DBT. The ideal candidate is an experienced data pipeline builder using ELT methodology . They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products.

Essential Duties and Responsibilities

• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Cloud Integration ETL Tools, Cloud Data Warehouse, SQL, and AWS technologies.
• Design and develop ELT, ETL, Event drivern data integration architecture solutions
• Work with the Data Analysts, Data Architects, BI Architects, Data Scientists, and Data Product Owners to establish an understanding of source data and determine data transformation and integration requirements
• Troubleshoot and tune complex SQL
• Utilize On-Prem and Cloud-based ETL platforms, Cloud Datawarehouse, AWS, GitHub, various scripting languages, SQL, querying tools, data quality tools, and metadata management tools.
• Develop data validation processes to ensure data quality
• Demonstrated ability to work individually and as a part of the team in a collaborative manner

Qualifications

• Bachelor's degree (or foreign equivalent) in Computer Science, Computer Engineering, or a related field.
• 8+ years of experience with Data Engineering, ETL, data warehouse/data mart development, data lake development
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Experience working with Cloud Datawarehouse like Snowflake, Google BigQuery, Amazon Redshift
• Experience with AWS cloud services: EC2, S3, Lambda, SQS, SNS, etc.
• Experience with Cloud Integration Tools like Matillion, Dell Boomi, Informatica Cloud, Talend, AWS Glue
• Experience with GitHub and its integration with the ETL tools for version control
• Experience with Informatica PowerCenter, various scripting languages, SQL, querying tools
• Familiarity with modern data management tools and platforms including Spark, Hadoop/Hive, NoSQL, APIs, Streaming, and other analytic data platforms
• Experience with object-oriented/object function scripting languages: Python, Java, Scala, etc., a plus.
• Experience with Agile/Scrum is valuable.