Mid-Level Data Engineer (Remote)

Jobright is an AI-powered career platform that helps job seekers discover the top opportunities in the US. We are NOT a staffing agency. Jobright does not hire directly for these positions. We connect you with verified openings from employers you can trust.

Job Summary:

Agile Defense is a company dedicated to supporting the nation's most important missions through innovative technology solutions. They are seeking a Data Engineer to enhance existing products and develop new analytics solutions, requiring strong communication skills and a solid understanding of the software lifecycle in an Agile environment.

Responsibilities:

• Strong oral and written communication skills.

• Able to communicate effectively with Product Owners (POs) and stakeholders to understand requirements/scope and be able to translate that into user stories.

• Able to effectively communicate and coordinate between stakeholders, POs, and team members including designers, business analysts, developers, and Scrum Masters across other development teams.

• Work to become a subject matter expert for the products that are supported.

• Able to assist with testing/communication/de-bugging with external integration partner system teams.

• After team onboarding, work independently to continue to learn the system and products by reviewing resources and documentation, testing, and engaging with design, development, Product Owner and stakeholders in great detail.

• Design, build, and maintain efficient ETL/ELT data pipelines

• Develop and optimize SQL queries for large-scale data processing

• Implement data quality checks and monitoring systems

• Create and maintain database schemas and data models

• Collaborate with cross-functional teams to integrate data from multiple sources

• Optimize database performance through indexing, query analysis, and partitioning

• Automate data workflows using CI/CD pipelines

Qualifications:

Required:

• 3-5 Years of Experience working in an Agile and Scrum software development environment performing the duties of a Data Engineer.

• Databricks notebooks, job schedules, SQL query tools

• Managing multi-hope data processing pipelines (185 separate nodes in data pipeline)

• Scripting in SQL, pyspark, and bash shell

• Postgres database management

• FlaskAPI

• AWS Cloudflare, S3, EC2, EKS, ALB

• Docker containerization

• Apigee

• Jenkins, GitHub and separate code environments

• Strong oral and written communication skills.

• Able to communicate effectively with Product Owners (POs) and stakeholders to understand requirements/scope and be able to translate that into user stories.

• Able to effectively communicate and coordinate between stakeholders, POs, and team members including designers, business analysts, developers, and Scrum Masters across other development teams.

• Work to become a subject matter expert for the products that are supported.

• Able to assist with testing/communication/de-bugging with external integration partner system teams.

• After team onboarding, work independently to continue to learn the system and products by reviewing resources and documentation, testing, and engaging with design, development, Product Owner and stakeholders in great detail.

• Design, build, and maintain efficient ETL/ELT data pipelines

• Develop and optimize SQL queries for large-scale data processing

• Implement data quality checks and monitoring systems

• Create and maintain database schemas and data models

• Collaborate with cross-functional teams to integrate data from multiple sources

• Optimize database performance through indexing, query analysis, and partitioning

• Automate data workflows using CI/CD pipelines

Preferred:

• Familiar with and experience using tools like Jira, Confluence, Mural, and Teams to communicate requirements, design, processes, and content.

• Experience working in an Agile software development environment and familiar with Scrum structure and ceremonies

• Knowledge of cloud platforms (AWS, Azure, etc.)

• Experience with Jenkins or similar CI/CD tools

• Familiarity with data visualization tools

• USCIS or DHS Experience

Company:

Agile Defense is an information technology company located in Reston. It is a sub-organization of Agile-BOT. Founded in 1998, the company is headquartered in Reston, Virginia, USA, with a team of 1001-5000 employees. The company is currently Late Stage.