Senior Software Engineer

Job Description: Job Description:

• Develop automation tools to validate and reconcile data between API responses and database queries using PostgreSQL and Snowflake.

• Build and maintain FastAPI endpoints utilizing Pydantic BaseModel for data validation, SQLAlchemy ORM for database integration, and Snowflake connectors for secure connections.

• Leverage Python Requests and JSON libraries to interact with REST APIs, parse structured responses, and automate API-based testing.

• Design and execute data validation frameworks using pytest and Pandas to ensure accuracy and integrity across multiple data sources.

• Create Streamlit dashboards for visualization of validation results, metrics, and exception reports in real time.

• Integrate automation workflows with Databricks and SQL pipelines to support large-scale data validation and performance optimization.

• Utilize Jira for Agile sprint tracking, test case management, and issue resolution while maintaining code quality through GitHub version control and CI/CD workflows.

• Document reusable FastAPI modules, validation schemas, and automation utilities to enhance standardization and accelerate development cycles.

Must Have Skills/Requirements:

• Minimum experience: BS + 12 years relevant experience

• Strong proficiency in Python with experience in automation and data validation frameworks (pytest, Pandas, Requests, JSON, os, logging, re).

• Hands-on experience with FastAPI, including Pydantic, BaseModel, and SQLAlchemy ORM for backend API development.

• Expertise in PostgreSQL and Snowflake, including secure connection handling via SQLAlchemy, snowflake-connector-python, and psycopg2.

• Proven ability to design and automate validation between API responses and database queries, ensuring accuracy and consistency.

• Proficiency in REST API integration using Requests, FastAPI, and Uvicorn with structured JSON schema handling.

• Familiarity with GitHub (version control, branching, pull requests) and Jira (Agile tracking, sprint and Kanban management).

• Strong understanding of CI/CD pipelines, API authentication (JWT/OAuth2), and data integrity testing.

• Experience optimizing SQL queries, FastAPI endpoints, and improving performance in high-volume systems.

• Knowledge of Databricks and integrating validation workflows within large-scale data engineering environments.

Nice to Have Skills/Requirements:

• Experience developing Streamlit dashboards for real-time visualization of data validation metrics and test results.

• Exposure to AWS services (EC2, Lambda, RDS, S3) for deploying data and API solutions.

• Understanding of Docker or Kubernetes for containerizing and orchestrating automation tools and APIs.

• Familiarity with data quality frameworks and ETL testing in enterprise environments.

• Experience with workflow orchestration and scheduling of validation pipelines.

• Knowledge of Snowpark or Spark for advanced data transformation and validation automation.