Senior Bigdata Engineer - Java & Pyspark

**Job Description:**We are looking for a highly skilled Software Engineer with extensive experience in large-scale data processing and migration projects. The ideal candidate should have a strong foundation in Spark/Databricks and a proven track record in big data application development. This role requires forward-thinking expertise in tooling and architecture to optimize and scale migration efforts.

Key Responsibilities:

• Convert Java-based data processing applications to PySpark, ensuring high performance and scalability.
• Work on large-scale data processing and migration projects, handling complex data pipelines.
• Leverage Spark/Databricks expertise to develop and optimize big data applications.
• Design and implement scalable and automated solutions to minimize manual migration efforts.
• Collaborate with cross-functional teams to drive best practices in big data processing.

Required Qualifications & Skills:

• Strong programming skills with expertise in Java and PySpark.
• Hands-on experience with big data processing frameworks such as Apache Spark and Databricks.
• Proven track record of working in large-scale data migration projects.
• Ability to architect and scale solutions to streamline migration processes.
• Experience in optimizing big data applications for performance and efficiency.
• Knowledge of cloud-based data platforms (AWS, Azure, or GCP) is a plus.

Job Type: Full-time

Benefits:

• 401(k)
• Dental insurance
• Health insurance
• Paid time off
• Referral program
• Relocation assistance
• Vision insurance

Schedule:

• Day shift
• Monday to Friday

Experience:

• databricks: 5 years (Required)
• Java: 6 years (Required)
• PySpark: 5 years (Required)
• Big data: 3 years (Preferred)

Work Location: Remote