Software Engineer (Apache NiFi)

Hello,

Please find the below Job Description.

Role: Apache NiFi SME

Duration: 6 to 12 months

100% Remote

Job Description:

Responsibilities:

• Design, implement, and manage data flows within Apache NiFi, ensuring efficient and reliable data movement across systems.
• Build scalable workflows for data ingestion, transformation, enrichment, and delivery.
• Configure processors, connections, and controllers to optimize data throughput and performance.
• Regularly monitor and maintain NiFi clusters, ensuring high availability and fault tolerance.
• Implement appropriate security measures, including user access control, data encryption, and SSL configurations within NiFi.
• Ensure compliance with organizational data governance policies and regulations.
• Work on integrating NiFi with other systems such as databases, cloud services (e.g., GCP, AWS, Azure) and other big data or messaging platforms.
• Ensure seamless connectivity between NiFi and external data sources or destinations.
• Implement monitoring tools and set up alerts to detect failures or performance issues.
• Generate reports on data flow metrics and performance for stakeholders.
• Stay up to date with the latest NiFi releases and patches, ensuring that the environment is using the latest stable versions.
• Optimize the performance of data transfer workflows by configuring NiFi’s backpressure settings, load balancing, and ensuring high availability across NiFi clusters.
• Monitor file transfer and API performance, troubleshooting bottlenecks and optimizing workflows as needed for both throughput and latency.
• Collaborate with cross-functional teams, including data engineers, security teams, and IT infrastructure teams from both entities, to ensure smooth data flow and integration.
• Provide detailed documentation on data flow architecture, security protocols, and API configurations to ensure that the integration process is well-understood and maintainable by both organizations post-acquisition.

Required Skills and Experience:

• Apache Nifi Expertise – MUST
• Apache Workflow configuration – MUST
• Must have knowledge of secure file transfer protocols (SFTP, FTPS, HTTPS) and encryption methods etc.
• Strong Troubleshooting
• Strong Communication
• Ability to operate independently and know when to escalate