Developer or AI Workflow Engineer for Eye Fixation Enhancement

Role Overview

We are looking for a skilled Developer or AI Workflow Engineer to join our project focused on enhancing eye fixation within the Uni3C ComfyUI workflow. The primary objective is to address and resolve the current issue where character seyes follow camera movement, ensuring they remain fixed on a predetermined point in 3D space, independent of camera dynamics. The character can be dressed or unclothed.

Responsibilities

• Analyze the existing Uni3C ComfyUI workflow to identify the underlying causes of the eye tracking issue.
• Develop and implement solutions that ensure eye fixation works consistently across various camera angles and character poses.
• Build and deploy an API capable of processing photos and videos to output character animations with stable eye direction.
• Explore and apply approaches such as using the http://eyes.pt BBOX detector, masking and inpainting techniques, ComfyUI MotionCtrl, and ControlNet conditioning to achieve desired outcomes.
• Deliver updated workflow files, comprehensive documentation, and a demo video showcasing the corrected eye behavior.
• Conduct live walkthrough sessions to demonstrate the solution and ensure seamless integration.

Required Skills

• Proven experience with ComfyUI and ControlNet.
• Familiarity with Uni3C or similar video synthesis frameworks.
• Expertise in character motion control, eye tracking, and inpainting techniques.

Nice to Have

• Experience with SMPL-X models or 3D-enhanced video generation.