Backend Developer (YouTube API, Web scraping, Microservices) - Part-time - WW

Project Overview

We are looking for an experienced developer to build a microservice that scrapes and searches YouTube for product-related videos to enhance an e-commerce platform with AI-driven recommendations. This microservice will handle video metadata extraction, transcription with timestamps, and sentiment analysis for videos related to specific e-commerce products.

The ideal candidate should have strong experience with the YouTube API, large-scale data scraping, and API integration, with a focus on backend development.

The project timeline is approximately one month for initial development, with the potential for ongoing maintenance and future expansions to platforms like Reddit or Meta.

Key Responsibilities

• Develop a microservice that scrapes YouTube for videos based on search terms (e.g., channel name, product names, product part numbers).
• Extract metadata from YouTube videos, including views, likes, dislikes, upload date, and video duration.
• Generate and store video transcriptions, including timestamps that correspond to spoken words in the video.
• Enhance service capabilities to perform sentiment analysis on video content.
• Integrate the service asynchronously with an existing backend system (no frontend development required).

Key Elements of the Microservice

• Search Terms: Handle inputs like product names, alternate product IDs, and channel names.
• Outputs: JSON format with details such as:

1. video_url: Direct link to the YouTube video
2. metadata: Thumbnail URL, title, description, channel info, views, likes, dislikes
3. transcription: Array of objects containing timestamps and corresponding video transcription

Technical Requirements

• Backend Development: Strong expertise in backend architecture and microservice design.
• YouTube API Expertise: Experience working with YouTube's API, rate-limiting, and large-scale data scraping.
• Preferred Tech Stack: Node.js, Python, React (open to flexibility based on candidate's expertise).
• Scalability: Ability to handle large-scale scraping efficiently while respecting YouTube API limits.
• Asynchronous Work: The service will function asynchronously and integrate with the client’s existing backend.

Qualifications

• Proven experience with YouTube API and data scraping tools.
• Proficiency in backend development with microservices and API integrations.
• Experience with data processing, JSON formats, and handling large datasets.
• Familiarity with sentiment analysis and natural language processing is a plus.
• Ability to work independently in a remote, asynchronous work environment.