Azure Data Engineer

About Softtek

Founded in 1982, Softtek is a global provider of process-driven IT solutions with 30 offices in North America, Latin America, Europe and Asia. With 15 Global Delivery Centers in the U.S., Mexico, China, Brazil, Argentina, Costa Rica, Spain, Hungary and India. Softtek helps improve and produce predictable outcomes for top-tier corporations in over 20 countries. Softtek is the creator and a leader of the nearshore industry.

For more information on what we do, who we are, and career opportunities, visit www.softtek.com

Role: Azure Data Engineer

Responsibilities:

• Create and maintain an optimal data pipeline architecture.
• Assemble large, complex data sets that meet business, functional, and non-functional requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, and redesigning infrastructure for greater scalability.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using big data technologies.
• Create analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
• Work with stakeholders including executive, product, data, and design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across national boundaries through multiple data centers and GCP regions.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product to become an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.

Requirements:

• Minimum of 4 years of experience as a Data Engineer, with a degree in computer science, statistics, information systems, or a related field.
• Advanced knowledge of SQL and experience working with relational and non-relational databases, as well as familiarity with a variety of databases.
• Experience in creating and optimizing data pipelines, architectures, and big data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Analytical skills related to working with unstructured datasets.
• Create processes supporting data transformation, data structures, metadata, dependency, and workload management.
• Experience handling message queues, stream processing, and highly scalable data stores.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• Experience using the following software tools:
• Hadoop, Spark, Kafka, etc.
• Experience with relational SQL and NoSQL databases.
• Experience with Azure Cloud and services such as Cosmos DB, Azure Data Lake, Azure SQL.
• Experience with streaming processing systems: Storm, Spark-Streaming.
• Experience with object-oriented languages: Python, Scala, C++, Java, Go.
• Experience with CI/CD integrations and deliveries.
• Experience with software containerization and automation: Docker / Kubernetes, Ansible / Puppet.
• Experience with data flow and extraction tools like Airflow, Big Query.
• Experience in agile frameworks

Required Language

• Advanced English

Possible Job location(s)

Remote - Costa Rica

“At Softtek, we recognize that talent is diverse, we respect each person's life condition, and we promote equitable treatment and opportunities without discrimination.

Join the Softtek team and visit our page https://www.softtek.com/transcend to learn more about our Diversity and Inclusion Policy.”