Senior Engineer - Data Engineering

Education:

Min. Bachelor's in Computer Science or Engineering, Information Systems, or related fields

Expectations in the job:

• Collaborate with other teams to design, develop and deploy data tools that support both operations and product use cases

• Work with the team in solving problems in big data technologies and prototype solutions to improve our data processing architecture

Responsibilities

• Works closely with the business’s Data and Analytics teams and gathering technical requirements
• Experience in building and maintaining reliable and scalable ETL pipeline on big data & / or Cloud platform through the collection, storage, processing, and transformation of large datasets
• Support production issues as relate to application functionality and integrations

Must Haves:

• Proficient in spoken and written communication skills (verbal and non-verbal)
• Proven experience in developing Big Data eco system/data warehouses and ETL pipelines (Min. 2 years)
• Proficient in scripting capability for analysis and reporting using any one of the languages Python/R/scala programming Proficient SQL development skills with ability to write complex efficient queries for data integration
• Analytical skills to support Business Analysts and ability to translate user stories / work closely with the tech team
• Must have strong experience in data warehouse concepts
• Experience in Database design/data modelling  in Data lake environment is highly preferred
• Strong problem-solving skills (Math skills required for data modelling)
• Ability to manage and complete multiple tasks within tight deadlines
• Working experience with any one of the cloud platforms namely AWS, Azure, Google cloud et al

Desired: 

• Familiarity with DevOps/DataOps practices like CICD pipeline
• Good knowledge on Agile principles and experience working in scrum teams using Jira

Tech Stack:

• Expert/proficient in scripting languages in python, pyspark, R, scala, SQL
• Experience in AWS/GCP/Azure (AWS EMR, Databricks, Spark, Azure HDInsight et al)
• Understanding Data Lake, Lake House Concepts, any new Big Data eco system.
• Good understanding of cloud Data Stack including services like S3, Redshift, EMR, Lambda, Athena & Glue.
• Good understanding of Hadoop/HDFS
• Preferred to have experience using Data modelling tools like ERWin, Visio et al.

Soft Skills

• Excellent Verbal and Communication skills needed.
• Should be a excellent team player.
• Should be comfortable with the basics of Agile and have worked in atleast one principle like Scrum.
• Should be comfortable to mentor junior engineers in the team (For Senior Data Engineers)
• Should be able to operate in an ambiguous environment with minimal guidance.