Data Engineer

Key Responsibilities

• Creating systems that ingest data from various sources
• Developing data pipelines
• Workflow flexibility and strong teamwork skills
• Ownership of end-to-end data pipelines and automate data quality reporting process.
• Collaboration with SME/Software engineers and Data Science experts to understand the needs and deliver efficient data driven solutions.
• Experience with distributed systems and cloud architecture
• Write efficient, well documented and highly readable code

Ideal Candidate Requirements

• 2+ years of experience in Big Data Engineering using tools like Spark, Hadoop, Hive, etc.
• 3+ years of experience with Software Engineering using Python / development using modern Relational and NoSQL databases like MySQL, PostgreSQL, and MongoDB
• Some experience with stream processing systems : Storm, Spark Streaming, etc.
• Some Data Modelling and Data Science experience building predictive models
• Experience with Unix and statistical software packages (R, SAS) for data manipulation
• Confidence with analytical tools such as R or Matlab
• Experience with data-science tools / workbench like Dataiku, Jupyter, Rapid Miner, etc. is a plus
• Experience with Docker and Linux shell is a plus.
• Bachelor's degree or greater in Computer Science or related areas or equivalent practical experience