Senior Data Engineer

Senior Data Engineer

Our client is looking for a Senior Software Engineer (Customer Data) to join our Data Engineering team. In this role you will play key part in developing our next generation Data Platform to support all our operation processes, data sciences and business intelligent data processing.

Technologies

If you have a passion for writing data processing software, including acquisition, cleansing, correlation, organization, analysis and machine learning, and the challenge of building a new data infrastructure from the ground up excites you, we would love to hear from you. The team works with:

  • The Hadoop ecosystem
  • Relational data modeling and batch processing on SQL databases
  • Distributed processing framework like Spark
  • Streaming data platforms like Kafka or AWS Kinesis
  • Realtime stream processing frameworks like Spark Streaming, Samza or Kafka Stream
  • Cloud computing systems like Amazon Web Services
  • Big Data analytics tools like Hive or Spark SQL

Responsibilities

  • Design, develop and maintain Java/Python based Data Platforms including Data Lake, Operational Datamart and Analytics Data Warehouses
  • Design and build scalable Java/Python based ELT/ETL workflows to transform and integrate data into Data Platform/Data Lake
  • Develop data strategy and roadmap for data technologies
  • Play multiple roles that span data architecture, design, data warehousing, and ELT/ETL processes
  • Work closely with product management, business, engineers, cross-functional analysts and data scientist to uncover new insights from our data
  • Design data models and ELT/ETL workflows
  • Enhance scalability, performance and stability of our existing infrastructure

Technical Requirements and Desired Skills

  • 5+ years of experience working as a software developer in Data Engineering, Data Warehousing/BI team
  • 5+ years of experience in Java/Python development
  • 3+ years of experience in SQL and Relational Databases
  • Experience with unit testing frameworks like JUnit or TestNG
  • Extensive experience working with structured and unstructured data platforms, ELT/ETL, and Data Modeling
  • Experience with data warehousing and BI platforms
  • Knowledge and experience with big data systems such as Hadoop, Spark is a plus
  • Experience with Machine Learning and Statistical Frameworks is a plus

Location

This is a permanent role based on the San Francisco Bay Peninsula.