Location options: Visa sponsor, Paid relocation Job type: Full-time
Experience level: Senior
Technologies
java, python, kafka, google-cloud-platform, spark-streaming
Job description
The Data team at Traveloka consists of Analytics, Data Platform, Data Science, Machine Learning / AI groups, individually comprising an expansively diverse team of data engineers, data analysts, data scientists, and product managers.
The Ingestion team, a branch of the Data Platform group, is the gatekeeper and steward for data streaming into our platform. We are responsible for specifying data contracts along with building, maintaining, and streamlining real-time as well as batch pipelines to strive for efficient and seamless ingestion of billions of events everyday from internal as well as external sources, accruing Petabytes of data in our data lake. With priority focus on automation, we also provide self-service platforms to add new data pipelines that cater to dynamic business requirements.
Responsibilities
Your domain of ownerships:
- Design and build efficient, scalable and reliable data infrastructure.
- Own, architect, and scale the ingestion pipelines.
- Ensure robust data quality in our data pipelines.
- Implement and constantly improve automation around deployment, monitoring, alerting and debugging to ensure reliable pipelines.
- Work with various stakeholders, understand the specifications for new ingestion, and integrate them with the ingestion infrastructure.
- Identify the right data technology for various use cases and implement them.
- Adopt and learn new technologies relevant to the team's growth and improvement of the tech stack.
Technical Qualifications
- Good knowledge of any JVM-based language such as Java and Python with an expertise in either one (rich experience on writing production grade code).
- Good understanding of different data formats such as Protobuf, Avro, JSON, Parquet, or ORC..
- Rich experience in designing large-scale streaming pipelines using Apache Beam, Kafka, Google Dataflow, Google Pub/Sub, Spark, or other streaming frameworks.
- Good familiarity with Cloud Technologies such as AWS and GCP.
- Good comprehension of container orchestration such as Docker and Kubernetes.
- Good grasp of software engineering concepts, design patterns, and algorithms.
- Hands-on experiences in using distributed version control system such as GitHub, which we use and the practices of CI/CD.
- Practical exposure to one or more workflow management and scheduling frameworks like Apache Airflow and Azkaban.
- At least a Bachelor’s degree in Computer Science from a reputable university or equivalent experience.
Nice-to-Haves
You’ll be a standout from the rest of the candidates with any of the skillset below:
- Good SQL skills.
- Good knowledge of various RDBMS- or NoSQL-based DB systems such as PostgreSQL, MongoDB, DynamoDB, and BigTable.
- Work background in travel or e-commerce industry.
To Apply:
- Send us your CV and point us to your blog, home page, or LinkedIn profile.
- Our friendly Recruiter will get in touch with you and tell you more about the exciting things the team works on and brief you through the interview process
