Middle Data Engineer - Qatar, دولة قطر - Talent Pal

Talent Pal
Talent Pal
شركة تم التحقق منها
Qatar, دولة قطر

منذ أسبوع

Aisha Al-Thani

تم النشر بواسطة:

Aisha Al-Thani

مسوّقة للمواهب لبيبي
وصف
Main responsibilities will include creating pipelines with complex ETL logic, making performance optimizations, investigating cases in collaboration with BI and Data Science team Our workflow emphasizes adaptability, collaboration, and focus on delivering value to customers and teammates We look at the entire development process and optimize it as a whole, involving a culture of continuous platform and personal improvement Skills Required Python programming skills, with a focus on code reuse, parallelization, and ensuring reliable pipeline functionality Skills in workflow management frameworks (Airflow) with the ability to define and run complex DAGs Extensive SQL skills, including DML and DDL (ALTER TABLE, GENERATE_ARRAY, LEAD, ) and query performance optimization Py Spark knowledge for ad-hoc Spark jobs, window functions, and Spark SQL for aggregate and join operations Understanding of Data Warehousing and Data Lake concepts, dimensional approach, ability to follow appropriate indexing, partitioning, and other optimizations to support efficient data retrieval 2+ years of experience with cloud providers or proficiency in using Databases/Cloud Storage APIs Responsibilities Create new batch/streaming pipelines for newly appearing databases Participating in integrated storage layer creation, investigating linkage between services Improve/expand pipelines with new tables or fields according to business requests Participate in technical debt management, redefine tables partitioning, create summarised tables and proceed migration, create metadata Use SQL and business context understanding skills to help engineers, BI analysts, and product managers get insights Our Stack GCP platform Big Query Databricks AWS streaming services Tableau, Redash Benefits 100% remote work with ability to work from office in the countries where we have offices - Qatar, Kazakhstan, Tajikistan Free courses and and personal growth activities The opportunity for growth in a mission-driven, well-funded and already profitable start-up Cultural enrichment interacting with Teammates from more than 50+ countries

This job has been sourced from an external job board.
المزيد من الوظائف من Talent Pal
انظر جميع وظائف Talent Pal