A data centered business is on the lookout for a Data Engineer to join their team. This is an opportunity to work on cutting edge of tech and cloud computing. They are a fun environment with a quality team that is working with some of the biggest and best datasets in the country.
You will be building world-class, scalable data pipelines and warehouses to add real value to customers and drive new product development, both internally and externally.
- Completed BSc (Computer Science) degree or similar (Software Engineering, Data Science etc. with experience in software)
- 2-5 years’ experience in Data Engineering (Hadoop and Spark)
- Data Processing products - Big Query, Redshift, Spectrum, S3, Athena, Kafka, Spark, Storm, Flink, Beam, Presto, Hive
- ETL processes and transformations
- Cloud experience ideally with Google Cloud Platform
- DevOps Stack development experience
- Apache-Airflow or other data pipeline tools
- Exposure to Scala or Java in context of data processing
- Experience and proficiency with Python
- Experience in the design and implementation of data flows
- Advanced SQL/PostgreSQL/Redshift knowledge
- Support sophisticated predictive data products by maintaining data science production environments (cloud-based, python), ensuring that the outputs from Data Science models are available and integrated into the system and integrate, coordinate and maintain data flows between various sources of data
- Manage and maintain cloud service integrations that perform key data functions by working towards replacing third-party elements of the data pipeline by using open-source tools
- Make decisions around the infrastructure, layout and processes of the data warehouse, including:
- working with the engineering team on how to best track and record data
following up on data inconsistencies to ensure that it is corrected
- Transforming, standardizing and collecting data from various sources
Reference Number for this position is LV46312 which is a permanent position based in Johannesburg North offering a salary of up to R900k per annum highly negotiable on experience and ability. Contact Liza on email@example.com or call her on 011 463 3633 to discuss this and other opportunities.
Are you ready for a change of scenery? The e-Merge IT recruitment is a specialist niche recruitment agency. We offer our candidates options so that we can successfully place the right developers with the right companies in the right roles.
Do you have a friend who is a developer or technology specialist? We pay cash for successful referrals!