February 7, 2019
Johannesburg Hyde Park, South Africa
Job Type


The business made waves in 2017 when they secured $2m venture backing out of North America for their AI platform that can deliver industry leading automated consumer behavior predictions for companies using their raw unstructured data.

They are currently recruiting engineers to help fill the massive demand for their product and to help their results become even better (scarier).

You will be required to work with a team of extraordinary engineers, to deliver businesses automated consumer behavior prediction platform to companies. The business results are multiples better than traditional statistical or ML methods. They are automating and commoditizing cutting edge AI results directly from a client data lake and your job will be to help their platform handle data at the massive scale needed. The data sizes can reach PetaByte ranges, and they still expect it to process rapidly.


  • 3 years’ experience with Hadoop v2 and MapReduce
  • Proficiency with the management of our Hadoop cluster and accompanying services including Hive, Spark, Kafka, Scoop and Oozie
  • Proficiency with Presto
  • 3 years’ experience in NoSql databases. Cassandra preferred
  • Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
  • Experience with integration of data from multiple data sources
  • Knowledge of various ETL techniques
  • 3 years’ experience in creating Lambda Architecture, along with knowledge of its advantages and drawbacks
  • Experience with Cloudera/MapR/Hortonworks


  • Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
  • Implementing ETL process
  • Part of ETL is analysing and understanding business client data well enough to integrate in to our API
  • Propose, design and implement Big Data Architecture including infrastructure
  • Monitoring performance and advising any necessary infrastructure changes
  • Defining data retention policies

Reference Number for this position is GZ42995 which is a permanent position based in Hyde Park offering a cost to company salary of R800K pa negotiable on experience and ability. Contact Garth on or call him on 011 463 3633 to discuss this and other opportunities.

Are you ready for a change of scenery? The e-Merge IT recruitment is a specialist niche recruitment agency. We offer our candidates options so that we can successfully place the right developers with the right companies in the right roles.

Do you have a friend who is a developer or technology specialist? We pay cash for successful referrals!

Drop files here browse files ...

Related Jobs

Are you sure you want to delete this file?

Subscribe To Personalized Notifications

You are subscribing to jobs matching your current search criteria.

Email Notifications

Email notifications will be sent to you Subscribe


Custom RSS Feed

Your personalized RSS Feed is below, copy the address to your RSS reader. Subscribe