The Big Data Engineer is a member of a team of technology experts specialising in various tasks associated with data platform administration and maintenance.
The primary tasks are focussed on ensuring excellent uptime and performance of the data platforms and as such, a Big Data Engineer has a good understanding of NoSQL operation in general, good knowledge of best practices, and a passion for ensuring the highest standards of uptime through proactive monitoring, pre-emptive tuning and general attention to detail.
In this role, you will be primarily responsible for NoSQL data platforms such as Kafka, Cassandra and Redis, with secondary responsibilities for databases from other vendors, so a desire to learn multiple platforms is key.
Automation of installation, monitoring and maintenance using tools like Ansible, Prometheus and Grafana is key with this role.
As a member of a geographically dispersed group of database experts, the Big Data Engineer is required to work on a shared backlog of time-sensitive tasks and deliver high quality data platform services to the Company.
This is only possible through efficient teamwork, prompt escalation and excellent communication skills.
Key Responsibilities :
Proactive participation in the Architectural design and implementation of the Big Data solution across all SGD Products
Proactive monitoring of production data platforms and resolution of system generated alerts
Proactive health checking and maintenance of the data platforms including resilience, maintenance, capacity planning and performance analysis.
Platform integrity checks including indexes, data integrity, disk / storage integrity
Reporting and analysis, including extracting data from multiple sources and identifying and resolving inconsistencies
Platform configuration, patching and upgrades according to SG Digital change and release management processes
Design and implementation of automation using scripting languages such as Ansible
Design and implementation of monitoring tools
Reporting on the stability and performance trends of platforms
Improvement and maintenance of existing documentation, such as various HowTo documents, Operations Manuals and Alarms Journals.
At least 5 years experience with NoSQL platforms
Expert knowledge of Kafka implementation, monitoring and alerting
Good understanding of the administration of Cassandra and Redis
Proficiency in SQL
System administration skills in Linux preferable and / or Windows
Knowledge of at least one other major database product including, but not limited to, Informix, Microsoft SQL Server, MariaDB / MySQL, Postgres
Good knowledge of shell scripting. Additional coding skills, such as Ansible, would be advantageous
Troubleshooting skills and an ability to determine the root cause of an issue quickly
A willingness to learn and take on new challenges, and a desire for continuous personal and technical development
An attention to detail
A performance focused culture right across the business. But that's just the foundation! We also believe in a work hard, play hard environment, where we embrace people who are tuned in to the sound of creativity and innovation.
As a truly international company, we can offer you the diversity and the opportunities of the world. Mix this with a people-friendly culture and we got a pretty good offer to give.
So don't just come and work for us, spend your time with us!