X4 Technology are offering an opportunity for Senior Market Data Engineer to come work with global data provider who simplify global trade information and providing valuable insights to the commodities, energy, and maritime sectors.
The Market Data team is responsible for acquiring, filtering, enhancing, and distributing externally sourced data from thousands of maritime trade channels. This data is crucial for our operations, enabling us to deliver customers a timely and precise overview of global trade.
Responsibilities of the Senior Market Data Engineer:
- Work alongside Product Owners and Engineering Managers to outline the strategy for improving our data pipelines, focusing on enhancing quality and reliability.
- Create and implement APIs, pipelines, and read models to efficiently publish our datasets.
- Optimize and maintain datasets and their schemas to ensure optimal performance.
- Guarantee the integrity and availability of data for all users.
- Provide state-of-the-art environments and tools to support the development team.
- Share your knowledge with team members, offering guidance and assistance as needed. Collaborate with fellow engineers to provide insights that can assist in re-architecting existing systems for better scalability and extensibility.
Requirements of the Senior Market Data Engineer:
- At least 5 years of experience as a Software Engineer or Data Engineer, with substantial hands-on experience using Kafka and Scala in a production environment, particularly with Cats or Spark.
- Proficiency in designing and implementing RESTful APIs.
- Experience in DevOps and Infrastructure as Code practices, especially with AWS.
- Familiarity with Data Processing Orchestration tools, such as Airflow.
- Fluency in English and experience in an international work environment.
- Familiarity with functional programming frameworks like fs2.
- Understanding of Big Data Management concepts, including DeltaLake, Medallion, and Lineage.
- Experience with Terraform, GitOps, and Kubernetes.
- Background in machine learning, OCR, or NLP.
- Proficient with ElasticSearch and GraphQL.
- Strong skills in Python and SQL schema optimization using PostgreSQL.
- Knowledge of AWS WAF principles.
This will primarily be a remote opportunity that might require the occasional attendance into their London based office. The salary is coming in anywhere between £65,000 - £100,000 DOE.
The interview process can start as early as Monday the 7th of October, if you feel like you would be a good match please apply immediately for consideration.