Our client is the digital arm of a 100 year old company in the food supply chain sector. They are building a variety of Digital B2B and B2C software solutions to help their clients to be able to feed our vastly growing population more efficiently.
Although the mother company is 100 years old, their digital arm functions like a startup and as such they are constantly learning and developing. They have their own data platform which is pooling in all the data from their global operations (data from 10 different software products), but it is not very user friendly and well structured at the moment. They see Data and Data Engineering as a high priority so now they are building a dedicated Data Engineering team in Berlin. You would be the sole Data Engineer in the team at the moment which will give you freehand in deciding how to go about structuring and curating data and reshaping the foundations of data engineering for them. You will also play instrumental role in making their data platform better, more usable and innovative going forward.
This is a fundamental role to the continued success of the business and, as such, the incoming candidate will be able to make a significant and tangible impact on the company.
What to expect
- Help shape the Data Infrastructure and Process enabling the use of data to create Insights and Data Products
- Develop, construct, test and maintain data architectures
- Handle all data-related activities such as data parsing, cleansing, quality definitions, pipelining, storage, maintenance of ETL scripts, etc.
- Support in building complex queries required by the product teams
- Handle data integration, consolidation and reconciliation activities for digital products
- Provide governance support to ensure the data is stored and managed as per industry practices
- Manage deployment of analytics programs, machine learning and statistical methods
- Support data preparation activities for analytics including pipelining, data cleansing and web scrapping
- Help shape the data team and its procedures. As a member of a newly created group, there is a lot to contribute to.
- Hands on experience in data engineering projects such as writing SQL queries, data pipelining, API provisioning, database optimization / maintenance etc.
- Knowledge of data architecture concepts such as database systems, schemas, data warehousing etc.
- Worked with scalable architectures (lambda, containers etc.)
- Able to code in Python, client's preferred tech stack
- Comfortable working with client's tools: AWS, GitLab, Jira, Docker, Kubernetes, Airflow
- Desired knowledge of AWS Data stack: AWS Glue, S3, Athenas, Redshift
- Desired Kafka knowledge