For one of our clients we are looking for a Consultant Palantir Data Pipelining
HDF - Palantir - Data Pipelining
The service is requested as part the client's Data Foundation, Palantir project for Adhesive Business. The project has the purpose to build up a comprehensive service for Adhesive business and IT people by combining different Microsoft Azure Services and other tools to provide a data lake with data processing capabilities. Huge datasets from different platforms are managed on this data hub.
Establish robust and performant pipelines using Python, Apache Spark within the Azure environment using Azure Databricks;
Implement quality checks on incoming data and perform quality assurance tests on the datasets and code;
Analyse business’ requirement in terms of data transformation in a qualitatively diverse data environment;
Consult business on best practices and technical effort in relation to product development and project achievements.