For one of our clients we are looking for a Consultant (f/m) Azure Data Lake Migration
 
Overview:

• Stabilize and enhance data pipelines loading data into our central data lake;
• Migrate the solution from Azure Gen 1 Data Lake to Azure Gen 2 Data Lake;
• Plan a proposal of a handover the solution to maintenance.

 
Tasks:

• Azure Data Factory regarding pipeline implementations;
• Azure Databricks regarding pipeline implementations
• Azure Data Lake 1 Authorization.
• Documentation of the current data processing pipelines which is subject to approval by the client;
  • Source code will be provided to the contractor in advance;
  • Independently conduct interviews with stakeholders and developers to understand requirements from business and from technical point of view
• Enhancement and maintenance of existing pipelines developed with Azure Data Factory and Azure Data Bricks based on the documentation above;
• Developing a concept to migrate the pipelines to our Gen2 data lake which is also subject to approval by the client.
• The service provision of the contractor has the goal to prepare the migration from Azure Data Lake 1 to Data Lake 2.

 
Start: ASAP
Location: remote
Duration: 3 months+