For one of our clients we are looking for a Consultant (f/m) Azure Data Lake Migration
- Stabilize and enhance data pipelines loading data into our central data lake;
- Migrate the solution from Azure Gen 1 Data Lake to Azure Gen 2 Data Lake;
- Plan a proposal of a handover the solution to maintenance.
- Azure Data Factory regarding pipeline implementations;
- Azure Databricks regarding pipeline implementations
- Azure Data Lake 1 Authorization.
- Documentation of the current data processing pipelines which is subject to approval by the client;
- Source code will be provided to the contractor in advance;
- Independently conduct interviews with stakeholders and developers to understand requirements from business and from technical point of view
- Enhancement and maintenance of existing pipelines developed with Azure Data Factory and Azure Data Bricks based on the documentation above;
- Developing a concept to migrate the pipelines to our Gen2 data lake which is also subject to approval by the client.
- The service provision of the contractor has the goal to prepare the migration from Azure Data Lake 1 to Data Lake 2.