# Microsoft Fabric Data Factory

### What is Fabric Data Factory?

Fabric data factory is derived from Azure Data Factory for the most part with some subtle differences.

For example in Fabric data factory the concept of IR (Integration Runtimes) or entities like linked services of ADF(Azure Data Factory) dont exist.

They are primarily designed for seamless integration with Lakehouse and Data warehouse in the Fabric ecosystem.

You can check out the differences between the two [here](https://learn.microsoft.com/en-us/fabric/data-factory/compare-fabric-data-factory-and-azure-data-factory).

In this blog we will look into how to create a fabric data pipeline that copies files and folders from an ADLS2 location to a different location on ADLS2 while preserving the source file/folder hierarchical structure.

### The setup

On ADLS2 we have two folders : **Source** and **Destination** in the storage container.

We would copy the files and subfolders from Source folder to the Destination folder.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724440189500/87a628ae-fdcd-4481-8444-a9a0f7d36a38.png align="center")

The source folder structure is as below that contains the following files and subfolders

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724444937061/bc77ddc2-3c50-4b0e-a6b3-8bec0b8c4f17.png align="center")

Log into your fabric tenant and under your workspace click the Data Pipeline option

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724439911662/3d438d8d-00ce-41a8-a045-23c30d2ea661.png align="center")

Then add the `GetMetaData` activity on the Data pipeline canvas.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724441191074/4d277dbb-072f-41ff-a2d2-56edf4e3e852.png align="center")

In the settings option of the `GetMetaData` activity create a connection to the ADLS2 source location.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724441313251/d1102a6f-6ac2-44a6-95f7-fa0dabbe3885.png align="left")

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724441376458/b71a3d3c-bc51-4134-a6f0-648f5249f3ab.png align="left")

Once the connection is done, set the `file path` under the `GetMetaData` activity settings.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724441814081/403cf891-489f-4d02-8f9f-961d70c28422.png align="center")

Next add a `For each Activity` and under its `Settings` option set the `Item` option to :

```abap
@activity('Get Metadata').output.childItems
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724442140807/17175010-db59-4077-9ee0-092f9bf2670c.png align="left")

The `Get Metadata activity` child items would act as items object for the `For each activity`.

Now add a `Copy activity` inside the `For each activity`.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724442282722/8fc5a97e-73f2-4e9b-8cdf-241a4931b343.png align="center")

Set the `source connection` for the `Copy activity`.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724445654535/72401aab-4223-409d-863e-2075ad453aa4.png align="center")

Please note that we've configured the wildcard paths to: `/Source`. This setting means that we intend to copy all files and subfolders within the Source folder. If we change it to `/Source/*` it would only copy the subfolders and its contents excluding any files located in the root of the Source folder.

Next set the `Destination connection` for the `Copy Activity`.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724442928243/31b8bbc3-cde0-4488-9108-1e6498240fba.png align="center")

When you want to copy the subfolders along with the files, ensure that the file format option selected is `Binary` in the Source and Destination connection option of the Copy Activity.

Now go ahead and Execute the pipeline.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724444642328/c555ab29-4e6e-46be-ad06-0c795e03e4e8.png align="center")

Check the destination folder and you should the all the folders, subfolders and files copied over.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1724445529151/08784d73-4ef7-4cef-af0d-f181fbbd7f8b.png align="center")

### Closing Notes

Using the above method we demonstrated how simple and easy it was to move folders and files using the Copy data activity. However, a major drawback of the above approach is that it preserves the source hierarchy in the destination.

If you prefer not to maintain the source hierarchy in the destination and want to dump all the files in a designated single folder in the destination then the process becomes very complex .I will explain in [another blog](https://www.azureguru.net/traversing-and-copying-complex-directory-structure-on-azure-using-microsoft-fabric-data-pipeline) how to achieve it. Till then stay tuned.