VOFS (DFS): Federative, disconnected, peer-to-peer filesystem for Grid4All

Overview

Grid4All is an FP6 project funded by the European Union. The main objective of the project is to research and develop a democratic Grid, that enables individual users to draw upon internet resources for their applications without having to invest in IT infrastructure. Read about Grid4All.

The final prototype package is available here

Our team is involved in Grid4All data storage, researching and developing DFS, a peer-to-peer distributed filesystem with a modern and democratic character, that is most suitable for the project. Currently, it is in an early prototype status.

DFS supports other Grid4All components. In particular, DFS supports the Telex framework for collaborative applications, by providing connectivity to its distributed instances through special filesystem structures called multilogs.

Apart from DFS we have implemented a fully functional and efficient generic multilog toolkit for general evaluation of the multilog design.

The most prominent features of DFS are:

• Peer-to-peer, web-like navigation. Other's computers are accessible just by typing their address in the path.
• Federation of files. You can create workspaces and populate them by linking in files anywhere on the internet.
• Federation of storage. Peers can contribute storage to each other, creating storage pools. Peers create files using this distributed pooled storage.
• Disconnected operation. Files are locally cached and remain available when disconnected from the network. Synchronisation is done automatically, if conflicts do not occur.
• Notifications for file modifications and publish-subscribe messaging
• Integration with the OS. DFS can be mounted in Linux and MacOS.

Read an article on DFS architecture

Creating a VO-aware filesystem with DFS

In Grid4All, DFS is used to create a VO-aware filesystem for VO members. It's basic function is to create shared workspaces from existing VO member files and storage, or other resources contributed to the system. The scenario looks like this:

Publications

• A. Chazapis, G. Tsoukalas, G. Verigakis, K. Kourtis, A. Sotiropoulos and N. Koziris, "Global-scale peer-to-peer file services with DFS," in Proceedings of the 8th IEEE/ACM International Conference on Grid Computing (Grid 2007), Austin, TX, USA, September 2007 [pdf]