GridTorrent
GridTorrent is an implementation of the popular BitTorrent protocol designed to interface and integrate with well-defined and deployed Data Grid components and protocols (e.g. GridFTP, RLS). Just like BitTorrent, GridTorrent is based on peer-to-peer techniques, that allow clients to download files from multiple sources while uploading them to other users at the same time, rather than obtaining them from a central server. By dividing files into fragments, GridTorrent can combine the best out of the two protocols:
- It exploits BitTorrent's peer and fragment selection , thus providing an optimized data transfer service.
- It takes advantage of the striped version of GridFTP protocol, as it has the ability to directly communicate with GridFTP servers, thus being backwards compatible.
GridTorrent Scenario:
In short, GridTorrent works as follows: A request to GridTorrent for a file triggers a query to the Replica Location Service. This procedure is repeated periodically, in order to detect any changes in the locations of file replicas or of any joins or departures of nodes. The file can be located in GridFTP servers or GridTorrent seeds or leechers, following the BitTorrent terminology. This is indicated by the prefix of the file’s PFN, which reveals the supported protocol, thus preserving backwards compatibility with the existing Grid Architecture. For GridTorrent peers, the file urls stored in the RLS are of the following form:
gtp://site.fully.qualified.domain.name/path/to/file
Upon receiving the list of peers, GridTorrent acts according to the protocol prefix of the PFN. If it concerns a GridTorrent client, the two involved peers initiate communication following the BitTorrent protocol specifications. In case the remote peer is a GridFTP server, the local peer issues a GridFTP partial get message for the data within the specific block it intends to download, determined by the local rarest-first piece selection policy.
GridTorrent is fully compliant with Grid Security Infrastructure (GSI). The user is offered the possibility to choose the security level of his transfers, by defining whether he requires or not user authentication, integrity check or data encryption.
GridTorrent Block Diagram:
The fundamental components of the GridTorrent mechanism are the following:
- The RLSManager, a component that communicates with the Replica Location Service.
- The PeerManager, which handles all the communication with other GridTorrent or GridFTP enabled peers.
- The DiskManager manages all disk I/O for storing and retrieving files.
- The GTPConnection handles the communication with a remote GridTorrent peer while GFTPConnection issues partial GridFTP messages and receives the blocks sent by a GridFTP server.
- The Server class is listening for new connections from remote GridTorrent peers.
Publications
- Athanasia Asiki, Katerina Doka, Ioannis Konstantinou, Antonis Zissimos, and Nectarios Koziris "A Distributed Architecture for Multi-Dimensional Indexing and Data Retrieval in Grid Environments" In Proceedings of the Cracow 2007 Grid Workshop (CGW'07), Krakow, Polland, October 16-17, 2007 [link]
- Ioannis Konstantinou, Katerina Doka, Athanasia Asiki, Antonis Zissimos, and Nectarios Koziris "Gredia Middleware Architecture" In Proceedings of the Cracow 2007 Grid Workshop (CGW'07), Krakow, Polland, October 16-17, 2007 [link]
- A. Zissimos, K. Doka, A. Chazapis and N. Koziris, "GridTorrent: Optimizing data transfers in the Grid with collaborative sharing," in Proceedings of the 11th Panhellenic Conference on Informatics (PCI2007), Patras, Greece, May 2007 [pdf]