MultilogToolkit < CSLab

---+ Generic Multilog Toolkit.

Read [[Grid4All][Grid4All]] to get some context.

[[%ATTACHURL%/multilog_toolkit.tar.gz][Download]] the first (alpha) version of the toolkit. 

The generic multilog toolkit implements a filesystem structure that includes a number of log files.
Each log file belongs to an editor on the network.
Editors log their updates to a shared document in their own log files and read all other logs
for the others' updates.
The toolkit, besides log manipulation, enables each editor to serve his own log from his site
so that each other editor can maintain a replica in their own site.
The replication process is designed to handle casual network disconnection of any editor.

It uses TCP/IP sockets for communication and Linux inotify to monitor local files.
It has four main tools; a log server, a log client, a log reader/writer (logtool)
and a multilog daemon.

---++ !LogServer

The !LogServer monitors a log directory for changes and serves read requests from the network.
Read requests are made at an offset relative to the entire log.
The !LogServer handles the different chunkfiles efficiently.
Clients may request to be subscribed for notifications when the log changes.
The !LogServer will dispatch a short message with the new log size to subscribers.
The local participant directly appends to the log.
The !LogServer uses the system's facilities to be notified of changes (linux inotify).

---++ !LogClient

The !LogClient is bound to a specific !LogServer.
The !LogClient creates an empty log directory and then updates according
to the !LogServer.
The !LogClient will only append to its log.
Throttling of synchronization granularity can be controlled by the local user,
via special access patterns to files.
The local participant directly reads the log.
If the !LogClient is disconnected from the network, this appears as idleness.
The !LogClient will attempt to learn about the !LogServer's state according to
a heartbeat.
When there is demand, the !LogClient will subscribe for notifications
so that latency is minimized.

---++ !LogTool

The !LogTool makes access to the structure of a log.
It provides reading and writing logs, handling log chunks efficiently,
and mechanisms for constructing and maintaing an in-memory state of the log.
The !LogTool is used by the !LogServer and !LogClient, but can also be
used from command line, for reading, writing, or waiting on a log.

---++ !MultilogD

!MultilogD is a daemonized script that parses a list of participant and
populates a multilog directory with one log for each.
!LogServer is started for the local participant's log and
!LogClient is started for all the others.
!MultilogD periodically polls the list of participants and adds any new
ones and re-launches daemons for logs, if they have vanished.

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
gz	multilog_toolkit.tar.gz	r1	manage	15.3 K	2008-05-29 - 11:53	GeorgiosTsoukalas	The first version of the generic multilog toolkit

This topic: CSLab > WebHome > WebSearch > Grid4All > MultilogToolkit
Topic revision: r1 - 2008-05-29 - GeorgiosTsoukalas

No permission to view TWiki.WebBottomBar