The first thing we wanted was a safety exit: the chance for us to "change components" with (almost) zero impact. On assembling different pieces of pre-existing software one often stumbles into surprises, so we wanted things to integrate loosely enough for us to change a piece of software with another without actually compromising the general architecture. We also wanted a high redundancy, which means that we wanted to have many copies of the data, so that if a node or part of it gets damaged there are still good chances to get data back.
The final result is shown in the attached graphic. Rising up from basement level what you have is:
- Freenet: an open source peer to peer network that has no central server and can serve pretty well the following layer. It can efficently cache the multimedia files on a large number of nodes, so that if you have any such file you can "publish it" and the network will provide to cache it on a number of nodes proportional to its popularity
- Mercurial: an open source distributed version control system. It manages a repository of the XML representations of our objects. The node writes its activity to this repository, that is pushed and pulled to/from the general repository in terms of minimal incremental changes to reduce the amount of traffic. It uses Freenet as a transport when an Internet connection is available, but it can also use just anything, from usual http synchs to hand transported flash drives (which frees our users from the need to have a permanent or even occasional Internet connection).
- Mysql: an open source relational database into which objects are assembled to be efficiently served to the GUIs.
- Sphinx: a free open-source SQL full-text search engine. It is deeply integrated with MySQL, so we put it in parallel. We use it as "contained" in mysql, as a plugin storage engine, but if needed it can be accessed as an independent entity.
- Ambaradaemon: this is what all external applications use to deal with our system. They have no notion of what software packages are doing what. They connect to this daemon and use a public API to send it commands and retrieve answers. So it is... a translator, if you please.
Any GUI that can manage front-end relations with the user. It can be any number of them, they can be either desktop applications or macros for Open Office or local intraweb applications, you name it. They all exchange messages with Amabaradaemon, who in turn delivers them in proper format to the package that executes them.
We shall now spend a few words on some of the components, to see more in detail what they actually do.