You might wonder how is Portal’s development progressing after Austin and Jason posted their updates a while ago. After posting my article on the research of file synchronizers I am looked through all the publications and articles I mentioned and a lot more. As it often happens with these kinds of inquiries you end up in similar or even completely different research areas. In this case I found out that some file synchronization publications overlap with the research underlying my company’s product SubEthaEdit which features collaborate text editing.
But let’s get back to identify the implications for Portal’s development. Soon after reading the fundamental papers about this topic I felt confident that you probably won’t invent a new file synchronization engine yourself. There are a lot of interesting tools out there which work quite well but either lack a cool user interface as envisioned by Farzad or just don’t quite meet the requirements Portal has. For example, rsync is a very popular tool among administators and power users to keep file trees in sync over a large number of machines. rsync tries to minimize the transfered bytes to keep two copies of a file in sync by doing incremental updates on the file. This is especially great for slow connections where you don’t want to transfer a complete file if only a few bytes have changed. The described approach is employed by the rsync tool and is generally known as the rsync algorithm. You may find the algorithm implemented in other software as well. But what else does rsync offer? It is basically used as mirroring tool. You can mirror a file tree from your machine to several others. It allows you to promote changes you made on your machine to others. If you change your files on the other machines and then want to promote them back then it get’s difficult because rsync doesn’t have great support for back and forth synching. It’s great as a uni-directional mirror utility. Using Portal you want to have bi-directional synching because you want to work with your files on every synched machine without worrying whether they can be promoted back. Therefore rsync is not an option for Portal.
My next candidate was Unison. It caught my eye early on because like rsync it is a tool that is widely used and not yet another research prototype. First of all it support bi-directional synching which we want for Portal. It uses a compression protocal similar to rsync which makes it suitable for slow links. Besides that it is resilient to failures. I think this one of the most important aspects if you don’t want to worry all the time whether my synching software is gonna destroy my data on the next update. There are so many failure scenarios in a synching application. You can experience all sorts of network failures. Your software might crash. Your files are modified while the synching process is still going on. But in all those cases you want your replicas to stay intact and be synchable in the future. Unison has put a lot of effort into making sure that your data is safe. Interestingly enough Unison features specification dealing with all those issues that is also proven.
What does all this mean for Portal? After my research on topic I decided to use Unison as the foundation for Portal. There are two options for accomplishing this. The first approach I pursued would have been a Mac OS X specific implementation of the Unison specification. The result would be an optimized solution for Mac OS X which could make use of all the operating specifics. Implementing your own solution takes a lot of time and you wouln’t see any flashy results because it would involve mostly file I/O and network programming. Comparing to an existing and mature implementation your own solution will surely inlcude many bugs and issues at the beginning. The second approach I’m now focusing on is to use the existing Unison implementation and build Portal on top of it. Unlike the first approach we now get much sooner to the point where we can experiment with application interaction, i.e. user interface. My next step is to build a prototype of Portal using the Unison tool under the hood.
