In my recent post entitled, Faetus v0.5 released!, I described how and why I built a Python-based FTP server that reads and writes to Amazon S3. This project was born out of necessity because I am using a 3rd-party file management application (which didn’t understand the S3 API) but needed the files to end up on S3. The system did talk FTP, so I wrote an FTP server that does just that.

As it turns out, the file manager DOES now support the S3 API. Therefore my immediate need for an intermediary FTP interface is gone. While I think it is still a very cool idea, I no longer have an urgent need for it. I will continue to fix some of the major bugs with it, but most of my tinkering will cease. Please let me know if you are interested in using Faetus and I’m happy to help you out if I can.

Faetus is an FTP server that translates FTP commands into Amazon S3 API calls providing an FTP interface on top of Amazon S3 storage.

This project came out of a specific need I have at PBS. I need to put files on Amazon S3 storage, but the 3rd-party file management system doesn’t know how to speak to S3′s APIs. What the system does understand is FTP. So I decided to write an FTP server that utilizes S3 storage as opposed to local filesystems.

The server is based on pyftpdlib, a Python library that implements most, if not the full RFC spec for. Especially nice is the fact that the the library implements the filesystem layer in an abstract fashion. I basically needed to replace all the filesystem calls with S3 API calls. I used boto to handle all the S3 API calls and I quickly had a working prototype.

After playing with my prototype for a few hours, I stumbled upon Chmouel’s ftp-cloudfs… who basically did the exact thing I had just done, but for Rackspace CloudFiles. For consistency’s sake, I rolled my code back, forked ftp-cloudfs, gave the project a questionable new name, and added my S3 code back in. La voila!

• Faetus project page
• Source code on GitHub