Duplicity's not a bad choice for making secure backups on Linux. It uses GnuPG to encrypt data and integrates well with typical Unix workflows. Best of all, it has support for many storage backends; the same tool can be used to back up to a USB stick or to Amazon S3.
I also use Tarsnap. Tarsnap has a smarter model than Duplicity for incremental backups that allows for deletion of old data. However, it's also tightly tied to the most reliable form of Amazon's S3 storage, which can make it relatively expensive.
Therefore, I use a hybrid model, where critical, extremely security-sensitive data is stored in Tarsnap, and the bulk of my personal data is backed up to a USB drive and cloud storage via Duplicity.
Even so, as I uploaded more Duplicity files into Amazon S3, I wanted to save more money. Duplicity doesn't have direct support for Amazon's super-cheap, super-slow Glacier service, but it's possible to ship objects in S3 buckets to Glacier without too much difficulty. Now, I spend less than a dollar a month on remote backup.