I was talking to someone today who knew about rsync's bandwidth control, but not about scp's. Yep, they both have it.
For rsync, you specify the bandwidth limit it KB/s.
rsync --bwlimit 200 foo server:bar
My common usage is:
rsync -avz -e ssh --bwlimit 200 --progress foo server:bar
For scp, you specify the limit in Kb/s (kilobits vs. kiloBytes).
scp -l 1600 foo server:bar
You can make SSH much faster with the ControlMaster configuration
directive. When enabled, only the first SSH connection to a server
will incur slow connection overhead. Each additional connection will
simply reuse the first connection via a tunnel.
scp gibson:www/ht will result in a multi-second delay Host *That's it. Now your first SSH connection will automatically act as a
ControlPath /tmp/%r@%h:%p
ControlMaster auto
About a year ago I wrote a script that crawled my Twitter network and made me follow lots of new people. Then it waited 24 hours to see if they followed me. If they did, I kept them in my list; otherwise, I chopped them. After a couple of days I had almost 600 followers. I just did this for fun, and I knocked it off after a couple of days for fear of being banned.
I wish I'd recorded stats from the beginning of the experiment, but I only started recording them in August (2009). This is what it looks like so far.
This is a method I use when working with slow ruby scripts that operate on huge datasets.
It caches the return value of a block of extremely slow code in a file so that subsequent runs are fast.
It's indispensable to me when doing edit-debug-edit-debug-edit cycles on giant datasets.
