Improve Time Machine performance with Big Bands

It may sound like some bizarre fusion between Duke Ellington and H.G.Wells, but, if you use a Time Machine via a NAS, Airport Extreme, or Airport Time Capsule (rather than a directly connected external drive), then you’ve probably experienced painfully slow time machine performance after several months of use.

Improving Time Machine PerformanceTime Machine backups are stored inside a sparse bundle disk image. This type of disk image file starts small and grows as required. Within the OSX file system, a sparsebundle is actually nothing more than a directory (a bit like OSX Apps are also directories). Inside the sparse bundle there are files called bands. The default band size is just 8Mb.

As more data is added to the sparse bundle, so more bands are created. So, a 800Mb backup will take up around 100 band files. This architecture means that time machine does not have to open an enormous file each time it needs to make changes – vastly increasing network performance.

Time machine performance boost

However, when you have around 800 Gigabytes of data and your sparse bundle contains something closer to 100,000 band files, then things can start to get ugly. It is possible to improve time machine performance by changing the band size to something larger, so not as many are required.

I usually go with a band size of 128Mb. While it’s possible to create band sizes that are much bigger than this, I found that the advantages are minor, and it may introduce other reliability problems along the way. I have been reliably using 128Mb bands for a few years.

Warning – While you can convert an existing sparesebundle file, it will make a long time – possibly several hours or more. If you can, copy it to your local machine first and perform the conversion there before deleting the old original from your network device and copying your new one back again. ALWAYS backup your original sparsebundle file if your data is critical.

For the purpose of this I’m going to assume we are starting afresh with an empty time machine compatible network share.

Improving Time Machine Performance

Open System Preferences > Time Machine. Click Select Disk.

Configure Time Machine

Select the time machine compatible network share and click Use Disk.

Configure Time Machine

This will switch on time machine. Turn it off by toggling the switch. Check the Show Time Machine in menu bar box – this just allows us to manually initiate the first backup.

Start first backup

Click the time machine icon in the menubar and select Back Up Now. Then watch the Time Machine preferences pane carefully. We want to wait until it has created the new sparsebundle file, and just started to back files up before stopping it. You will see Looking for backup disk… and then Preparing backup… and then Backing Up. When you see Backing up, you can click the X cancel icon as shown.

Creating the new Time Machine  sparsebundle

OK, so now we need to open the Terminal application, and to make things a little easier, move to the directory where time machine’s sparsebundle file is. So, open Terminal, and from the Finder, drag in the time machine network share (yours will be named differently).

Drag time machine share into Terminal app

This will place the path to the time machine backups in terminal. e.g. in my case, this is /Volumes/timemachine/

Just move the cursor to the start of the line and insert the cd command, thus:

and hit return. This will change the current working directory. You should now be able to list the directory by typing ls -la

The output should include a sparsebundle file named after your Mac’s Computer Name (from the System Preferences > Sharing pane). Here’s mine:

OK, so now we know we are in the right directory, we can run the following command to create a copy of the sparsebundle but with 128Mb bands.

Just substitute your sparsebundle filename for the one called steves-mac, above. The output from my conversion looked like this:

Once that’s finished (mine took just under a minute), you will need to copy three special time machine files

com.apple.TimeMachine.MachineID.bckup
com.apple.TimeMachine.MachineID.plist
com.apple.TimeMachine.SnapshotHistory.plist

that don’t get copied by hdiutil. Run the following command, again using your own sparsebundle name.

Now you can remove the original sparsebundle file with:

and rename the new big band sparsebundle back to the original filename

If you are interested, you can now type

to see the actual band files inside the sparsebundle. Most of them will be around 134217728 bytes in size (128Mb).

That’s it! You can close the Terminal app, and click on the time machine icon in the menubar and restart the full backup, which should pick up where it left off. Either that or just re-enable time machine in the preferences pane, and let it run automatically.

,

8 Responses to Improve Time Machine performance with Big Bands

  1. carlota September 18, 2014 at 20:28 #

    Hi Steve, I get the following message back form Terminal

    hdiutil: convert: only a single input file can be specified.

    I am sure i might be doing something wrong, but would you be able to break the procedure in 2 stages?
    Thanks

  2. Debi Billing October 12, 2014 at 01:31 #

    Thanks for the great instructions with the screenshot and command examples! Much appreciated!

    Since I wasn’t familiar with Unix, it took me a long time (two days) to figure out the reason I kept receiving the error, “hdiutil: convert: no output file specified”. I had an apostrophe and spaces in my Time Machine sparse bundle, but I had tried enclosing it in quotes and escaping the apostrophe, all to no avail. Then, based on some comments on another site with similar instructions, I renamed my file to eliminate the apostrophe and spaces. When that didn’t help, it finally hit me to wonder if it was case sensitive, and sure enough, it finally worked. Yay!

    Then when I attempted to delete my original sparse bundle, I received the error, “rm: Debis-MacBook-Air.sparsebundle: is a directory”. So, I did some searching and found I needed to use the command “rm -rf Debis-MacBook-Air.sparsebundle” instead. That worked and then I renamed the new file back to my original name with the apostrophe and spaces (instead of dashes).

    Just wanted to add these comments to maybe help others, and also suggest you may want to update your instructions, to avoid someone else having to spend the time struggling like I did. =)

    Thanks again!

    Debi

  3. bop August 25, 2015 at 16:45 #

    Sure that

    sparse-band-size=262144

    ist 128Mb?

    • steve August 26, 2015 at 06:20 #

      Yes, on my machine with 10.9

      • jpap November 30, 2015 at 02:05 #

        The default block size is 512 bytes; the band size is given in terms of the number of blocks.

        So 128 MBytes == 128*1024*1024 bytes == 128*1024*1024/512 blocks = 262144 blocks.

    • Zack November 27, 2015 at 20:53 #

      It is correct. If you look at the documentation for hdiutil it says that “sparse-band-size specifies the number of 512-byte sectors that will be added each time the image grows”.

      So, if we do a little math:

      262144 / 2 / 1024 = 128MB

      Above we divide by 2 to covert the number of sectors into kilobytes (2 * 512 = 1024) . We then divide by 1024 again to convert kilobytes into megabytes.

    • A Mac User December 16, 2015 at 19:24 #

      The sparse-band-size parameter is the number of 512-byte sectors (chunks), not the number of bytes. Since 512 is 1/2 1,024, then 262,144 B / 2 = 128 KiB.

      Many UNIX and UNIX-derivative commands use 512-byte sectors as a sizing parameter because, “in the ol’ days,” that was the size of a hard-disk sector so became a common unit of size for managing data.

  4. tmuser September 29, 2015 at 23:13 #

    Thank you for your step by step tutorial. One issue is that if you originally selected the have encrypted backups, the converted bundle will not be encrypted, at least this is what happened to me. You can verify it with “hdiutil isencrypted “. I tried redoing the conversion using -encryption and -stdinpass and created an encrypted bundle and eventually TM accepted it after it prompted for the password.

Leave a Reply

Time limit is exhausted. Please reload CAPTCHA.