Personal tools
You are here: Home Linux and free software How to create big-ass empty files in milliseconds

How to create big-ass empty files in milliseconds

Posted by Rudd-O at Jan 23, 2009 11:10 PM |

A small trick that will help you when you need a file that is multi-gigabytes in size, but you don't have hours to wait for it to be written to disk.

dd if=/dev/zero of=bigassfile bs=1M count=1 seek=9999

See what I did there?

This command, issued in a console, literally means: Duplicate data from /dev/zero (a zero generator) into bigassfile.  Do it in blocks of 1 mebibyte each, write only one block, but skip forward to 9999 blocks in the bigassfile.

So what does Linux do when it sees this?  Pretty simple: it creates a bigassfile that is ten thousand megabytes in size, but only bothers to write the very last 1 mebibyte, leaving the first 9999 MiBs empty.  Since only one mebibyte is ever written to disk, it's nearly instantaneous.  When you list the file, it will show as having ten thousand mebybites, but if you use an on-disk size analyzer, the file will only occupy one mebibyte.  As you write data to the file, Linux will use free disk space to supply the big-ass file with room to save the incoming data.

So, remember the trick: give a block size, ask the command to write only one block, and skip N - 1 blocks, where N is the size in blocks of the file you want.

There you go.

Note that this trick does not work in Windows FAT16 or FAT32 filesystems.

Document Actions

typo and optimization

Avatar Posted by Seth Heeren at Dec 24, 2009 01:33 PM
There seems to be a rather crucial error in the code example. I assume you meant seek=9999 instead of skip=9999. Otherwise you'll just waste CPU time skipping over 9999MB of zeroes from the input file <grin?/>

Courtesy of Mike Hommey on the zfs-fuse list this could be simplified into

     dd of=bigassfile bs=1M count=0 seek=10000

You see, if you're gonna copy 0 blocks, you don't even have to say it has to be filled with zeroes. Awesome.

Yer rite

Avatar Posted by Rudd-O at Dec 24, 2009 01:33 PM
Fixing...

Add comment

You can add a comment by filling out the form below. Plain text formatting.

Info
You are not logged in. You may optionally enter your username and password below. If you don't enter anything, this comment will be posted as 'Anonymous User'.
(Required)
(Required)
(Required)
Enter the word