Storage Speed on the PandaBoard

In my experience so far with the PandaBoard, I noticed that the system feels sluggish using the SD card.  I'm using a Class 6 SD card which means it can sustain 6 MB/sec writes.  The USB 2.0 bus is capable of a max 480 Mbps, or 60 MB/sec; in practice, USB tops out at around 30 MB/sec due to CPU overhead.  Still, that's quite a bit faster than the 6 MB/sec the SD card can do.  With that in mind, I wanted to compare the performance of the SD card against a USB 2.0 hard drive.

The bonnie++ tool is great for testing storage performance, so I installed it with a quick yum install bonnie++ and ran it against both the SD card and the hard drive.  I was using the 2.6.35-g6d019da-dirty kernel.

It's generally a good idea to store the output of bonnie++ on a different drive from the one being tested so as not to interfere with the results, so first create a RAM-disk:

[root@fedora-arm ~]# mkdir /bonnie
[root@fedora-arm ~]# mount -t tmpfs none /bonnie

The SD card holds the root filesystem, so I tested it first; note that you have to run bonnie++ as a non-root user.

[jbastian@fedora-arm ~]$ bonnie++ -d /tmp -n 0 -m "SD Card" -q 1>>/bonnie/bonnie.csv 2>/dev/null

It takes a while for bonnie++ to do its thing, so I fired it up and grabbed a bucket of popcorn and a movie.  When it finished, I ran it again on the USB hard drive: just change /tmp to /mnt/usb.  The output is just appended to the bonnie.csv file.

[jbastian@fedora-arm ~]$ bonnie++ -d /mnt/usb -n 0 -m "USB HD" -q 1>>/bonnie/bonnie.csv 2>/dev/null

The output is CSV, but bonnie++ includes a tool to convert it to HTML to make it more readable:

[jbastian@fedora-arm ~]$ bon_csv2html /bonnie/bonnie.csv > bonnie.html

I used tidy -i to reformat the HTML a bit so I could remove the columns for the file creation tests (which I did not run), and the results look like:

Version 1.96

Sequential Output

Sequential Input

Random Seeks

Size

Per Char

Block

Rewrite

Per Char

Block

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

/sec

% CPU

SD Card

2G

41

93

3125

5

3371

4

336

99

17598

15

27.4

1

Latency

1168ms

33487ms

32391ms

44466us

2628ms

663ms

USB HD

2G

43

99

9017

16

4538

6

369

99

12357

9

93.9

6

Latency

215ms

5849ms

5889ms

52858us

108ms

7643ms

As I expected, the SD card clearly wins with random seek times: 663ms vs 7643ms.  This is one of the major advantages of flash storage over traditional spinning magnetic media since flash does not have to move a head over the disk and then wait for the desired sector to spin under the head.  It's interesting, though, that even with the significantly slower seek times, the hard drive managed to squeeze out 93.9 seeks/sec compared to only 27.4 seeks/sec on the SD card.

The throughput results are more interesting. On sequential output, the hard drive is the winner with 9017 K/sec compared to the SD card's 3125 K/sec.  Writing data is one of the weaknesses of flash storage since it has to re-write an entire block if you need to flip a bit from a 0 to 1. For sequential input, the SD card managed a respectable 17598 K/sec vs the hard drive's 12357 K/sec.

Overall, the SD card performs nicely on large sequential reads, but the USB hard drive seems to outperform it in general usage.  Unfortunately, neither drive came close to the 30 MB/sec I was hoping for.

Just for fun, I also ran zcav, another tool provided by bonnie++ to test the read rates over a hard drive.  On a normal hard drive, the read rates will be faster on the outer edge of the disk since the outer tracks have a longer circumference and thus have more sectors compared to the inner tracks, thus more sectors pass under the head in one revolution of the disk on the outer tracks.

On an SD card, the rates should be constant since there is no spinning disk.

The zcav needs access to the raw device, so I'll run it as root.  Again, store the zcav output in the RAM disk so it doesn't interfere with the testing.  I only tested the first 8 GB of the hard drive since the SD card is only 8 GB in size, and also because testing all 200 GB would take all day!

[root@fedora-arm ~]# zcav -l /bonnie/zcav-sdcard.out /dev/mmcblk0
[root@fedora-arm ~]# zcav -r 8192 -l /bonnie/zcav-usbhd.out -f /dev/sda

The output is just a series of numbers in three columns:

#block offset (GiB), MiB/s, time
0.00 14.59 17.547
0.25 16.10 15.897
0.50 15.09 16.966
...

I think this will be easier to make sense of as a graph, so gnuplot to the rescue:

[jbastian@fedora-arm ~]$ gnuplot
...
gnuplot> set terminal png
Terminal type set to 'png'
Options are 'nocrop font /usr/share/fonts/dejavu/DejaVuSans.ttf 12 size 640,480 '
gnuplot> set output "zcav_graph.png"
gnuplot> set title "ZCAV"
gnuplot> set xlabel "Block Offset (GiB)"
gnuplot> set ylabel "MiB/sec"
gnuplot> set yrange [0:20]
gnuplot> plot "zcav-sdcard.out" using 1:2 title 'SD Card' with lines, \
>             "zcav-usbhd.out" using 1:2 title 'USB HD' with lines
gnuplot> quit

And the graph:

The SD card seems to be reading at about 15 MiB/sec and the USB hard drive at about 10 MiB/sec.  This is a little below the first bonnie++ run showing 17 Mib/sec and 12 MiB/sec respectively, but the SD card is still consistently faster than the USB hard drive at sequential reads.

For comparison, I ran bonnie++ on a desktop system (Core 2 Quad @ 2.66 GHz, 6 GiB RAM, 320 GiB SATA hard drive) running Red Hat Enterprise Linux 6.1 and it had a nice 58 MiB/sec for sequential writes, 72 MiB/sec for sequential reads, and a 1066ms latency for random seeks.  Here's the full set of numbers compared to the PandaBoard:

Version 1.96		Sequential Output						Sequential Input				Random Seeks
	Size	Per Char		Block		Rewrite		Per Char		Block
		K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	/sec	% CPU
SD Card	2G	41	93	3125	5	3371	4	336	99	17598	15	27.4	1
	Latency	1168ms		33487ms		32391ms		44466us		2628ms		663ms
USB HD	2G	43	99	9017	16	4538	6	369	99	12357	9	93.9	6
	Latency	215ms		5849ms		5889ms		52858us		108ms		7643ms
Desktop SATA	11G	578	97	58851	11	25655	4	2636	94	72324	4	176.9	1
	Latency	13998us		3126ms		411ms		25294us		862ms		1066ms

My conclusion: I will continue using the SD card to hold the root filesystem.  I was hoping to get a performance boost out of a hard drive, but I think the USB bus is the bottleneck for the hard drive.  Hopefully the PandaBoard 2.0 (or whatever it will be called) will have a SATA port on it!