SD Card Speeds

After yesterday's benchmarks revealed a possible problem with the SD card performance, Mark Salter asked me to run a few more tests to see if the DMA API patch set is hurting the SD card (but helping USB storage).

I recompiled the 2.6.40.3-2.02.fc13.armv7l.omap kernel but I disabled the DMA API patch set.  I also tested with smp (the default) and nosmp.  And just for good measure, I re-tested the 2.6.35-g6d019da-dirty kernel because the last time I tested 2.6.35 was in May and I've updated a few packages since then.

It does not appear that the DMA API patch set is to blame for the slower SD card performance.  The random seek times were longer than 76000ms for all variants of 2.6.40.3 compared to just 664ms on 2.6.35.  The sequential reads were fastest with 2.6.35, but today's test was quite a bit faster than May's test; I'm not sure what to make of that.  Likewise, sequential writes were also fastest with 2.6.35.

Thus, some other change between 2.6.35 and 2.6.40 must be responsible for the drop in SD card performance, but what?

Version 1.96		Sequential Output						Sequential Input				Random Seeks
	Size	Per Char		Block		Rewrite		Per Char		Block
		K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	/sec	% CPU
SD Card 2.6.35 (May 2011)	2G	41	93	3125	5	3371	4	336	99	17598	15	27.4	1
	Latency	1168ms		33487ms		32391ms		44466us		2628ms		663ms
SD Card 2.6.35 (Sept 2011)	2G	39	91	3118	5	3465	4	448	99	19104	16	27.9	1
	Latency	3900ms		42424ms		54958ms		19929us		667ms		664ms
SD Card 2.6.40.3 + DMA patch + SMP	2G	22	65	1862	4	2544	3	339	99	19626	16	15.6	1
	Latency	2793ms		29928ms		39111ms		50446us		665ms		76023ms
SD Card 2.6.40.3 + DMA patch + nosmp	2G	19	58	1669	3	2513	3	343	99	20829	13	17.2	1
	Latency	2858ms		38268ms		42161ms		51148us		667ms		76651ms
SD Card 2.6.40.3 + no patch + SMP	2G	26	76	1669	3	2271	3	411	99	19592	16	15.7	1
	Latency	2855ms		34616ms		58124ms		22279us		666ms		82254ms
SD Card 2.6.40.3 + no patch + nosmp	2G	18	54	1603	3	2395	3	382	99	20894	13	16.5	1
	Latency	5020ms		33958ms		29546ms		51301us		666ms		77587ms

Storage Speed on the PandaBoard Revisited (Again)

Since my last post, I took a short vacation and I've been helping in the Virtual Fedora Activity Days (VFAD) to rebuild packages for Fedora 15 hardfp (hardware floating point).

Research into the slow storage speeds on the PandaBoard (and other Cortex-ARM9 systems) has continued and a patch set developed to hopefully improve the situation.  David Marlin built a 2.6.40.3-2.02.fc13.armv7l.omap kernel with the patch set, so I downloaded a copy and ran bonnie++ again.

The results were mixed for the SD card.  Sequential reads had a nice improvement, but sequential output and random seeks took a big hit.  The random seek times are particularly worrisome with an increase from 663ms to 76023ms!  Something doesn't seem right.

The USB hard drive, though, saw some nice improvements overall. Sequential write latencies dropped from 5.8 seconds down to 0.38 seconds and throughput more than doubled: 9 MiB/s vs 23 MiB/s!  Sequential reads latencies increased a bit, but throughput again more than doubled: 12 MiB/s vs 27 MiB/s!  Random seeks also saw a nice improvement from 7.6s down to 4.9s.

Version 1.96		Sequential Output						Sequential Input				Random Seeks
	Size	Per Char		Block		Rewrite		Per Char		Block
		K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	/sec	% CPU
SD Card 2.6.35-g6d019da-dirty	2G	41	93	3125	5	3371	4	336	99	17598	15	27.4	1
	Latency	1168ms		33487ms		32391ms		44466us		2628ms		663ms
SD Card 2.6.40.3-2.02.fc13.armv7l.omap	2G	22	65	1862	4	2544	3	339	99	19626	16	15.6	1
	Latency	2793ms		29928ms		39111ms		50446us		665ms		76023ms
USB HD 2.6.35-g6d019da-dirty	2G	43	99	9017	16	4538	6	369	99	12357	9	93.9	6
	Latency	215ms		5849ms		5889ms		52858us		108ms		7643ms
USB HD 2.6.40.3-2.02.fc13.armv7l.omap	2G	39	99	23519	26	11118	15	327	99	27314	18	150.6	12
	Latency	322ms		383ms		341ms		54567us		697ms		4910ms

I also tried a simple test with hdparm as shown on lwn.net:

[root@fedora-arm ~]# uname -r
2.6.40.3-2.02.fc13.armv7l.omap

[root@fedora-arm ~]# hdparm -t /dev/mmcblk0p4

/dev/mmcblk0p4:
 Timing buffered disk reads:   50 MB in  3.10 seconds =  16.11 MB/sec

[root@fedora-arm ~]# hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads:   66 MB in  3.05 seconds =  21.64 MB/sec

Compared to the older kerrnel:

[root@fedora-arm ~]# uname -r
2.6.35-g6d019da-dirty

[root@fedora-arm ~]# hdparm -t /dev/mmcblk0p4

/dev/mmcblk0p4:
 Timing buffered disk reads:   50 MB in  3.09 seconds =  16.20 MB/sec

[root@fedora-arm ~]# hdparm -t /dev/sda1

/dev/sda1:
 Timing buffered disk reads:   26 MB in  3.04 seconds =   8.56 MB/sec


The SD card's performance is about the same, but the USB hard drive is much faster with the new patch!