NotQuiteReadyboost

[GJones]

First of all, a caveat: I do not know what exactly happens when bad sectors start appearing on a swap device. I suspect crashing programs would be in the works, and possibly kernel panics. And if you use a flash device as swap, it will wear out faster, which means it will develop bad sectors sooner. This means that
a) you should absolutely use a dedicated medium for swap, if you try what I outline below, lest you destroy a good USB stick and your data with it; and
b) if something goes otherwise goes wrong, and your data vanishes as a result, you should not blame me.

Anyway...

Because of my netbook's lack of RAM, I was thinking about putting a swap partition on a USB stick. USB stick reads are (supposedly) fast, paging occurs in the background (which I think should mitigate the effects of the slow write speed), and my experience with zram indicates that high-speed swap is almost as good as more RAM.

But I don't have any old/spare sticks at the moment, and it seems like a shameful waste of a USB stick anyway... Which is why I decided to use a 2 GB MicroSD card. Procedure looked something like this:
- Partition card with one huge partition, and run mkswap on it
- Put in an fstab entry for the SD card. Unfortunately I can't make it user unmountable, as mount/unmount doesn't work for swap.
- Provide sudo access to swapon/swapoff for users in the plugdev group.
- Insert card and reboot. The new, hopefully-fast swap space is now enabled!

Obviously this is the lazy budget way to do things. Maybe tomorrow I'll see if I can cobble up a udev rule to mount swap partitions on USB devices automatically. Anyway, for now the ugly sudo hack is necessary so you can turn off the SD card swap before removing it, and not crash your machine! So I figure one might as well go the whole way with ugly hacks.

Results

With SalixOS 13.37 KDE, 32-bit...

- iostat shows reads and writes to the device... Currently more reads than writes, probably because Linux was swapping a little during the last update.
- When swapping starts, performance takes a bit of a dive as expected. Based on pure perception, it's not as much of a dive as when relying only on hard disk swap space, but that isn't saying much.
- Firefox performance when not under load seems a little better (i.e. less freezing when rendering pages). But again, that's not saying much.

All in all the results are less than impressive, even considering that I'm using KDE 4, which skews things a bit in the general direction of sluggishness. hdparm reveals a likely reason why:

Code: Select all

root@grayarea:~# hdparm -t /dev/sdb

/dev/sdb:
 Timing buffered disk reads:  56 MB in  3.04 seconds =  18.40 MB/sec
root@grayarea:~# hdparm -t /dev/sda

/dev/sda:
 Timing buffered disk reads: 188 MB in  3.02 seconds =  62.33 MB/sec

sdb is the SD card, sda is the hard drive. As you can see, hard disk IO is about 200% faster. And I obtained similar hdparm results with my USB sticks.

My thoughts on this are as follows, in approximate order of estimated likelihood:

- Maybe these are poor quality USB media. Pricier sticks and cards might results - which would be too bad, since the whole point of this experiment is to be a cheapskate.

- Maybe this netbook's high-speed USB support is shoddy. Some computers definitely get better IO speed with USB media than others.

- Maybe Linux's paging mechanism isn't optimized for use with flash media. It's possible that what I'm doing is actually not even close to Windows Readyboost in terms of what it does. IIRC Readyboost is more aggressive about caching stuff on the USB medium, maybe it also incorporates some pager optimizations for such media to avoid unnecessary overhead and IO.

- Maybe my (mostly default) setup is not optimized for swapping to and from a flash medium. For instance, increasing vm.page-cluster might improve the performance of a flash device as swap space, by reducing the number of writes (which I'm guessing would be a bottleneck). This doesn't touch on the read speed issue though.

- Maybe the whole concept of Readyboost is actually stupid, and most removable flash media just do not perform adequately for it to have any benefit. Were this the case, though, I would not expect Microsoft to spend any effort adding Readyboost to Windows Vista and 7. Whatever else you can say about MS, they're not entirely stupid.

I'd be interested if anyone else has input on this, or is willing to try it out on a low-spec computer...

[Shador]

When swap data is lost I expect nothing bad to happen at first. If you're lucky you're system is going to continue operation almost as normal. Of course it depends entirely on the data that is lost If it's just cache data, nevermind. If it's application data, that application is probably going to crash when the next page fault for the affected page occurs. Some other data like system/kernel memory could of course crash the system.
That's at least my experience with the FreeBSD based FreeNAS that creates some swap space on each raidz-member disk. On such a setup it happened once or twice that I lost one member disk or intentionally pulled it for testing purposes.

Something you could try to make better use of swap is increasing the swappiness and playing with the swap priorieties.

[GJones]

Thanks Shador...

It turns out Readyboost does something completely different, not even close to swap-on-flash - it uses the flash device as disk read cache, to minimize seek times. Slow read and write speeds are a known issue, but are mitigated by nonexistent seek times. The result is (apparently) a good improvement in responsiveness on slower computers that would normally have trouble with Vista/7, and no benefit on fast computers. As far as I know, Readyboost does not use flash devices for page cache at all.

That said, flash-as-page-cache might be useful in its own right. Bandwidth isn't as good as with the hard disk, but seek times are orders of magnitude smaller, and if you're retrieving little 4 KB pieces of programs as opposed to the whole thing, it's probably going to be faster than the disk. However, that's assuming you are using it as page cache, as opposed to swapping out active programs (since swapping out lots of stuff -> huge IO wait times). And I don't know how much faster it would be.

Guess I'll experiment with this and different swappiness settings. Get the feeling it may be a lost cause though.

[GJones]

Hmm. Just for the heck of it, I tried using the real Readyboost with a Micro SD card on Windows 7... It might make things a tiny bit faster, hard to tell. IOW, similar results to swap-on-flash under Linux.

OTOH, it does give me an extra hour of battery life, bringing Windows' battery life on par with Linux's. I figure this is due to Windows being an incurable swapper... Yay I guess.

[GJones]

Okay, got to ask - does Linux provide any way to measure how much stuff is being paged out of RAM? I know that swap usage in top/htop does not indicate this...

[Shador]

Why shouldn't it? If the kernel decides by some algorithm that a page needs to be replaced there are two options: drop the page, swap it out. The decision depends on swappiness and the kind of data. i.e. application data can never be dropped. But for example cache data or some file mapped into memory (e.g. an executable to execute it) can. Probably other factors have an influcence too.
So the swap usage indicates what you seem to refer as "paging out of RAM".

If you want a bytewise listing the command free can give you that.

[GJones]

Oh. Thanks. IOW what I've read about paging vs. swapping is also wrong? Yay.

Anyway that would mean that swap-on-flash is completely useless by design. Thanks.

[Shador]

IOW what I've read about paging vs. swapping is also wrong?

Paging: memory management strategy, all memory is divided into pages. For example when starting an application virtual addresses are assigned for each "block" of application data (actually that works for any file with the mmap system call), but the data is not necessarily loaded into memory. Now when accessing a virtual address (e.g. because the system wants to start executing the data in the respective page or some application tries to read/write to that page) the system looks in a table for the actual physical memory address where the page is stored in RAM. If it is not yet loaded into memory a page fault occurs. The OS now tries to determine where the data is actually stored e.g. on swap or for the application binary on your root fs and loads the requirexd page into memory. If it is unable to solve the page fault e.g. because an invalid virtual address is accessed, you get a page fault message in your system log.

Swapping: Just the paging that occurs between the dedicated swap space and the RAM. I.e. we're moving data from the RAM to the swap space/HDD or vice versa.

Anyway that would mean that swap-on-flash is completely useless by design. Thanks.

No? Just that doesn't make it more or less useful than swap-on-hdd. Other characteristics of the medium that is used for swap like response times, transfer speed, shared or separate data channels, ... can of course an influence. But as so often when optimizing performance on this level the influence can just be guessed. In the end only testing reveals the actual usefulness,. And suddenly everybody has a theoretical explanation as to why it works.

[GJones]

Okay, thanks for the clarification. I'll have to see what else can be done about seek delays on low-end computers though... Login and application start times on Linux have gotten longer lately, and it's started to get on my nerves a bit.

My suspicion is that a lot of this is due to Linux using thousands of tiny shared libraries scattered all over the disk - unlike Windows, which uses larger binaries. I figured that caching code on some device that doesn't suffer from seek delays, such a USB stick, could mitigate this; even if the device doesn't offer much bandwidth. That's basically how Readyboost is supposed to work. However
- It seems that using a swap partition is not how one would go about doing this
- Readyboost seems not to work that well, at least on my underpowered netbook

Perhaps it would be better to forgo the use of a separate device, and instead have some kind of library cache in a single contiguous file? That would probably cut down on seek times. It would be a complicated way of handling the problem though, and probably beyond my skills to implement.

[Shador]

My suspicion is that a lot of this is due to Linux using thousands of tiny shared libraries scattered all over the disk - unlike Windows, which uses larger binaries.

The shared libraries should actually have a positive impact. While they might be scattered over the disk they only need to be loaded once for all applications that need it. As the static libraries are built into the actual binary you need to load the same (or just slighly different) libraries a dozen times. So with the Microsoft you just get higher memory usage and need to read more data than with the common linux shared library way. The only advantage is higher compatibility and robustness on different (broken) systems.

Login and application start times on Linux have gotten longer lately, and it's started to get on my nerves a bit.

You seem to be talking about cold-start time here. No cache can help you here, because caches just cache data after they have been used at least once. But after a cold start the cache is still cold and first needs to fill up. A cache can only provide benefit after the second, third, ... start of an application.
If you want to improve cold-start times only probabilistic preloading techniques like readahead can help and the only thing they can improve, if at all, is the very first startup at the cost of possibly loading unneeded stuff and thus wasting disk bandwidth.

Actually there are some filesystems, but I just know about high-end ones like ZFS, that allow using devices (commonly SSDs) as cache devices. And a filesystem cache has slightly different goals than the page cache becuase the latter is usually only kept in memory.
Maybe there's some overlay/fuse filesystem which you can put on top of another filesystem to get caching to a dedicated disk device and not only in memory as the page cache does.