You could try the GrSecurity
kernel patchset, which does things to mitigate the impact of such kernel holes. It also comes with a MAC system, but doesn't have to be used with that. Note that if you go with GrSec, you might want to disable mprotect() restriction under PaX options, because that will make Firefox and Opera not work at all.
Otherwise I don't think much can be done to prevent kernel exploits. You could enable stack smashing protection at compile time maybe, not sure how much that's worth? Also make sure vm.mmap_min_addr is set to a reasonable value (65536 should do it on x86), and that address space layout randomization is not disabled (or that kernel.randomize_va_space is
set to 2). ASLR can protect you from certain userspace exploits, and is NOT fully enabled by default on Salix for some reason.
That said, if you really think there's a danger of getting compromised, you might be better served by a distro that GPG-signs its packages and issues kernel regular kernel updates.
P.S. Use Noscript. Or at least enable click-to-play for all plugins, and disable Java plugin if possible. At the moment the main use of Java applets seems to be in writing cross-platform malware installers. Worse, most Java exploits are IIRC due to design flaws, not memory management issues, so GrSecurity and such will NOT protect from them. (Not without the MAC part anyway.)