Hello Lin,
Usage of memory for buffers and cache is handled by the Linux kernel, and I have so far not seen a situation where it would do a particularly bad job.
The system making a noticeably "slow" impression is more often related to swapping. The interesting number would be the memory usage of your appliance processes (virtual and resident memory) compared to the installed memory.
Keep in mind that on a Virtual Machine, you have another independent layer of memory usage and swapping in your virtualization engine. If your virtualization is overbooked and running low on memory, guest VMs may be swapped and show degraded behavior without the guest OSs even knowing.
Cheers,
~Ralph
