The problem
Have you tried copying a VERY large file (ie 2-10 GB), maybe a big SQL backup or a Virtual PC file, or just a movie?
Have you noticed how this totally damages the Windows system file cache? What I mean is this, it looks like the system has swapped everything to disk because of the large file copy.
Already open applications respond as if they are being fetched from disk (noticeable I/O).
Normal performance is temporarely severely affected due to this since everything appears to be reloaded from disk.
The way the Windows file cache works, is fine for normal app use, but files today can be extremely big and the fact that Windows thinks everything must be available for fast reading is not always in the users best interest.
Most users don't have 2 gb available for file cache anyway - and normal users/apps are not even interested in being able to read a given file quickly yet another time.
A possible solution
One way to fix this would be to have a low-level driver that copies file(s) without affecting the Windows file cache.
How to activate:
Several ways could be interesting
- Hot folders. Most interesting way from a server usage stand point.
- By using hot folders the user could for instance monitor file activity for FTP, or general file server usages.
- This way a server would still be able to be of general use even though large files are being served.
- Hot-keys
- Most usefull for general users.
- Hold down CTRL+ALT (or what-ever) and then issue a file copy operation.
- All files that begin to be copied are now surpassing the file cache. This might affect other files, but during normal operations, who cares if a few files skip the file cache.
- Scheduled
- Server usage again. During backup schedules and general maintenance.
- What typically happends is this: The backup starts and the system is generally totally crippled by the heavy I/O. Existing critical apps gets swapped out (such as Exchange server, IIS, etc) and the next time a mail arrives / a user hits a website upto 20-30 seconds of penalty can occur.
- By size
- Files being accessed that are larger than XX mb/gb are ignored by the Windows cache.
Another extra small feature this app could have would be to be able to copy files SLOWLY. Sometimes you want to copy files to/from a server that is being used heavily without severely affecting availability (yes yes, I could write a program that does that quite easily, but it would fit nicely into this app's context).
Anyone?