SUMMARY: Slow access to directories with high number of files

Hi everyone,

Sorry for the delay, I could not post the summary before

When I pointed the slowness of Solaris managing directories with more than
500 000 files, I received the following answers :

- Move the files into /tmp/swap fs (into memory) -> Since I had more than 2
GB of files, this was not an issue, but may be this idea can help someone
else with more memory and/or less files.

- Use ReiserFS, which is really really fast on Linux but since it is tightly
integreted into the Linux Kernel, I do not think we will see a Solaris port
soon.

- Use a virtual FS -> Files
If the files are really small - moving it into tmp/swap fs - i.e. into
memory may help. Assuming you have enough memory of course.

-Basically there is not much you can do.  This is simply one of the
limitations of Unix.   Here is a couple of tunables that may help *a
little*:

- improve performance by increasing the DNLC to buffer more
info in cache.

- Try mounting the directory as tmpfs.

- Read http://www.princeton.edu/~unix/Solaris/troubleshoot/diskio.html

- Use Veritas VxFS

I did try some of these solutions but none improved performance on our
developpement machine (which was not slow because not under the same load).
Someone rebooted the production server for another reason and it began to
run faster. We were running the same version of the software for about a
year without a reboot on this machine. It is running fast since the last
reboot.

Regards,

Vincent Brousseau


> -----Message d'origine-----
> De : sunmanagers-admin@sunmanagers.org
> [mailto:sunmanagers-admin@sunmanagers.org]De la part de Vincent
> Envoye : 27 novembre, 2001 22:21
> A : sunmanagers@sunmanagers.org
> Objet : Slow access to directories with high number of files
>
>
> Hi everyone,
>
> I run a very old C application on a Netra with Solaris 8. This application
> creates a huge amount of files in the same directory each day. I can count
> more than 500 000 files that I can not suppress in the busiest
> directory of
> the application. Each access to this dir is VERY slow and slows all the IO
> subsystem. I know it's way too many files, but is there a way I
> can optimize
> the way Solaris access a directory with a lot of small files? I
> do not have
> access to the code of the application so I can not recompile, that would
> have been too simple.
>
> Thank you, I will summarize.
>
> Vincent
> _______________________________________________
> sunmanagers mailing list
> sunmanagers@sunmanagers.org
> http://www.sunmanagers.org/mailman/listinfo/sunmanagers
>