Summary: Centralized log file server

From: Malloy, Jim <JMalloy_at_ibasis.net>
Date: Tue Jan 15 2002 - 11:24:46 EST
Many, many responses.  My thanks to all.

All responses are included below (in no particular order).  Short version:

        Use syslog
 
        Use rsync or rdist 

        NFS mount or use scp or rsh - makes scripting easier and provides
additional
        security (in the case of rsh and scp).

        Plan space, archive time requirements first

--Jim

+----+----+----+----+----+----+----+----+----+----+----+----+----+----
Karl Vogel [vogelke@dnaco.net]
   I'd use scp/ssh instead.

+----+----+----+----+----+----+----+----+----+----+----+----+----+----
R. Lonstein [rlonstein@pobox.com]
        You could accomplish this with cron, but I've been down that road
both
        as the SA and as the developer and have the following comments:
            + Rotate your syslog files based upon age and size. Plan for
extra
              space. Archive at least nightly, preferrably at rotation. I
              particularly like the BSD implementation of newsyslog,
Solaris's
              is barely adequate. See the following links...
 
http://www.cert.org/security-improvement/implementations/i041.09.html
                http://www.weird.com/~woods/projects/newsyslog.html
            + Pull, don't push.
            + Use ssh with a public key/private key for each host configured
to
              perform a single command (like run rsync, below).
            + Use rsync for application logs.
            + Require that errors, warnings, etc. be emitted to syslog. Use
a
              script to tail application logs and relay them under 'local'
              of syslog.

        The most important advice I can give is:
            + Plan & prioritize first (then review the plan). Don't collect
gigs
              of logs if no one will read them. If someone is expected to
read
              them keep them as small as is useful. Better still, automate
              combing the logs and then do random spot checks. Determine how
              long you hold logs, do backups, provide access. You will end
up
              dealing with these eventually and it's easy up-front.

+----+----+----+----+----+----+----+----+----+----+----+----+----+----
David Foster [foster@dim.ucsd.edu]
        Any logs coming from syslog can be configured on remote systems
        to be saved on those systems *and* the log server.

        For other logs, I'd suggest using 'rsync' to transfer them, it's
        a great utility for keeping directory trees sync'd up. Also, better
        to use SSH instead of RSH as the transfer shell, for security
        reasons.

                RSYNC: File distribution:
                        ftp://samba.anu.edu.au/pub/rsync
                        http://samba.anu.edu.au/rsync/
                        ftp://minnie.cs.adfa.oz.au/rsync/
                        Article:
http://linux.oreillynet.com/pub/a/linux/2001/11/01/rsyn
        c.html

                OpenSSH : Open Source Secure Shell
                        http://www.openSSH.com/
                        CERT Doc :
 
http://www.cert.org/security-improvement/implementations
        /i062_01.html
                        RSAref :
ftp://ftp.zedz.net/pub/crypto/libraries/rsa/ (not neces
        sary)

                        Windows Client:  http://www.openssh.com/windows.html
                        Windows SCP:  http://winscp.vse.cz/eng/
                        Mac Client:  http://www.macssh.com
                        Mac Client:
http://www.lysator.liu.se/~jonasw/freeware/niftyssh
        /
                        Scanssh: (Scan subnet for SSH server protocols)
 
http://www.monkey.org/~provos/scanssh.tar.gz
                        Replacing telnet/rlogin/rsh with SSH:
 
http://securityportal.com/direct.cgi?/research/ssh-part1
        .html
                        VanDyke Tech. VShell: SSH2 server on Windows
 
http://www.vandyke.com/products/vshell/index.html

        Dave Foster

+----+----+----+----+----+----+----+----+----+----+----+----+----+----
Andrew_Rotramel@cch-lis.com

        I keep my log files on a central server. I have the log staging
area(s)
        available via NFS and use a cron script to copy the logs over to it
each night
        or each weekend. Right after copying the logs over, I cat /dev/null
>
        logfile.name to zero out the log file. About once each year I write
the central
        log archive to tape for long term storage.

        Andrew

+----+----+----+----+----+----+----+----+----+----+----+----+----+----
Paul LaMadeleine

        If all you want to do is replicate the logs on a center server (I'd
assume
        in hostname specific directories) that I'd use rsync.  You can check
it out
        at rsync.samba.org.

                 Paul

+----+----+----+----+----+----+----+----+----+----+----+----+----+----
Mark Wishneusky [wishneum@stiusa.com]

        Those sound good...  Also, what about mounting the drives
temporarily (in a
        script at like 3:00 a.m.), copying everything, and then umounting
        again.  Another could be to use rcp.

+----+----+----+----+----+----+----+----+----+----+----+----+----+----
Omar Onsi [oonsi@hotmail.com]

        try the "rdist" command. read about it, i think it is what you are
looking
        for.
        it is used to synchronize files on all nodes from one server.

+----+----+----+----+----+----+----+----+----+----+----+----+----+----
Jeffrey C. Keyser [signature@adelphia.net]

        Rather than ftp, rsync or scp the logs. They'll be encrypted during
        transmission and if you configure it properly, you won't be prompted
for
        a password. Therefore, you won't need to write an expect script.

+----+----+----+----+----+----+----+----+----+----+----+----+----+----
Kevin P. Inscoe [kevin@inscoe.org]

        Rather then ftp the logs if those applications could write to the
        syslogd it would sure make life a *lot* easier. You can use facility
        switches in /etc/syslogd.conf to direct multiple outputs to multiple
log
        files but going through syslogd is how I would try to do it. It is a
        basic syscall in C and available in Perl as well. You could even
write a
        wrapper and put it in a shell script. You  are aware of course you
can
        push syslog entries to a central server which is also a good
security
        practice. Make it so you cannot login to the log server(s) except
local
        console and remove ftpd you would have the makings of a fortress
right
        there. This way if your systems are compromised you will always have
the
        logs and they can't be modified. :-)

+----+----+----+----+----+----+----+----+----+----+----+----+----+----
Christian J Hedemark [chris@yonderway.com]
        Set up MySQL (RDBMS) with a next generation syslog facility, and use
syslog
        as the default logging facility for as many applications as you can.
That
        way your developers can have a robust SQL interface to all of the
logs.
        Forget the few hours lag.  We're talking fractions of a second.

        For any other applications that can't/won't do syslog you can set up
a perl
        script with DBI to move the contents of the logs into the same SQL
database.

+----+----+----+----+----+----+----+----+----+----+----+----+----+----
Jeff Horwitz [jhorwitz75@yahoo.com]
        maybe you've already considered this and didn't mention it, but have
you
        considered sharing the log directory via NFS?
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers
Received on Tue Jan 22 17:58:05 2002

This archive was generated by hypermail 2.1.8 : Thu Mar 03 2016 - 06:42:32 EST