SUMMARY: how do I REALLY delete a file?

SUMMARY: how do I REALLY delete a file?

There are definitely degrees of how deleted is a file. The responses ranged
from REALLY deleted (hammers and industrial shredders) to sort-of deleted (rm).

THANKS TO:
Christopher Barnard, Tom Grassia, David Breneman
Rich Teer, Casper Dik, David Foster, Chuck Norem
ric, Ray McCaffity, Jeff Woolsey, Anthony Florendo
Florian Laws, Crist Clark, Derek Smythe, Gary Carr
Peter van Gemert, David Magda, Chuck, Chintu
Bhaskar, Gerard Henry, Hutin Bertrand, Stoyan Genov
Markus Mayer, Aaron Lineberger, Rich Kulawiec


DELETE A FILE: PHYSICAL DESTRUCTION

As many people mentioned, the safest option for really deleting a file is to
remove the hard drive and physically destroy it. While I relish the thought of
beating Sun equipment with hammers, actual physical destruction of hardware is
not always an option. Your employer may have very specific policies that must
be followed. It may not be enough that you personally destroy the disk. You may
be required to turn the disk over to a 3rd-party data destruction service to
satisfy procedural, compliance, and auditing purposes.

If you have a degausser laying around, you could degauss the disks. In addition
to wiping the contents of the platters themselves, degaussing can render the
drives unusable by destroying timing tracks, server motors, and spindle motors.
http://www.spectrumwest.com/Attach2.htm had some interesting details.

OT: Degaussing a hard drive is certainly different than degaussing a
steel-hulled ship, but the ship's degaussing system was pretty interesting:
http://www.fas.org/man/dod-101/sys/ship/weaps/degaussing.htm


DELETE A FILE: LOW-LEVEL FORMATS

This approach assumes you want to delete an entire disk drive or at least an
entire file system. These could not be used to target specific files while
leaving the rest of the file system intact. Definately worth including though:

Sun Blueprint re: scrubbing disks:
http://www.sun.com/blueprints/0600/scrub.pdf

Repeatedly format > analyze > [purge|write|compare]

Overwrite raw device with something such as:
dd if=/dev/zero of=/dev/rdsk/c#t#d#s#

Wipedrive doesn't seem to be available for Solaris, but might be of interest to
folks who also use Linux, OS/2 (TONS of people I'm sure), and Windows variants.
Bcwipe is a similar tool. I should have specified this was a Sparc-based
system. Perhaps these tools can wipe x86 Solaris installs on PC hardware.(?)

Someone mentioned a linux utility named "purge" but I was unable to (quickly)
find a Solaris varient to provide a URL.


DELETE A FILE: FILL-UP FILE SYSTEM

A popular suggestion was to delete all the files/data we wanted destroyed.
Then, fill the file system with junk. One problem with this approach is that it
assumes we already know ALL of the sensitive files that should be deleted and
don't forget one. Unless all of your sensitive data is positively stored in a
known area, you risk leaving a file alive by mistake. Rich Kulawiec's version
of this answer captured the process very well:

1. Remove all files of interest.

This will leave disk blocks on the free list which may contain
some data that you'd rather not see leave your custody.

2. AS ROOT, run this on each filesystem where the files from (1)
used to live:

	yes > junk

and let it run until the filesystem is completely full.

Why "as root"?  Because the last 10% of the actual free space
on each filesystem is reserved for root.  (See "tunefs".)  This
allows root user to manipulate the filesystem and recover from
some space exhaustion problems -- even when the filesystem appears to be "full"
from the viewpoint of end users.

Why run the command shown above?  Because it simply echoes the
string "yes" (or any string supplied as its argument) endlessly.
Redirecting it into a file named junk means that it will continuously append to
that file...until it can't.  Since you've run it as root, the only condition
that will satisfy this is "exhaustion of all free space on the filesystem".

At that point, you have allocated and rewritten every block on
the free list...which means that all the blocks which used to
contain interesting data now contain very boring data.

3. Of course, "yes > junk" is rather simple-minded and relatively
slow - something that did block writes would run a heck of a lot
faster.   So for example, you might want to use something a
notch more sophisticated, along these general lines:

	touch junk
	while (1)
	dd if=/some/big/file/full/of/crud bs=20k >> junk
	end

to do 20K writes.  This should run considerably faster, and of course you could
also do your variant of this with shell, perl, C, python, whatever you wish. 
The general idea though, is that you want to force the system to allocate every
(currently) free block so that you can scribble on it.  This should put
recovery of the data beyond the ability of most people.


DELETE A FILE: BACKUP/RESTORE

The general idea with these suggestions was to delete all the sensitive files,
backup the file system, wipe the file system (using an approach from above),
then restore the 'approved' files. Again, the risk with this approach is that
you could leave a file alive by mistake.


DELETE A FILE: SHRED, WIPE

Casper usually knows his stuff, so I'll place some trust in his response. It
seems that UFS might operate in a way that would allow the use of
file-destruction utilities such as 'shred' or 'wipe'. Again, the risk with this
approach is that you could leave a file alive by mistake.

Me:
>I found GNU's shred, but the docs state "that shred relies on a very important
>assumption: that the filesystem overwrites data in place. This is the
>traditional way to do things, but many modern filesystem designs [such as
>Solaris' UFS] do not satisfy this assumption." I'm on Solaris 9 with UFS
>mirrored by SVM/SDS.

Casper <<EOF
Solaris ufs actually does satisfy that assumption.  As long as you do not
truncate the file first, the data is overwritten in place.

You must, of course, make sure that the data is flushed before you remove it
because the shared page cache might optimize stuff like:

	open("file", ...);

	over write file
	close()
	unlink("file")

to just:
	unlink("file") (discarding dirty pages in the process)

but judicious fsync() use will fix that.

ZFS, however, does change the picture; it does not allow overwriting
files.
EOF
Thanks Casper!

You can find wipe at http://wipe.sourceforge.net/
You can find shred at 

http://www.gnu.org/software/fileutils/doc/manual/html/fileutils.html#shred%20invocation


ULTIMATE DECISION:

Comments from two folks reminded me that I don't even have to make this
decision!

"I'd pass the decision to management: what's more important, how quickly the
new owner gets the machine, or the confidentiality of your data?  Get their
decision in writing (email will do)."

"...it is "management's" data. It's your job to offer your opinion and the
facts.  Let them assess the risks and make the call... as long as if their
opinion differs from yours, that you make sure to get it properly documented
that _they_ made the call so you don't take a fall if it comes back to bite you
guys."


FUTURE CONSIDERATIONS:

Cleanly seperate ALL data storage areas from OS drives. This can easily be done
for 'normal' data, but can be more challenging if some of the files you want to
delete include application files that are spewed all-over the OS file systems
during a software installation.

Only save data in an encrypted format. Again, good for normal data, could be
tougher for application files.

Thanks again for everyone's input!
Merry Christmas and Happy New Year!
-John Christian


ORIGINAL POST BELOW:

Hi gurus,

How does one really delete the contents of a file? It's my understanding that
rm unlinks the inode, but doesn't actually zero-out the data blocks associated
with the file. How can you really destroy the contents of a file? What if
you're doing a rm -r of hundreds of files?

I found GNU's shred, but the docs state "that shred relies on a very important
assumption: that the filesystem overwrites data in place. This is the
traditional way to do things, but many modern filesystem designs [such as
Solaris' UFS] do not satisfy this assumption." I'm on Solaris 9 with UFS
mirrored by SVM/SDS.

BACKGROUND: I need to transfer ownership of a host to another company. The
"safest" way to transfer the host (IMHO) is to beat the hard drives with a
hammer before throwing them into an incinerator and telling them to buy new
ones. BUT, they will be performing similar work and could save time by starting
with the host built to our specs. SO, I could format > analyze > write, then
rebuild the system from scratch to our specs. BUT, mgmt wants this all done
really fast and has asked "Can't we just delete the files and databases we
don't want them to have?" Due to the nature of the data, I want to be sure the
data is REALLY destroyed.

Just because I'm paranoid doesn't mean they're not after my data.

Thanks for any direct solutions or hints on where to find more information!

-John


		
__________________________________________ 
Yahoo! DSL  Something to write home about. 
Just $16.99/mo. or less. 
dsl.yahoo.com 
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers