Domino 6 Archiving questions--HELP NEEDED!

Our company is looking to begin archiving to a mail server, and have so many unsanswered questions! ANYBODY who could provide any help to these questions, and additional information, would be so appreciated (we’re starting from knowing nothing!):

Archive Size - What should the maximum size of each archive be, should there be a size requirement, and how does the size requirement effect the retention.

Mail Retention - How long are we legally required to keep. Can you specify, for example, that you only want 2 years worth of data per employee, and the rest be constantly deleted?

Cleanup - How does Lotus Domino perform or handle the cleanup of archives and mail files?

Bandwidth - If we are archiving accross the network what type of load will it apply. what are the requirements to archive automatically from administrator on off hours.

Domino/Notes Requirements - When archiving to a network server are there any requirements that Lotus has put in place?

Backup - Do we need to have backup (Probably a dumb question), if so how. The size of the archive and disk could dictate the backup drive type and size.

THANK YOU, in advance

Subject: Domino 6 Archiving questions–HELP NEEDED!

[Archive Size - What should the maximum size of each archive be, should
there be a size requirement, and how does the size requirement effect the
retention.
]

Size limits are up to your discretion as administrators. How much diskspace do
you have? A project I’m working on now is allocating 1.4 TB for mail archives,
and with the price of drive space now getting below $1/GB, it’s not really much
of an investment.

[Mail Retention - How long are we legally required to keep. Can you
specify, for example, that you only want 2 years worth of data per employee,
and the rest be constantly deleted?
]

Legal requirements for mail retention are dependent on what industry you’re
in. For example, financial trading companies have much more stringent legal
requirements from the SEC than, say, a construction firm. But yes, you can
specify that you only want to retain 2 years worth of email – though I’d
generally suggest a bit more elaborate a scheme.

[Cleanup - How does Lotus Domino perform or handle the cleanup of archives
and mail files?
]

The Compact task processes each mail file, referencing the individual Archive
profile for that NSF or the mail owner’s Policy settings in his/her Person
doc. It’s the policy settings that allow for tremendous flexibility. Check
the Administration help for details.

[Bandwidth - If we are archiving accross the network what type of load
will it apply. what are the requirements to archive automatically from
administrator on off hours.
]

The traffic isn’t particularly heavy. It’s basically a very busy replication
session. If you have a server backbone, it won’t be a problem. If you have a
switched network, it won’t be a problem.

But it’s normal to run Compact at off-hours anyway.

[Domino/Notes Requirements - When archiving to a network server are there
any requirements that Lotus has put in place?
]

What’s a “network server?” Are you saying you’re archiving to a File Services
server on your network, such as NT or Netware? Is this server-based or
client-based?

[Backup - Do we need to have backup (Probably a dumb question), if so how.
The size of the archive and disk could dictate the backup drive type and
size.
]

Sorry to say it, but dumb question to ask anyone else. :slight_smile: I mean, do you want
to write your archives to tape? The backup policy doesn’t need to be the same,
but it’s really up to what level of service you want to offer your users.

These days, it’s almost cheaper to build yourself a disk-mirrored archive
cluster, and forget the tapes. Redundant servers with tons of drive space will
cost you, what, another $3000-5000? You’ll spend more than that administering
a tape backup in the first six months.

Good luck. Remember to check the admin help on archive policies. They are
AMAZING in R6.

Subject: RE: Domino 6 Archiving questions–HELP NEEDED!

Your fast response was amazing! Thank you so much for your insight.

A few more questions/comments:

-Our Tech team was all amazed that you are allocating 1.4 TB for mail archives! That is huge. What we were wondering was how many users you had! How much is allocated per user?

-I checked out the admin help on archive policies, and they’re great, as you said.

-We would really like to know more about Compact. What exactly does it do and how does it work? For example, when you run Compact, does it only compress the file, or does it strip out headers, etc.?

-You asked what a network server was, and you were right in saying that we are archiving to a File Services server on our network (NT). From what we read in the admin help, it shouldn’t matter if it’s server-based or client-based. What are the pros and cons? With server-based, we can specify all archive settings at the server, but is this just for R6, or previous versions (5.0.11) as well? Not sure if you would know this since you are working with R6.

-Does Domino 5 support centrally control of the mail file archiving using policies, or do we absolutely have to upgrade to 6.0?

-Mail file clean up reduces the size of the source mail file by deleting archived documents or reducing them in size. What happens to the attachements?

We really do appreciate the help. Thanks again for all of your insight!

Subject: RE: Domino 6 Archiving questions–HELP NEEDED!

[Your fast response was amazing! Thank you so much for your insight.]

Archiving policies are currently my Hot New Thing ™ so it’s easy for me to
want to stay on top of them.

[-Our Tech team was all amazed that you are allocating 1.4 TB for mail
archives! That is huge. What we were wondering was how many users you had! How
much is allocated per user?
]

Depends on how you look at it, really. As many as 18,000, if we end up
archiving the whole domain there. Could be as few as 5,000 (which is still a
lot).

The thing is, why not stack a whole lot of disk? Even high-end SCSI drives,
which you can pop into Sun boxes as readily as NT boxes, only run about $3/GB.
So we’re talking less then $5,000 for all that diskspace. Is that really so
much when administering an enterprise messaging system? Even if you want to
mirror it, you’re talking $9,000 at the high-end. It’s still a line-item in
terms of a multi-million dollar installation, where you might be spending
$300,000 just on client licenses, and $800,000 on consulting services.

And think of the benefits. You never have to tell users that they have a
disk quota. You just archive off for the next three years, at which point
drive space will be 50% of today’s cost. As long as you’re not trying to tape
archive it, I’ve never really understood the resistance to throwing huge
amounts of drive space on a server. When you think of the data as
“production,” then it’s more expensive to administer. But if you think of it
as similar to paper archives, and therefore not mission critical, then you can
ease up a lot on the priority of it.

[-I checked out the admin help on archive policies, and they’re great, as
you said.
]

I’m really impressed by this aspect of R6. Kudos to the engineers on it!

[-We would really like to know more about Compact. What exactly does it do
and how does it work? For example, when you run Compact, does it only compress
the file, or does it strip out headers, etc.?
]

These days, Compact is a bit of a multi-purpose tool. When it was first
created, it was a process to recover allocated diskspace for an NSF. So if you
had an NSF with 100,000 documents, and you deleted 90,000 of them, then running
Compact would get back the drive allocation on the NSF, reducing the file size.

Later, Compact also became the tool you used to upgrade the On-Disk Structure
of an NSF. So when new versions came out, you’d run Compact, it would
restructure the files on disk, and you’d get the efficiency gains on the new
version.

Now, Compact also include the -A switch, which tells the server to process
archiving rules on the database at the same time. This means that if your
archiving rules say “copy documents older than 90 days to this other database,
then delete them in the source,” Compact will perform that at an API level,
then recover the space in the source NSF. Check out the Admin Help for
“Compact” for more details.

[-You asked what a network server was, and you were right in saying that
we are archiving to a File Services server on our network (NT). From what we
read in the admin help, it shouldn’t matter if it’s server-based or
client-based. What are the pros and cons?
]

Well, if you’re having clients write to a network file server, then you have
significant traffic, plus the security of the resulting archive is dependent
on your network-level security. That is, if the CEO archives his mail to the
NT file server, and I come along and access his home directory on that server,
unless you’ve taken careful steps, I’ll be able to access his archive from my
Notes client with full access rights.

[With server-based, we can specify all archive settings at the server, but
is this just for R6, or previous versions (5.0.11) as well? Not sure if you
would know this since you are working with R6.
]

As long as the source server and the archive server are both R6 (they can be
the same server if you want), then you’re fine. The database itself doesn’t
have to be an R6 template or anything.

[We really do appreciate the help. Thanks again for all of your
insight!
]

No problem. As I said, this is currently an area of strong interest for me.

Subject: RE: Domino 6 Archiving questions–HELP NEEDED!

That was some good information that you passed on, and we’re coming to our final sets of questions for you.

-In response to the final question and answer that you gave:

With server-based, we can specify all archive settings at the server, but is this just for R6, or previous versions (5.0.11) as well?

As long as the source server and the archive server are both R6 (they can be

the same server if you want), then you’re fine. The database itself doesn’t

have to be an R6 template or anything. We really do appreciate the help. Thanks again for all of your insight! No problem. As I said, this is currently an area of strong interest for me.

-My question to this is, do both the archive server and the primary mail server have to run Domino, or could the archive server act as just a File Server without domino installed?

Thanks!

p.s. also to comment on one of your comments, our company completely regards email as mission critical!

Subject: RE: Domino 6 Archiving questions–HELP NEEDED!

[-My question to this is, do both the archive server and the primary mail
server have to run Domino, or could the archive server act as just a File
Server without domino installed?
]

Important point. In Domino-land, “Server” means a Domino server. It means
nothing else, ever. Domino has no awareness of file servers, except by path
designations or drive mappings. You can map a file server volume on your
Domino server, then specify that as a local archive destination, but I
wouldn’t recommend it. It’s going to create a security vulnerability and a lot
of extra network overhead.

[p.s. also to comment on one of your comments, our company completely
regards email as mission critical!
]

You can regard email as mission critical without regarding two year-old
archives as mission critical. It’s just a question of your policy statement.