Major security hole with reader fields?

HCL Domino Domino Forum
1

Major security hole on a local replica with reader fields?

For over 8 years of development with Lotus Notes, I have always been under the unshakable belief that Lotus Notes security was VERY strong, as long as you know how to use READER fields on your documents.

If your name (or role, or group) is NOT in a reader field on a document, the document simply doesn’t exist for you.

Or does it?

Create a document with a reader field.

Put a role in the reader field.

Create a view that shows the document.

Create a new form with a computed for display value:

@DbColumn (“”:“NoCache” ; “” ; ; 1);

Create a local replica of the database and make the option “Enforce a consistent access list across all replicas” is selected

Make sure you are a member of that role

Replicate the database while you have access to document

You now have a replica of the database containing a document protected with a reader field.

If you remove yourself from the role in the ACL, you should never be able to see that document anymore:

If you access the view, it shows no document

If you create a new document using the form, the computed for display formula is activated and the @dbcolumn WILL COMPILE ALL THE VALUES of the documents you cannot access!!!

This behaviour is incorrect only on local. On the server, the @dbcolumn function only compiles the data from the documents you are allowed to read.

I was even more astonished when I preformed a search on the R4/R5 forums, and found out that this behaviour was first observed in R5, over 2 years ago, and was never corrected!!!

I am aware that no security model is really secure locally, but this is a MAJOR security hole, because it also has a serious impact on application development: you now have a different behaviour in a local VS a server replica, and you cannot use the reader field as a natural “document sorter” anymore. (If you do, you must then assume that a mobile user could replicate the database and eventually see lookup results he is not supposed to see).

Will IRIS patch this security hole once and for all ?

Nicolas Abesdris / PCLP

2

Subject: Few comments and questions?

Nicolas, I agree, this bites!

It appears that Lotus is depending on security being complete and in place before local replicas are made, so that the replicator would filter the docs or, Is it possible that the DBColumn data was in a cache, not sure there is one locally.

Please post any updates, I also assumed that a dbcolumn would honor Reader Name fields, when cocsistent ACL is enabled. Is it possible you have more than one Readers field?

Do you get the same results from DBLookups, this would truely expose the document, and not just the index like DBColumn appears to be?

Also, this question stems from long past versions of Notes, But is the view column you are using Categorized?

Thanks,

Don Strawsburg

3

Subject: RE: Few comments and questions?

Hi Don,

Is it possible that the DBColumn data was in a cache, not sure there is one locally.

Well, as far as the @dblookup and @dbcolumn are concerned, I always use the “NoCache” option unless dealing with massive databases. I don’t know if this setting could be innefective on local, but I don’t think it changes the problem, because I made extensive testing and noticed the same behaviour, even in the case of new documents or new views.

Is it possible you have more than one Readers field?

No, only 1 reader field - with computed multiple values. Contains only roles.

Do you get the same results from DBLookups, this would truely expose the document, and not just the index like DBColumn appears to be?

Unfortunately, I get the same problem with @dblookup. If I know the name of a specific field, for instance, I could go get this data with a simple dblookup even though it is not in the view.

I am mostly concerned on a development basis: it means I could prepare a dialog list field for a user to display only 4 choices, and find out later than the user actually had 100 choices displayed!

If other fields contains @dblookup based on the content of this dialog list, the form could extract and calculate all sort of values from a document that technically shouldn’t exist for this user.

is the view column you are using Categorized?

I tested the security hole with both categorized and not categorized views. Notes reacts woth both exactly like it would normally, returning a multiple value if you lookup a category, etc.

The only difference is that, for the lookup, all the documents in the views on a local replica are visible, regardless of reader fields, period.

I’ll continue to monitor this tread to see if anything new comes out.

Thanks to all

Nicolas Abesdris / PCLP

Quintessence e-solutions Inc.

4

Subject: Major security hole with reader fields?

hasn’t this been solved yet?

I am having the same problem.

Rebuilding view indices doesn’t solve the problem.

Thank you in advance

5

Subject: “Enforce a consistent access list across all replicas” is NOT a true security feature

I think the key here is to distinguish between features that are truly intended to be rock solid security and those that look like security but are really just there either for convenience or to discourage the casual user. Real security includes things like encryption, the ACL, and yes even author/reader fields. ‘Convenience’ features include things like hiding/restricting a view, restricting a form and “Enforce a consistent access list across all replicas” among others. This is because if I have physical access to the data all bets are off unless you have used encryption. Reader fields do work as advertised in that if you didn’t have access to the document you would NOT have been able to replicate it locally and at a later date if access was removed for you from the document you would not replicate changes. The reason that this situation arises is that you are depending on the “Enforce” setting to provide security. I do agree that the lookups shouldn’t return the data and that is a problem but that is less a problem of security than it is an app dev problem.

The key is don’t ever depend on “enforce” as true security because if I have physical access to the data and it isn’t encrypted I can virtually guarantee that I can get at the data.

Best Regards,

Rob

6

Subject: But it means I cannot relay on a consistant behaviour!

Hello Rob,

Yes, I know that you can never relay on a “security” feature on local, because a malicious user can always access the data somehow.

But I was under the impression (obviously false) that a reader field was not simply hiding a document; that it was actually encrypting it. (now I feel silly… I guess I was overconfident!).

Yes, I can encrypt locally - but only for the entier database. I thought the data was encrypted, then decrypted on-demand as you open a document with a specific ID. My conception of the Notes security model suddenly took a serious drop in the confidence level.

Also, as a developer point of view, what REALLY matters to me is NOT that the user “sees” data he is not supposed to see; it is the IMPACT on development. It means I cannot use the reader field to selectivly hide information and expect this behaviour to carry to the local replica. This is very frustrating!

The document is hidden from the view - how hard is it to make sure it stays hidden from a lookup that reads this view??

7

Subject: Well, that’s always true.

If you have a database unencrypted on local, “a malicious user can always access the data somehow.” It’s just a fact, Nicolas. Encryption alone doesn’t really assure the data, either. Encryption can be broken. So when you talk about access controls, you’re talking about something qualitatively different from local access and encryption, both.

But there’s another point to what you’re asserting: someone who actually had authorization has to replicate a read-restricted doc down to local. You’re saying someone with the authority put the data on local. Well, isn’t it obvious? “Don’t do that”?

Documents won’t replicate down from the server if the user doesn’t have a reader level of access to the doc.

How secure do you want the data to be? It’s being placed locally on a database. There are other more complicated ways of extracting more data about the unreadable documents in an app. I’m pretty sure the issues involved are documented by Lotus somewhere, though I’m not sure where.

8

Subject: RE: Well, that’s always true.

Encryption alone doesn’t really assure the data, either. Encryption can be broken.

Well, of course! If you bring it down to this, everything canultimately be hacked.

The point is to make sure the structure is consitant and logical enough that you know you can program an application and expect it to behave the same way, both locally and from the server.

If the data is SO important that NO risk can be taken at all, then yes, by all means, let’s use the new checkbox “Prevent user from copying or replicating” and voila, the database is “safe” on the server.

That is, as long as your physical machine is safe… !

My point was that we have a powerful security mechanism that allows a designer to create roles and assign specific portions of design elements - but also specific documents - to be visible or hidden.

The whole point of this is voided if you can only relay on it on the server.

But there’s another point to what you’re asserting: someone who actually had authorization has to replicate a read-restricted doc down to local. You’re saying someone with the authority put the data on local. Well, isn’t it obvious? “Don’t do that”?

I wish it was that simple!
What if a user has a new position and has his access changed ? I am to go re-install each application localy replicating on his portable? The whole point of the roles was to be able to change a user’s access with a single click on the ACL, no? And what if the laptop is shared amoung 2 people who use the same database with two different ID? Yes, Domino 6 now has roaming users (I haven’t tested this feature yet) but at least until version 6, users had to switch ID and are accessing the same local replica! (anyway, to my knowledge, the Notes client always had serious trouble replicating 2 local replicas of the same database on the same disk).

If the documents are hidden from the views, they should be hidden from the lookups that access the views, it seems quite logical, isn’t it?

Documents won’t replicate down from the server if the user doesn’t have a reader level of access to the doc.

Obviously!

How secure do you want the data to be?

I don’t need it MORE secure, I need it secured the way it is supposed to be. If a reader field is supposed to make a document unreadable by people that aren’t inside the field, then it should behave that way consistantly everywhere. Either it does, or it doesn’t. If it doesn’t, then fine! we will adapt the design to this, or we will take it into account when thinking a database security model. If it does - but partially only - then it is a security hole.

Hopefully, IRIS will patch the problem in the next incremental!

9

Subject: RE: But it means I cannot relay on a consistant behaviour!

“I thought the data was encrypted, then decrypted on-demand as you open a document with a specific ID. My conception of the Notes security model suddenly took a serious drop in the confidence level.”

If you have a field that is encrypted, eg the Body of a mail memo has a “red coloured” field box, then what you said above is true.

In my opinion, what you mentioned does have an impact on appl. development. It means that if you want to secure your data, make sure you encrypt that field containing the information instead of just hiding it using Readers field.

My 2 cents.

10

Subject: Nothing to do with Enforce consistent ACLs

There’s a long list of ways to solve this…

  1. Set any update on the documents using that Readers field with that role
    prior to altering the members of the role. This will cause the documents to be
    removed from the local replica of the user, making them no longer available in
    the view lookup.

  2. Clear the local user’s replication history programmatically upon noting a
    change in the role. This is possible (even easy) under R6. This will force a
    complete replication of all documents, and remove those documents the user no
    longer has access to.

  3. Use an approach other than @DbColumn to build the data set you’re concerned
    with. Combing a document collection via script will behave as you expect with
    regard to Readers controls.

It would be interesting to see a patch from Lotus on this. With Enforce
Consistent ACLs on, the client caches the group/role membership information to
determine the behavior of the local replica. If the replication history
cleared for the database in the event that this cache changed, then you’d get
the behavior you expect, since the documents would be removed from the local
replica.

Your remark that [If you do, you must then assume that a mobile user could
replicate the database and eventually see lookup results he is not supposed to
see] would be better phrased as “If you do, you must anticipate that a
mobile user who could once see a document may not have that document removed if
you are using an abstracted security mechanism.” While this is clearly a
development concern, it’s hardly a security hole. After all, even if the
behavior were altered, all the user has to do is backup a copy of the NSF after
each replication to keep all documents to which their access is revoked.

11

Subject: Major security hole with reader fields?

Have you reported this problem to Lotus Technical Support? My understanding is that this is a discussion forum only and may or may not be monitored by Lotus/Iris. If you don’t get a satisfactory reply in this forum, I suggest that you open a ticket with Support.

I experienced a similar issue with the FilePrint command. It was discussed in this forum in October 2002, but apparently never reported to Technical Support.

12

Subject: Major security hole with reader fields?

While not a reliable way to get attention from development, this posting got forwarded on and there is now an SPR #KSPR5K4U7Z: “Reported security issue with Reader Names fields” if anyone wants to track this through support.

As noted by other responders, this should not be considered a security hole because local access controls are advisory and removing someone from a role cannot reliably deny access to a document someone has already copied locally. But it is anomalous and undesireable, and we’ll try to figure out why it’s happening and whether there is an appropriate fix.

13

Subject: RE: Major security hole with reader fields?

Thank you - it is a relief to see that IRIS will take appropriate corrective action regarding this issue.Can someone tell me where I can enter this SPR code to go read the status of this fix?

Thank you!

14

Subject: Unable to reproduce

I followed your steps to reproduce and everything worked correctly for me. A likely problem is that when you turn off a role in an ACL in a local replica, it doesn’t take effect until you reopen the database and sometimes the client keeps the database open longer than you might expect (so you keep the role).

You might try exiting and restarting the client after removing the role from the ACL. Beyond that, we would need to know what version of the client, server, and designer you’re using and a precise script that demonstrates the problem.

There are other known problems that sound a lot like this one, but with luck this one may be a phantom.

15

Subject: Steps to reproduce (view must be hidden?)

After a little bit of experimentation, I found out this behaviour seems very erratic. It could be a cache issue, or it could be related to the hidden views. Anyway - I managed to reproduce the bug when I hide the view and switch my access from one role to another on local.

Here are the exact steps to reproduce:

I use client designer 6.01 and the development is happening on local:

1 - Create a new, empty database on a LOCAL designer station.

Because my server is on a remote location, designing directly on it can sometimes be slow, so I am designing my applications on LOCAL first, and only replicating the changes to the server.

2 - Change the ACL to Enforce a consistant ACL across all replica.

Create 2 roles: Role [Some] and role [All]

We intend to use [All] to make sure we always retrict all documents, and we open some specific documents through the [Some] role.

3 - Create a simple form, with 2 fields:

“Data” is a regular editable field

“Limit” is a checkbox type field with predefined value “RESTRICTED”

4 - Add a READER field, computed, multiple value, that computes to:

@If ( Limit = “RESTRICTED” ; “[All]” ; “[All]” : “[Some]” );

5 - Now, create a view showing all documents. Let’s call it “AllDocs”.

The first column is sorted and displays the content of the field “Data”.

This view will be used by the end user.

6 - Duplicate this view, change its name for “(AllDocsLookup)” in order to make it into a hidden view.

I also add an alias named “AllDocsLookup” so I don’t have to refer the view by its name between brakets.

7 - Create a new form. Let’s call it “TestLookup”.

On this new form, include a single editable dialog list field called “Choice”

And lets reference that view with a dbcolumn:

and save the form.

7 - Create some documents in the database.

I created 4 documents.

Document 1 and 4 are not restricted (they include the [Some] role)

Document 2 and 3 are restricted to the [All] role only.

If I make sure I am only flagged to the [Some] role, as a user, I can only see 2 documents in the regular, visible view:

But in order to be really sure, let’s take a look at the hidden view, using the SHIFT-CONTROL trick:

As expected, I can still only see 2 documents out of 4.

Lets now create a new document using the TestLookup form:

The dialog list compiles to ALL the document of the view, regardless of the reader field security:

It seems to be linked to the fact that the view is hidden, but It could just be a coincidence because of cache clearing or something?? anyway - I hope this should give you a good start on where to find the bug?

Thank you

Nicolas Abesdris / PCLP

Quintessence e-solutions Inc.

16

Subject: Y’know… this behavior actually makes a certain amount of sense.

Let’s think about this for a minute…

What does @DbColumn return? The contents of one column in the view index, right? @DbColumn is fast, because the information is retrieved directly from the index, which means the individual data notes are not accessed on the use of @DbColulmn.

But this call to a Notes view is like any other. It doesn’t refresh the view index until it’s considered out of date. But why would this index be considered out of date? After all, none of the documents in the database changed with the replication. Only the role membership changed. Therefore the indexer considers the view current.

This should be testable in the above scenario by NOT building the index before calling @DbColumn for the first time. Alternatively, you could wait until the second replication, when the documents are on the local hard drive but considered hidden, then cut & paste the view on the local version. In either case, you’re forcing the view index to be rebuilt after the documents are considered unavailable to the current user.

If this turns out to be the case, then you simply need to force a view index rebuild before the @DbColumn. While that’s not necessarily a cakewalk, it isn’t insurmountable, either.

17

Subject: Ways to force a view index refresh?

Hum… well, it is worth testing, for sure!
How would you handle refreshing the index of a view programatically from the @dblookup or the @dbcolumn call?

(That is, as a workaround until the problem is corrected in a further release of the software) ?

18

Subject: RE: Ways to force a view index refresh?

You couldn’t directly from the @function context. But you could on the
QueryOpen event for the form via Lotuscript.

Something like…
[]
Dim sess As New NotesSession
Dim db As NotesDatabase
Dim view As NotesView

Set db = sess.CurrentDatabase
If db.Server = “” Then
Set view = db.GetView(“[your view]”)
Call view.Refresh
End If
[]

I really don’t know if it will work, but it would give us more information.

19

Subject: RE: Unable to reproduce

I will try to come up with the exact procedure I followed to find the bug.

If I try to reproduce it myself on a blank, clean database, I can’t reproduce the bug - yet I am definitly encountering it in my current application development:

here is a screenshot of a keyword field:

It presents me with 3 choices.

Here is the @dbcolumn code:

KeyList := @DbColumn (“”:“NoCache” ; “” ; “$Pick_Sponsor” ; 1);

DescList := @DbColumn (“”:“NoCache” ; “” ; “$Pick_Sponsor” ; 2);

FullList := KeyList + “|” + DescList;

@If (@IsError (FullList) ; “” ; FullList)

Here is the $Pick_Sponsor view, as seen with my current ID:

As you can see, the view only displays (correctly) one single document, since my access role currently only permits me to see one document. (Before you ask, there is a hidden column that allows the DBlookup to return a corresponding ID list).

Here is the view seen from the designer point of view:

As you can see… the view only displays what I am supposed to view (on local) when opened directly, but the dblookup or dbcolumn is returning ALL the documents.

If I connect on the server replica, directly on server, I get a different result for the very same dialog list field:

Here is the same view opened when I have the proper role:

Now it shows all the documents, correctly, since I am allowed to see them all.

If you search the R4 / R5 forums with the keywords: Reader AND Local AND dbcolumn you will also find some people having similar problems.

I feel a bit better to see it is not a consistent behaviour: it means it truly is a bug and not an overlook security hole…

Thank you to take care of this.

Should you need any more information, please do not hesitate to contact me.

Thank you!

Nicolas Abesdris / PCLP

Quintessence e-solutions Inc.