i’m interrested if in Lotus Domino 8.5 it will still be the restrictions of 64 KB for Dialog boxes, because it’s annoying if there are big lists. Is there a workaround?
What about the 64 GB Database limit in 8.5. It’s really fast to arrive there if you have attachements. Is there changing something in the future? Only depending on OS or storage? Thanks for reply
Subject: 64GB size limit
At least at the current time we don’t have any plan to change the 64GB database filesize limit. The logical database size limit has been substantially altered with the various compression features and, most substantially, with the introduction of DAOS in 8.5. Its fairly easy to create a multi-TB “database” with DAOS storage of attachments as long as the base database file doesn’t exceed the 64GB limit.
Subject: RE: 64GB size limit
Thanks for feedback,
The problem is not the multi-attachments of the same file, but as example you have a media database with streams, then if you have about 30’000 you will already reach the limit of 64 GB. And this streams are onyl once stored.
Subject: that should still work with daos - if applied, the files get stored seperately from the nsf.
Subject: seems to work
Thanks for feedback. I read the process to convert Databases before 8.5, and it seams to be really very easy.
That would really be the solution do downsize the Lotus Notes databases. Thanks a lot.
Subject: Glad to help - and let us know how it works out.
Subject: DAOS is a server only feature isnt it?
If you use DAOS as a solution for the 64 GB database limit you cannot make local replicas of this database. So i think there is a need to extend the limit in the future.
Subject: no replica without attachement?
Is it not possible to do a local replication without the attachements extracted by the DAOS? Because i have a media-database for music that will go up to 300 GB? Do i have another solution in Domino 8.5? Thanks for help
Subject: I’m not sure if DAOS will help on the 64 GB limit
Based on what I’ve read about DAOS, there is a physical size and a logical size.The DB will still be 300 GB of logical size although it may be 20 GB of physical size…
So I’m not sure if the 64 GB limit applies to the logical size.
Subject: if you have one of this ‘monsters’, why not extend the data amount in a test environment and share the result for us?
Subject: It is right now - How many Databases > 10 GByte do you have? How many of those do replicate the full set of data and attachments to Clients for offline use?
I hope it is o.k. that I ask.
Besides this, you are right, every database limit has to be watched and there is most likely still several ones that need work on in the next years one way or another (e.g. lets imagine bringing daos offline, as a switch to an database just like having an full text index or alike)
Subject: 1 Database about 37 GB
Hi Benfried,
I have till now only 1 DB that have the size of about 37 GB. From this 37 GB are about 30 GB’s only music file attachements. Means if all attachements can be hold external the DB size will go down to approx. 5 GB. Till now i have 1 full local replica. Same is in the Dialog lists, if you want to select record labels with 64 KB you are lost. But it helps not to keep multiple copies (different written) of the same Record label.
Thanks for information
Subject: O.k. This is what I would plan for:
I have been working for TV/Radio stations before, sounds like your local replica is to make sure you can still broadcast, even if the servers would be hit by a longer power outage or something. Based on this assumption, this is what I personally would go for:
a) Not sure when You do expect to reach 55 GByte+ - But if within the next 2-3 years, I would consider installing for the machine with the local replica actually a server, too (both, a server and a client on same hardware) - Server to server replication will work well with DAOS from what I know, I did not find the time to test myself, yet.
Different Notes Domain (the ‘local’ Server needs no more than just one server document in its names.nsf, IBM Software Upgrades can be run for this machine independently from rest of network, which might help down the road)
I was doing so in times whenever having to do development work with bigger amounts of data on various notebooks for 10 years now, and it is just more efficient - e.g. way less waiting time for updated views after replications on the replica, if all view properties are set right - the system just ‘feels’ faster.
b) Not sure what you exactly do in the Dialog lists, and what exactly goes wrong there, - but my guess would be, that you end up simply running into limits of Field content lengths. (If I remember right, actually 32K limits, not 64 K).
I was able to trace down the fields guily in similar situations before (remove all follow up fields to see when you don’t get the error any longer).
Technology wise it is usually best if you can replace collecting thousends of strings in a multi value field for the basics of a selection list by using @Picklist. (available in script, too, if you prefere so. For me, in every single case, this was way faster, and getting rid of the complete problem at all. The actuall selection would move from a dialog list into a button).
Technology wise only second choice, but could solve issues, too, and with the advantage, that the dialog would still look absolutely identical:
a) Sometimes (- depending on the application) it is possible to store the data in multiple fields - the problem is usually ‘just’ storring the data in the field itself, not the handling (not the @DBLookups/@DBcolums, not the list they return, and not the Dialog list to select data).
b) Sometimes it is possible to do some pre- selection before even getting into the dialog to reduce the amount of data.
c) sometimes the data can be storred in different fields (lets say if you use @DBColumn to get a column with multiple fieldvalues and some separator, you might want to considder to place those in different fields to share the load at the critical point.
Subject: why search for Solutions if it could be done standard?
Hi Bernfried,
Thanks for you very interesting answer. You are absolutely right.
My DB grew in a few month from 4 GB to 37 GB. I have about 100’000 samples to store in that will end up as mp3 in about 300 GB Database.
The question is why set up 2 Servers if the Database could be in general increased to OS limit as Oracle, SQL and others.
Same for the 64 KB Dialogbox limit. Instead of splitting filds, and afterwards splitting again the limit could be enhanced up to 128 or 256? I don’t see the problem, because they increased it in Version 6 form 32 KB to 64 KB.
That’s the reason i wrote this in this forum. To sensibilise the Lotus Engineers for real usage limitations. I hope they will pick it up for the new 8.5 version.
Subject: Nothing wrong with this - besides …
The fact that you are hitting 8.5 development at a point where it is most likely already at feature freeze. Asking for new features / higher limits is of course not a bad idea, but It takes in most cases way bigger time frames than 6 months ahead and again in most cases it takes either many asking (IdeaJam?) or a very very big customer to get it thru, or a use case which would show that one would use many customers if they don’t go for it or having very good friends within Lotus at the right positions - it takes many years to get there and tickets to the US conferences.
Some would argue that IBM/Lotus is not not there to raise limits you did run into by not doing the right capacity planing of all details half a year ago (Why attach the files to the database in the first place? Why not use @Picklist in the first place, which was done as a tool to work around such limits in Dialogs years ago?
Some might argue, that I was suggesting the wrong workaround to you - if your Database is that important, it should of course be on two servers, and those servers should be clustered, but at different locations. - I still would prefer to go for what I suggested to you.
Please don’t take this as offence. As you did already read I have been running into some of the same issues - in some cases by not being careful enough. I just wouldn’t lean back and expect IBM/Lotus for me to do this things - In the very most cases not even if we are talking about true bugs (versus ‘works as designed stuff’) - I take the time to do the most easiest sample to reproduce the error I can imagine and post a sample database as a possible bug report - and while doing so, in 98 out of 100 cases I learned enough of the problem when doing this to get going again with a lost day or so. - Going support, getting my code verified and asking for a hotfix (which the customer might even have to apply to each and every client) just takes longer, and is way more inatractive - this is what I found when doing notes since Notes 3 Beta (and long before Lotus was IBM).
Subject: Thanks…
Hi Bernfried,
Thanks again for your detailed explanation. Sure i’m not waiting for IBM/Lotus to change the limits. We work with the possibilities that are fact. But as told engineers are depending on ideas & real world limitations to go further with the development of the product. And the limits are there for a long time. 15 years ago it was nearly not possible to reach them, but today it’s a fact it depends of the needs.
For myself i can live with the proposed solutions that you gave me. But that means not, that it’s an easy change to do in a program code. As we are also developers and i know that you live from feedbacks to progress, and that’s what i intend to do.
Thanks a lot to you for your cool ideas.
Subject: I did not try to say it is easy - nor it should be the way it is. Thanks for your kind words - and good luck!