Hello as there will be no development of the NSFDB2 feature, I would like to know what is planned in the relational / transactional area for Domino.
Of course I am quite surprised to see the only great server orientation of this last years be abandoned.
link :
Subject: NSFDB2 plans
Lotus Domino introduced the ability to use DB2 (aka NSFDB2) as an alternate data store in Domino 7. While the NSFDB2 capability offers some benefits for accessing relational data, we have found that the scaling aspects of NSFDB2 aren’t up to the standards that customers expect from Lotus Notes & Domino. There has been a significant investment in the NSF (Notes Storage Facility) data store, and NSF remains the strategic data store for Lotus Notes and Domino applications and data. The NSF store is optimized for use of storing and retrieving Notes & Domino data and provides the performance that our customers need and expect when accessing Notes & Domino applications. As a result, we are discontinuing further development efforts of the DB2 data store capability within Lotus Domino. This includes discontinuation of additional features, updated DB2 versions, and platform expansion for the DB2 data store. This change allows us to focus on the core NSF capabilities and customer requested capabilities that offer value to our customer and business partner community. As a result, we discourage new uses of the NSFDB2 feature within Lotus Domino. Existing NSFDB2 applications will continue to be supported but we encourage leveraging the use of NSF as the data store for Notes & Domino applications. Traditional connectors for relational database access (e.g. LEI, Decs, ODBC, etc…) are still available for Notes & Domino applications.
Can you provide more details on the current use of NSFDB2? We are looking into whether the use of Query Views will provide the desired functionality.
Subject: the second Garnet move
your response is clearly inadequate.we and our customers have been waiting for years for a an integrated relational capabilities.
and once again you are doing a Garnet move
well, what is the message ?
that Domino is doomed to stay a mail or document repository platform ?
You should know that in every (decent) application there are simple relational needs : Triggers, Constraints, Sequences, Primary key, Foreign Key, Commit, Rollback
Agents are not designed for that. We need something else.
“standards that customers expect” are theses capabilities that you did not really offered with DB2NSF so it is only designed for reporting.
So how do you address them ?
ps : Using also an external database + Domino + LEI (13000$) is too complicated. It’s easier to use a J2EE application server
Subject: NSFDB2 support available until 2017 (at least!)
To clarify - NSFDB2 continues to be supported, for a minimum of 5+3 years from Domino 8.5’s availability, and could possibly extend beyond that. In other words, the NSFDB2 feature will be supported at least through 2017.
Through the discussions we’ve had with customers, we heard about performance issues using NSFDB2. DB2, as its own application wasn’t the issue - but rather the interaction of using a relational back end with Domino data. While some applications appear to be suitable (querying, for example) implementations of NSFDB2, there are performance considerations for other applications. Many of the customers we’ve heard from have expressed interest in query view capabilities, which is something we’re investigating.
Domino has invested in making the NSF component robust and scalable and is suitable for enterprise quality messaging and applications. We are working on a FAQ concerning NSFDB2 and expect it out soon.
Subject: Query views…but requesting which datastore ?
You say you are investigating keeping query views…but which datastore could be queried ? if those can only query DB2 datastore then it’ll be far LESS usefull that being able to query NSF.
BTW, Query view only solve a small part of the problem…NSF needs to provide, beside dynamic queries, programmatical transaction support, data joining capabilities and referencial integrity.
Subject: Lotus Notes & Domino Strategy 2009
Interesting presentation but there is no information about the future of NSFDB2 and NSF…
Subject: Where exactly will it be available?
“Domino has invested in making the NSF component robust and scalable and is suitable for enterprise quality messaging and applications. We are working on a FAQ concerning NSFDB2 and expect it out soon.”
Where exactly will it be available? Could you also clarify about the future of NSF?
Subject: how would you handle huge #docs and apps that really want an RDBMS model?
I thought nsf2db was doing well in the performance area, so I’d like to hear more about what happened to it…
Subject: Very disappointing news
It is a very disappointing news. Is it for sure?Then maybe it is the right time to say: “Lotus Notes goodbye”? Our hopes have been balked.
http://www.ideajam.net/IdeaJam/P/ij.nsf/0/FDACCCF38EF88F2F862573AF0042B2C4?OpenDocument
Subject: I need View Contention to go away
I recently blogged about my frustration that a fix for View Contention wasn’t going to have a future with nsfdb2:Forside
We have lots of documents, lots of users and lots of document updates.
The Domino View indexer locks views when indexing them, leading to Contention on that resource which effectively locks all activity in our application.
Query Views was going to be out fix for that problem.
Now we are looking at storing part of our data on a separate sql server.
Subject: Writing out to RDBMS
@Jens, we had the same issue and we were under the gun to provide a globally acceptable solution. In our case both the view indexes and the full text index were being used by the application, and as you probably have experienced, in extensive testing the server/application had no issues. What the business unit did of course was to say they wanted all the reports done on a certain day and time, thus HEAVY load on the server - which just stopped doing index and view updates. The application seemed to be operating fine, but returned stupid results. The business unit didn’t care what happened to the data, or what process stopped, just that it was incorrect. The Domino processes that stopped working made us look like monkeys, and there was serious consideration to move the whole application away from Notes to Oracle/web custom development.
We then wrote a pile of code in our Notes app to take all the data out to Oracle, wrote another pile of code to take user selected parameters to execute a select statement on the Oracle data, and another pile of code to return the rows back into notes with the correct security enabled. Effectively we did our own rdbms enablement, but ALL CUSTOM for the application.
I guess the nsf is pretty cool, but wouldn’t IBM have a real powerhouse application if it used an RDBMS natively. Just think of the numbers of Domino servers that would be installed if Domino could use MySQL or PostgreSQL! Can you also image what that would do to increase development productivity in any organization from SMB to corp development shops?
Subject: I would suggest that you didn’t talk with the right customers!
I’m sure every notes shop would tell you to improve the nsf - I hate dealing with a corrupt database, but when you actually look under the hood, perhaps these databases are being used in a more relational manner so that having a relational database as the back end makes more sense!
One of the most severe limitations in getting to documents is having a view with the ‘correct’ first sorted column. I have databases with more views that any other design element, and most of these views are lookups. Sometimes the lookup column is simple, other times it is a complex concatenation of drill-down key values. The query view is a simple concept that allows designers to develop views at run-time not at design-time.
Secondly, when one is trying to select a set of documents based on certain filter values, we are forced to use a full text index. How is it possible to return a filtered set of data in an orderly view (like select * from table where x=y ORDER BY z) unless there are pre-built views for every value of z?
Finally, what happens when you want to combine information from different forms (simple join in sql)? I’m told you can do this with xpages, however, I can’t figure out how to use the view control to make the join. Maybe you can make the join for one record, but what if you want to launch a spreadsheet with all the documents from the above filter? What I generally do now is store the fields from the second form on the first and then write AGENTS to update the first form when values on the second change. Pretty unproductive…
I’d say the relational back end should be made a requirement, rather than trying to force domino designers into a convoluted mechanism for getting to the right data. I’m wondering right now how easy it would be to get help to craft the right sql statement, than it would be to find an xpages expert. Based on the amount of time either has been around, the answer is a no-brainer.
Subject: Domino is a repository
sometimes I start to believe that all the development part of Domino is a mirage.
I don’t think it exists any critic application (or complicated one) on Domino,
only critic repositories (and emails are repositories).
An application has business logic, not repositories.
(I really would like that somebody from Lotus prove me the contrary !)
So we are there because of
an innovation problem (cannot get rid of NSF) or
a strategy choice : IBM wants to keep the relational / transactional needs on other platforms rather than extend Domino.
I’ll tend to say that it is a strategy choice.
good or not it has been chosen.
(OS2 and the smartsuite were also strategy choices)
so, to summarize :
mailing & repository : go for Domino
easy relational : go for php or J2EE
relational / transactional + business logic + enterprise integration : go for J2EE
Subject: Domino/Notes is really business workflow
I would say, on the contrary, Domino is a strong platform for rapid workflow and security development.If all you need is just a repository, then your best bet would be a RDBMS.
I have designed extremely complex applications which span multiple platforms ( Oracle, MQSeries, XML, SAP) and were a complete success. Doing the same applications using another tool/software would have taken a huge amount of hours ( = cost )and number of people.
With Notes it is much easier to design complex applications than in any other software. If there was another platform , better than Notes I would be the first to switch to it.
Am I frustrated some things in Domino? absolutely, but still , for time being, it is the fastest platform to do a complex application with minimum cost.
Subject:
Subject: As to me, the best thing about IBM Lotus Notes/Domino is NTF
As to me, the best thing about IBM Lotus Notes/Domino is NTFTemplates! Your whole application can be inside one notes template file (NTF)! It is the best feature of LND. That is why I would prefer if NSF had all relational database capabilities and features within itself. They could be optional.
Subject: There’s more discussion on NSF’s possible SQL-like directions on my blog.
http://www.bleedyellow.com/blogs/erik/entry/the_future_of_nsf_part_14
http://www.bleedyellow.com/blogs/erik/entry/the_future_of_nsf_part_22
Subject: the backend is poor
let’s consider this study case : try to implement it in Domino, it will be nightmare !
we have 3 tables
Employee (employee id = eid, lastname, firstname, birth date)
Department (department id = did, name)
DepartmentEmployees(did, eid)
A) how to implement the concept of unique key and foreign key (and prevent the dupplicates)
B) how joining tables and displaying result (without dupplicating information!)
department A
employee 124
employee 35
depatment B
employee 5
employee 6
etc …
C) deleting the “department A” deletes all records in departmentEmployees table
D) if something goes wrong in the deletion process we can rollback
E) after data selection (query view), being able to process datas (by java beans) and then display result (display view)
example :
data selection : employees <= 30 years old / employees > 30 years old
process datas : profit made by department
display view : a graphic chart
F) more enterprise integration : mix data placed in other databases. distributed transactions etc …
Subject: Rating something as “Poor” is relative to the application
Hi, I agree for the scenario you painted, which is highly transactional, Notes is not up to that in its present form. The Notes datastore (inside the NSF) is optimized for unstructured data. This makes it ideal for some things, and yet “poor” for other things. I wouldn’t say its “poor” across the board in general. In various Notes training 101 course they talk about what Notes is and what its good for. They use an airline booking application (lots of realtime transactions) as an example of an application that is outside the scope of Notes.
What is required to extend Notes to the type of transactional applications you are referring to is an RDB backend. NSFDB2 was an attempt to go there. You can do additional things using LCLSX which can talk to an external DB. I am unclear if the LCLSX api has transations and rollbacks, but I recall seeing that at one time. Whether the next iteration of xpages with the ability to talk to external non-Domino data will make Notes/Domino a candidate for transacdtional applications like you describe is yet to be seen. Possibly those on the Notes Dev team who are working on the external data programming while I write this will be able to shed light on this. However, I doubt they can share freely at this time as I suspect IBM will not want to deliver too much info on this prematurely, until they determine exactly how far the implementation can go because they are already feeling the heat from those customers who put their eggs into the NSFDB2 basket and feel they were burned. I supposed that’s what this thread is about ultimately…to jolt IBM into responding with their plan.
Subject: NSF versus a relational DB
Add these requirements to the “employee” example you gave:
granular field-level security to access certian employee-specific data
replication (especially offline)
each employee has a variable number of user-defined fields for tracking metadata
…and all of a sudden NSF is much more attractive. Read my blog entries – there is a lot that you can build into Notes (though, yes, it would be nice if it was supported natively) that makes it behave more like a true relational database.
In my experience it’s much harder to make a relational database as flexible as NSF. IBM learned this the hard way. After years of trying to migrate NSF to DB2 (NSF<>DB2 was a step towards that final destination) IBM decided to quit doing it.
Keep in mind that IBM has said they are no longer moving forward on enhancing NSF/DB2, but they simultaneously said that they ARE moving forward on enhancing NSF. And they know that people want relational-type capabilities.
I’ll reiterate from my blog about the main relational capabilities that people tend to want in NSFs (my current thoughts in bold). Bottom line: IBM could probably address quite a few of these concerns, and perhaps they will, or already are.
SQL queries - This has been somewhat doable for years with NotesSQL, and now with NSFDB2. It needs to become a core part of the product. It could, with more effort from IBM. With the move to Javascript as a primary language, I wouldn’t be shocked if IBM implemented SQL as a major query language.
JOIN capabilities - This is partially addressed with #1, and you can write backend code or use XPages in 8.5 to do the UI-equivalent but again, indexing speed is key. There is currently NO way to store the index that results of a JOIN – at least not without NSFDB2 and again, it needs to become a core part of the product. The lack of JOIN capabilities is really the main architectural disadvantage of NSF (besides just getting big as we’ve discussed). There has been a tiny bit of progress here in 8.0 with the new NotesDocumentCollection.Intersect() methods and few others. This might suddenly become possible if view index storage and contention was relieved by moving indexes outside of the NSF (commented on my blog).
Enforcing Referential integrity - If a doc is “dependent” on another doc, not much is keeping you from deleting the 2nd doc, therefore causing an invalid relationship in the database. This would likely be a lot of work, and when taking replication and doc security into account I’m not even sure it’s completely doable. But for many applications this is a “nice to have” that can be addressed in other ways and not an absolute requirement at the db level.
Transactional model - IMHO, NSF can’t really go there while maintaining it’s current capabilities, and this would be the #1 reason to stick with a relational database. People don’t enjoy replication/save conflicts in their payment systems. If you really, really needed this, you could code your own transaction-esque model, too.