XPages: why to use them?

HCL Domino Domino Forum
1

Hello,

I’ve been learning XPages in the last few weeks and started creating my first XPages for real use (8.5.1FP2 Designer and server). The more I learn about XPages the more convinced I am that they are useless. Everything I’ve seen so far can be done with forms and agents as or more easily.

Here are the main problems I’ve faced:

  1. Reuse of what we’ve done before. I’ve built a large repository of reusable LotusScript libraries, subforms and agents. NONE of these can be used with XPages and IBM wants me to throw all these to trash and start from the beginning again.

  2. Header and Footer. A company specific layout typically has a header and footer (implemented as subforms before) where header opens some div tags and the footer closes them. How do I do this with XPages? I’ve tried e.g. custom controls (“Item not found”) and placing the HTML retrieved with Ajax to div with .outerHTML (footer does not render correctly).

  3. Security. In an HTTP agent I can choose to run it with user’s rights or signer rights. Now in my XPage I’m creating an Excel (.xlsx) file in the server side which I need to send to the user via HTTP. In this process I need to (in my understanding) write this file to the disk but the user does not have rights to do that. Am I correct that XPages code always runs with user’s rights?

  4. Slowness. Development typically involved hundreds or thousands iterations where I save a small change and test it with a browser. When I save a form or agent I can get the result to my browser page in the same second. When I save an XPage (have to use mouse for saving bec usually Crtl-S does not work) it takes about 30 sec to build before it shows up on my browser! This slows down development a lot. Even the subsequent page loads are slower than they are for the same thing in a form and agent, even if I set the XPage to stay in memory.

  5. You still need to create the form! Why? Form is also just a presentation layer.

Fix these and XPages may become usable. One positive comment though: the error messages of SSJS are very descriptive!

I guess XPages are useful when there is a need to share components with Mashup Center etc. but if there is not, is there any reason why I should use XPages?

Of course forms and agents seem to be in maintenance mode and it’s not fun to develop with something that Lotus seems to have declared dead. So please fix XPages fast or start developing forms and agents again (e.g. “ stay in memory” check box for agents!).

thanks,

  • Panu
2

Subject: It’s not as bad as you think…

(Disclaimer: 8.5.2 is beta at the moment, may not have anything I’m talking about, etc. etc)

  1. Reuse of what we’ve done before.

You can call agents on docs in 8.5.0 and 8.5.1, though this means you have to save the doc prior to processing. This may not be possible for a variety of reasons, but it’s something. 8.5.2 will have a method to run an agent on an in-memory doc. This is the true “bridge” that finally connects legacy code to XPages.

  1. Header and Footer.

There’s a variety of ways to do this - check the blogosphere.

  1. Security.

Yeah, this was a hurdle. 8.5.2 will have a new method, effectively allowing you to spawn a NotesSession object as either the web user OR the signer. So you could even mix-and-match, calling some agents/libs/whatever as the web user, and others as the XPages singer, all from within the same Xpage. This capability leapfrogs XPages past all classic Domino web dev in that area.

  1. Slowness.

Yeah, the compile delay kinda sucks. But subsequent performance should be faster than agents in most cases.

  1. You still need to create the form!

No, you don’t – it just helps to add some pseudo-structure to the DB so that Designer can help you work with your fields and anticipate data types, etc.

3

Subject: It is as bad as you think…

"You can call agents on docs in 8.5.0 and 8.5.1, though this means you have to save the doc prior to processing. "

  • True, this is possible. But my experience doing this shows it has a number of serious flaws:
  1. One must save the document first. This is a whole can of worms, some my doing and some not.

a. In my instance I wind up committing data to the database that’s invalid, because my validation logic is in LotusScript classes. Some of that logic could be put in JS, but some of it is impossible to put into JS without scrapping the entire system. So I commit invalid data and flag it as such in the Agent, because I have no choice.

b. It seems that saving the XPage document demands concurrencyMode=force if the LS Agent is going to attempt to actually commit data. Since I’ve never written an Agent (except LS to generate web pages) that does not commit data, this must be set or one gets save conflicts, even if the LS is coded save(true, true).

c. The above, (b), seems to cause a commit to fail if there aren’t any changes to the document. Since LS has no way to know if anything has actually changed (no NotesDocument.isDirty, for instance), the normal way to do things is - when in doubt - save anyway. But if one does that from an XPage then

if(not doc.save(true, true)) then complain

will in fact complain, causing the code to fail. Every time. So I get to manually manage “dirtiness” to get code that’s been working for years to keep working.

  1. Once the Agent is done running, the SSJS on the XPage has no ready access to the results, as fields on currentDocument reflect the old values before the Agent ran. Deleting currentDocument has no effect. I surmise the XSP command processor keeps an in-memory copy during the Agent’s execution, that it never bothers to refresh, which Is Not Good.

  2. It seems having an Agent process code using LS2J when called from postSaveDocument on an XPage causes all manner of weirdness. The only way to fix this was to commit text describing the desired operation to a sessionScope variable, along with parameter data, then close the XPage and have the next page loaded check sessionScope, then process it using AJAX. LS2J is legacy code … if legacy code is supported in an Agent from an XPage then this shouldn’t be happening. Instead XPage + LS2J = nightmare.

  3. The postSaveDocument event randomly fails to fire, so the XPages where a LS Agent actually can do something without freaking out, random doesn’t do anything. To be more freakish, the querySaveDocument event does run, and even though that event does nothing what-so-ever but log that fact, pSD never goes off. i have never seen it function reliably over months, though it’s possible once one stops rebuilding the application constantly it will in fact work reliably. But during development … uh uh.

  • Do not drink the legacy migration XPage Kool-Aid.
4

Subject: OK Thanks

  1. Reuse of what we’ve done before.

Looking forward to see this functionality in 8.5.2.

  1. Header and Footer.

Good. It’s just that many things I try to do will not work the way I think they should. I have to learn to do everything the XPages way.

  1. Security.

This sounds great!

  1. Slowness.

OK, have to do some more testing because my impression is the opposite.

  1. You still need to create the form!

I understand these points but simple data binding without the form should also be made possible.

5

Subject: My thoughts

I appreciate your concerns about rip and replace. But just because XPages is available, doesn’t mean you have to completely redevelop your apps, as Tim Tripcony blogged recently http://www.timtripcony.com/blog.nsf/D6Plinks/TTRY-85LHLS. At the time I went on an XPages course last year I was revamping an existing web application, and had already got through a large amount of the work. It’s still predominantly classic web, but I’ve used XPages for areas where the use of repeat controls made certain things MUCH easier (like editing multiple language keyword documents in a single interface, and changing an extremely unwieldy and inflexible agent that generated response forms and were limited to 250 questions to a repeat control that performed much better, had better scalability, and was easier to support). Users cannot tell any difference, and it was not too difficult, even on my first app, to reproduce the same look and feel.

Equally I have a very large client and web application with lots of lotusscript script libraries and custom classes. I haven’t converted it to XPages and don’t intend doing a full conversion any time soon. What I do plan on doing is possibly using XPages for the views or some other simple aspect of the application. Again, repeat controls could open a wealth of opportunities for innovative presentation techniques that could add significant benefit to the application.

For other Notes-based applications, I’ve created a separate searchable view of the content to plug into an intranet. No design elements have been added to that Notes database, all are in the intranet database.

For me, the key is using the right tools for the job. An XPage with a partial refresh and a few lines of SSJS allows me to develop much more quickly functionality that with AJAX would require javascript to call an agent, plus javascript to handle the response, and any errors. Thinking of performance, that is an agent running on the server. Not an issue if it’s the only one, but have that application and server used by a lot of people, and the agent manager may become anissue. Factor in that someone (can’t remember the blogger) researched that SSJS runs 4x quicker than LS or Java, and it’s more scalable too.

If you’re looking at generating Excel and are using XPages, you may want to have a look at this blog post from Chris Toohey, if you haven’t already seen it One-click Microsoft Excel (or Symphony Spreadsheets) via XPages and SSJS | dominoGuru.com.

Ctrl + S should work as long as the focus is in the Design or Source pane. I certainly had issues when the focus was in the wizards, but use Ctrl + S to save my XPages. Saving takes a long time on my home PC, but that’s about 3 years old and was not designed to cope with The Eclipse client. My work PC was spec’d when we migrated to R8 to cope with the Eclipse client, and performance of saving is no different to what I used to have with classic Notes design elements.

I would disagree that Forms are the presentation layer. In XPages architecture they are for data entity structure, as equally are Views. You don’t need a Form for XPages, but it enables the Designer client to understand your data model, so you get the right drop-downs. Equally, you could have a database with just a default view, and use repeat controls to display the content, doing a database.FTSearch to get your content, although you would need to sort it. The benefit of a view is it defines the sort order and index, making it easier to organise your data as you wish.

6

Subject: Thanks for your comments

I have no plans to convert [anything] to XPages. I’m experimenting with them in my new development. Of course I would very much like to use the reusable components (LS classes and subforms) that I’ve created over the years.

Thanks for the interesting info about SSJS performance. It would be interesting to see a test report (didn’t find with short googling). As XPage is a servlet in the end, doesn’t that mean that the SSJS is also converted to Java when the server builds the XPage/servlet?

I need to create an Excel 2007 report with charts. That is multiple XML files in a zip file. I don’t think I can just print that out like Chris does.

Maybe the focus is not in right place or something but often after editing code Ctrl-S does nothing. And I didn’t mean saving time but the time to load the page the first time. It looks like Domino server builds the XPage when it is loaded the first time (or it builds it after save, I’m always loading right after the save).

As far as Forms I mean we should be able to add data sources without forms (which are design elements and have nothing to do with data).

7

Subject: use the MS OOXML SDK V2 for Excel 2007 files

which gives you an API to directly manipulate the OOXML files. Have fun :slight_smile:

8

Subject: Is that Java based?

Actually I started to code the XML generation with SSJS already and now it seems I need to port that code to LS.

I took a look at MS SDK but did not find any Java API. There is a dll which might enable LS access but I cannot place any dlls to servers.

I found this which is interesting:

http://poi.apache.org/spreadsheet/quick-guide.html#NewWorkbook

but it does not support charts yet.

9

Subject: no Java yet

we do it all server side thru a processor engine we have built. But otherwise, you are changing XML and have no idea if that effects something in the document.

10

Subject: I’m generating the XML

This is quite straight forward actually. First I build the report manually in Excel and get the ready XML that way. Then I just place the real life values in that XML in my code and maybe add more charts with different values. So I’m not changing the XML tags that Excel generated but just changing the values inside them.

11

Subject: Some of my thoughts

“Of course I would very much like to use the reusable components (LS classes and subforms) that I’ve created over the years.”

As far as I know, from 8.5.2 you can call existing WQO/WQS agents with a document context, and have it work as before. Having some rather complex LS apps, rewriting it to SSJS would take forever and introduce errors for sure, so rewriting is not acceptable for us either.

“Thanks for the interesting info about SSJS performance. It would be interesting to see a test report (didn’t find with short googling)”

Having rewritten the front-end part of a CMS system from LS WQO to xpages, our typical response times went from 7-900ms to 2-300ms with complex templates. However this is not a straight rewrite, and we did it because LS was not performing good enough, and as we piled on functionality, perfomance took a dive. We are using Freemarker as a template engine, called via an xpage to java, using an xpage as our controller…but not using JSF for rendering.

“As XPage is a servlet in the end, doesn’t that mean that the SSJS is also converted to Java when the server builds the XPage/servlet?”

It does seem, from some experimenting, that XSP does not run in a servlet container, but something else, or at least a stripped down container. The result is the same, and the JSF Xml is used to generate Java classes that are then compiled. This is what’s causing the delay from saving to displaying the result, and the delay can be quite painful.

“I need to create an Excel 2007 report with charts. That is multiple XML files in a zip file. I don’t think I can just print that out like Chris does.”

I would look into creating Java classes to handle this, as there are Java based solutions that can handle it. Then you can call this Java solution from your SSJS.

“It looks like Domino server builds the XPage when it is loaded the first time (or it builds it after save, I’m always loading right after the save).”

It has to rebuild all affected classes everytime you save.

"As far as Forms I mean we should be able to add data sources without forms (which are design elements and have nothing to do with data). "

I believe you can create your own Java data sources and use these, there are examples out there, I think.

Now, when it comes to xPages, I love the flexibility that I can now have, but I think it does come at a price of increased complexity (not necessarily a bad thing). Also I am not a great fan of the JSF component model that treats web as a stateful eventdriven object model, and requires XML to do anything. I am myself a bigger fan of stateless html :slight_smile: Also xpages seems to have some problems with high load, we have 2 PMRs that either panics the server or hangs the http-task. One of these are fixed in 8.5.2, the other is still under investigation.

12

Subject: Good comments

It would be great if we could use LS in 8.5.2. Even if the performance would not be as good as with SSJS this would give much smoother transition than the current XPage functionality.

About Excel: that’s a good tip but the main problem is that even if I use these Java classes, I think I would still have to save the file to disk in order to serve it to the user via HTTP.

[“I believe you can create your own Java data sources and use these, there are examples out there, I think.”]

OK but this sounds complicated. I would simply like to bind to any existing or non-existing items instead of having to go thru a form.

13

Subject: Dojo Option for Charts

Another option if you’re using charts are the dojox charting scripts. I’ve written a few tutorials on my blog (http://hermes.intec.co.uk/hermes/blog.nsf), which are also on the Notes Application Development wiki. There are some limitations, but SSJS allows you to build the javascript series very flexibly.

14

Subject: Yes, I’m using them in my other report

But this one needs to go to Excel. I decided to do it with LS (have to port the stuff I’ve already written in SSJS) because I need to write the file to disk.

15

Subject: Good summary…

  • I just wish someone had put that out months ago, before I was committed to XPages. Lotus may treat the old way as dead, but they have provided no reasonable path forward. What serious development shop doesn’t have tons of legacy code they’d like to keep, and maybe change out piecemeal? It’s essentially total lack of legacy integration alone is enough reason to avoid XPages like anthrax, for application migration.

  • For totally new work? (shrug) Could be the cat’s meow everyone says it is. Haven’t done that, but I can say the viewPanel control is an utterly useless load of rubbish that doesn’t belong on any web page. True, it’s simple to use, but its presentation is about as inflexible as rebar, and the results are about as pretty as rebar.

  • I drank the Kool-Aid. Avoid XPages for legacy application migration. You will not be sorry.

Hope this helps…

16

Subject: My comments

Hi Panu,

Here are my comments.

Overall though, yes XPages are a big change, but they are a positive step.

  1. Reuse of what we’ve done before. I’ve built a large repository of reusable LotusScript libraries, subforms and agents. NONE of these can be used with XPages and IBM wants me to throw all these to trash and start from the beginning again.

You’re right that currently you can’t directly use LotusScript in XPages, however, you can call agents and pass in documents. Also porting LS to SSJS really doesn’t take long and makes for a big performance improvement.

  1. Header and Footer. A company specific layout typically has a header and footer (implemented as subforms before) where header opens some div tags and the footer closes them. How do I do this with XPages? I’ve tried e.g. custom controls (“Item not found”) and placing the HTML retrieved with Ajax to div with .outerHTML (footer does not render correctly).

You need to have a look at custom controls and the editable area control. The editable area allows you to insert other custom controls (the equivalent of a subform) inside each other. You can create a “layout” custom control and then insert the content inside it on each XPage. You can download Tim Clark and my Lotusphere presentation which steps you through creating a new XPages application from here: http://mattwhite.me/storage/show112.zip

  1. Security. In an HTTP agent I can choose to run it with user’s rights or signer rights. Now in my XPage I’m creating an Excel (.xlsx) file in the server side which I need to send to the user via HTTP. In this process I need to (in my understanding) write this file to the disk but the user does not have rights to do that. Am I correct that XPages code always runs with user’s rights?

You are correct for the moment. In 8.5.2 you will have the ability to run code as the XPage signer.

  1. Slowness. Development typically involved hundreds or thousands iterations where I save a small change and test it with a browser. When I save a form or agent I can get the result to my browser page in the same second. When I save an XPage (have to use mouse for saving bec usually Crtl-S does not work) it takes about 30 sec to build before it shows up on my browser! This slows down development a lot. Even the subsequent page loads are slower than they are for the same thing in a form and agent, even if I set the XPage to stay in memory.

That is very surprising. I find it is best to develop locally rather than on a server, but I really don’t have a problem and that’s running inside a virtual machine. What spec hardware are you using?

  1. You still need to create the form! Why? Form is also just a presentation layer.

The form is now the equivalent of a table in a relational database, it is a great way of separating design from content. And it also gives you the cool abilities to have multiple forms tied to a single XPage.

I think overall, if you spend a little time reading the resources out there for getting started with XPages that you’ll see that XPages are a huge step up from the 15 year old technology that we’ve been forced to use in Domino web development for the last few releases. I agree there is a steep learning curve but it really is worth the effort.

Matt

17

Subject: Thanks for the comments

  1. Reuse: I hadn’t thought about starting LS agents from XPages, thanks for the tip. Of course in that case the XPage might become totally useless. Most of the code I have are LS classes which cannot be used from SSJS nor Java. If I port something I will definitely port to Java (one reason being lack of proper OO support in JS and its editors). In my understanding also custom Java classes can be called from SSJS. It would be interesting to see LS vs. SSJS performance test results.

  2. Header & Footer: Thanks. My problem is that custom control does not allow tags in source that are not closed. But I guess open tags could be implemented the way you describe which I don’t understand yet. Need to take a look at the presentation.

  3. Security: Thanks for the info.

  4. Slowness: I’ve developed locally and on two servers where everything else is very fast and I’m always very frustrated bec of waiting. I tested now with a clock and the waiting time after save was 15 sec and subsequent loads took 3 sec each. My estimate of 30 sec was too much but even 15 sec feels like forever when doing development, especially when I’m used to the 0.5 sec I get with forms and agents.

  5. Form: I agree that form could be used here like a table. But there should also be a possibility to create/access/edit documents without creating a form (a separate useless design element) for them.

  • Panu
18

Subject: Trust me, no Kool-Aid here…

Regarding saving: yes, there are a bazillion scenarios where you can’t or don’t want to save a doc before processing. But to be fair, it’s not 100% “there is no way to have LS work with your docs.” It’s just 97%. :wink:

I consider 8.5.2 to be the first version of XPages that I consider actually appropriate for serious legacy app migration (and it might really take an FP on top of that). The fact that XPages work in the Notes client as well in 8.5.2 is an added bonus, if for no other reason than it further encourages IBM to dump all their dev resources into them from here on out. They are the New Way, whether we like it or not.

But I said it a year ago, and I’ll say it now - IBM should have had an SSJS<>LS bridge a long, long time ago.

"2. Once the Agent is done running, the SSJS on the XPage has no ready access to the results, as fields on currentDocument reflect the old values before the Agent ran. Deleting currentDocument has no effect. "

Ok, that I didn’t know. If that’s the case, that sucks, big time. Did you open a PMR about that? I need to check that with 8.5.2 and the new API call that lets you run on an in-memory document.

19

Subject: Yet everyone seems to say it’s 97% possibile, not 97% IMpossible…

  • This is what I find most vexing. I could expound at length (the prior post was the highlights) about all the problems I’ve hit while vainly attempting to integrate complex legacy code into a XPage. Why doesn’t anyone say that anywhere? Instead everyone says “you can call legacy Agents but you have to save first”. That is kool-aid, plain and simple, because absolutely nothing I’ve seen about integrating my OO legacy code is as easy as saving first, then calling an Agent. Nothing.

  • And now everyone is talking about how amazing R8.5.2 will be. Whatever. R8.5.2 doesn’t exist, from a real-world development standpoint, and it won’t exist for months until the beta is done. I used to trust that if all else failed Lotus would at least maintain backward compatibility. Even if R8.5.2 fixes legacy integration, that faith is no longer there; if R8.5.2 fixes legacy integration then that’s what R8.5 should have been out of the gate, not something that wanders onto the scene over a year later, after reading all the XPages spin and being led into thinking legacy integration works now.

  • If I were a shop looking to migrate a complex enterprise-level legacy application, today, I would avoid XPages completely, unless I had the budget to scrap the entire application and redo it from the ground up. If someone would have bothered to say that months ago, before I started this nightmarish gauntlet I would have avoided XPages like anthrax.

  • Yet still I seem to be the only one against XPages and legacy migration, on R8.5.1FP2. Could be me, but in nearly thirty-five PMRs and a direct line to Lotus support, I would imagine someone would tell me I’m using their technology inappropriately by now. Instead I add to the SPR database I can’t see.

And the nightmarish gauntlet continues…

20

Subject: Just for others reading, not everyone agrees with David

David, all we/I am doing is app modernization with taking existing apps and moving them to XPages. Are their challenges? Yes. Is it worth it? 10000%.

Today, traditional notes dev is dead. It does not meet the needs of the requirements coming from the business users. They want web 2.0 apps that are self service that connect with everything. You can do that with Domino Web Dev, but we all know its takes hacks. Xpages gives us a modern development process - and can be compared head to head with .NET, Java, and alot of the web frameworks like Django and the other PHP things.

I curse Xpages on a regular basis. Far less than when I first played with them in the 8.5 beta. I do the same with VS.NET when building VSTO and C# apps. And the same on other platforms. Let’s be real - all tech sucks. period. Everything has rough edges. Things will get better with every release. And new hardware is your friend: I got a new laptop and desktop with SSD hard drive and it made a giant difference with the Notes client and Designer. DDE now loads far faster than VS.NET 2010 and saves and runs far faster.

There is plenty of use for Xpages right now. There will be more in 8.5.2 and even more after that. Just remember back to when 4.0 came out and we all struggled with LotusScript. Designer has warts all over it and agents didn’t even have run as web user. It took until 6.0.X until we were totally comfortable. Same goes here.