Andrew Pollack's Blog

Technology, Family, Entertainment, Politics, and Random Noise

STTN - Store more than 32k of non-rich text on a document, and let it still be visible -- even in Notes 6.5.4 and 7.0.1

By Andrew Pollack on 06/01/2006 at 01:51 PM EDT

I don't do "Show and Tell Thursday" all that often. I'm just too heads down right now. Today though, I thought I'd share a tip after having to resolve some customer issues related to a problem that can crop up in newer versions of the Notes client.

Background - What is "Summary" data?

When you save a Notes document, it cannot have more than 32k of "Summary" data on it. Summary data is all the values stored from all the fields on your forms that aren't rich text. It also includes any noteitems written to your document that don't have fields defined on the form. This includes the "Form" item, and anything you write with lotusscript, java, or even formula language using the "field itemname := " syntax. In order to be used in a view, data must be "summary" data on the document.

The Problem -

I have this form which is used to kick off a search in NCT Compliance Search. It builds a list of databases which are to be searched very deeply for all documents related to keyword. That's not like just doing a full text search and getting the top results. In this case, there may be thousands of databases and hundreds of thousands of documents to return. Compliance search writes the results to a database as a background process. Its mostly used for complying with legal requests for records.

The list of databases can be thousands of entries long. In some customer cases, its way too long to be stored as summary text. At save, I store the list as a comma delimited string in a hidden rich text field. That works fine and is easily parsed by the background java agent into an array. Its very quick. The problem is, I want to update in real-time a list on the form of the selected databases. Rich text is not good for that. The form has buttons and things that let you use file masks and wildcards to pick the databases to be searched, so its important that you be able to see the list before you submit the search.

Prior to version 6.5.4 and 7.0.1, I was able to write the data to a set of fields, taking care that none were greater than 16k, and display the values of those fields on the form. The data probably wasn't stored correctly on the document at save time, but it wasn't used anyway. Starting in these new versions, however, an error was raised complete with an unavoidable dialog box.

To try to solve the problem, I wrote a loop in the queryRecalc() event to set all those fields to 'non-summary' but that didn't help.

The Solution -

It turns out that even setting the values to non-summary didn't help because at recalc and save time, the field definitions override what you set in the scripted events, and they go back to summary fields. To get around the issue, when I write the list data out I write it to fields which have no prototype field definition on the form. Fieldvalue_1 -> Fieldvalue_N. As I create these items on the note, I set them to non-summary (notesItem.issummary=false). On the form itself, I have field prototypes set as "ComputedForDisplay" which are named differently, but show the values of the actual note items. Since the field prototypes are computed for display, their data is never saved, and thus never made summary or counted.

To clean up, in the querySave() even I have a loop which removes the noteitems I created as fieldvalue_1 -> fieldvalue_N since they are no longer needed.

There are  - loading -  comments....

My own thoughts on this are...By Richard Schwartz on 06/01/2006 at 10:26 PM EDT
Do I detect chunking going on here? ;-)
Ouch! The difference here is...By Andrew Pollack on 06/01/2006 at 10:58 PM EDT
...the use of the rich text field to store a large volume of data is
transient. Its a one way, one time use to move the data from the user
interface side to the back end agent.

The "Chunking" to which you refer was truly an evil genius hack from the very
bowels of hell, and used to store and interact with critical data.


Other Recent Stories...

  1. 05/05/2016Is the growing social-sourced economy the modern back door into socialism?Is the growing social-sourced economy the modern back door into socialism? I read a really insightful post a couple of days ago that suggested the use of social network funding sites like “Go Fund Me” and “Kickstarter” have come about and gained popularity in part because the existing economy in no longer serving its purpose for anyone who isn’t already wealthy. Have the traditional ways to get new ventures funded become closed to all but a few who aren’t already connected to them and so onerous as to make ...... 
  2. 04/20/2016Want to be whitelisted? Here are some sensible rules for web site advertisingAn increasing number of websites are now detecting when users have ad-blocking enabled, and refuse to show content unless you "whitelist" their site (disable your ad-blocking for them). I think that is a fair decision on their part, it's how they pay for the site. However, if you want me (and many others) to white list your site, there are some rules you should follow. If you violate these rules, I won't whitelist your site, I'll just find content elsewhere. 1. The total space taken up by advertisements ...... 
  3. 12/30/2015Fantastic new series on Syfy called “The Expanse” – for people who love traditional science fiction[] “The Expanse” is a new science fiction series being broadcast onthe Syfy channelthis winter. It’s closely based on a series of books by author James S. A. Corey beginning with “Leviathan Wakes”. There are 5 books in the “Expanse” series so far. If you’re a fan of the novels you’ll appreciate how closely the books are followed.TIP: The first five episodes are already available on If you’re having trouble getting into the characters and plot, use those to get up to speed.The worlds created for ...... 
  4. 10/20/2015My suggestion is to stay away from PayAnywhere(dot)com  
  5. 08/07/2015Here is one for you VMWARE gurus - particularly if you run ESXi without fancy drive arrays 
  6. 08/06/2015The Killer of Orphans (Orphan Documents) 
  7. 06/02/2015Homeopathic Marketing: Traveler on my Android is now calling itself VERSE. Allow me to translate that for the IBM Notes community... 
  8. 03/17/2015A review of British Airways Premium Economy Service – How to destroy customer goodwill all at once 
  9. 02/26/2015There's a bug in how @TextToTime() and @ToTime() process date strings related to international standards and browser settings. 
  10. 01/21/2015Delivering two new presentations at Developer Camp (EntwicklerCamp) 2015 in Germany 
Click here for more articles.....

pen icon Comment Entry
Your Name
*Your Email
* Your email address is required, but not displayed.
Your thoughts....
Remember Me  

Please wait while your document is saved.