Andrew Pollack's Blog

Technology, Family, Entertainment, Politics, and Random Noise

Announcing NCT Compliance Search

By Andrew Pollack on 11/20/2004 at 11:26 AM EST

In Q.A. testing now is the first in a series of NCT Search 3.0 products. Compliance search differs from a regular search in a key way. It is designed not to sort results and give you the top most likely useful information -- it is designed to give you ALL the results, in total, gathered in one spot.

The reason for this is usually regulatory compliance or legal discovery response. Suppose you have 10,000 users in your company each with a mail database and then another 50 or 60 project databases and discussion forums. A lawsuit is filed and the first thing the other side asks for is any electronic document which references their side in any way. Compliance with this request is difficult, expensive, and time consuming. Do you send an email to all employees asking them to search their mail files? Can you be sure you've gotten them all?

NCT Compliance Search gives you a database template which allows you to enter the search terms and kick off a process in the background at the server. The search request form allows you to easily choose the databases by server, directory, and filename as groups and with wildcards with the ability to exclude specific databases that may meet the other criteria.

The search is multi-threaded, and the administrator of the tool sets the number of concurrent threads so as not to overwhelm the server. The search is performed on each source database and the results are placed in a "target" database. The target default is the same database that contains the search request however any database can be used. If you're searching only mail databases for example, you may want to use a blank mail file for a target.

Response threading can be maintained for context if you like. The tool has an option to automatically collect all the documents within a "thread" as defined by a parent/child relationship tree if even one of the documents in that thread is found to match the search term. When this is one, the thread is re-created with all the same relationships in the target database. This is valuable for keeping context when reviewing the results.

Of course, not all documents will look right when being shown in another database. NCT Compliance search allows you to "Render" documents as they are found as they would be if printed or if selected and "mail-forward" was chosen in the U.I. The options for this are to render none, render non-mail documents, or render all documents. You can also choose to render the documents and also copy the fields themselves so all the data is available if needed. Rendering isn't perfect, design problems within the source db can make rendering fail -- NCT Compliance Search logs the condition and copies the document. If the document cannot be rendered OR copied, the log is noted with this along with the database name and the noteid.

A log file is generated in each search which indicates among other things each database searched, the status of its full text index, the index update frequency, and the access level under which the search is granted. If documents are protected by read-access names and not available within the context the search is running, a notation is made in the logs. By release time, we plan to support launching the search under the authority of "Full Access Administrator" if the user doing the search has that right or if an administrator of the tool provides permission.

Encrypted documents are not presently handled other than being copied to the target still in their encrypted state. Our plan is to handle this in two stages. First, by noting the log files with enough information for the person performing the search to quickly identify who owns the document and thus get them decrypted if need be; and second to support some kind of protected key storage.

Because this kind of search can be resource consuming on the server, we have built in a method for stopping individual or all currently running background search processes smoothly.

So what do you think? Anyone interested in a beta?

There are  - loading -  comments....

My own thoughts on this are...By David Bailey on 02/09/2005 at 04:17 PM EST
Thanks, Andrew.

I think I have a client that could use this. I'll be in touch.

Other Recent Stories...

  1. 01/26/2023Better Running VirtualBox or VMWARE Virtual Machines on Windows 10+ Forgive me, Reader, for I have sinned. I has been nearly 3 years since my last blog entry. The truth is, I haven't had much to say that was worthy of more than a basic social media post -- until today. For my current work, I was assigned a new laptop. It's a real powerhouse machine with 14 processor cores and 64 gigs of ram. It should be perfect for running my development environment in a virtual machine, but it wasn't. VirtualBox was barely starting, and no matter how many features I turned off, it could ...... 
  2. 04/04/2020How many Ventilators for the price of those tanks the Pentagon didn't even want?This goes WAY beyond Trump or Obama. This is decades of poor planning and poor use of funds. Certainly it should have been addressed in the Trump, Obama, Bush, Clinton, Bush, and Reagan administrations -- all of which were well aware of the implications of a pandemic. I want a military prepared to help us, not just hurt other people. As an American I expect that with the ridiculous funding of our military might, we are prepared for damn near everything. Not just killing people and breaking things, but ...... 
  3. 01/28/2020Copyright Troll WarningThere's a copyright troll firm that has automated reverse-image searches and goes around looking for any posted images that they can make a quick copyright claim on. This is not quite a scam because it's technically legal, but it's run very much like a scam. This company works with a few "clients" that have vast repositories of copyrighted images. The trolls do a reverse web search on those images looking for hits. When they find one on a site that looks like someone they can scare, they work it like ...... 
  4. 03/26/2019Undestanding how OAUTH scopes will bring the concept of APPS to your Domino server 
  5. 02/05/2019Toro Yard Equipment - Not really a premium brand as far as I am concerned 
  6. 10/08/2018Will you be at the NYC Launch Event for HCL Domino v10 -- Find me! 
  7. 09/04/2018With two big projects on hold, I suddenly find myself very available for new short and long term projects.  
  8. 07/13/2018Who is HCL and why is it a good thing that they are now the ones behind Notes and Domino? 
  9. 03/21/2018Domino Apps on IOS is a Game Changer. Quit holding back. 
  10. 02/15/2018Andrew’s Proposed Gun Laws 
Click here for more articles.....

pen icon Comment Entry
Your Name
*Your Email
* Your email address is required, but not displayed.
Your thoughts....
Remember Me  

Please wait while your document is saved.