In Q.A. testing now is the first in a series of NCT Search 3.0 products. Compliance search differs from a regular search in a key way. It is designed not to sort results and give you the top most likely useful information -- it is designed to give you ALL the results, in total, gathered in one spot.

The reason for this is usually regulatory compliance or legal discovery response. Suppose you have 10,000 users in your company each with a mail database and then another 50 or 60 project databases and discussion forums. A lawsuit is filed and the first thing the other side asks for is any electronic document which references their side in any way. Compliance with this request is difficult, expensive, and time consuming. Do you send an email to all employees asking them to search their mail files? Can you be sure you've gotten them all?

NCT Compliance Search gives you a database template which allows you to enter the search terms and kick off a process in the background at the server. The search request form allows you to easily choose the databases by server, directory, and filename as groups and with wildcards with the ability to exclude specific databases that may meet the other criteria.

The search is multi-threaded, and the administrator of the tool sets the number of concurrent threads so as not to overwhelm the server. The search is performed on each source database and the results are placed in a "target" database. The target default is the same database that contains the search request however any database can be used. If you're searching only mail databases for example, you may want to use a blank mail file for a target.

Response threading can be maintained for context if you like. The tool has an option to automatically collect all the documents within a "thread" as defined by a parent/child relationship tree if even one of the documents in that thread is found to match the search term. When this is one, the thread is re-created with all the same relationships in the target database. This is valuable for keeping context when reviewing the results.

Of course, not all documents will look right when being shown in another database. NCT Compliance search allows you to "Render" documents as they are found as they would be if printed or if selected and "mail-forward" was chosen in the U.I. The options for this are to render none, render non-mail documents, or render all documents. You can also choose to render the documents and also copy the fields themselves so all the data is available if needed. Rendering isn't perfect, design problems within the source db can make rendering fail -- NCT Compliance Search logs the condition and copies the document. If the document cannot be rendered OR copied, the log is noted with this along with the database name and the noteid.

A log file is generated in each search which indicates among other things each database searched, the status of its full text index, the index update frequency, and the access level under which the search is granted. If documents are protected by read-access names and not available within the context the search is running, a notation is made in the logs. By release time, we plan to support launching the search under the authority of "Full Access Administrator" if the user doing the search has that right or if an administrator of the tool provides permission.

Encrypted documents are not presently handled other than being copied to the target still in their encrypted state. Our plan is to handle this in two stages. First, by noting the log files with enough information for the person performing the search to quickly identify who owns the document and thus get them decrypted if need be; and second to support some kind of protected key storage.

Because this kind of search can be resource consuming on the server, we have built in a method for stopping individual or all currently running background search processes smoothly.

So what do you think? Anyone interested in a beta?