Andrew Pollack's Blog

Technology, Family, Entertainment, Politics, and Random Noise

Rabbits down holes

By Andrew Pollack on 05/17/2005 at 12:35 PM EDT

Sometimes you chase the rabbit too far down the hole. In my never ending quest for high performance, I built an in memory index that works very much like a sorted view column in that its very quick to find an entry, and even support partial matches -- in fact, the constructor accepts a regular expression used to split the entry into the parts for granularity of partial matches. We tend to think in terms of any letter, but in my case splitting on periods was good enough and produces a far smaller index.

As you can guess, it won't scale. Oh, sure it scales enough for anything I can currently see putting in it, but go not too much further out and you run out of heap space. So, its got to go off to disk. Now I find myself halfway through the paper process of creating a fast on disk structure to match the index, and I realize -- its just not worth it. Its a great tool, but its tech that needs to be shelved so I can finish this project. Maybe it will reappear incarnated as my own full text index or something.


There are  - loading -  comments....

Jakarta Commons?By Stephan H. Wissel on 05/18/2005 at 08:57 AM EDT
Hi Andrew,
Did you have a look at Apache's Jakarta Commons? They have all kinds of stuff
on caching, collection and things we tend to reinvent.
:-) stw
Unfortunately, the license prohibits it.By Andrew Pollack on 05/18/2005 at 07:15 PM EDT
Most of that stuff is released under GNU -- which means that if you use it in
any thing you do, anything related must also be given away. As much of what I
do is commercial, I can't use that stuff.
Typically it's Apache 2.0By Stephan H. Wissel on 05/19/2005 at 07:48 AM EDT
Hi Andrew,
on Apache most of the time you have an Apache 2.0 license. The same as in
Tomcat. So it should be possible to reuse it. Even M$ uses a .NET port of
Lucene in their desktop search.
Have a second look. E.g. the collections are Apache license. What parts were
GPL you liked?
;-) stw
I meant ALL of the commons is Apache 2.0 lic... By Stephan H. Wissel on 05/19/2005 at 07:49 AM EDT


Other Recent Stories...

  1. 03/26/2019Undestanding how OAUTH scopes will bring the concept of APPS to your Domino serverWhile a full description of OATH is way beyond what I can do in this quick blog entry, I wanted to talk a bit about how "SCOPES" interact with the already rich authorization model used by Domino. Thanks to the fantastic work by John Curtis and his team, the node.js integration with Domino is going to be getting a rich security model. What we know is that a user's authorizations will be respected through the node.js application to the Domino server -- including reader names, ACLs, Roles, and so on. The way ...... 
  2. 02/05/2019Toro Yard Equipment - Not really a premium brand as far as I am concernedDear Toro Customer Service, I arm writing about the following machine: Toro Power Max 1120 OXEModel:38654S/N:31000#### Specifically, bearing part #:63-3450 This is the part ($15 online / $25 at the local dealer) that caused me to raise my objections on-line. This piece of garbage is supposed to be a bearing. It carries the shaft which drives both stages of the auger. The shaft passes through the bearing (which is what bearings do) after the auger drive pulley as the shaft goes through the back (engine ...... 
  3. 10/08/2018Will you be at the NYC Launch Event for HCL Domino v10 -- Find me!Come find me in NYC on Wednesday at the Launch Event if you're there. I really do want to talk to ...... 
  4. 09/04/2018With two big projects on hold, I suddenly find myself very available for new short and long term projects.  
  5. 07/13/2018Who is HCL and why is it a good thing that they are now the ones behind Notes and Domino? 
  6. 03/21/2018Domino Apps on IOS is a Game Changer. Quit holding back. 
  7. 02/15/2018Andrew’s Proposed Gun Laws 
  8. 05/05/2016Is the growing social-sourced economy the modern back door into socialism? 
  9. 04/20/2016Want to be whitelisted? Here are some sensible rules for web site advertising 
  10. 12/30/2015Fantastic new series on Syfy called “The Expanse” – for people who love traditional science fiction 
Click here for more articles.....


pen icon Comment Entry
Subject
Your Name
Homepage
*Your Email
* Your email address is required, but not displayed.
 
Your thoughts....
 
Remember Me  

Please wait while your document is saved.