Andrew Pollack's Blog

Technology, Family, Entertainment, Politics, and Random Noise

Small progress taking the first real steps in moving Second Signal off IBM Domino

By Andrew Pollack on 01/21/2013 at 01:08 PM EST

I've been dealing with some serious analysis-paralysis with this project. Between not having a deadline and having so many choices to look at -- with virtually no experience with any of them -- I've found it very difficult to take the next step. As a result, the site is beginning to suffer from inattention and that needs to stop. I've finally decided to break off a big chunk and get working on a back end database.

You don't know how spoiled you are working with IBM Domino as a development platform until you start trying to get all the benefits it offers in any other way. Replication, directory management, indexing, and security are all built in. The combination remains, in my opinion, the most efficient and cost effective way to build dynamic web sites. What a shame that the market just doesn't care. I've spent a lot of time looking at other platforms and I'm convinced that no shelf-ready solution is going to do things just the way I want. No matter which way I go, I'm looking at a piecemeal solution by comparison. I've decided that I need to break the decision process down further and start with with the base level of my architecture. The back end database.

With the architecture choices I've made, the first step is the back end database. Everything builds up from there no matter which specific products I pick at each layer. For the database, I'm really liking what I read about mongoDB and I plan to test that out first. If I don't go that way, I'm probably going to go with a traditional relational database even if that limits my ability to store semi-structured data. In that case I'm leaning toward PostgreSQL based on what I've read, but until I've worked with it more and played with the tools I'm still considering both MySQL and MSSQL as fall backs. I'll really only pick MSSQL if I become too frustrated with other choices. My biggest concern in picking mongoDB will be deciding that it has the stability, reliability, tool availability, and industry acceptance that I really want.

Since I have an immediate need to starting using a better back end database for some other things I'm doing, I'm going to tackle this first -- even before choosing the tools for the rest of the platform.

The next layer up in the site will be the most important. I plan to build the whole thing as a set of interfaces accessible though SOAP:XML. All access to the back end data will be through these interfaces. There will be no direct web page calls to the back end data store. In all likelihood I'll be using Java servlets under Tomcat to provide these interfaces -- though that hasn't been decided yet. This part of the architecture will incorporate the security layer as well, which is why I've chosen a direction strong on object modelling. Within this architectural layer will be three layers of object modelling. The one closest to the back end database where the reading and writing actually happens; a second layer that provides access controls; and a third layer where where application logic happens.

On top of all that comes the application platforms. This will include some native Visual Studio .NET applications for running on Windows as well as whatever frameworks make sense for the various development tools I use for mobile applications and web site interfaces. A JSON layer overlay of the SOAP:XML web services may be used here as well.

The next time I write on this subject, hopefully I'll have a lot of positive things to say about the new back end database, and can make my final decision on the middle ware layer.


There are  - loading -  comments....

re: Small progress taking the first real steps in moving Second Signal off IBM DominoBy Craig Schumann on 01/21/2013 at 07:49 PM EST
I'm intrigued about your choice around SOAP? Why not model the API layer on
REST? Since you are considering using servlets anyway, seems like added
overhead of SOAP to be something that can only get in the way. MongoDb
represents everything in JSON anyway so you could probably get pretty far with
a light-weight API serviced by Tomcat.

Not trying to change any minds, just curious.
re: Small progress taking the first real steps in moving Second Signal off IBM DominoBy Andrew Pollack on 01/21/2013 at 10:29 PM EST
Craig, good question - but SOAP:XML gives me a layer that's really nice to work
with from a tooling perspective on several platforms. Keep in mind the web
site in this case is providing an interface for the users but also for several
application specific programs that interface with the radio equipment and the
telephony system.

JSON is touchy as hell and like everything javascript, it's very easy to screw
up. I really see JSON as useful only for moving objects into javascript from
something else. In short, JSON is a semi-elegant hack, but not more. I
consider anything coming from the browser (or application client) side to be
suspect, and JSON just begs to be exploited if you don't read it carefully.
I would work from the API levelBy Stephan H. Wissel on 01/21/2013 at 08:15 PM EST
Your external API (SOAP as of your planning) is the contract that is hard to
change (and shouldn't change anyway). So I would start from there:
- define a XML Schema and a list of actions, both incoming and outgoing
- based on the action you can define a webservice both in SOAP as well as in
REST
- based on the XML Schema you can define your Java beans ( see
http://xmlbeans.apache.org/ )
- based on the XML Schema you can define your JSON (later GSON is a nice tool
to swap JavaBeans with their JSON representation)

--- you haven't coded anything yet ---

Now you implement down from the contract the persistence - and - for testing
your mock objects and up the other bits and pieces. If you realize along the
way, that you want a different DB, your schema work doesn't change. You want to
do everything in dotNet, Node, Python... they all have tools to deal with an
XML Schema and XML Data
re: I would work from the API levelBy Andrew Pollack on 01/21/2013 at 10:41 PM EST
That's all true -- but only as far as it goes. Several of the web services are
already in use, however. The remaining ones aren't, but the objects they
represent are fairly mature in the system now. They'll need fleshing out, but
that outer layer of web services is actually pretty simple in the case of my
application.

The first and most pressing need is to pick the database because in some ways
that's going to drive a lot more. On top of that mongoDB is so different from
a traditional RDB that if I go that route I have to entirely re-think my object
management layer as well as the read-write layer. That's two out of three
layers in that model.

Looking at mongoDB, I'm already seeing potential problems with it that are
sufficiently large as to make me also need a traditional RDB. I'm not sure I
want to do that. In a very short time this afternoon, I plugged in a new
PostreSQL database server and got it talking very nicely with my ASTERISK PBX,
where it's already storing all the call billing data, and connected to it from
my workstation with nice GUI managemetn client. The PostgreSQL platform also
has a reasonably elegant warm standby replication solution out of the box and
is very well accepted in the industry. Based on that, I think I'm seeing my
choice.


Other Recent Stories...

  1. 05/05/2016Is the growing social-sourced economy the modern back door into socialism?Is the growing social-sourced economy the modern back door into socialism? I read a really insightful post a couple of days ago that suggested the use of social network funding sites like “Go Fund Me” and “Kickstarter” have come about and gained popularity in part because the existing economy in no longer serving its purpose for anyone who isn’t already wealthy. Have the traditional ways to get new ventures funded become closed to all but a few who aren’t already connected to them and so onerous as to make ...... 
  2. 04/20/2016Want to be whitelisted? Here are some sensible rules for web site advertisingAn increasing number of websites are now detecting when users have ad-blocking enabled, and refuse to show content unless you "whitelist" their site (disable your ad-blocking for them). I think that is a fair decision on their part, it's how they pay for the site. However, if you want me (and many others) to white list your site, there are some rules you should follow. If you violate these rules, I won't whitelist your site, I'll just find content elsewhere. 1. The total space taken up by advertisements ...... 
  3. 12/30/2015Fantastic new series on Syfy called “The Expanse” – for people who love traditional science fiction[] “The Expanse” is a new science fiction series being broadcast onthe Syfy channelthis winter. It’s closely based on a series of books by author James S. A. Corey beginning with “Leviathan Wakes”. There are 5 books in the “Expanse” series so far. If you’re a fan of the novels you’ll appreciate how closely the books are followed.TIP: The first five episodes are already available on Syfy.com. If you’re having trouble getting into the characters and plot, use those to get up to speed.The worlds created for ...... 
  4. 10/20/2015My suggestion is to stay away from PayAnywhere(dot)com  
  5. 08/07/2015Here is one for you VMWARE gurus - particularly if you run ESXi without fancy drive arrays 
  6. 08/06/2015The Killer of Orphans (Orphan Documents) 
  7. 06/02/2015Homeopathic Marketing: Traveler on my Android is now calling itself VERSE. Allow me to translate that for the IBM Notes community... 
  8. 03/17/2015A review of British Airways Premium Economy Service – How to destroy customer goodwill all at once 
  9. 02/26/2015There's a bug in how @TextToTime() and @ToTime() process date strings related to international standards and browser settings. 
  10. 01/21/2015Delivering two new presentations at Developer Camp (EntwicklerCamp) 2015 in Germany 
Click here for more articles.....


pen icon Comment Entry
Subject
Your Name
Homepage
*Your Email
* Your email address is required, but not displayed.
 
Your thoughts....
 
Remember Me  

Please wait while your document is saved.