What filesystem type should I use for my Informix database on Linux?

The post below is a translation and repost of an article by Eric Vercelleto on his Le Village Informix Blog last August.  Here is the link to the original for my French speaking friends:

http://levillageinformix.blogspot.ca/2011/08/choix-du-type-de-file-system-sur-linux.html

This is not a literal translation, but I hope it is faithful to the Eric's meaning and intent.  Thanks to Sonny Bernier for showing the original to me and many thanks to Eric for permitting me to reprint it here and for all that he does for the Informix community in France and around the world.

Choosing the type of file system on Linux: if you have any doubt ...

Hi all,    It is under the threatening sky that I take my pen to dig into a test I have wanted to remake for some time: 
    That is the impact of the choice of the type of file system on which to implement your chunks for Informix Dynamic Server on Linux.

     As you probably know, raw devices have for some time been considered obsolete technology on most Linux distributions. It is typical of the conflict of interest between database vendors who advocate its use, and the editors of the OS and hardware manufacturers who go in the opposite direction. 

    How to implement chunks in IDS is not without effect, insofar as it directly affects the performance of input-output, and so the overall performance of the Informix server. As a reminder, it's always good to divide up the data on the maximum number of disks, and to mirror the physical devices.  That's why it is better to create the chunks on disks configured with some redundancy (ex: RAID 0/1, RAID 5, journaled file system, etc. ..), but there are costs in terms performance associated with doing so.

     I hold, that it is obvious that there is no reason to use redundant storage for your data dbspaces unless your databases are not logged or if you do not back up your logical logs.   However, in both of these cases, certainly, you should revise your strategy unless the loss of a day's work or more would not cause you problems. 

    As far as the security of your data is concerned, IDS protects you very well.  In the ability to "backup" the logical logs continuously to an external device, and of course the ability to perform regular storage space backups with ontape or onbar, you can face a total crash with a high degree of confidence about the sustainability of your employment. 

    The big advantage of this system is that IDS ensures data consistency (through transactions), and the restoration of your data- after the physical loss of a server - and of completed transactions based on information kept about these same transactions through the logical logs and their backups. The actual interval between each logical log backup to an external device is important, since it will determine how much work can be lost in case of crash of all disks in the instance and the loss of transaction details since the last logical log backup. 

    In short, it is unnecessary to secure your hosts file system of your data, it does not help much, except possibly for the rootdbs and / or the dbspace that contains physical and logical logs to ensure availability (through RAID 0/1 for example) .  (Editors note: Eric holds that the server archives and logical log backups are sufficient to prevent data loss removing the need for redundant storage.  Those who know me know Eric and I disagree on this point, but on the balance of his post we are aligned. -- Art)

     So, to determine which is the best filesystem type for Informix dbspace chunks, I performed a test which consisted of the dbimport of a database of small size (1.7 Gb), but which has very complex indexing. 

   The test plan was as follows:

1. Creation of a dbspace on a linux ext4 file system (journaled), with the onconfig parameter DIRECT_IO set to 0, then dbimport the data into a non-Logged database.
2. Creation of the dbspace on a linux ext4 file system (journaled), with the onconfig parameter DIRECT_IO set to 1, then dbimport the data into a non-Logged database.
3. Creation of the dbspace on a Linux ext2 file system (not journaled), with the onconfig parameter DIRECT_IO set to 0, then dbimport the data into a non-Logged database.
4. Creation of dbspace on a Linux ext2 file system (not journaled), with the onconfig parameter DIRECT_IO set to 1, then dbimport the data into a non-Logged database.

    I can hear the question: "What is DIRECT_IO". This is an onconfig parameter that allows Informix to override the cache layer of file systems, so IDS writes and reads using the Kernel Asynchronous IO bypassing the filesystem's caching mechanism.  This offers virtually the same benefits as if one is using raw devices, namely safely writing directly to the disks, and a performance gain from using Kernel IO. This parameter is available with the Growth and Enterprise Editions (Ed: and in the new Growth Warehouse and Ultimate Warehouse Editions), however, unfortunately not :-( in Innovator-C Edition. 

    To make this short: 
Test 1: execution time =   91m 41s 
Test 2: execution time = 195m 57s 

Test 3: execution time =   54m 49s 
Test 4: execution time =   49m 54s 

    The final winner is the EXT2 file system with direct_io activated.

   The surprise was the test with ext4 with direct_io activated.

   This turns out to be a combination to be avoided. 

   Note that each test was repeated two times, taking care in between restart the Informix instance. The execution times for each test were very consistent every time. 

    You now have the elements needed to make decisions for your future implementations.See you on our station for new challenges.

Posted by Eric at Vercelletto, August 2011

    On the subject of the need or lack thereof for redundant storage, it's not so much that I think Eric is wrong.  Essentially, he is correct.  Given Informix's famous archive and logical log recovery mechanisms, it is certainly possible to completely recover from a hard crash with minimal data loss.  Where we differ is that I am to lazy to want to go through a full restore like that if there is any way that I can avoid it.  Fully redundant disk storage (ie RAID1 or RAID10) is one such way and I minimize the possibility of ANY data loss that way.  Eric, thanks for a great post and thanks for the efforts of performing the testing. 


    I did some similar testing and reported the very similar results during one of my sessions at the IIUG Conference in San Diego.  I would include EXT3 with EXT4 as being slower than EXT2 unless you turn off the data journaling leaving only meta-data journaling active.  In addition, note that EXT4 and EXT3 with journaling set to write-back mode, are unsafe (see my presentation slides for details).

Bitemporal Data - Is this the next big thing?

I know you.  You are all thinking "What the heck is that?  What's Art babbling about now?" The concept of bitemporal data deals with the temporal state of our data from two separate perspectives.  Hang in there, I've got to spew the buzz words before I can explain them.  Here is what that all means.  

Data changes over time.  But there are two separate timescales.  The first is the date/time at which a value was modified, and, as part of that, what the value was/is before and after that transition time.  The second timescale or perspective answers the question: For what range of dates/times was each version of the value valid?  Here's an example that I encountered once, in a previous lifetime:  Sales teams.  I had to build a reporting system for sales teams that could cope with multiple levels of reporting (I won't even deal here with the fact that the number of reporting levels was different from one division to another - that's a separate article).  The composition of the teams, the sales divisions each team belongs to, the sales region that owns which groups of divisions, and who manages each level all of those elements were changing periodically.

What happens when a salesperson moves from one team to another in the middle of a reporting period?  How do you account for the change if the reports have to show sales earned while a member of each team attributed to each team's manager and to the correct division.  Meanwhile the individual's sales need to all be attirbuted to that one salesperson.  To make matters worse, the team change was made on the 15th of the month retroactive to the 15th of the previous month and the reporting period is from the 1st of the month to its end.  Compounding it all, last month's reports have to be recalculated.  Lastly, what happens next year when the auditors want to know why the sales figures for the two divisions were recalculated after the fact, want to see an audit and the original figures?

Honestly, when I did this, management pooh pooh'd the most difficult parts of this whole scenario and produced simplified rules that got me off the hook to do it right.  But in general, that may not happen to you if you are hit with something similar.  So, what's the solution?  Bitemporal data versioning.  The ISO/IEC (JTC1 SC32 WG3) SQL committee recently published SQL extensions to cope with bitemporal data.  MS SQL Server, Oracle, and DB2 are each partially compliant since they have had some supporting features for a little while (not long actually) but us poor Informix zlubs got nothing.  OK end of complaint which isn't the purpose of this post.  

You can implement BTD (getting tired of typing it all out so BTD will have to do now) yourself without server support, though support makes things SO much easier, especially at query time.  Each table needs two pairs of date columns:

• First_Effective_Date
• Last_Effective_Date
• Date_Added
• Date_Invalidated

With these columns in place and the Date_Added and First_Effective_Date made part of the primary key of the table you can implement my nightmare scenario and survive the exercise.

"Wait", you say, "I get the effective dates, why the added and invalidated dates?"  Well, that implements the 'B' part of BTD.  What if on the 15th we change that salesman's team to team Victor.  Then on the 1st of next month we generate the sales report for this last month and regenerate the previous month's adjusted sales report.  Senior Management looks at the report at noon on the 2nd but accounting updated their reports at 9AM.  At 12:15 the Sales Manager calls his assistant and says: "Dude, you screwed up!  I said move John K. from Team Jones to Team Hector not Team Victor.  Go fix this and get me new reports!"

So by 9AM on the 2nd there will be three versions of the sales reports for the prior month and two versions of the reports for last month.  They all have to be reproducable for auditing purposes at any time.  That's why we need the four dates.  Yes, we could live with just one effective data and one valid date, but the queries to determine which row is the one we need to report will get VERY hinky indeed as you would have to look ahead for future effective and valid dates that might invalidate the version you are looking at.

Is this all immensely complex and immensely confusing?  You bet.  Would an implementation of the BTD SQL standard in Informix ease the pain?  Probably.  Do you  REALLY care about BTD?  Well, you should because one day, when you least expect it, BTD is going to jump out and bite you on your... ankle. ;-)  

There is a LinkedIn group called Temporal Data (which includes BTD: http://www.linkedin.com/groups?gid=3885228&trk=myg_ugrp_ovr) that you can join if you want to learn more and join in the fun.  There are several experts that post and lurk there and if you look through the posting history, you will find links to BLOG posts and other documents explaining the issues and solutions more thoroughly. 

It seems that my friends at Oninit UK have worked with i2Global to port the excellent open source CRM SugarCRM to use Informix as its repository.  It will be no surprise to us that with Informix standing behind it, SugarCRM now scales much better than it did with its default repository RDBMS.  Duh!

When is IBM going to get it that they have the best database without peer in Informix and finally get 100% behind it? 

Solution to a problem

I received an email this morning with a link to a Linked-In post that pointed to someone's BLOG.  The Blogger was proposing a solution to a problem someone had presented to him at work that he thought was rather elegant and so posted it for the consumption and edification of the Linked-In community.  The problem:  In a single SQL statement register an attendee for an event unless the attendee is already registered for an overlapping event.  The solution was, to me, overly complex and absolutely non-portable in that it takes advantage of Oracle's conditional INSERT statement, so I said to myself: "How would I solve this better and in a fairly portable fashion?"  So, presented here is my solution:

Setup:
-- Table attendee( attendee_id serial, name char(40) );
-- Table event( event_id serial, title char(80), start_time datetime year to minute, end_time datetime year to minute, location char(10) );
-- Table attendance( attendee_id int, event_id int, register_time datetime year to fraction(3), attended boolean );

Assume all tables have the appropriate RI constraints and indexes in place.

I propose that a simple INSERT TRIGGER (and yes an UPDATE trigger would also be appropriate but I leave that to the reader as an exercise) is the correct, simple, and portable solution:

CREATE PROCEDURE attendee_check() REFERENCING NEW AS neu FOR attendance;

DEFINE clash INT;

       SELECT count(*)
       INTO clash
       FROM attendee AS a, event AS e1, attendance AS ae, event AS e2
       WHERE a.attendee_id = neu.attendee_id
         AND e1.event_id = neu.event_id
         AND ae.attendee_id = a.attendee_id
     AND e2.event_id = ae.event_id
     AND e1.start_time <= e2.end_time
     AND e1.end_time >= e2.start_time
     AND e2.event_id != e1.event_id;

       IF (clash > 0) THEN
             RAISE EXCEPTION -746, 0, "Overlapping Event Rejected!";
       END IF

       RETURN;

END PROCEDURE;


CREATE TRIGGER attend_trig_i INSERT ON attendance
       REFERENCING NEW as neu
       FOR EACH ROW
       ( EXECUTE PROCEDURE attendee_check( ) WITH TRIGGER REFERENCES );



-- Attendees
insert into attendee( 0, 'Art' );
insert into attendee( 0, 'Fred' );
insert into attendee( 0, 'Barney' );

-- Events
insert into event values( 0, 'Event 1',  '2011-12-28 09:00', '2011-12-28 10:00', 'Here');
insert into event values( 0, 'Event 2',  '2011-12-28 09:00', '2011-12-28 10:00', 'There');
insert into event values( 0, 'Event 3',  '2011-12-28 09:30', '2011-12-28 10:30', 'Elsewhere');
insert into event values( 0, 'Event 4',  '2011-12-28 10:30', '2011-12-28 11:30','Elsewhen');

-- Test:
> insert into attendance (attendee_id, event_id, register_time, attended ) values ( 1, 1, current, 'f' );

1 row(s) inserted.

> insert into attendance (attendee_id, event_id, register_time, attended ) values ( 1, 4, current, 'f' );
1 row(s) inserted.

> insert into attendance (attendee_id, event_id, register_time, attended ) values ( 1, 1, current, 'f' );
  268: Unique constraint (informix.attendance_pk) violated.

  100: ISAM error:  duplicate value for a record with unique key.
Error in line 1
Near character position 51
> insert into attendance (attendee_id,event_id,  register_time, attended ) values ( 1, 2, current, 'f' );

  746: Overlapping Event Rejected!
Error in line 1
Near character position 1
> insert into attendance ( attendee_id, event_id, register_time, attended ) values ( 1, 3, current, 'f' );

  746: Overlapping Event Rejected!
Error in line 1
Near character position 1
>
select * from attendance;

attendee_id    event_id signup_time             attended

          1           1 2011-12-14 10:48:11.221        f
          1           4 2011-12-14 10:49:21.361        f

2 row(s) retrieved.

>

Simple, elegant, and except for the details of the syntax of the stored procedure (and in some cases the trigger) it is portable and can be implemented in most RDBMSes, unlike the original solution.  The final advantage is that this solution will prevent overlapping registrations no matter who writes the application making the insert which is now a simple insert.  The application developer is not required to know in advance that he has to meet this requirement (as obvious as it should be ;-).

The best database for every purpose!

As a consultant I often am asked "What is the best database for everything we do?" Unfortunately, there is no easy answer for that, and I know this is not the response you all expect from me. Bear with me on this and I will explain.

The best database, and to be clear by 'database' I mean Database Management System, is not MySQL, PostGreSQL, Ingres, Oracle, Sybase, IBM DB2 branded products, IBM's Informix branded products, Voltdb, OpenSQL, Berkeley DB, Progress, Unify, MS SQL Server, or any of the literally hundreds of other RDBMS products on the market whether they are free or for-cost. No single database product can serve all of the needs of any large organization nor for most smaller organizations. That's a fact! While I think that IBM Informix Dynamic Server Ultimate Edition is the cats meow of RDBMS products, the performance, feature, and innovation leader in my book, it is not best for every purpose in every organization.

So, the question remains out there: "What is the best database for ...", what do I recommend? Because if my clients have to use a different database server product for each group of applications that fit best, or even groupings that are sometimes second best, they are still going to have to maintain and be able to manage and tune several products from several companies won't they? The list of skill sets that their DBAs will need will be very broad and I will have to hire more DBAs than I might need if I could just use one product.

I've been thinking about this for a while, and I keep coming back to the marketing presentation I made to IBM two years ago and to how appropriate that was to this question even more so than to the question of how IBM should market its database products. If my clients could get the "right" database, or close enough, for every situation from a single vendor, and if that vendor could make the skills needed to manage all of those database products they market sufficiently portable, that would reduce my clients' costs of ownership substantially making this single vendor solution even more attractive.

OK, so what product lines are out there? Oracle effectively has MySQL for very small projects and their Enterprise Class Oracle server in its various "editions". Sybase has an Enterprise class server and a mobile server. Ingres has only an Enterprise class server but not much third party software support left. Progress is a good SMB server only. Same with Unify. PostGreSQL has many nice enterprise class features but not all and many organizations won't bet the ranch on an open source product that doesn't have corporate support behind it.  Microsoft has only SQL Server in its two editions. That brings us to IBM which, as I pointed out two years ago, has a plethora or database management systems appropriate for any size organization and any task:

• C-ISAM and IMS for application level databases.
• IBM Informix Standard Edition for small business.  Clean simple OLTP performance for simple schema databases serving up to several hundred users with zero maintenance.  This is the original Informix database server and it is still a viable product for the right organization and project.
• IBM Informix OnLine for small to medium business.  Clean simple OLTP performance for simple schema databases up to several TB in size serving up to several thousand users with extremely low maintenance and outstanding reliability and uptime.  This is the RDBMS that revolutionized the industry by finally allowing one to archive a database without having to take it offline or block transactions.
• IBM Informix Dynamic Server (Innovator, Growth/Choice, and Ultimate Editions) for:
• Medium to large business
• OLTP performance and better than five-9s reliability
• Medium to large Data Warehouse
• Large Data Mart and DSS applications
• Object Relational features
• Industry leading expandability and extendability
• Industry leading uptime and reliability with near zero unplanned outage and very low planned outage requirements
• Industry leading TCO requiring 1/2 to 1/4 the DBA support of other RDBMS products
• Databases up to 128PB serving many thousands of concurrent users
• Industry leading replication and clustering technology that scales better than the competition
• Unmatched embeddability with configurable footprint
• Heterogeneous clusters of  hundreds of servers on different hardware and OS platforms without special backbones or infrastructure
• IBM DB2 Mainframe for mainframe based applications
• IBM DB2 LUW which fills most of the same niches as Informix Dynamic Server 
• Strength in 3rd party application support
• Very large data warehouses.
• Distributed server
• DB2 Pure Scale highly scalable homogeneous clustering
• IBM Informix Ultimate Warehouse Edition with the Informix Warehouse Accelerator for accelerating complex data warehouse style queries by up to 300X without losing Informix's OLTP performance and features and without requiring expensive special purpose hardware.
• Netezza provides performance even beyond Informix Ultimate Warehouse for dedicated DW applications that need ultimate speed.

So is IBM missing anything?  Yes!  It is missing a unified structure of management and maintenance tools that can cross and span the entire range of database products so that an organization's DBA skills can also span the product line.  Put that in place and two things happen:

1. I have the answer to my clients' questions and
2. IBM can approach the market with this now unified product line that no one can touch

 Instead of producing a white paper entitled "Six reasons to move from Oracle to IBM" which should really have been entitled "Six reasons to move from Oracle to IBM DB2" they can produce a white paper entitled "Sixty reasons to move from Oracle to IBM" that can highlight all of the benefits of using a unified database product line within your organization.  This makes IBM a one-stop-shop and makes my life as a consultant a whole lot easier.

IBM Finally Wakes Up

It looks like IBM took my test and passed! On June 8, 2011, IBM in cooperation with SmartGRIDNews.com, ran a webinar presentation to over 450 Energy Industry executives and technologists. The main thrust of the presentation was avoiding the pitfalls of implementing Smart Meter systems, however, more than 50% of the time was spent explaining how Informix is the only product that can handle the volume of data that smart data produces. (See: http://www.smartgridnews.com/artman/publish/Video-Webinars/). Kudos to IBM's Richard Wozniak (an old Informixer BTW) for a clear presentation.

Dumb and Dumber

This is a quiz. Just like the SATs/GREs/ACTs, first we have the story, then the questions to see if you are paying attention:

Let's say I have a product that has a feature that a large segment of the market needs. Let's add to this scenario that there is no other product that can fill the particular need as well. There are competing offerings, but none will work as well, all will cost more to purchase than I will sell mine for, and all will cost the customer more to maintain and manage than using my product. Being a poor historian I remember the bit of wisdom erroneously attributed to Ralph Waldo Emerson: "If you build a better mousetrap the world will beat a path to your door." (it was actually coined by Elbert Hubbard seven years after Emerson's death as something Mr. Emerson might have said if he had thought of it). Taking that sentiment to heart I post a note in my BLOG, announce the product on my product's Facebook Wall and LinkedIn page and wait for the sales to roll in.

When that doesn't work I hire some new sales people, move others from other products, and shuffle a successful sales leader from another product line over to head up a "SWAT Team" to go talk to all of my existing customers and try to get them to start using this peachy feature so I can generate some PR success stories about them that might prompt new customers to take a look at my product and this nifty feature.

Question 1:
The most likely outcome of this scenario is:
a. I will sell thousands of new licenses for my product in just weeks.
b. I may get a few existing customers to begin using this neat feature, but it will generate no new net licenses.
c. Nothing.

Answer: [ ]

Question 2:
The best word to describe me as a business man is:
a. Entrepreneur
b. Genius
c. Market leader.
d. Idiot

Answer: [ ]

Question 3:
The response from existing customers to my efforts to get them to use the nifty feature will likely be:
a. "Hey, this is really cool and fast and saves storage too! Thanks! When can I shoot my PR video?"
b. "Dude! We don't do that kind of stuff here, so I have no need for this nifty feature. Sorry."
c. "I know all about this feature, have for years. Don't need it. Go away!"

Answer: [ ]

AUS -versus- dostats

Do you like the idea of using the task manager to take care of update statistics tasks but prefer to use dostats than AUS (Auto Update Statistics) in 11.xx?  No problem, for several releases dostats_ng.ec has supported creating an AUS-like refresh function.

For a while now I have noticed that clients that are trying to use AUS to manage their statistics on larger databases are experiencing problems with the system catalogs and some tables having data distributions that are out-of-date or insufficiently detailed.  Many are going back to using dostats with our help.

In this entry I am going to show you how to completely replace AUS with dostats.  The steps are simple:

1. Get utils2_ak and build and install dostats (by default it is now built from the next generation dostats_ng.ec source).
1. Get utils2_ak: This one is easy.  Go to the IIUG Software Repository (www.iiug.org/software) and download the package.
2. Build the package: Usually easy, on Linux and any platform with GNU Make and ESQL/C configured to use GCC, just type 'make'.  The file BUILDING and notes in the makefile (Makefile.nognumake) explain what to do if you have to use UNIX make and the native cc compiler on various platforms.  But this is also easy, it is usually a 5 minute task to modify the makefile for your platform - don't forget to modify the makefile in myschema.d if you want to build myschema as well.  If you are on AIX, you will also have to get my portable at utility "ar2" and compile and use that to extract the source for myschema from it's System V/POSIX format ar archive file.  If you have trouble - email me.
3. Install:  There is an 'install' target in the makefile, so you just have to edit the target path into which the files will be installed and "make install".
2. Disable the AUS tasks in the task manager.
1. Easy Peasey.  You can do this in OAT or just go into dbaccess or OAT's SQL editor and run:
   UPDATE sysadmin:ph_task set tk_enable = 'f' where tk_name matches 'Auto *';
3. Install new tasks.
1. There are two tasks, the refresh and the evaluator.  The refresh is VERY easy because dostats takes care of that for you.  Configuring the evaluator is a bit trickier.  Fortunately, I've scripted the whole shebang for you.  The setup_dostats.ksh script below will take care of everything. 
2. There are three scripts below, setup_stats.ksh, evaluate_stats.ksh, and initial_stats.ksh  Just install all three scripts in your path and make two edits to customize the scripts for your environment:
1.  In setup_stats.ksh change the start_time to the time you want the refresh task to kick off and estart_time to the time you want the next day's evaluator task to kick off.  Normally you want the evaluator to run after the end of a normal day's processing cycles but before the refresh task.  The refresh task should probably run before large end-of-day processes to optimize their runs.  The evaluator takes about two minutes to process a 2000 table database.
2. Change the location of the environment setup script you use in all three scripts.
•  There is a line in each file, below the legend:
• # Environment "dot" script
• Change that next line to execute your environment file that gives access to the instance and which must also include the bin directory containing dostats and the scripts themselves in its PATH. 
3. All three scripts take two arguments, the name of the database you are scheduling, and the path to the directory into which you want log files for the runs to be written.   The script setup_stats.ksh installs a stored procedure in the target database that runs the evaluate_stats.ksh script at the appropriate time, creates a table in sysadmin, dostats_stmts, to hold the set of commands for the refresh to run, creates the SPL procedure that executes the refresh commands and installs it as a task to be run daily at the appropriate times, and creates an initial set of commands for the first run.
4. As configured the evaluator script will use whatever thresholds for aging and browsing you have already set in OAT for AUS to use.  You can modify those settings in OAT or by editing the values stored in the tables in sysadmin, or you can edit the evaluate_stats.ksh script and replace --aus-thresholds with the -a -A [days] and -b -B [pct_chng] parameters.

The third script is just for you to use to unconditionally wipe clean your existing distributions and create a complete new set using dostats.  This will "prime the pump" for the aging threshold so that every table has the same initial starting point for its stats.  You will want to run this script once initially during a quiet time on your system and again say monthly or quarterly just to get everything on an even keel again periodically.

 

##################################################################
# setup_stats.ksh - Install a procedure to generate updated data
#     distributions for the named database.
#
# Written by: Art S. Kagel, Advanced DataTools Corp.
#
##################################################################

dbs=$1

logdir=$2 

export start_time='03:00:00'

export estart_time='00:00:00'

# Environment "dot" script

. ~informix/ADTC/aliases.art

#
# Install the initial evaluator procedure and populate the dostats_stmts table with
# an initial set of commands.  Also schedules the refresh job to run at 3h
#
{
    dostats -d $dbs -Q 100 -w 10 --aus-thresholds --isolation d -m --procedure dostats_refresh_$dbs --schedule --frequency d --first-time $start_time
 2>&1

#
# Install evaluator task to run at midnight daily
#
dbaccess -e $dbs - <<EOF
create procedure evaluate_${dbs}_stats();
define cmd char(200);

let cmd = '/home/informix/ADTC/bin/evaluate_stats.ksh $dbs &';
system cmd;
end procedure;
database sysadmin;
delete from ph_task where tk_name = 'dostats_evaluator_task';
INSERT INTO ph_task( tk_id,        tk_name, tk_type, tk_group, tk_description, tk_execute,
    tk_start_time, tk_stop_time, tk_frequency, tk_Sunday, tk_Monday,
    tk_Tuesday, tk_Wednesday, tk_Thursday, tk_Friday, tk_Saturday ) VALUES
( 0, 'dostats_evaluator_${dbs}_task', 'TASK', 'DOSTATS', 'dostats run for evaluate_${dbs}_stats',
'EXECUTE PROCEDURE $dbs:evaluate_${dbs}_stats();', $estart_time, '10:00:05', ' 1 00:00:00', 't', 't', 't', 't', 't', 't','t' );

EOF
}  | tee $logdir/setup_stats.$dbs.$(date +%Y.%m.%d.%H.%M.%S).out

echo "Report also written to: $logdir/setup_stats.$dbs.$(date +%Y.%m.%d.%H.%M.%S).out"

================================================================================

#! /user/bin/ksh

##################################################################
# initial_stats.ksh - unconditionally generate a full set of data
#     distributions for the named database.
#
# Written by: Art S. Kagel, Advanced DataTools Corp.
#  

##################################################################

dbs=$1

logdir=$2

# Environment "dot" script

. ~informix/ADTC/aliases.art

{
    dostats -d $dbs -E -Q 100 -w 10 -n 100 --aus-thresholds --time-display --drop-distributions --isolation d -m  2>&1
}  | tee $logdir/initial_stats.$dbs.$(date +%Y.%m.%d.%H.%M.%S).out

echo "Report also written to: $logdir/initial_stats.$dbs.$(date +%Y.%m.%d.%H.%M.%S).out"

============================================================================

#! /user/bin/ksh

##################################################################
# evaluate_stats.ksh - Install a procedure to generate updated data
#     distributions for the named database.
#
# Written by: Art S. Kagel, Advanced DataTools Corp.
#

##################################################################

dbs=$1

logdir=$2

# Environment "dot" script

. ~informix/ADTC/aliases.art

{
    dostats -d $dbs -Q 100 -w 10 --isolation d -m --procedure dostats_refresh_$dbs  2>&1
}  | tee $logdir/evaluate_stats.$dbs.$(date +%Y.%m.%d.%H.%M.%S).out

My next new feature request

I've thought of my next feature request.  Speaking with other users, especially those in high volume near 24x7 environments, I often hear the lament that after a server restart it can take many minutes and often hours before the server reaches its normal steady state with active rows in the cache and all data dictionary entries, data distributions, stored procedures, common statements, and other cached objects loaded up.  During this ramp-up period performance is significantly reduced since most queries require a physical IO to read in the pages that are needed and possibly fill in other cache entries.

Based on this lament, my feature request is that in the background, after every checkpoint, the engine dumps out the list of cached data and index page addresses so that during a restart the engine can - optionally - read in the list and repopulate the cache.  If this goes through the normal page processing algorithms I expect that the data dictionary and distribution caches can be filled in as well.  The statements cache and stored procedure caches may have to be handled separately, though the procedure cache should be simple enough if the procnames are saved along with the page addresses at checkpoint time.  The statement cache may be the costly one since the entire statement texts will have to be saved and restored to memory, but this is also doable.  At least from an outsider's point of view.

I ran this by some of the Informix development team last night here at IOD and no one thought that this one was difficult to pull off.  The only thing we lack is a strong user case.  What do YOU think?  Is this a useful feature?

Fear the Panther

So, after much waiting and absolutely no pre-event hoopla, Panther is finally here.  You will see much in the press and in the IBM official announcements about the features of Panther (Informix Dynamic Server version 11.70) that IBM thinks are important to the market place. Jerry Keesee is calling Panther "The Last of the Big Cats" hinting that IBM will be finding another theme for the next release.

I don't want to repeat all of what you will hear from IBM, but I do want to provide my own announcement of those features that I feel will be the most important in the market place over the next 12 months.  This is a tremendous release as far as features are concerned.  Without further ado, here's my list, categorized as IBM has done theirs:

OLTP Data Security

What do I mean by OLTP Data Security?  Well, what is the greatest fear in an OLTP environment?  To me it is loss of transactional data.  If a user has completed a transaction and thinks it is secured in your database, you cannot afford to lose it.  If that transaction was a $20 ATM withdrawal, your company is on the hook for the money.  If it was a multi-million dollar electronic funds transfer, even more so.

If a user is in the midst of a transaction and your primary server crashes, do you tell the users "Sorry.  System problem.  Please try again later."  You will lose a customer, or several hundred customers.

"OK.", you ask, "How can Panther help?"  Panther has a new feature that IBM is calling "transaction survival".  I call it "uninterruptible transactions".  If your primary MACH11 server goes down any transactions begun by a user connected to any secondary server (HDR, SDS, or RSS) will not be lost but will be picked up by the surviving SDS secondary server which is promoted to be the new primary server and the transaction will continue unharmed.  There is no application change or ONCONFIG parameter needed to enable this feature and make it work.  Your existing applications don't have to be modified to reconnect to the new primary server.  This is all automatic and effort-free.  No other RDBMS server in the industry has this feature.  Oracle does have a similar feature but it's not real and it's not automatic.  You have to enable it in your application because it is actually implemented in the front-end libraries which fake survival by recording the transaction locally on the client in a flat file and replay the transaction from scratch after reconnecting to the new server.  The feature is so lame that Oracle doesn't even talk about it.

To me transaction survival is a biggie!  The biggest feature in Panther.

OLTP Performance

First, ADTC's testing shows that Panther is about 11% faster than 11.50xC7 without taking advantage of any of the performance enhancing features mentioned below.  However, Panther has several features to further improve performance of OLTP systems. 

One is the removal of the requirement for a foreign key constraint to have an index on the foreign key.  The supporting index can now be optionally disabled at constraint create time (or later) and its storage will be released.  The selectivity of these indexes on the foreign keys pointing to lookup tables with few rows is low and the utility of requiring the index has always been questionable.  The engine does not need these indexes to enforce the constraints unless cascading deletes are enabled.  You likely don't need them for searching because most systems have composite keys containing those foreign key columns that are used for searching and filtering.  It is rare in an OLTP style query for the lookup tables to be selected as the primary query tables requiring an index on the dependent table to support the join.  Removing this requirement will reduce the overhead of inserting and modifying data in tables with many code columns without having any impact on query performance.

Another one is the new Forest of Trees indexes.  This is a new indexing method that combines many of the advantages of a Hash index with those of a traditional Informix B+tree index.  For indexes on keys who's first column(s) have low selectivity but are still important to the correct processing of many OLTP queries (think geo, country, region of an index supporting a company's geographic reporting structure as an example) the b+tree indexes can become rather deep with relatively few unique elements on a level.  What Forest of Trees indexes do is to let the Database Architect specify one or more of the leading columns to be used to create a hash key.  Then a separate b+tree is created for each hash value containing only the remaining key parts following the hash columns.  This results in several flatter b+tree indexes.  You can't perform range scans on the hash columns (for that you will also need a pure b+tree index) but you can do so on the remaining columns in the index key.

Multiple index scans in the optimizer is another big win for OLTP.  Often you have a complex join between three or four tables using the filtered rows from the independent tables to act as filters on the dependent table and several filters on multiple columns in the dependent table.  The optimizer in earlier releases of IDS had to select the "best index" often resulting the engine having to read many rows of data and perform the final filters using the actual row data even though indexes on the filter columns are available.  The old XPS optimizer could use more than one index on each table in a query.  Now Panther can as well.  This can reduce the number of long key compound indexes which can also improve insert, delete, and update performance.  For databases that are delivered as part of a third party application where you can't control the schema, this one may be a HUGE win since the composite indexes that could make such queries reasonably efficient may never have existed in the first place.

Data Warehouse Performance

New Star Schema  support in the optimizer.  What does this one mean?  This is another offshoot of XPS and the multi-index support above.  In Panther, when you have many dimension tables, you can create a single compound index on all of the dimension table keys in the fact table.  The optimizer will realize this and use the filters on the dimension tables to generate a list of valid combinations of the dimension tables' keys into a temp table and join that temp table to the fact table using the compound index to quickly locate the rows that satisfy the criteria on the dimension tables.  Whoosh!  At ADTC we've tested this one quite a bit and the results are impressive.

Administration

I put the new Heterogeneous Server Grid feature under this heading because I can't think of a better one.  IBM is actually making quite a bit of noise about this one and rightfully so.  If you have lots of old hardware around, don't throw it out.  Don't sell it to a junker or to a used equipment reseller.  Configure it as an IDS grid node and add to your company server farm's net computing power at near zero cost (since the boxes are already paid for and amortized).  I'm waiting for confirmation about whether this means you can have an HDR or other MACH11 secondary that's build from different hardware or different OS.  Is suspect not, but who knows what the future may bring.  Heterogeneous grid was a big request from a number of very large Informix customers, so if Heterogeneous MACH11 members are what you need, speak up.

Zero downtime rolling upgrades.  Speaks for itself.  Jarrod down in New Zealand is jumping for joy over this one I'm sure.  He and his one assistant have to upgrade a very large number of servers during the few weeks a year that his company's servers can afford to be offline even for a few hours.  Being able to configure their server grid to be able to do this anytime to roll in a fixpack that removes a critical bug will be a god send for him and many of us out there.

The IIUG will soon be posting its new Features Survey which we will then hand to IBM to help them design the next release of Informix Dynamic Server.  If you don't think that you have a voice in how IBM determines features for Informix, think again.  Every one of the features mentioned here was requested by users like you/me/us either directly or through an expressed need that had to be filled.  Several are features that were asked for and received high marks on the previous Features Surveys.  So, when you see the Insider or email announcement of the Survey, fill it out. 


In the next release I'd like to see:

• Transaction survival when the server your session is connected to crashes!
• Bitmap indexes with multiple bitmaps used to generate a net bitmap of satisfying rows.

What would you like to see?  Comments welcome.