Cubegeek

Cognos Conference Call Notes

Guys I talk to say Cognos missed their number while Hyperion hit theirs. Here's some details of the Cognos conference call.

The Wiki Problem

I have agreed with Seth Grimes to contribute to a BI wiki. I'm not sure exactly how and when, but it's a good idea and I intend to get a lot of material out there.

The problem is that the current Wikipedia entries around OLAP surround the idea that OLAP is 'a projection of a relational database'. If you check out this entry, I am rather at a loss at how to deal with it. What I think I'll do is to start with the particulars of Essbase and move backwards to the general and then make the kinds of corrections at the high level that make sense with regard to the way this industry actually matured.

In addition I'll keep a rolling commentary going on here under this new category MDBMS. Since Essbase is evolving into something which is truly unique I think it's important to keep things open.

You see Essbase is implemented both at IBM DB2 OLAP and as Essbase. It has completely abstracted its storage model, so even in Essbase 7X as available from Hyperion, you can use a block storage or an 'aggregate storage' option. In addition there is Hybrid Analysis which, from what I understand is actually a projection onto relational (which if Essbase puts onto MySQL could be a huge hit). IBM DB2 OLAP server has bolted the Essbase system onto DB2 relational (by the guys at the Santa Theresa Labs). As far as I know this is fairly unique in the whole of the database world, much less the 'OLAP' pigeonhole.

My point is to show that Essbase is an organic fully-featured database platform, not simply an OLAP engine, nor a projection of relational tech.

The Virgin Pagoism Problem

We've decided to get the boy a cellphone for his fifth grade graduation. And in doing so demonstrate our appreciation for his maturity and tie another leash around him. And now to the Dad question. How much is this going to cost me?

Virgin Mobile has a pay-as-you-go system that looks attractive for a low priced entry. We can get him a $25 phone without a contract. Seems nice. There are two plans, but which to choose? Plan A costs 25 cents a minute for the first ten minutes a day and 10 cents a minute for the rest of the day. Plan B costs 10 cents a minute all day plus 35 cents a day.

It sounds like a fairly simple math problem, however I am at a loss to understand how to put it into a function and determine the crossover point. I know how to develop a simulation in Excel, but having done so, I still don't know how to characterize the breakpoint at which one plan cost more than the other. Virgin, in their ad says that the breakpoint is at 200 minutes per month, and they are truthful in their advertising, but I'm distrustful of the number and I want to know the scheme under which they developed the revenue model.

Here is my spreadsheet which allows you to enter a factor (the bolded number) which drives a 30 day simulation of daily calls. You can see that Plan A is cheaper if you call just a little bit, but you can also see that there are months where you could call less than 200 minutes and Plan A would still be more expensive than Plan B.
Download file

Clearly, the boy is going on Plan B to start. He'll make a mountain of calls when he first gets it and then he'll slow down. When he slows down to a trickle, we can switch to Plan A. Virgin's pricing is unique - I like it. I can also clearly see that this plan is way more affordable than that of Boost Mobile which is a straight .25/minute and .15/minute on nights and weekends.

The 'nights and weekends' pricing schedule was clearly developed around demand schedules of the first pricing revolution started by MCI in the 80s. I've got to believe that the capacity of the cell network has far outstripped that of those days. That's why Cingular has introduced 'Rollover' and other interesting pricing schemes. The simple excuse that business use during the weekdays creates a supply constraint thus prices must rise simply doesn't cut it.

Eric Schmidt of Google has suggested that the alternate content that phone carriers can charge for, like ringtones, sms, photomail, video content, gps, themes and games are so profitable that they can more than pay for voice carriage. Voice could be free just to get people on the platform of subscription services. Already Sprint has such a rebate system in place that most of the handsets are free. Clearly any digital system capable of delivering video games, has umpteen times the bandwidth required for monaural duplex voice.

Me myself, I've got the Treo 650 on Sprint PCS. I get unlimited SMS, internet, email pop client and all the Palm goodies, and 1000 minutes for about 65 bucks a month. So I'm not complaining at all considering that finally all of that works. Lots of folks have griped that Sprint has disabled some of the bluetooth and wi-fi capabilities of the Treo, but I don't mind. It's just a leash.

In the meantime, pay as little as possible for voice. Hope the spreadsheet helps. Now, what's the formula?

Apple on Intel

I just realized that on June 9th I predicted that Apple could make money on Intel. That same day, they announced that they were switching from IBM. I'm sure it was coincidental, and I'm pretty sure that I hadn't heard the news. Anyway, it's exciting to me because I've been trying to get Darwin to run on my ThinkPad forever. It don't work. I'll try again when I get VMWare installed.

Now check this out:

..Dell (the company) has for several years fearlessly—and lucratively—sold servers loaded with Linux, the operating system Microsoft reviles and dreads. And as the industry's top dog it wields more bargaining power with Microsoft than other PC-makers. So I emailed Michael Dell, now the company's chairman, and asked if he'd be interested in the Mac OS, assuming that Apple CEO Steve Jobs ever decides to license it to PC companies. (For now, Jobs says he won't.)

"If Apple decides to open the Mac OS to others, we would be happy to offer it to our customers," Dell wrote in an email. It's the first time any PC industry executive has openly shown enthusiasm for selling machines with Apple's software. Though that's all Dell would say for the record, I suspect his interest is not unknown to Jobs. So, as I said in this column last week (and in an article in the new issue of FORTUNE), the ball is in Jobs' court.

So the question is whether or not Apple can be as successful in the consumer electronics business as Sony has been in the computer business. The iPod says a resounding yes, and that's essentially going to be an annuity for Apple forever. Sorry Diamond Rio. But Jobs is going to have a tough job following up on the iPod, unless...

You see the next killer app is obvious. It's the phone. An Apple Phone that did the equivalent of the Treo 650 would be the hardware that changes everything. An iPhone that integrates all of the iLife applications and iPod storage could compete on a huge level, if the price was right. It would be a huge leap for Jobs, but so was his move into the music business. It wouldn't be a huge leap for Apple customers who have demonstrated their love for insanely great hardware.

So the move for Apple into Intel suggests to me that Jobs is ready to start specifying hardware requirements again. Doing it for a short stint into the computer world could get him to the place Sony is, with enough savvy to start branding all sorts of consumer electronics.

Think about it for a minute. We have a bunch of no-count consumer electronics companies who build products with no soul whatsoever. Circuit City & Good Guys house brands, Magnavox, Allegro, LG, JVC and a dozen others. Just anybody who makes a DVD player is a competitor for Apple if they step up. The Apple brand could be devastating.

The New OLAP Survey

The new OLAP Survey is here. I'm not sure how it differs from the old ones but I'm going to take it. Thanks to Mosha for posting to the Cubegeek portal.

The official customer invitation to fill OLAP Survey 5

We would very much welcome your participation in The OLAP 5 Survey. This is
the largest independent survey of OLAP users worldwide. The Survey will
obtain input from a large number of users to better understand their buying
decisions, the implementation cycle and the business success achieved. Both
business and technical respondents are welcome.

The OLAP Survey is strictly independent. While the OLAP vendors
assist by inviting users to participate in the Survey, the vendors do not
sponsor the survey, nor influence the questionnaire design or survey
results. As a participant, you will not only have the opportunity to ensure
your experiences are included in the analyses, but you will also receive a
summary of the results from the full survey. You will also have a chance of
winning one of ten $50 Amazon vouchers.

English version: http://www.survey.com/olap5sur2.html
German version: http://www.survey.com/olap5survey2.html

Introducing Case Stories

I pretty much decided this morning that the Cubegeek blog will be the working version for the book that I'm going to write. I've been thinking about it for half a dozen years and now I'm just going to do it. Since I've been blogging for at least 3 years, I'm very comfortable with the form and I know I'll aggregate plenty of material.

One of the ideosyncratic points of my book will be that it's meant to be something of a guide for apprentices and masters alike, but angled towards practitioners, not theorists. So when I start talking about 'Case Studies' I'm going to use narrative, not academic jargon. I'll call them 'Case Stories'.

The difference between a Case Story and a Case Study is that the story is portable enough to become something of an urban legend. In computing this is a good thing because we are about building knowledge in the field and working with knowledge in the field. Probably the most famous story in our field is the legend about 'beer and diapers'. I'll not repeat it here. The point is not in the details of the story, but in the fact that it communicates well the concept of correlations in retailing.

What it important to know is that these kind of conceptual breakthroughs are possible - that people in different industries are trying to use computers to help them solve problems that are far from obvious. It is not important that one understands correlation algorithms. I'm trying to teach things that cannot be googled, you see. You can find out on your own time the extent to which the originating beer and diapers story is or is not true or applicable, but you cannot deny the force and impact of the story. This is the difference between a study and a story.

As you build applications in the field, you will be forced to study. You will find details and results that will only make sense in the context of your study and you will write code to manage and nail down that problem. But above all that work will be a tale, a story, that builds your reperotoire of understanding. This is what helps you associate the particulars of your geekdom with the practical forces of the market and the industry. That's where we're going.

Every solution has to be sold. People will live with problems their entire lives until some charismatic character convinces them to change their life. This is certainly true of computer systems. You can be that charismatic character if you can pull a narrative out of your bag of tricks. Case Stories are a means to that end.

Beating the Monolith

A story I like to tell was about my visit several years ago to State Farm in LA. The guy we met with was a real hot shot. He had a very good handle on his data, and had presided over the creation of about 37 dimensions of analytic data using FOCUS. He was ready for OLAP.

The problem was that he wanted to put everything in one cube. It took about 2 hours of intense discussion until we made a breakthrough. It went something like this: We wrote all of the dimensions he had on the board and talked about their meanings and hierarchies. Like most insurance companies, State Farm had about 12 dimensions of customer demographics, and were looking to add more. I figured out that our guy was thinking of his dimensions in clusters and his measures too. This was my first clue. So I asked him why he needed 12 demographic dimensions in all his models. "Because it's OLAP!", he said. Wasn't that the point?

As he started talking about the dimensions of Housing Type we came across an interesting attribute. It was 'Distance to a Fire Hydrant'. What? It turns out that when it comes writing lines of fire insurance, this is important to know. Breakthrough. I asked if it makes a difference whether the homeowner is male of female. Nope. Smoker or non-smoker? Yep. But the most important dimension, as far as the fire analysts were concerned, was type of roofing. It doesn't matter who you are, if your roof is made of straw, your fire insurance premium is going huff and puff and blow you away. So we ended up chopping down the number of dimensions of interest down to a reasonable size, and therefore had a cube design that wouldn't break the bank.

I've long argued against the human cognitive ability above 9 dimensions, but I think that subject matter experts (if no one else) can manage that many variables in their heads. The point of augmentation is to get the computer to help us manage more information than we normally would do, so I'm not so tough on that score any longer. Especially since I now know tools that can handle that many with ease.

Yet we designers still have to fight the tendancy for people to expect a be-all end-all cube. The way projects are managed contribute to this. Here are some helpful tips.

First off, get used to the idea of having purpose-built cubes. If you think one cube is going to serve everyone, you're going to be disappointed. You want a system that enables you to deliver everyone their own garden in the cube farm.

Secondly, get used to putting up or shutting up when it comes to hardware. It just may be the curse of Essbase, but running it on old beat up or shared hardware is not acceptable. A new colleague was raving this morning about Teradata performance. He said, all you have to do is snap in a new node and anything you need, you get. A Teradata node is 1 million dollars worth of hardware. You don't even talk to Teradata until you're ready for that level. In many respects that is what Essbase demands - first class hardware. Not a million dollars worth mind you, but any time I see Essbase on a server smaller than the ones in my house...

Thirdly, people have to stop assuming that Essbase can be maintained on a part-time basis. I've always said that Essbase is the Porsche of its class, and yes it's true that anyone with a driver's license is capable of driving a Porsche... off a cliff. Hyperion has done a good job in upgrading the look and feel of EAS such that it is appropriately intimidating to newbies, and it expresses the complexity inherent. That complexity needs to be managed by people dedicated to the database technology.

These three hurdles point, inevitably to the creation, development, production and retirement of databases. Not just putting everything into 'Hyperion'.

What requires a bit more understanding is designing for Analysis, which is a discipline most database architects have not studied. Eventually I am going to write the book. Cubegeek readers will be the first to know.

Thus endeth the lesson.

Essbase V5 Tuning Top Ten

(From the Archives, March 1998)

The following are simple methods that will increase performance under Version 5. None of these involve partitioning which can add much greater performance but require extensive administration.

1. Dynamic Calc
2. Block Density
3. Cache Settings
4. Intelligent Calc
5. Paging & Memory Limits
6. Dense / Sparse
7. Calc Script Efficiency
8. Server Contention
9. RAID 5
10. Load & Calc Method

Essbase Capacity Planning

This entry is a pointer to a couple capacity planning documents. The first one I created was back in 1997, which I updated last in 1999.

The 1999 Capacity Planning Document is embedded in the extended entry. (Format sucks)

The updated version will be linked here.

Open Source BI

The other day, my son asked me if I would buy a Corvette for $200. Of course not, was my reply. Who in their right mind would want to buy such a headache? Such is the kneejerk reaction of most sensible people. You get what you pay for, right? So when it comes to open source software, that has been my reaction as well. But the reasons are a bit more complicated than simply price. Especially when you're talking BI and data warehousing.

Seth Grimes is a man I've always listened to but never met. I hope that he has a blog somewhere. In the meantime, I catch bits and pieces of his mind in articles like this. Today he's talking Open Source.

A trio of vendors is preparing to release open-source reporting tools, looking for profits by selling support and enhanced versions of free, open-source software. The availability of open-source tools will bring welcome competition and price pressure to a market that has long been dominated by proprietary software such as Business Object's Crystal Reports and Cognos's ReportNet. The three would-be market disrupters, business intelligence (BI) veteran Actuate and start-ups JasperSoft and Pentaho, want to emulate MySQL's open-source database-market success by offering low entry costs and products with developer appeal.

Actuate has a good opportunity here because of their market name recognition, but if I'd have to bet, my money would be on Pentaho. The reason? I like their team - OLAP veterans who got bashed around by the market but know their stuff. They already have the vision about the full suite of products based on their knowledge of what industry customers are already buying. That will be their great advantage, because when it comes to cost, these products will only save money if they can snap into place in current architectures.

The other vendors are going to come in sideways under the Java wire, and I don't think it's going to work. Show me a Websphere shop that's going to accept ear files from just anyone. Of course, such things may change, but fitting into existing BI architectures is the strongest strategy.

The other thing that the Pentaho folks have got going for them is the strength of their engineering guy who was a principal at Appsource whose Wired for OLAP was acquired by Hyperion to become their Analyzer product. It was a slick and highly intuitive and functional product from day one, and I don't doubt Pentaho's ability to do it again from scratch.

The strike against all of these open source vendors will be the light experience base of the consultants who would be apps builders. As much as we would all like to have the most hip Java goodies on new promising platforms like Eclipse, the fact of the matter is that newer platforms means newer staff, and newer staffs just don't know their way around DW and OLAP projects.

We should also never underestimate the power of sales, marketing and partnerships done right. That challenge will be a large burden for the new kids on the block to overcome, but I wouldn't be surprised to see them capture, over the long haul, a good 10-15% of the market. There will be no leapfrogging however. Small vendors that die off from the inevitable consolidation of marketshare of the top tier players will provide the first customers to switch over as BI becomes commodified. But that leaves the big boys way ahead.

This is ultimately a battle for new customers and the mid-market. It will be interesting to watch.