Cubegeek

Essbase Grids in 11?

One of the crazy thoughts that I had last week was that about grid computing with Essbase. Somewhere among the scores of RSS feeds I consume, somebody said something to the effect that Google has the entire internet in memory. Huh? What? Hmm.

How would I go about meeting the Google challenge for Essbase? Well, it's quite obvious that way down in the internals level, there are very cool memory models that do smart paging from I/O that almost nobody ever uses. That's because nobody has, in the implementation world anyway, scoped out the efficiency of Essbase's caching system. Which is to say precisely this. I was instructed that Essbase guesses the proximity of one query to the next and sometimes pages in some guestimate blocks into virtual memory from disk and then flushes them back smartly after some time. This process can be tweaked by setting commit points etc. There's got to be somebody on this planet with a development tool that visualizes the efficiency of virtual memory models. They have to had pointed such a tool at Essbase at some point in its development. Boy would I like to get my hands on that.

So what if I had enough memory to handle an entire cube? OK that's a no brainer, just adjust the size of the cache to an obscenely large size and make sure nothing gets flushed so often. But what about huge cubes or a set of them? In that case, we already have transparent partitions and distributed OLAP. In other words, you can put a distributed cube across multiple Essbase servers and keep pieces of them 'in the air' so to speak - transparent partitions fully in memory. The only question is whether or not it's really efficient to get across n partitions on n servers, each with sufficient memory to keep the entire partition in fast RAM.

It turns out that the Essbase team has looking very specifically at enhancements to transparent partitions in the 11 release. This, ladies and gentlemen, actually gets my blood up. I don't know if this kind of gridding is what they had in mind, but I sure as hell want to try it out.

Imagine if you will, a nice 9 dimensional cube about 40GB large staged across four 64bit machines each with 12GB of RAM. You could easily get a queryable section of that distributed nicely. I mean even with that distributed services software you could have done that and query balanced stuff with disk redundancy, but what if you wanted to go super fast on calcs or queries and just keep the whole thing in RAM? Push a few levers and boom, the whole thing is 100% in RAM.

Now in order to go to the next level, you'd want to look at some automatic partitioning strategy, maybe even dynamic sized partitions. I've got to believe that the ASO engineers have looked very precisely at this problem because they know more ways to slice up multidimensional data than we mere mortals ever dream. And once again, in 11 it appears that they have looked very closely at how to get BSO partitions and ASO partitions on better speaking terms. Sounds delicious to me.

So one of these days, I'm going to have some lab time. Bwaahhahahah!

Cubing with OBIEE

I just got back from a short tour in Northern California meeting with a score of Oracle reps who sell in California. I am on the road to OBIEE and OBIEE+ integration with Essbase and discovering some of the painpoints with regard to understanding this integration and selling the tools into established customers, new customers and customers with competing technologies. It was very enlightening.

One of the specific pain points that I picked up on, is determining when to transition Planning users into full-use Essbase and/or getting OBIEE users to decide to use Essbase at all. I specifically recall the term 'cubing' used. I kind of like it because it sounds like a buzzword with some traction out there. I heard it in the context that major customers who have OBIEE are already 'cubing' with Essbase, meaning making Essbase the data source for their OBIEE reports rather than making OBIEE go directly against relational and other sources.

Today I want to talk about this video which draws attention to three flaws with generic OBIEE implementations.

The salient points are clear.

Use a data warehouse.

Don't depend on caching.

Don't use too many case statements in your queries.

I can only say that the first one was a surprise, the other two are cascading surprises. If, in order to use OBIEE and get it performing you need to build a DW of any stripe, then hallelujia days for Essbase are not far away, even though Essbase is not as simple as it used to be. Building on Essbase eliminates all of the limitations of watching your queries and building aggregate tables and all that carefulness one needs with stars and snowflakes. Plus using Essbase makes maintenance of those hierarchies very simple and query times consistent, no matter how complex or unbalanced they get. (I have yet to know if OBIEE handles unbalanced parent-child hierarchies well, for example).

I have been assuming, but only loosely, that OBIEE like Discoverer, depends to some extent on materialized views that are known by Oracle Apps specialists and that these are tidy enough to support some realtime querying without the need for exporting to a DW type architecture. If that is the case for turnkey stuff, but not for what demanding customers demand, then the technical case for 'cubing' with Essbase can be quite easily be made.

The clues given by the speaker, that one has to be smart in ETL design suggests to me that the technical case is indeed strong. And from what I've heard from some reps, the economic case with Essbase can be more flexible than it ever was under Arbor or Hyperion.

What is clear to me is that if one desires to enable EPM, then the key differentiator in using Essbase in the Oracle BI stack is its capability to handle write-back and it's capacity for managing very specific security requirements. These are things you cannot do with any ease in under star and snowflake designs. That matters whether or not complex KPIs are in play.

That's what I've got so far.

On The Trail of OBIEE

I'm officially on the trail of OBIEE and figuring out its strengths and weaknesses. This is going to be fun. I'm meeting with a bunch of Oracle guys in the upcoming weeks and, who knows, I might even bump elephants. (That's a term from the Warren Buffett biography I'm reading from some meeting of the big dogs in Sun Valley).

One of the elephants is clearly Mark Rittman. The more I get close to finding out what OBIEE is and how it works, the more his name appears. So today I just ingested this piece of his over on the Oracle network. Basically he shows, for OBIEE folks, how to build an Essbase cube. It's a very good introduction to how to think of Essbase if you come strictly from the relational world, or if you're new to BI.

It's kind of astonishing to me how small the overlapping sets are of relational experts and multidimensional experts. Funny, I remember thinking to myself, in my 8th year of the profession how I'd like to start a company called 3DB with relational, multidimensional and object oriented database guys. Come to think of it, I started way back with hierarchical databases, so it shows you how old I am. But the idea of 3DB was that we could conceive of any model of reality and stick it into the most efficient place. Now these webby mash-up guys are saying even database technology is passe. Well, we'll see about that. In the meantime, there is still some great advantage to be had if one truly understands best practices in relational and multidimensional. The OBIEE stuff is all about what it looks like on the front end.

Speaking of which, I hope I get to use my iTouch client of Oracle Indicators with all this. I'll be a walking advert, just like I was for the Palm and AvantGo a decade ago. (sigh).

Anyway, that's the update. I should be signed into a server this week and get some hands-on. In the meantime, I will try to contain my bulldog proclivities about the superiority of Essbase especially given my involvement on the losing team of selling Essbase technology to the guys at Siebel back just before nQuire was purchased by the Siebel guys.

Discoverer At Long Last

Now that I'm with a serious and world-class Oracle consulting shop, I get to vent my bile about all of the Oracle stuff I've had to deal with over the years. There are lots of reasons to dislike Oracle. If you're IBM you dislike them because old DB2 heads with VLDB experience on mainframes want to outperform on the DW side and Oracle clutters up the joint with all their application stuff and client server stuff. If your an Essbase guy you dislike them because they're Oracle and you don't get a shot at OLAP until their star schemas fail. "If Oracle can't do it, it must not be possible".

Well I've had to deal with the latter for a while, but along the way I had plenty of success, especially when I get a chance to do a trial and showoff Essbase with the customer's own data in under a week. This was my specialty and I never lost a trial. Not coincidentally there were two products that I was always able to slam, they were nVision - the viewer to Peoplesoft data and Oracle Discoverer, which was essentially what we considered at Hyperion to be the product competitive to Cognos Powerplay and what is now Hyperion Web Analysis.

For the first time in my life, I actually got to see Oracle Discoverer, and I am pleased to say that I get it. In fact, I've been able to decode a good deal of the OBIEE product offering with a few hours' exposure to Discoverer and Daily Business Intelligence aka DBI. It is rather what I expected it to be which is a web-based version of pretty much what Hyperion does with "IR" aka Interactive Reporting, aka Hyperion Intelligence, aka Brio Query. The advantage that Discoverer has is that it is able to use a fairly vast library of materializable views which are in the bowels of every Oracle App. In other words it seems to have a massive advantage over that thing I used to know as the BO Universe.

What nobody has ever done, and probably needs to start doing like NOW, is get some Essbase heads like me, for whom MDBMS design is second nature, together with some Oracle Apps people who understand those things they call Application Responsibilities (or is is Business Areas) - basically that huge tangle of Oracle Apps tables from which a Discoverer guy would cobble together those reports that Discoverer does well. It would be brilliant. I can't tell you how long it's going to take for me to get the time to do this, but when it happens, watch out.

What Oracle has apparently done, is taken the tactic of putting together a massive pile of templateable views and tables so that super users can put together a fearsome array of simple reports into a big fat pile of web based reporting objects. In other words they have reverse-engineered by brute force the sum total of their best guess of what people want to see and made it relatively easy to get Discoverer at the metadata and thus made all their apps reportable. What nobody has done is gone at this from a multidimensional design perspective - true OLAP - and gone at the data in bigger chunks which would be applicable to a true hub and spoke datamart strategy. The result is that you have more reports than anybody could possibly want, but still a very high cost of ownership to get the very specific custom query spaces that real customers eventually want once they get over the fact that they have BI.

Obviously the solution is Essbase, and when I (or somebody like I) can get enough people in the Oracle world to see this, the disco balls will sparkle and it will be a whole new ballgame. I'm bold enough to say that all of the analytical goodness of 'Siebel Analytics' or whatever that technology is called (maybe Noetix views?) does not hold a candle to the pure power of the Essbase engine. The only advantage that it has is that, according to the above tactics, it is already in place and ready to generate queries from Heck. But my nickel says no way anything pulling from materialized views or even cached versions of those views are going to be as fast as Essbase cubes.

My gut says that as Fusion matures, Oracle is going to allow customers to have it their way. That means they will build architectural paths through Essbase as well as paths around it. Sales guys will sell what they know how to sell. But the customer who understands how to build a warehouse from the mart backwards is going to have a huge advantage if they take the Essbase path. Why? Because Essbase was talking to desktop based widgets three years ago. All the API stuff is done, debugged and proven. Little Hyperion only exploited this in two or three major applications. Big Oracle can do it everywhere.

For those of you who are new to Essbase but familiar with DW, let me try to be concise as I say Essbase is a fully functional MDBMS with multi-user concurrent read/write access with security down to the cell level. It scales. It partitions. It parallelizes. It calculates forwards and backwards. It allows input at any level of any hierarchy and I know a company (which I happen to work for) that is chock full of people who have 7plus years of experience designing and building this monster. It is quite simply THE survivor of the OLAP wars and it is now in version 11, a bona fide Oracle property.

There's one catch of course. Like Teradata VLDB experts, people who know it aren't cheap. Then again we're not stupid either. The biggest problem for a guy like me with 12 years of Essbase is that most customers don't dream big enough to keep me awake. So I'm kind of on a mission to get to the point where I get to do some interesting sized projects.

My bottom line is this, and if you think about it, it should make perfect sense. If you can go to Google Finance and get your publicly traded company's stock price and topline financials faster and easier than you can from your internal systems, you are doing something radically wrong. The news is that the BI of the future is here today - it's just kind of in various pieces in the massive Oracle product stack and in the minds of the massive Oracle sales and consulting worlds.

The fastest path is coming.