Cubegeek

Choices Choices

Python vs. Ruby: A Battle to The Death from Gary Bernhardt on Vimeo.

The Basics

This weekend I picked up a copy of Kernighan & Ritchie's Second Edition, remembering that I actually never read the first, although I should have. In my twisted career, I was the guy who was helping my friends study for the LSAT or doing their COBOL finals, but I never got to see the Sun workstation under my nose. That was college. And then I remember selling my AT&T/Olivetti machine to my buddy before I could get a C compiler. And then I mastered a few 4GLs and nobody was doing UNIX in the business world. So through a bizarre series of events and coincidences, I have never had a C compiler of my own, and thus no C programming experience. That changes now.

On first glance, the XCode compiler is massive yet deceptively simple. I remember when you stuck directives in source code to link different memory models for the build. Here there are completely different frameworks under which code is developed. For the short term, I'm working with the CLI model and going through K&R.

My goal is to be somewhat accomplished in a few years. I think that's achievable, and I'll be posting notes here along the way. I expect that my discipline for working in tiny cramped computing spaces for the past n years will make me a very efficient programmer, like a Russian. We'll see. Reading up on the iPhone developer's cookbook from Addison Wesley, I've been told that I have to work within 20MB of RAM. For me, given the state of the industry when I last did serious programming, that's a whole lot of space. I should have a ball.

I'm moving from being a data architect guy towards developing some API competency, and I have a feeling that I'll enjoy doing it in C more than in Java. There is nothing more aggravating to me than the stream of gibberish that ensues when a Java program crashes, but I may learn to deal with that too. 

Open Source Property Implications - Part 1

I've begun reading 'The Success of Open Source' as I try to get my head around the idea, or the fact, that open source databases have leapfrogged enterprise engineered databases in many important ways. It's a disorienting feeling I have, so I need to reconcile it with reality. That's the thrust, but I think there are important implications as well.

If there is one thing I've always liked about my industry and also hated it is the way in which programmers have been organized to work and create wealth has managed to be casual. I thought about this today while looking for a bathroom that was not being serviced in the spanking new building where I'm working. And so I'm curious to know some of the underlying principles, if there are any.

You see the building I'm currently working in has been architected for collaboration. Nobody owns a desk, anybody can use any 'cube' which is more like an open plan workstation, to dock their laptop and do whatever it is they do. There are lockers disbursed through the building so that if you have stuff, that you can park it securely as your itinerant work takes you wherever in the building. There are small private rooms with phones and glass doors so that you can do a conference call. Some are as small as a large telephone booth, others can seat three or four folks.

Now the unwritten rule of this architecture has to be written, and you will find spontaneously printed notes all around the joint reminding employees that camping is not allowed. In other words, it defies the spirit of the collaborative environment to claim one of these small private rooms for yourself. But enough people have camped for the signs to become necessary. I even noticed one printed in large red letters in the elavator this morning.

This environment comes complete, as one would expect in Silicon Valley, with an area with beanbag chairs a ping pong table and subsidized soft drink machines. It's hardly what one would think of as panoptic. Yet despite its thoroughly contemporary style you'll still find a catering truck pulling up to to the front of the building every day at lunch. I've been to a large number of companies in my time, and I would argue that this particular campus is on the more advanced edge of the IT high tech sort. In the building I visited yesterday, there are no corner offices. In the corners are conference rooms. Everything about the place says, young, smart, tansformative & green. Yes green. Outside the front door of my building is a solar powered trash can. Don't ask.

Is propriety at risk here? Yes, a little bit, but I cannot tell to what extent. My immediate reaction to the no camping sign is split. One could easily be chagrined that old habits die hard. On the other hand re-engineering is not necessarily quality improvement. To work in a building that encourages collaboration and play is a hallmark of the new style of business organization you find in Silicon Valley. But is hierarchical organization necessarily bad? Is the desire to be territorial wrong? Is it necessarily a good thing to have a rumpus room in a corporation?  I think it all depends at root on one's understanding of the ways and means people are organized around property and how that is perceived by workers and their management.

For example. If I understand the open source movement correctly, it produces a high quality product because the inner workings of products are subject to arbitrary and massive scrutiny. In architecting a building where people work around that same principle, you don't own your office because your work should be subject to arbitrary and massive scrutiny. Camping implies propriety, of keeping to yourself, of privacy.

--
The downside of proprietary engineering development that I am intimately familiar with originated with my experience at Xerox. Xerox fumbled the future because it slavishly attended to its customers - customers who were not future-oriented. Likewise when you make your money designing to spec in a bespoke manner for a limited set of customers you are likely to satisfy them very well but you will be inefficient for the masses. Open source takes advantage of the masses and a new arrangement between customer and provider. So I am aware how propriety inspires loyalty and trust and how loyalty and trust can be the enemies of innovation.

There are great implications about this new kind of property which is inherent in the open source movement, but the manifestations of the broader social implications are just beginning to appear.

Double DBA Insider

This week I got a call from one of my colleagues on the East Coast. Apparently some Federal agency needs a bit of assistance and is kicking off an interesting project. I actually like the East Coast but am working to build up a few things in my backyard. He asked me of my particular expertise, as we hadn't yet worked together and I filled him in. It wasn't until I got off the phone that I realized something. Now I'm an insider.

One of the primary reasons I have not expanded my relational DBA and architecture skills is because despite the fact that I have the background and basic capability, I have never been given the keys. I can recall numerous occasions in which people on projects would be stuck and I would be finished - 'and if we could only clone you...' (and get me for a lower rate) everything would be cool. I would twiddle my thumbs waiting for them to find the rare and wiley Informatica expert; when they finally got them on board, I would tell the guy what to build. I would have a logical model of exactly what I needed the feed from the data warehouse to be for my ODS and staging areas and I would twiddle waiting for their DBA to materialize and then materialize some views.

And so for a guy with as many years as I have in RDBMS, considering I did my first subselects and star schemas in 1991 on DB2, you'd think I'd know a lot more about VLDB, partitioning and various other fine points of physical database design. But I don't. But now I'm an insider, and I can pick all that up again and then some because the guys in our shop are original gangsters of Oracle. They knew Scott when he was a tiger (and there's some old school inside Oracle for you).

Life is interesting.

Snoop Poddy Pod

For Schneier's Movie Plot Contest, my first entry.

A recent study at the University of Wisconsin shows that 75% of minor children with iPods listen to inappropriate music. What's on your child's iPod? Think you know? Think again. Teens can easily alter the names of the songs on their iPod. Now thanks to a revolutionary new product, Snoop Poddy Pod, you can know exactly what kinds of messages your child is subjected to. Our Snoop Pod database analyzes the digital signature of thousands of offensive songs, updated daily, and checks to see which of those are synced to your child's iTunes. Snoop Pod parents are sent a weekly confidential email with the real scoop. Take back control with Snoop Poddy Pod.

Reversal of Fortune

It's official. I'm moving back to the Essbase world.

It has been about six weeks now since I have moved out of my old digs where I was VP of Services for a Microsoft partner. It was an invigorating experience, rather like a shower that ran uncontrollably hot and cold, alternately luxurious and chilling. I'm sure that I learned a few things, but I haven't bother much to decontexualize them. As folks have been speaking to me recently about my change, it has been difficult to express what has been good or bad without speaking out of school. Nevertheless I did have a good analogy that I used a bit while I was employed there regarding my disposition.

Half of my job was gardener, the other half of my job was catcher. So I wore two gloves, one for tending to flowers and fertilizing, the other for catching fastballs, curveballs and the occasional spitball. Occasionally I would use the wrong glove for the wrong task and a flower would get clobbered. More often it was the other way around and I would get seriously burned by the pepper. Builds character. Of course I wish all of my former employees and colleagues all the best, except for one, and that's all there is to say about that.

Today I'm completely decompressed and back to running on slightly modified versions of my old anti-Microsoft bias and gripes. There's a part of me that cannot wait to get back into Unix, and especially on the Macintosh platform. There's another part of me that cannot wait to get back to big hunkin' enterprise customers and their money and sophistication, and there's another part of me that is almost ready to get back to heads down, hands on work, but not really.

As it turns out I'll be doing that hands on in the short term and I'll be working on front-ends for a change. The thing that has really got my attention is, as I've said, multitouch and new visual paradigms for business intelligence. I've decided that 'glasstop' I will now call 'tabletop' computing and that's what I'll be working on as I come up with some theories and product ideas. They will grow to technology and technique ideas as I get my hands on a Mac later this year and start learning UI programming. This very simple application by CNN of Perceptive Pixel gives you an idea of what I think will be irresistible to BI customers in the next generation.

Immediately, however, I'm going to be working with Tim Tow of Applied OLAP on his new Dodeca product, which is a soup to nuts .NET smart client whose features I helped brainstorm a couple years ago.   It's a comprehensive development environment and the most sophisticated thing you can put on top of Essbase. Tim and his team are world class and I'm really proud to be working with him. In fact, I just bought 3/4 Terabyte drive just to celebrate the data mongering possibilities I will start to exploit as I get back into serious Essbase.

I have to say that in my career so far, the project that helped generate some of the Dodeca features was my most extreme and extraordinary. It was conceived to replace the cost estimating system for a global 100 company that allowed 17,000 Essbase models to be living all at once. That's was not centrally maintained and driven by a DW, that was with 5,000 cube modelers, with dimension building access who were generating their own data. It's the one that got me up with the HP folks in Cupertino running Essbase on the big 64bit Superdome where I did some screaming performance stuff with parallel java servers and the whole nine yards. When I tell you this front end is the greatest harness ever for Essbase, I kid you not. From what I've seen and know about its capabilities, it's everything that Alphablox and Arcplan promised to be, but better. Most importantly, It allows you to build custom workflows, custom forms and custom distributions of all that with more security granularity than any other front-end to Essbase. Again, all in a secure .NET framework. The guys at Applied OLAP have rethunk client distribution for the BI world, and everybody who is doing anything at all in Excel and VB really needs to check it out.

Only Dodeca enables this kind of Essbase farming. There's nothing else anywhere that scales as an application framework for Essbase. Did I say that it does distributed master data management too? Yeah. That's part of it. The really fabulous idea that makes this even more interesting is something of a future, but you know me, that's where my head goes. The fabulous idea is that you can begin to think about the move from unstructured data to multidimensional structured data because Dodeca is going to make it easier to do so on a different scale than people are accustomed to. But that's all I'm going to say for the moment.

It's wonderful to be as old as I am and still get excited about stuff.

ADP Screws the Pooch

Back in the summer of 1988, I went on vacation to Mexico with the woman who, unbeknown to me at the time, would eventually become the Spousal Unit. We hung out with a lot of folks from Texas, which was something of a surprise. it was my first vacation to Puerto Vallarta, and I had some strange notion that only Californians went anywhere in Mexico besides Cancun.  Well, one of the couples we hung out with was in some business whose details I forget. But our better halves both complained that it seemed like pulling teeth to get us away from work for a holiday. He explained that the toughest thing about being lead dog in a company was going to bed worrying if you were going to be able to meet payroll. I never forgot that.

Fast forward to a couple months ago. My company did payroll through Wells Fargo. Somehow Wells got involved with Intuit and then they did payroll. We decided to switch to ADP - the experts. Or so it seemed. Last pay period, they engaged when they weren't supposed to - so all of our employees got their paycheck doubled. This month was supposed to be their first payroll. Guess what? Nothing. All of their systems are down today, and none of the direct deposits went through.

We have been told that this doesn't just affect our company, but everybody that ADP services. Lovely. Here's to you ADP. Get it right please.

Google BI

The pieces are coming together and I'm preparing to move away from my current technology set. I'm going to make a blind jump into the future and prepare myself for that thing which will be Google BI. The first step is a fairly good understanding of Bigtable and figuring out how to apply what I know vis a vis best practices and apply it to that technology. I will chronicle that here and try to become a better programmer.

Stay tuned.

Cubegeek Wiki is UP

The Cubegeek Wiki is now operational and ready to grow. Ideas and contributions are welcome.

Inside Dirt: ASO vs MSAS

The Cubegeek has gotten some inside scoops that are big news for Essbase fans. The first is open information which I have no compunction from sharing, but it is significant when you think about it.

Item the First: Hyperion & MDX
George Spofford, who is widely acknowleged as the Joe Celko of MDX, has been hired by Hyperion. In that capacity, he is responsible (in some way) for the commitment to MDX in the Essbase product group. It is thus very likely, especially given Hyperion's past participation in open OLAP standards (APB-1, XMLA), that Essbase is going to implement the full set of MDX functions.

Spofford's new book is entitled 'MDX Solutions : with Microsoft SQL Server Analysis Services 2005 and Hyperion Essbase'.

Item the Second: ProClarity & Essbase
I have it on very reliable information that ProClarity is very seriously looking at adding Essbase support to their product line. This is fabulous news to me, having been certified back in 2003. ProClarity guys are top dogs in MDX programming. Back in 2003, they were acknowledged as the people who had made the best (and only) wizards that generated optimized MDX. More properly said, they were the company, along with Panorama, that debugged the Microsoft API for them. I have a good feeling that they're not going to have that kind of headache when dealing with Essbase. I hope Miss B. at Hyperion goes after them so that they feel welcome into the Essbase Ready family.

There have been many times when I would have loved to stick Essbase on the back-end of ProClarity, especially when it comes to iterative development of new KPIs. ProClarity is a sweet front-end and its server layer is very good about handling things other than query stuff - like documents and spreadsheets. It's a low-maintenance middle tier and handles books of reports very well. Anyway, I'll keep my ear to the ground on this one.

Item the Third: Performance
A document exists that shows some performance benchmarks of Essbase ASO vs MSAS. I've seen this document with my own eyes, and it has names on it of real people - some of whom I know personally. Now this document is older than 6 months, and who knows what has changed since then. But the bottom line is clear. ASO kicked MSAS butt.

The notable notes that I took from this was that unoptimized MSAS is fairly crappy. Who knows where it is now, but even after MSAS was optimized ASO queries processed 40 times faster. There were, as always, qualifications on this. When the Hyperion guys say that sub-second queries are the only acceptable queries, parsing 'better' goes beneath the radar of ordinary mortals. However when it comes to benchmarking, this is computer *science* after all, not computer politics.

Additionally, the results showed that MSAS used a lot less disk than the equivalent ASO models. On the other hand the results showed that ASO scaled much better for concurrent queries. So on the whole, for little applications with few users, it's a push which makes cheap (free) MSAS the winner. When you scale up, ASO wins. That means you have to define 'scalability'.

The great thing however is that the what most people will want from ASO is that it does what Essbase used to be unable to do.