Cubegeek

Pano: The New Blade

I just found this online:

Pano's solution to the unnecessary desktop software is a 100% hardware client that has no CPU, no memory, no operating system, and no software, so it requires zero maintenance and has no security risks. Combined with server-based virtualization, Pano delivers a superior Windows experience.

You might think that since there is basically nothing new that anybody needs from WinXP that it ought to be fairly easy to slim it down, virtualize it and make some custom hardware to run it faster. In fact, a lot of people thought of that before, several years ago with blade computing. Keep a monitor and a keyboard and mouse in front of you and keep your actual PC hardware back in the raised floor. It's basically the same idea, minus the virtualization, and it was a good idea back then. It's even a better idea now.

Stewie and I had a cranky agreement about the corporate fetish for the thin client. We both agree that we took a lot of smart C and C++ programmers and told them they were inferior to HTML newbs. Back in 2000, I was suffering the entire madness of software development being presented through environments as kludgy as Cold Fusion. Sure the web is exciting, but not that exciting. Besides, very few enterprises scaled up anyway. Extranets, yeah. Then again, that was throwing away the smart C and C++ (And Delphi) programmers, and the industry was cursed yet again.

Now we have dime a dozen general purpose hardware breaking its neck for one application. The browser. What could be more retarded? And for this we need dual core? Give us a break Mr. Dell. The time is now to go virtual.

It seems to me that these guys at Pano have a pretty great idea, and it's not just because I'm more and more in favor of the Cloud.

The Sea as Hash

This is a titanium semisphere in a clean welding facility. It is part of the construction of the sort of sphere required to handle the pressures of deep sea exploration. There have been, to date, according to open sources, only about a half dozen watercraft capable of diving below 20,000 feet. It's a rather astonishing fact considering how much of 'the planet' mankind has supposedly spoiled.

I got a chance to play volleyball on the beach yesterday evening and having been away from my hometown for 8 weeks, I was a bit overwhelmed by the opportunity to sit and stare at the ocean for a while. As I sat, I was singly impressed by the utter perfection of the horizon line, how flat it seems, how crisply it differentiated the pale blue sky from the deep green sea. A thought crossed my mind about the limits of mankind. If God wanted man to fly, some said, he would have given us wings. If God had not wanted us to fly, replied the engineer, he would have made gravity way stronger.

God does not want us to chop off half the moon and throw it at Mars. God does not want us to breathe chlorine instead of oxygen. But God might not mind us drinking the oceans dry. But could we do that? Every time I see commercials about the preciousness of water I merely think that water is uneconomical, not scarce. The oceans are there for the draining, if only we had the tools. Nobody is building them - nobody is even thinking about building them, that we know.

It has been years since I've last seen anything about the Cousteaus and decades since I last heard tell of Alvin, grandson of the Trieste bathyscaphe. There has been a bit of drama around this or that Russian sub drowning in the Barents, a sea we never heard of before and thought nothing of since. But there isn't much imagination over here about the depths of the ocean, not really.

So it occurs to me that those depths might very well be the safest places to store this or that on the planet. I wouldn't suggest nuclear waste or anything of that nature, but if you really wanted to put something away for a while, it wouldn't be too difficult to put it inside a cast-iron ball and drop it into the Atlantic, or the North Sea for that matter. You might know exactly where it is - hide it in plain sight, but it would take quite an effort to get it back. Dropping things into the deep is rather like a very cool and cheap one-way algorithm.

It Seemed Like a Good Idea At the Time

There's something about soccer that best captures the spirit of HD television. But I have to tell you that a lot of sports is looking really good this season. I've been watching and recording the HD broadcasts on Universal HD and they are spectacular. Until I speed them up, and then I realize I don't know what I'm watching.

For a moment this morning I had an insight about machine intelligence from the Culture. I was fast forwarding through a bunch of commercials and then through some of the soccer game, and I realized that I was catching a bit more information on HD than I used to on the standard definition Tivo I used to own. But I was surely at the limits of human comprehensibility - there was no way possible for me to make sense of the soccer game itself. I could only recognize that it was a soccer game and nothing more, even though it was not literally a blur - several images were clear as stills - I couldn't process it with my organic brain as fast as the DVR could play it.

Suddenly I remembered a conversation between ships from 'The Excession' in which they had decided to take some action after a discussion that went for several pages in the book. But one of the ships suggested that before they do, that they ask their passengers what they thought, after all things had changed. One ship objected saying that it had polled the passengers 2 minutes ago and they had assented to go. The other said, well you know how long it takes them to decide things. You have to give them time to think.

A comic routine I heard on XM 150 the other day made a running joke about the common excuse 'It seemed like a good idea at the time'. It is the ultimate excuse. There are a lot of reasons that our human psychology makes the decisions it does, we are subject to certain inevitable biases, not the least of which is that we often make patterns that don't exist. We make errors in cognition due to a lack of time or focus or based on irrelevant data that somehow seems relevant at the time. For example, this very blog piece started off as something about the Olympics, but it is actually more interesting (now) as a thought piece on human vs machine cognition.

Being in the BI community, it is my job to see how machines can help humans make better decisions. The better I understand the business, the better I can tailor the system I'm building to assist the a limited scope of decisions to be made by the audience. When I described my gig to Unk last week, I got down to a specific job that I'm doing for a specific customer. It turned out that he had a similar issue during his days shuffling budgets as an administrator - and so he actually repeated that case back to me flawlessly.

There are lots of reasons we decide poorly, but when we are under time pressure, I think that is the reason we make the most mistakes. I'd make a final observation which is that some folks decide forcefully but then are amended quickly. That tends to be my decision making style - to set up a framework in which a good number of choices can be taken (in a fast OODA loop context) and harness as many observers in the process as possible. The quick decision coupled with authoritarian style can be very effective within organizations, but also dead wrong. In a proper framework, quickly wrong, might not be so bad.

Speed is key.

A Few Words About Agility

In my world, multidimensional design for analysis starts with knowing what the customer cares about. Ultimately I'm oriented towards solving a business problem - coming up with the right analysis for the kinds of problems the business is working on. When I have that kind of focus from the customer, time disappears. Most of the time, however, I don't have that focus, and so time is an issue. Thus, as a builder, I have to lock in schedules and project plans and all that. That's where agility comes into play. I'm talking about agile methodologies here. Agility leads to Focus.

Let's step back for a moment and try to recognize a few things.

• The Right Tools
• The Right People
• The Right Focus

If you have those three things working in sync, then is is just a short matter of time before a productive system is built. An agile methodology helps you get the right focus. What does that mean? It means you have a way of quickly determining what to deal with and what to throw away. That counts for code, it counts for requirements, it counts for data.

The keys to agile methodologies are informal communication, rapid cycling and close proximity. They are all inter-related. You basically need all three.

Informal Communication
This means that you do not create documents ahead of time. You create a lot of messages in real time. You talk a lot. You meet a lot. You repeat yourself. You ask a question as soon as possible. You get up and leave your desk to talk to the DBA when you get seven records instead of one. You call the AR guy to ask exactly what it means, this 'debit memo' thing. You whiteboard and you take pictures of your whiteboard. You have a conference call and you make a recording of the conference call with Audacity and then convert it into an MP3, burn it to disk and hand out copies. You blog your status reports. You have everybody's IM and you use it for stupid questions all the time.

When you work like this and people get used to it, you create a no-excuses environment. This is not a place for people who tend to drag their heels and be bureaucratic. This is not a place for passive-aggressive behavior. That shows up right away. When people don't want to communicate, when they hold things back, when they can't say what they really think, Informal Communication exposes them. When you have highly detailed documentation and requirements ahead of time, those people can hide, and they will sabotage your project. Why? Because if they are not 100% right the first time, getting from good to great becomes impossible. They will be too busy bowing at the temple they've created to see that Indiana Jones (you) have come to change things. Big Fat Documents become a stack of technical bibles, and when people swear on that stack of bibles, they build a Temple of Doom. That is the enemy of Agility.

Step Back a moment.

You've got to have the right tools and the right people. The tools have to offer ways to do things quickly so that fairly major changes can be made. Tools that encourage you to experiment are good. You want manual transmissions, not automatics. You want manual steering, you want to be able to turn the traction control off.

Now you have to have people who are willing to take the risk to use those tools. People who are ready to experiment, pop the clutch and see what that baby can do.

Back to Agility.

Rapid Cycling
The most important thing about an Agile methodology, in my view, is that it allows you to make mistakes quickly. A quick mistake is instructive, like when the Kung Fu master puts you to the mat in half a second, and then you get up and try again. Or like when Rafiki hit Simba with his walking stick. You learn quickly. The theory I'm getting to is the OODA Loop. How rapid is rapid? Well obviously you have to make enough mistakes in a short enough period so that you get it right by the time you need to. Your design for analysis has to be done rapidly enough so that it captures the end user's best guess as to what they need to analyze. I say best guess, because as you build, and they see new data for the first time, their understanding of the value of that data changes. You and your end user need to co-evolve that understanding. That means you will make mistakes.

On non-trivial tasks, you cannot control how many mistakes you're going to make. What you can control is how quickly you react to and correct them. This is key to developing a design which is going to be more adapted to your customer's evolving understanding of their own analytical needs. The bottom line is to make change your friend.

That is not to suggest that change doesn't have costs, but you first have to understand how quickly your design and development team can react and then what the pace of your customer is. Then you can communicate (remember?) what the incremental cost is for each change. What you don't want is the 'You want it when?' joke cartoon floating around. Get an understanding about how much can change, how quickly from your customer and then get inside his OODA Loop.

Every problem about IT's backlog is exactly this problem. The end user changes his mind faster than IT can develop products to support his mind. Unless the end-user's business is catching bullets in midair on Monday, milking cows on Tuesday and building nuclear powerplants on Wednesday, chances are that your IT's OODA Loop is too slow, not that the user changes his mind too quickly.

• Don't go into a skunk works mode.
• Don't build black boxes.
• Don't be afraid to tell your customer your secret sauce.
• Do reveal all your assumptions.
• Do ask stupid questions.

The more you cycle, the more you hone in on exactly what is important and why. This is an important learning experience for you and your customer. Even if you discover that you don't have the tools, or the people to actually realize the system.

Close Proximity
Almost nobody I started programming with liked working in a big open space with other people around. Not even in college did the best programmers work in study hall collaborating with other programmers. We all expect programmers to have extreme focus and that means isolation and silence. It doesn't work for Agility. Bullpens are best.

The programmer who can work in a bullpen, in a conference room with five different conversations going on is on his way to mastery. When the guy doing ETL is sitting right across from the Business Analyst who is rubbing elbows with the Front End designer, who is IMing the DBA, that's when the magic can happen. And these are exactly the same reasons why design work for analysis cannot be offshored. You need proximity. You need to be able to eyeball the work of other people and they need to be able to eyeball your work as well. End users need to be able to drive the system before UAT. What? Huh? Yes. Database architects need to be able to see the green screens of data entry personnel. Nobody can be too busy or too remote - these are not acceptable excuses. The more that everybody knows about what everybody else does, the more robust the system becomes.

Today we discovered that we need to make a combination key for invoices and vouchers for our data mart. Within 1 hour of that discovery we had three volunteers to do it in three different places, scrubbing before the ODS, modifying the SQL from the Cube to the ODS or changing the Essbase Load Rule. Now we have choices, That's Agility.

Agility Leads to Focus
When these kinds of tight, informal communication, that is rapidly cycling and in close proximity, meetings happen spontaneously, everybody in the room knows exactly where the kinks are in the system. The critical path reveals itself. This stuff tends to drive project managers crazy, especially the waterfall guys. Everything seems to get done, but nobody in particular is accountable. Everyone is accountable and the PM can't get around quickly enough to find out what's going on. This is an excellent situation, but it is one that is uncomfortable when you don't understand the dynamics. There is not much of a stepwise progression in this analysis, design, development loop. It's all happening at once. Lots of builds are getting done quickly, and people have to whiteboard where the new lines are going. And of course nobody wants to stop and write up a technical document.

Let's not forget however what the aim is. The aim is not to get rid of documentation or accountability. The aim is to gain focus. The aim is to insure that the system that gets built is exactly what the user wants without the problem of locking down specifications in a stack of bibles. The aim is to create and sustain a dynamism in an extended team that gets many different kinds of minds figuring out how to see in a way the company has never seen this piece of business before in an actualized analytical system.

That is the most exciting thing about designing and building analytical systems. That's why I almost never forget a good engagement.

BI Clouds

Unlike many of my new midwestern colleagues, I spend a lot of time missing planes. In fact, because of a 30 minute delay in Indianapolis yesterday, I won't be able to get home to Los Angeles until about 6pm tonight. Bad luck piles up over time, an early start beats fast running.

So while it might not seem like such a big deal to you, I'm wondering why more companies have not invested in BI clouds. I asked a few folks over the past week and got some blank stares when I talked about cloud computing related to MobileMe. But I hope that will not long be the case, especially as it is clear that Rolta (my new Daddy) is seriously invested in SOA, CA Unicenter type controls, Managed Services and Hosting. It's just a matter of time before it becomes clear that there are a lot of candidate companies for some extraordinarily nice IT around EPM and BI. The sooner the better. Any delay now may keep the whole industry stumbling.

It's all next to impossible to get people in corporate IT to stick to some standards established by the know-it-alls. And I must confess that I've spent a whole lot of time just learning exactly how many PM Methodologies are inferior to the sort Leah Wheelan got us working on a decade ago in AFSG. But the hub and spoke and active data warehouse and Agile JADs can only go so far - that is until you move to the next customer who wants to do it their way. Knowing this, I like most folks haven't even bothered much to listen to the know-it-alls. I mean how many projects have been delayed because the customer says 'yeah we need MDM but nobody wants to pay for it, besides, we've got deadlines'. Meanwhile the hierarchy gets rejects every pass and people invent department codes at will.

Such things would be eliminated by a company that hosts EPM - because we would own the standards, and the hardware, and the network and the security, and before you get into our heavenly cloud, St Peter would make sure you had your master data in order.

I've been thinking about hosted BI since 2003, not that me thinking about it matters much. But I think I've finally found some folks who can and will step up to make that kind of service offering a reality. As with most things, I remain cautiously optimistic and keep asking 'what can go wrong'. As the business case is made or not, I will report.

Welcome TUSC Bretheren

Well now that I have incremented my famousity another iota on the scale, I would like to say hello to all the new folks in my extended professional family. The extension comes as a surprise to me and a number of folks, but not as a shock. The surprise is, of course, the acquisition of WhittmanHart Consulting by Rolta TUSC.

I spent the last 48 hours or so immersing myself in the people and framework of TUSC which is my new parent company. It's going to be, very possibly, the best technical fun in my career. It has been a long time since I've worked with folks who have experience at Bell Labs or in a company run by engineers, so I get the feeling that I'm going to enjoy this time immensely.

But sticking to the title at hand, I just wanted to introduce Cubegeek to TUSC folks or rather EITC folks and tell you what it's all about. It is basically that which I find open-sourceable about the cool things I actually do in my job, some theoretical ideas of things I'd like to see, some biased opinions about the industry and what think is right or wrong in it, and a healthy dose of everyday geekery. I expect that we will soon be hosting some internal blogs with secret sauce, and there is no doubt in my mind that blogs and RSS are the best way to share such info - of course there is not now a secure RSS standard, I would like to see that change.

So feel free to comment on any of the things you see here. I'm always following the few that I get here and the many that I get on my other blogs.