Cubegeek

The Beginning of the End of an Era

If you've heard me blather about my technical accomplishments for any length of time, then you'll hear me brag about the Boeing CPS project. It was by a good measure the most ambitious project I have ever been involved in. The short end of the story was that I showed how I could get 500 databases to run concurrently in memory. I performed that minor miracle with the help of a a guy who knew LoadRunner inside and out and a team of techs at HP. You see this was done on a massive HP Superdome machine. I did this about four years ago at HP's Cupertino campus.

It turns out that HP has just sold off the property where such levels of supercomputing research, testing and debugging was done. This thus marks the end of an era, because the dudes I worked with up there had a bit of the shakes at the time. So if you ever heard me snark about Mark Hurd, it had something to do with the vibe they had knowing at the time that things were not going as well as they should have been.  I was very impressed by the staff for the three weeks that I was up there, but I could see that we were sorta alone. I mean here was some of the coolest big iron available, when the enterprise IT world was still freshly adapting 64bit computing - the one thing that restored my flagging faith in Windows. I could work with 32 processors and 64GB of Ram. How sweet was that?

Well now that we know some of Google's four year old magic, it's not very sweet. And if that campus was the last place HP engaged massive computing with companies like Boeing for doing stuff that required that level of compute power, then this sale to Apple represents another piece of inevitability for cloud computing.

iAds: Exactly Wrong

I'm sure there are marketing folks who can't wait, but I get a queasy feeling watching this cringeworthy video that this Christmas season is the beginning of the end of a certain romance geeks have with Apple. 

We all know the popular darling Apple has become - the consumer behemoth that it might become, with all sorts of clever encroachments on computing.. I don't know. Think about it. Ten years from now, when the majority of enterprise IT will be done in the Linux based cloudscapes, where will Apple have gone? Gone to market, every one. When will they ever learn?

 

The DevOps Manifesto

Is here.  I like especially the following part...

On most projects I’ve worked on, the project team is split into developers, testers, release managers and sysadmins working in separate silos. From a process perspective this is dreadfully wasteful. It can also lead to a 'lob it over the wall' philosophy - problems are passed between business analysts, developers, QA specialists and sysadmins. Furthermore, we see a replication of this silo structure within the teams - it’s not uncommon to see dedicated database and network people in the same system team, alongside sysadmins. Often the larger silos aren't in the same office, the same city, or in sometimes not even in the same country. The result is an ‘us and them’ mentality - groups of people who are simultaneously suspicious of and afraid of each other.

The Devops movement is bold enough to believe that there’s a better way - a way of building teams, and building software that can solve these problems.

How does Devops help?

The Devops movement is built around a group of people who believe that the application of a combination of appropriate technology and attitude can revolutionize the world of software development and delivery.

The demographic seems to be experienced, talented 30-something sysadmin coders with a clear understanding that writing software is about making money and shipping product. More importantly, these people understand the key point - we’re all on the same side! All of us - developers, testers, managers, DBAs, network technicians, and sysadmins - are all trying to achieve the same thing: the delivery of great quality, reliable software that delivers business benefit to those who commissioned it.

Breaking this down a bit - one of the key statements there is ‘sysadmin coders’. There’s been a certain amount of popularity recently around the concept of ‘polyglot programmers’. These are developers who know multiple languages, and multiple approaches to programming, such that when appropriate they can move swiftly between object-oriented Ruby and a functional language such as Erlang or OCaml. This is simply an example of a larger observation - there is no one IT skill that is more useful or more powerful than another. To solve problems well you need all the skills. When you build teams around people who can be developers, testers, and sysadmins, you build remarkable teams.

I'll be at the Meetup tonight in Santa Monica. Maybe I'll recognize a face or two from Shopzilla.

Computing Interfaces, Human Scale & The Death of Interoperability

Right about now I'm asking myself why it is that IBM doesn't own a phone company. It used to. It was Rolm. Tipped off by Enderle, I am thinking about what moves and what doesn't move in the computer industry.

What's moving now is the peripherals market, and the hottest peripherals are now owned by Apple. They are enabled by the network market and the hottest networks are still boring, but the changing of new and old protocols are making it interesting. Matters of net neutrality, Flash vs Ogg vs H.264 and the adoption of cloud strategies are what's making it interesting. Database technology is undergoing a renaissance too.

Only Google seemed to have its head in the game of comprehensively coming up with a 'new' computing paradigm which is network centric and going after the desktop. That was until the introduction of Apple's iPad. Apple now has an opportunity and is showing evidence of the vision to disintermediate Microsoft's desktop domination.

Think about it this way. The desktop has been, for a generation, the dominant computing metaphor. It is how must computing work is done today and it is the desktop metaphor including the mouse and keyboard that is how must of us do what we call 'computing'. 30 years ago it was the green screen. 10 years before that it was the punched card. What are you touching when you are using a computer? That is the question. Those who define and dominate win, and it starts with the generation of data and ends with the consumption of data. That, broadly speaking is what the computer industry is all about, it is about interface domination. What goes on in the black box is only of concern to the industry professionals - the battles in the marketplace are won and lost on interface.

For example. What is gaming? Gaming is an interface market with competition between the iPhone + iPad, XBox360, PS3, DS, Wii, and the Desktop. The game console wars are pretty much over and the winner is Microsoft. They dominate and so now everyone must write to XBox. However Microsoft has opened up a new front with Project Natal. Interface.

All this has to do with HP + Palm because HP has perhaps determined that it needs to be its own software integrator. HP can now extend its compute capacity into a new interface. This is key.

As the number of interfaces goes up, the influence of architects of APIs increases as well. What Apple is doing to squish out competition on it's platform begins to defy the mantras of interoperability. Apple can afford to do so because what I call 'development economies of scale' make it such that organic developers are not really competitive. Once a small group of in-house programmers have access to resources that allow them to develop technologies at a sufficient sophistication, a labor market for programming becomes surplus, and then barriers to entry build. Who actually needs to build a better spoon? You cannot get many jobs in spoon design. It's not because there is no spoon, but because the spoon has already been perfected.

Interoperability was important when teams of developers were still working on technologies that had not been completed to human scale. What do I mean by that? A technology that is complete to human scale means that you have a technology whose marginal improvement cannot be perceived by most humans. It's not about distinguishing them from magic, it's about distinguishing them at all. For example, it has long been the case in the audio electronics industry that the capability of the technology to record, transmit and reproduce sound has gone beyond the capabilities of human perception. Any good salesman at the stereo store will tell you that only 1 out of 50 people can actually hear the difference between a good $500 speaker and a good $5000 speaker. Why pay for the difference if you can't hear the difference? The clever stereo salesman would steer the prospective customer into believing he could hear the difference, but most of the time the customers could not - especially if they liked Cindy Lauper.

So how do people choose speakers today? Fashion, style, interface. The technology is complete to human scale - as such there is no need in the mainstream, to innovate. That's why 30 years ago, stereo components were the hot item, whereas today video components are hot. People can certainly perceive the difference between black and white and color. And now we are seeing the difference between low and high definition define the market. When everybody has HDTV, we'll all just buy for fashion, style, interface. The technology is done. It doesn't need to move.

While the technology is still developing to human scale, small innovators can make a difference. Market makers will corral them and interoperability will matter. Once there is no more need for integration, the industry won't care about interoperability just like the TV industry no longer needs pin socket standards for the tubes in their audio amplifiers. Single companies can integrate the technology from end to end, streamline, lower costs and improve quality - but sell interface.

HP, with its acquisition of Palm now has an interface for mobility. It can now build phones. Does anybody care about the developers of Palm apps? No. They will now be HP apps and the API can be streamlined to fit HP's needs for mobility. 

So what technologies are done and not moving in the computing industry because they have now reached human scale - the limits of human perception? Well, that's an interesting question that I'll start to think about. And while I do so, I'll keep an eye towards what happens when another interface solves another problem moving that problem towards human scale. This is how I can identify shifts in markets and opportunities for different vendors to leapfrog.

For example. I think a class of applications will emerge in the multitouch paradigm that will approach human scale that have never reached it in the desktop metaphor. I think that multitouch gestures can become a vocabulary for expression that will allow a much greater throughput than could be expected from keyboard & mouse. Guitar Hero showed a way to game that closer approximates the human scale of musicianship. There could never be a better way to play guitar via keyboard. You need to approach a guitar. Furthermore, as I've indicated previously, I think that sorting, distributing and shuffling will be better accomplished through multitouch interfaces than with mouse & keyboard interfaces.

Flash & Cold Fusion

My two cents.

Flash on Apple? No. What difference does it make? It's the difference between legacy applications and new stuff, which is to say if the trend continues that Android will suck up all of the Flash world and Apple rejects it, Apple will be in the better long term position.

I seem to recall that Flash was the stuff of Web 1.0, back when it used to belong to a company called Macromedia and the greatest authoring tool in the world was Dreamweaver and HomeSite. Are you kidding me? And back when Adobe purchased Macromedia, what I recall was that the whole edifice was crumbling under its own weight. Adobe was the ship almost too late to save a drowning witch. Considering the number of other video standards that are out there, the big deal with Flash just doesn't seem to be a big deal to me. The best developers will easily adapt to new tools.

Just because you can name a product Cold Fusion doesn't mean you can defy the laws of thermodynamics.

The Journalism Problem

Last night I listened to the NPR reporter interview the director from the service organization that just got dropped after a foster child was murdered here in LA. They guy took his time and explained patiently how his organization, which had been consistently rated in the top 10% of such providers to the government offices whose business is foster children and adoption got puked on for no good reasons.

It struck me that this was a no-win situation, especially as he mentioned that there is an ongoing investigation. An ongoing investigation means that somebody knows, its just that a bigger somebody with authority doesn't know yet and so officially nothing can be said. But that's somebody still knows. The problem is getting to that somebody is what makes for sensationalism. It is journalism and communications in a nutshell.

The NPR journalist took calls after the director of the service organization finished his statements. A woman from Orange County wanted to know the demographics of foster children and how it was that a family that keeps pit bulls can be approved as foster parents and potential adoptive parents. She wants blood. The LATimes journalist breaking the story is up next. I got out of my car and turned of the radio in disgust.

What's going on here is that the radio station and the newspaper, as part of their business model, need to highlight what's potentially wrong, which is by definition sensational. There's nothing wrong with that, it's important. What's wrong is that the newspaper cannot own that audience because the real audience for this story is longitudinal. The kind of outrage and blood the caller expressed is like an online flamewar with a newb. She obviously knows nothing about the subcontracting details and the authority delegated between the full-time bureacrats in the child welfare department, the care-giving subcontractors and the board of supervisors. None of us do. All we know is a two year old kid was murdered and the system didn't protect her.

The director mentioned the scorecarding system that the County uses to rate its subcontractors as well as the three government officials who would have to be involved in the approval of transfer from foster parentage to adoptive parentage. What we don't know is how well the system works so much of the time that we pay no attention to it. And if we are genuinely interested, where would we look? Most importantly how could we use a different form of communication to translate the energy of outrage which is currently owned by the radio station and the print media journalists into the energy of responsibility owned by the director's care-giving organization?

You see? The length of the attention span of the media in its current instantiation does a disservice to every organization that gets scandalized because the energy of the anti-scandal owned by the media outweighs the energy of reform.

Part of this imbalance is inevitable and cannot be fixed. Consumers cannot be producers. Anybody can enjoy listening to music, but only a few can play it, and only fewer still can compose it. We have a faux system of journalism that promises to enable democracy in conflict with this fundamental human fact. Everybody can be outraged at the death of a two year old, but only a very few of us can prevent it. The Director did not, and maybe he could not. However consumers can become prosumers if they are informed in the right ways. And so voters can become reformers if they are given the proper background. But this is not a job for journalism as it is currently constructed - it is more a job for the communications directors of such agency's as the one now under fire.

If this story generated 50,000 hits at the LATimes, how long with those same eyeballs stay glued? How long can the investigating journalist on the story write about it and follow up? How can those eyeballs be moved to a longer attention span with a prosumer / reformer / watchdog capacity that is credible even in the eyes of the Board of Supervisors, the child welfare agency and the subcontractors kiretsu? This is the problem to be solved in all of journalism and in all of communications. Journalism has to itself move from pique to education or the burden needs to be gracefully transitioned to the communications abilities of the first parties. When the care-giver's agency can take those 50,000 hits and disseminate on a long term basis, then we have a win.

Until then it will be outrage as usual and frustration with the system. This is what the current media generates.

--

Here's the technical view:

• Curt Monash on Ike Pigott
  
• Information Ecosystem Framework
• Retention of Everything

Last night I listened to the NPR reporter interview the director from the service organization that just got dropped after a foster child was murdered here in LA. They guy took his time and explained patiently how his organization, which had been consistently rated in the top 10% of such providers to the government offices whose business is foster children and adoption got puked on for no good reasons.

It struck me that this was a no-win situation, especially as he mentioned that there is an ongoing investigation. An ongoing investigation means that somebody knows, its just that a bigger somebody with authority doesn't know yet and so officially nothing can be said. But that's somebody still knows. The problem is getting to that somebody is what makes for sensationalism. It is journalism and communications in a nutshell.

The NPR journalist took calls after the director of the service organization finished his statements. A woman from Orange County wanted to know the demographics of foster children and how it was that a family that keeps pit bulls can be approved as foster parents and potential adoptive parents. She wants blood. The LATimes journalist breaking the story is up next. I got out of my car and turned of the radio in disgust.

What's going on here is that the radio station and the newspaper, as part of their business model, need to highlight what's potentially wrong, which is by definition sensational. There's nothing wrong with that, it's important. What's wrong is that the newspaper cannot own that audience because the real audience for this story is longitudinal. The kind of outrage and blood the caller expressed is like an online flamewar with a newb. She obviously knows nothing about the subcontracting details and the authority delegated between the full-time bureacrats in the child welfare department, the care-giving subcontractors and the board of supervisors. None of us do. All we know is a two year old kid was murdered and the system didn't protect her.

The director mentioned the scorecarding system that the County uses to rate its subcontractors as well as the three government officials who would have to be involved in the approval of transfer from foster parentage to adoptive parentage. What we don't know is how well the system works so much of the time that we pay no attention to it. And if we are genuinely interested, where would we look? Most importantly how could we use a different form of communication to translate the energy of outrage which is currently owned by the radio station and the print media journalists into the energy of responsibility owned by the director's care-giving organization?

You see? The length of the attention span of the media in its current instantiation does a disservice to every organization that gets scandalized because the energy of the anti-scandal owned by the media outweighs the energy of reform.

Part of this imbalance is inevitable and cannot be fixed. Consumers cannot be producers. Anybody can enjoy listening to music, but only a few can play it, and only fewer still can compose it. We have a faux system of journalism that promises to enable democracy in conflict with this fundamental human fact. Everybody can be outraged at the death of a two year old, but only a very few of us can prevent it. The Director did not, and maybe he could not. However consumers can become prosumers if they are informed in the right ways. And so voters can become reformers if they are given the proper background. But this is not a job for journalism as it is currently constructed - it is more a job for the communications directors of such agency's as the one now under fire.

If this story generated 50,000 hits at the LATimes, how long with those same eyeballs stay glued? How long can the investigating journalist on the story write about it and follow up? How can those eyeballs be moved to a longer attention span with a prosumer / reformer / watchdog capacity that is credible even in the eyes of the Board of Supervisors, the child welfare agency and the subcontractors kiretsu? This is the problem to be solved in all of journalism and in all of communications. Journalism has to itself move from pique to education or the burden needs to be gracefully transitioned to the communications abilities of the first parties. When the care-giver's agency can take those 50,000 hits and disseminate on a long term basis, then we have a win.

Until then it will be outrage as usual and frustration with the system. This is what the current media generates.

--

Here's the technical view:

• Curt Monash on Ike Pigott
  
• Information Ecosystem Framework
• Retention of Everything

On MDX, Briefly

I've worked with MSAS a bit and I've had engineers who worked for me tell me things that confirm my brief experiences as I got further from the technology. 

Most people who get up the rather steep curve for MDX admire it for its elegance and prefer it to SQL on that basis. Nevertheless the experience is that it is generally not worth it to learn the language if you have appreciable experience in SQL. 

If you have become something of an expert in tuning databases in the generations of products before the Greenplum and Vertica days, say with Essbase, Microstrategy, Teradata, Oracle Express or Sybase ASE, then you will have some familiarity with arcane 4G languages and dialects of SQL without many cross-product similarities. MDX was the language destined to solve all that, a sort of OLAP Esperanto. Unfortunately, the number of programmers deciding to hack SQL to perform against this and that sort of schema tended to dominate the few cross-platform specialists. In the end, it is my opinion that the lack of a predominating visualization stack obviated the need for widespread MDX adoption. As fat visual programming OLAP clients matured, enterprise customers began demanding thin clients. As full featured stacks became available, developers wanted LAMP, and so on. Now we are met on a battlefield testing whether PHP and MySQL will long endure, with people like me wondering how they ever got started considering the maturity and performance of products like Sybase and Essbase.

My direct experience is that flatly, for applications of any sophistication, on MSAS, stored procedures with T-SQL always performed better than their functional equivalents in MDX. In their aborted product stack, Performance Point, Microsoft engineers created a middle language PEL(?) that would supposedly choose which language was best suited for a task and then generate that code. All jokes about code-generators aside, my engineers always had to second guess PEL and ended up writing it all in T-SQL. So as a practical matter, there was no sense in learning MDX if you had already mastered T-SQL on the platform for which MDX was specifically designed. It would have been nice if MDX performed with speed commensurate with its elegance, but it simply didn't. This was two years ago. 

I know MDX lives on in Essbase but that Essbase expresses it differently than does MSAS. It does rather boil down to 'it depends', because language to language different platforms have different strengths, etc. 

I expect that we will not learn the definitive answer to this question until there is a shakeout of DBs that survive the transition to cloud infrastructure. And in that regard I think a Greenplum or a Vertica or a Hadoop-based solution will win out. In other words it won't come down to the semantic layer. The market never forced it to because there is no real OLAP standard (chicken or egg?). 

In the end I say MDX iff you love MDX.

ODTUG Followup #1

I'm still very busy kicking off a new project and haven't had much time to spare but I did want to do a short follow-up. 

First, thanks to everyone who showed up at my session. It was very gratifying to see a packed room. I hope the lesson in data management was valuable to you. And as I promised, I will be publishing parts of the toolkit shortly. I will do a separate post for each of the four tools I talked about and I will be replicating those over at the IN2HYPERION site as well. So you need to check there as well. 

Secondly, it was great, as usual to see my old friends and colleagues and meet new people face to face. Kathy Horton, Jon Rambeau, Mike McCarty, Ed Roske (whom I didn't get a chance to speak to but lobbed a softball question), Mark Rittman, Tim Tow, Rohit Amarnath, Yuri from Brooklyn, Rudy Zucca, Eric Helmer, Andrea, Ron Moore, Rob Donohue, Doug Burke, Sean Bernhoit & Darin Pope. 

More blogging this weekend. Stay tuned.

Get In2Hyperion + Cloud Battles

There's a new blog on the block. Just a bit of horn-tooting. Please notice. It's called In2Hyperion. It's put together by the good, smart folks at Rolta TUSC and I expect that it will reward your attention. 

What's new with me? Well, I think it is inevitable that we're all going to get into big data, and big data is all going to live in the clouds. The coming battle, as I see it, is going to be the super IT folks vs the super cloud folks. It's a showdown that has been long brewing. There are going to be winners and losers, and the ultimate winners will be companies. 

The super cloud folks are those old heads who have had to work with massive scalability of the sort the most successful websites have had to use. The super IT folks are those old head who have had basically to manage outsourcing, capital budgets, traditional IT, and of course big ERP and big DW. If you ask me, they are two radically different perspectives with few skillsets that overlap. Here's an example.

This week on Cranky Geeks, the cranks assembled had the nerve to suggest that Oracle bought Sun merely to destroy MySQL, and that if the world has any justice then some anti-trust agency is going to force Oracle to give it up. I really, really think that's a laughable criticism. That's because I firmly believe that Oracle wonks have absolutely zero to fear from MySQL because it doesn't scale. Now I'm just going to accept that as axiomatic, because obviously some guys somewhere have figured out how to do some masterful multi-tiering perhaps with Hadoop or some-such. And yes there is this thing stuck in my head about hearing that a lot of web folks think that a million rows in MySQL is 'very large'. So I'll admit the prejudice. But here's what I understand very clearly about Oracle, they will not kill things that refuse to die especially when there's market share at stake.

On the other hand, it's difficult for me to believe that Facebook is running on Oracle behind the scenes. These web jockeys know *something*. Which is why there is a war coming for the enterprise dollar.

My interpretation of the Oracle Sun acquisition is that Oracle is going to have its own hardware and will get into the cloud business ultimately. That will leave HP to buy SAP or vice-versa. IBM will face its greatest challenge in five to seven years. There will be other players - likely Cisco if you ask me, but the great irony is that Scott McNeely's prediction (The Network is the Computer) will finally come true, except that he won't be running the company that realizes it. Intel will probably get into the cloud data center business fabbing up custom stuff rather like whomever used to make those big switches for AT&T way back in the day. BTW, there's a great scene of a switch in THX 1138 (minute 62). 

The advantage the cloud and web-guys have is that they know how to build for scale. The disadvantage is that most of their data is pretty dumb and they're not particularly interested in applying rules and discipline to it. That's the huge difference, from my way of looking at things, between corporate IT data and public web data. So the gap that needs to be filled, which may auger for a real evolution in enterprise computing is the judicious migration of critical applications to utility computing.

Imagine this. Oracle drops off a containerized data center at your company's data center. They hook up power and network and in a short time several of your applications have been ported over. The container hooks up to a truck and off it goes to Oracle's grid. You shut down your data center and never worry about capital expense again.

Agent Smith says it is inevitable.

Countdown to the Purge

Well, it just seems that the more I look at this stimulus package the more I find to dislike. But I am looking at the way it may affect my particular industry. I just discovered today a nice hunk of smelly protectionism, that will serve to benefit the American IT industry just fine - well, depending on how you look at it. 

The bottom line is that a big splash of cold water is going to be thrown on H1-B visas. What you have here is a political compromise that changes the rules of the game with some politically correct exceptions for cover. 

One thing we know in IT and especially in IT consulting is that labor markets move very quickly and talent is at a worldwide premium. That's why we have so many folks from all over the world employed in this business. Any old Joe won't do. I could get into the particulars about my segment of the industry and my opinions about outsourcing and staff augmentation. Suffice it to say that it's very easy to stereotype a lot of Indian workers. But the fact is that Indians who make up a visible bulk of guest workers are all over the map in terms of talent. That doesn't matter as we throw the baby out with the bathwater here. 

This is just another zero-sum, wealth transferring, centrally planned wrecking ball headed at the economy. Unintended consequences? Yeah right.