I have this really difficult time writing about my industry and I hope to resolve it sometime this year. In the meantime I do attend a lot of Meetups here in LA to try and catch the flavor of a distribution of souls who operate where they do. It's rather tough to do all this working at home since most of my career I've had sales and marketing folks arranging those sorts of meetings for me. So these days I'm feeling the odd pain of not having a sales and marketing department and an army of fellow employees with new acronyms and strategies on their lips after three days of team building at the big hotel and convention center...
What I do have is the rare opportunity to meet practitioners out here on what seems to be the edge of technologies and practices the market may or may not come to grips with in the next few years. That also being in the relatively new field for me of Open Source makes it doubly complicated. I mean, everybody can know the same tools, have the same capabilities. Scary. Weird. So I'm always a bit hesitant to talk about exactly what it is I do, what it is I want to do, and I tend to stick to what I think is worth doing. That generally makes me sound like an idiot because I take clues from so many places. Cool when you're already a millionaire, silly when you have a day job. Nevertheless I'm a shameless data guy and I like talking with data guys. I kind of assume that everyone can build what I would like to see built and what's all keeping us from it is ... well the same lack of appropriately funded sales and marketing people.
Tonight was a big meet in West LA sponsored by an outfit I happen to think a great deal of - that being Factual, where all the men have long hair. OK 75% of the men have long hair. I do trust my gut, and my gut tells me they're going to do very well in the long term. I had that same gut feeling about Informatica and Comscore in my previous lives. Then again, I was wrong about Acta, or did they get acquired by SAP? I forget. I have said it before, if I had to work any other place it would be there or Evernote or SpaceX. Jeez. I have expensive taste. There is actually one other place that ranks close to my current paradise and that is in the CUSP program at NYU directed by none other than Steven E. Koonin, one of the T50.
For those of you who don't know, I have decided that there are probably only 50 people alive in the world whose books and speeches are worth reading and hearing. I call them the T50 among whom are Koonin, Nassim Taleb, Clay Shirky, Thomas PM Barnett, Neal Stephenson, David P Goldman and Danny Hillis. Hmm. I've got to put up a T50 page.
Anyway, in all things data, I'm trying to find a secure way to share everything I know without giving away my lunch, and quite frankly I don't have the math, the money or the market share to be too loose-lipped. So I find it a precarious business, and I'm not quite secure enough. Still, I know what I like and what I like is what I am exemplifying in a couple semi-open projects. H/T goes to someone who will probably never read Cubegeek about staking out territory whether or not you can defend your claim.
I am compelled by my history and my interests to value highly that data which might serve to inform the public about the real statistics behind every claim putatively made for the public benefit. The example I gave tonight was the price of shrimp in the Gulf of Mexico. And I am interested to see that the next generation of journalists will participate in the curation of open source data on some platform and standard that I have something to do with. That is the aim of Project Longitude. If you recall the transfixing tragedy that was the failure of the Deepwater Horizon and all of the conspiracy theories that arose, you may recall the arguments in support or against the seemingly arbitrary figure the President sought in damages against BP, the owner (?) operator(?) of the underwater gusher. One of these was the matter of the depletion of wildlife and economic impact to the region. I happen to think there ought to be a goodly number of people who would like to curate that information who might communicate with commodities traders. At some point, especially with journalists, information that is no longer breaking news loses its value, and so could be open sourced. The problem is making a good format. I would like to create a market for 'antique' data and make some hub for structured time series data on a relatively low cost, on demand cloud platform.
Right now, there is structured data of high value currently churning through corporate America. I know, that's how I paid for everything I own. I have been fortunate enough to start in the market before it got specialized. So I have done backends, front-ends, sales, tech support, consulting and management all around corporate financial data. This has allowed me the dubious distinction of being able to see that whole big picture from experience, rather than shutup and be a deeply experienced Java programmer. (yea I saw that coming too.. before they even had Swing). But now the price of much of data care and feeding is making a lot more data reasonably available to a lot more people, and I've packed my Conestoga and hitched it to what I hope to be a bull market.
We'll see.
I talked to (that) Randal Schwartz about my old old idea of XRepublic and tried not to bore him with the details. As I explained it, I realized that I'm a bit more cynical about the prospects myself. This prepared me for his response which was that we need a better quality of citizen before a better set of tools will produce a better quality of politics. I agree and disagree. We had a good laugh about a parliamentarian bot, or something programmed to autoboot defiers of Godwin's Law, or at least I thought it was funny enough to mention here. But I do believe that certain behaviors can be created with the proper UX. I mean, if Farmville doesn't make one good actual farmers, the virtual farms do actually grow and they do represent the results of your efforts. Good democratic practices can be gamified even if the result isn't real democracy. The experience has to be more instructive than what currently exists, exactly nothing. But sadly, it reminds me that I had a much less pleasant conversation on the same topic with Dave Winer 6 years ago. Maybe that's my clue that this may not be a good idea. Yet.
Most edifying by far were conversations with Lars and Jason about the shared pain of data management in AWS. I've been going through a bit of that, and I think the ideas I've been kicking around for a nicely kitted out VM or appliance spec will find a lot of uptake. That one is Project Kleiglight, now internal to Full360 as I push for it. In the meantime, I suspect that there are a number of smarter ways for me to accomplish a bit of that without springing for DMExpress from Syncsort. Or maybe not. But this goes back to thinking about the dimensions of the product / service offering and interpreting what Factual and Infochimps are doing in their architectures that might have already covered that ground. TBD.
All told it was a great Meetup at Steingarten, and cheap parking too.
