This is my blog, but it's also a kind of unofficial blog for Full360. Very interesting things are going on.
About a year ago, at a hotel room in the Venetian Hotel, we began a discussion about what we wanted to build into the Full360 elasticBI platform. We have been very successful. This year, we've had more time and people put into that process and in consideration of what we want, what our customers have had us build and what's happening in the market, we have a very different outlook than just a year ago.
It's really time to say that BI is just a part of what we're all about. It's a big part, but..
I think the simplest way to understand is to know that we have responsibility to funnel information to executive decision makers. That means, as systems integrators, we get to know a lot about all of the different kinds of source data in enterprise data flows. We have made it our business to re-architect enterprise paradigms to the AWS cloud. Our success in understanding and implementing that has far reaching consequences for what we can do, and therefore who we are.
This year at AWS Re:Invent, Amazon introduced several groundbreaking products, not the least of which is their new BI tool, QuickSight. As we have been talking to people and prospects, we've heard them say 'slice and dice, bar chart blah blah blah', and we get that. So we've always said that we are the cloud architects on the back end, which we are. We thought that by saying, "We are next-generation data warehouse', that it would be quite enough to distinguish us. It does but only when you start looking closely at the technology and what we're doing with it. I'll fill you in on that later. The point here is that our DWAAS platform and methodology is forcing us to invent new terms that the market can understand. I am continuing that responsibility. Fingers crossed.
Our experience is that we have been two steps in front of Amazon and four steps in front of the industry. This year we are only one step in front of Amazon and expect that they will start formalizing products around technology integrations we have done next year. What is that technology integration? Let's call it ETL. We have taken ETL to the next level - higher performance, lower cost. But we haven't launched it as a product, rather made it available as a service. Everything we do is a service in a systems integration and managed services framework. That's how we do business. We are focused on function in the context of the AWS cloud environment and architecture. That's not software you load on servers so much as it is API calls in an highly customized environment configuration. The bottom line is that with the elasticBI Framework we can get data from complex source to high performance destination at a scale and speed unthinkable in traditional architectures. While I'm on that point let me describe Multi-Tier DW.
Multi-Tier Data Warehousing is what we all our practice of combining multiple data storage and database technologies in a single analytic application. Imagine this example. A Full360 client has a website that generates 1 million unique customer hits a day from customer interactions. We use our Dragonfly realtime rules engine to score these customers as they use the website and add new attribute data at peak rate of 30,000 REST API transactions per minute. This is powered by VoltDB, our fast data in-memory database partner. This data is staged into a multi-node Vertica cluster that integrates internal financial data and industry benchmark data purchased from a third party. Additionally we have added custom widgets to the website to capture additional data. It is made immediately available through our custom IOS app to mobile employees around the globe. History is automatically archived to S3 for low cost high availability and then efficiently batch loaded into Amazon Redshift to process billion row queries on demand. We serve this data on one or more of our BI partner tools, perhaps Tableau, perhaps TIBCO Spotfire, perhaps Jaspersoft. That's a multi-tier DW solution.
But notice how we have extended back by providing our own microservices in our client's website. Notice how we have extended forward into our client's user base with IOS apps. Notice how with Dragonfly we have added a new dynamic dimension of data to our client's CRM. Now we could stop there and say, as a systems integrator we have demonstrated a larger aegis of control. We could stop there and say by employing in-memory, object-store and column store data management we have evolvled into next-gen Data Warehousing. But that would be incomplete, because we have done more than that.
So it leaves us with something of a marketing problem. Let's talk about the more.
One of the first operating principals of Full360 was that we are a DevOps company. We have reduced the cycle time of our data management. We can move applications from development to production in a matter of hours. Need to double your database staging mechanisms? Right away, sir! One of the things we do in our standard delivery is employ Cloudwatch to not only monitor server performance (remember everything isn't a server these days) but service performance. So we monitor the latency of the data process and can dynamically scale when necessary. We know when our customer's data sources are choking long before our customers do.
Another principle is that we will use Open Source and be vendor agnostic using the best technology and best architectures available for our client's projects. Architecturally speaking, nobody holds a candle to AWS. Azure might be worth consideration in a couple years time, but right now we're all about AWS. They are thought leaders as well as innovators. For Full360 that means we incorporate excellent technologies like Hashicorp's Vault, Terraform and Consul, to help secure, configure and communicate in the elasticBI platform. For our virtualization, we've been using combinations of Vagrant, Docker, veewee and Virtual Box. For our orchestration we've been using Chef and CloudFormation and Terraform is next.
This is just the beginning, but I need to keep this blogpost bite sized. I'm going to help you understand in the short term.
