Cubegeek

Some Scalability Tips

Artlessly stolen from the High Scalability Blog:

AFK Partners has release what they feel are the Best Practices for Scalability:
1. Asynchronous - Use asynchronous communication when possible.
2. Swim Lanes – Create fault isolated “swim lanes” of hardware by customer segmentation.
3. Cache - Make use of cache at multiple layers.
4. Monitoring - Understand your application’s performance from a customer’s perspective.
5. Replication - Replicate databases for recovery as well as to off load reads to multiple instances.
6. Sharding - Split the application and databases by service and / or by customer using a modulus.
7. Use Few RDBMS Features – Use the OLTP database as a persistent storage device as much as possible.
8. Slow Roll – Roll out new code versions slowly, to a small subset of your servers without bringing the entire site down.
9. Load & Performance Testing – Test the performance of the application version before it goes into production.
10. Capacity Planning / Scalability Summits – Know how much capacity you have on all tiers and services in your system.
11. Rollback – Always have the ability to rollback a code release.
12. Root Cause Analysis - Ensure you have a learning culture that is evident by utilizing Root Cause Analysis to find and fix the real cause of issues.
13. Quality From The Beginning – Quality can’t be tested into a product, it must be designed in from the beginning.

Waterfall Process Mapping

I found this perusing through visualization blogs. This particular method is used generically to measure webpage download performance, but it could measure any number of processes. The 'Waterfall Chart' is a cool way to visualize parallelization bottlenecks, and simple too. I don't know why Excel sorts the chart backwards, but..you'll find it under custom charts as 'Floating Bars'.

Obviously, you can tag each process multidimensionally depending how you want to go through attribute tagging. The metrics for this example is simple. However you could also consider a multistage process as a contiguous stackbar. 

In this example, Process 9 is the bottleneck as no other process can start before it finishes, and the visual clue is very easy to see with this sort by start time. 

To create this data would be simple, given some conversion into UNIX time, or seconds since x, indexed to an arbitary start date of the first process.