Issue 1 - In Which Our New Subscribers Wonder if They've Made a Terrible
Technical content of this issue - 2 out of 5
Welcome to Volume VII of Biff's PDC Newsletter - hard to believe I've been doing
this for 11 years. This year we start with 140 subscribers, the highest
total in years - this is largely due to the tireless efforts of our marketing
staff, as this year they actually sent an email invitation to all our technology
coworkers at FINRA. This helped swell our ranks with brand new subscribers
with no exposure to BPN in the past (as opposed to veteran readers long
suffering from exposure), including dozens of co-workers that I have never met
as well as some corporate officers.
People have asked if the larger and more powerful subscriber list will affect
the editorial content of the newsletter - but nothing has ever appeared in the newsletter that couldn't appear in the 8:00
hour on broadcast television (when the FCC Commissioners are still awake), so
I'm not worried in the least. On
the other hand, this Newsletter started back when Plural had an Human Resources
department of one person - my current employer and largest demographic has a
significantly more sophisticated HR presence.
As a sanity check, I ran some of last year's newsletter by the BPN
General Counsel just to be safe:
In short, if you're seeing this text then your mail client isn't formatting correctly and
you should check out the letter with the web site link below.computer.
From what I understand so far,
it is not a set of disks that you
install in your enterprise, but a
hosting service run by Microsoft. Based on
expertise accumulated by
Microsoft while building and running sites like MSDN, Microsoft.com and Hotmail,
it was architected by Dave Cutler of
NT and VMS fame, among others.
I got much more detail at the
talk immediately following the plenary session, which you will find below.
Oh well, damn the torpedoes, full speed ahead. It didn't get me fired at
Plural, I'll just hope for the best
(before I get email from TB or MB, Plural was technically a cost-cutting measure
having nothing to do with this newsletter). For newcomers, the newsletter
is a mix of narrations of my observations on life at a major technical
conference along with summaries of the technical sessions that I attend
throughout the day. If you shun the whimsy, you will want to skip ahead in
each issue to the session summaries. If the summaries bore you, you will
want to focus on the narratives at the beginning and the letters at the end of
each issue. Personally, I read the entire thing!
New feature for Biff's PDC Newsletter in 2009-
Valuable Advertising Space!
No advertisements, just the space!
For the flight out, I was lucky and got to fly Virgin America again this year.
If you get a chance to fly this airline, I highly recommend it, besides a little
extra leg room each seat has a video system with digital TV, music and games.
Right now they have a special with free WiFi while in the air. I must
point out that during the flight, I never needed to reboot by Windows notebook,
but did have to ask the Flight Attendant to reboot the Linux based video system.
I actually enjoy the isolation of long flights as a chance to catch up on work
and reading. Unfortunately about halfway through the flight I noticed on
another seat that Dodgeball was on TV and had to stop and watch. I never miss a chance to see Rip
Torn heave a wrench at the smarmy guy from the Apple commercials! All in
all, it was a nice, flight, but for one visual image. I have been flying for many
years and probably taken hundreds of flights, but this one was the first time I
have come across a couple in an obvious "get a room!" situation. It's
nice to see a middle aged couple so into each other - however those of us around them
could probably have done without actually seeing it.
Workshop - Architecting and Developing for Windows Azure
OK, tomfoolery gets set aside while we discuss our first session (well, not
completely aside). This was actually a day long workshop the day before
the actual conference begins tomorrow. I'm not going to go into alot of
background detail on Azure (Microsoft's cloud offering) because there's plenty of stuff
out there on the web, including my writeups from last year
Since I wrote those articles late at night while sitting alone in an empty hotel
room drinking cheap liquor from a plastic cup, there's probably been much better articles
posted online since.
The session was led by Chris Auld, a non-Microsoft employee from New Zealand. Although there was a good deal of code in this presentation, it's real focus was
a general look at the concept of the cloud and how attributes of a cloud affect how
you create your application. In the course of a day it covered alot of ground, but I'll try
to narrow it down to a number of coherent topics.
He started off discussing the technology stack that makes up a cloud
architecture. There are many
cloud technologies out there, each of which hits a different portion of the stack. Here's his take
Software as a Service
Platform as a Service
Azure, Google App Engine
Infrastructure as a Service
He did not list Azure, Google App Engine or Collabnet on his slide, those are my interpretations for reference (funny, he never mentioned Google App Engine the whole day...). He sees
vendors like Peoplesoft and SAP moving towards offering their services in the cloud for companies
that prefer a pay as you go model rather than the large infrastructure and support investment. The rest
of this discussion assumes we are discussing an Azure like scenario.
Costs - He put up a couple graphs showing the revenue vs. cost models for traditional apps vs. cloud apps. I will
try to describe them rather than try to reproduce them here. The key point was
costs for cloud apps on Day 1 were much lower since there was no big infrastructure investment, so the gap between
revenue and investment was small. Then the cost curve went up smoothly, as the cloud infrastructure
can be configured to use more bandwidth without more investment. The traditional graph showed a much higher
initial investment, for a greater gap between revenue and cost on Day 1. Although the cost curve did not rise as steeply
in the traditional model, it did not rise smoothly. At some point you outgrow your initial infrastructure and
have another lump sum investment cost. This leads to uneven cost per user (eg -
the 999th user is added for essentially no cost, but the 1000th user requires a
$100,000 hardware investment). This really made clear that
the cloud based "pay as you go" model enabled much better planning. By the end of the week I should be able to post
links to all the session slides from the week.
Elasticity - The next point he stressed was the ability of the Cloud to provide elasticity. The TicketDirect app normally requires
about a couple hundred servers to handle the load while providing a good user experience. When tickets for rugby season go
on sale, however, the app requires several thousand servers to provide a good user experience. For a single company to
try to handle this on their own infrastructure actually ends up with the worst of both worlds - they probably don't have enough
horsepower to handle the worst of the peak, plus they're paying for huge capacity they don't need the rest of the time. A
pay as you go environment allows a company to pay only for the bandwidth they need when they need it.
Greenness - He asserted that environmental issues are starting to make their way into terms and conditions. He talked about
the vicious circle of a data center, where you pump power in to run the computers which creates heat. Then you pump more power in
to cool the data center. The by product of all this power use is computation. He showed some stats about the efficiency of
power to computation in a relatively new data center they had built in New Zealand, then showed that Microsoft's Azure data centers
beat that efficiency by 30-35%.
Security - He acknowledged that that a huge concern for companies was the concept of putting their data in the cloud and
the security issues involved. His assertion was that Microsoft's 300-400 person team focusing on security could do a better job
than each company having their own small group focusing on the same issues. He brought up an additional point about the risks of putting
data in off shore data center, not because of security but because of jurisdiction. Data under different jurisdictions is at risk
of being subpoena'ed. For example, do you think the Swiss Banking industry would have problems with their data residing in New York City,
where it is under the jurisdiction of New York state and the Federal government?
Scalability - As we got deeper into technology, he spent some time going over basic concepts that I won't repeat here (such
as the difference between scaling up and out). What was interesting is that Azure gives some ability to do both. You install your
app on instances in the cloud. Instances can vary in CPU cores and memory, from 1 CPU core to 8 CPU cores. He then ran a quick example
of a prime number generator on one big instance versus several small instances, with the smaller instances performing about 20% faster. His
point was that if your app doesn't run faster on multiple cores on your desktop it's not going to run faster on multiple cores in the cloud.
He can't give more guidance on small vs. large until the pricing structure is more settled.
Caching - At this point he hit several points that were not unique to the cloud, but for any well designed, high throughput, scalable app.
Caching is a good thing, there's several ways to do it in ASP.Net and Azure introduces one or two more. 'Nuff said.
Asynchronous Patterns - Next came some slides discussing Asynchronous Patterns
- moving any operations that you can to an asyncronous queue so you can return
control to the user sooner and level the perfomance of the operations. As he went over the steps required in tracking a message
containing a job from the queue and ensuring that it was executed once and only once it became apparent he was designing a homegrown transaction
system. Fortunately someone didn't wait for the question period but asked
immediately about Azure and transactions. Turns out there are no distributed transactions
in Azure so you need to roll your own unless you're operating completely in the data service. Wow, big drawback for Asynchronous Patterns using the queue.
I guess if you track your tasks in SQL for Azure you can be transactional.
Partitioning - Again - partitioning is used in traditional apps as well. The key to this section was the difference in partitioning
between Azure Data Tables and SQL for Azure.
Azure Data Tables are completely stored as name value pairs. The functionality is limited, as there are no
indices, no schema, no stored procedures. It is more along the cloud model used by the Google Apps Engine as
I understand it. Data is accessed with REST calls. Becaause of the nature of the architecture, Azure can handle partitioning for you automatically -
if you provide the partition key it will automatically rearrange the keys across partitions to balance the
SQL for Azure is SQL Server in the cloud. It has schema, indices, some stored procedure report, etc. Data is accessed via SQL calls. As he
explained partitioning in SQL for Azure it became apparent that you're pretty much completely on your own.
Pricing - It's important to understand the pricing issues involved with Azure, as it can have significant
impact on how you design your application. For instance, data storage works like this-
- SQL Azure - No per query charge, $9.99 per GB per month for storage.
- Azure Table Storage - Pay per query ($.01/10000 REST calls, $.15 GB per month (65 times less than SQL Azure
Because of this, vertically partitioning large data columns (CLOB fields, images) into Azure Table Storage can
make a huge difference in overall system cost. Before implementing an Azure app, spend some time learning the
pricing model and you can realize substantial savings in your TCO.
Letters! We Get Letters!
This section is one of the favorites every year (by "one of the favorites" long
time readers will realize I mean "one of my favorites"), but it depends
upon the emails and questions I get from you. To prime the pump, I include
a couple of memorable letters from years past.
MB from MD writes - "I will be out of the office on business on Tuesday,
September 13th and Wednesday, September 14th. If this is urgent, please call me
on my cell phone @ (number withheld)"
Thanks for the kind note and the heads up, I will continue to send the
newsletters and they will be waiting when you return. -Ed.
SH from CA writes - Hey, I took music theory and a Physics of Music course in college,
and C# and D flat are not the same note, unless you are referring to the
tempered scale which is, in fact, an average [of] the correct tunings for all possbile
keys, i.e. none of the notes are correct."
Yes you are correct. Now please stop acting like the guy in class that noone ever
liked or I will have to remove you from the subscription list, whereupon you will
have to get copy of the newsletter from the STCTM Discussion Public Folder like the people
that try to get free newspapers out of the recycle bins at the subway stations. -Ed
If you have anything you want to say, any technical questions or anything comment that gives
me the opportunity to make a witty retort, send your mail to
The Conference Begins!
Employee of the Month for November, Biff's PDC Newsletter