Innovations in Trading and Technology (Amazon crashes and clouds burn. On April 21, Amazon.com had problems with one or more of its shared-services data centers – in other words, its cloud was grounded. And it wasn’t the first time.
While we don’t know the full extent of the damage caused by Amazon’s cloud failure, for the firms that rely on the Amazon infrastructure, it wasn’t pretty – applications, systems and websites went down, leading to frustrated users, angry customers and missed sales. That said, shared services – that is, clouds – are the future of computing.
For years enterprise software has been in decline. Firms have been thinking about how to transition from proprietary to open, from single to multitenant, and from individually deployed to the cloud, where not only the data is virtualized but the processing is as well. This was the idea with grid and utility computing, and now the cloud.
The problem is, while shared infrastructure is the future, systems still go down. That’s why most Wall Street firms are not putting mission-critical services in shared clouds; rather, they are leveraging private cloud infrastructures for their too-important-to-fail applications. But this will change.
When I first started writing about grid technology, compute-intensive applications were beginning to share processing power within a single data center. Then, as the capabilities of processors improved, our grids became data-constrained – the ability to compute was compromised by the ability to move data into and out of the database. From compute grids we moved to data grids, where the database became virtual and we leveraged distributed memory, only committing to the database when transactions needed to be permanently recorded.
Comments | Post a Comment
8 Comments to "Just When You Thought It Was Safe To Go Into the Cloud":
Anonymous
20 July 2011
Securities trading systems experience spikes in demand for processing capacity and bandwidth at the same time—either at regular intervals, like at market open and close, or because of ad hoc events, like a major economic event triggering global market volatility. Just like highways are at their slowest when most congested—either at regular intervals, like at rush hour, or because of ad hoc events, like a car crash or road works—a securities trading cloud would be slowest when markets are at their busiest and most volatile, which is when you can least afford to get stuck in ‘traffic.’
Isn't that a good reason for securities trading firms to be wary of cloud infrastructures (public or private)?
colpclark
20 July 2011
Larry, the systems that went down when AWS puked weren't designed or written in such a way to be fault tolerant. A lot of the engineering that we in Capitla Markets take for granted is just making its way into the web world. Also, It's important not to confuse a PaaS that provides HA, DR, etc. out of the box (is that still such an elusive goal?) with commodity cloud services. In regards to definitions, I don't refer to cloud offerings like NYSE's community plaform as a private cloud - I refer to that as a Public Cloud offering a Private Service. And I believe we'll see transactional related wholesome goodness there very soon.
Comments (20)
david.brukman
20 July 2011
Anonymous, that's a very good argument for capacity management, commonly known as traffic engineering; a concept that applies to roads as well as financial services. As long as that is engineered properly (e.g. sized for anticipated peaks), and backed up with SLAs, cloud infrastructure should be able to cope.
Comments (7)
GCCWolff
20 July 2011
We see the cloud, clouds as you point out, as a driver of Web 2.0. Our research has identified 4 central themes that define next generation web. We call Web 2.0 Web 4C. The four C's are crowds, clouds, connections and commerce. Cloud computing is essential to moving services onto mobile and tablet devices, Larry correctly points to some serious issues with open cloud architecture as well as the fact that it is coming.
This article is a welcome addition to the converstaion!
Anonymous
22 July 2011
David, if you're sizing the cloud for anticipated peaks (occuring concurrently), doesn't that negate some of the cost benefits?
david.brukman
22 July 2011
Anonymous, regarding cost benefits and peak-engineering: indeed some of the benefits go away, but not all. First, you get economy of scale, particularly if you include non-server costs like adminisration, monitoring, network access, geographic resilience. Second, you still get lower cost and time of scaling up if there is spare capacity to be "reserved". For that matter, you get a better utilization of freed resources if you need to scale down, too.
A close parallel is a shared network with multiple qualities of service including some high-priority ones that require reserving bandwidth. Typically, it's still cheaper than ordering a separate physical circuit for each high-priority service.
Comments (7)
anonymole
22 July 2011
Trust vs. convenience
At first we all had private email exchange servers now we trust the cloud with our email. We all used to host our own web servers. Now we let providers host our web sites. We used to serve our large content, now we let content servers distribute our videos and such. As the convenience of cloud based hosting, whatever the kind, surpasses the trust barrier we end up adopting the cloud based hosting solution. As long as we have trust concerns, we will strive to control those services that we cannot afford, psychologically or financially, to release to fly in the cloud.
Comments (83)
Electrade
26 July 2011
The requirement for transactional efficiency, security and reliability continues to rule out cloud for any high value, real time, financial transactions or critical real time data processing. I feel that the cloud of today to be an unrealistic propostion to a transactional financial institution that cares about its reputation and ability to operate efficiently.