Oracle Exalogic - initial analysis
29 Sep 2010 by Simon Haslam (in Virtualization) | Comments (2)
|The biggest middleware announcement last week at Oracle OpenWorld (OOW) was arguably the new "Exalogic" server. Rumours had suggested that there was a middleware machine of some sort in the pipeline, though its name was kept extremely well under wraps by Oracle. Here is my analysis of the Exalogic product based on the OOW keynotes and a product presentation, discussions with an Oracle representative on the Exalogic stand and my understanding of Oracle's existing Fusion Middleware 11g and virtualisation software. Note: given that the product was just announced last week, and only one has been seen publically outside of Oracle's engineering labs, inevitably this article contains assumptions and quite a lot of speculation!|
The Exalogic (EL X2-2) hardware is a collection of x86-64 servers and a shared storage appliance, all linked together using InfiniBand (IB) QDR interconnects running at 40Gb/s. Up to 8 Exalogic/Exadata machines can be connected to the same IB 'fabric'; if further machines are required another switch (pair, presumably) can be added to enable very large clusters indeed to be built.
|Let's look at the hardware that makes up this machine - its key components are:
* HCA = Host Channel Adapter - equivalent to the HBA in Fibre Channel implementations
All the servers are connected to the two active-active IB switches for inter-node data and storage access. InfiniBand has the concept of "virtual lanes" which is used to segregate data on the interconnect, for both performance and security reasons. The data centre ethernet (e.g. ultimately connected to the internet) connects via a bridge to the IB network, and the other ethernet network is used for the management ports (such as the ILOM on the servers).
The mirrored SSDs on the servers are "used for OS images and high-performance local swap."
Like Oracle's Exadata, this product uses standard rack mount servers (1U in this case) rather than the blade servers and chassis familiar to consolidation products from some other vendors.
The Exalogic comes in 3 sizes: full rack (30 servers), half rack (16 servers) or quarter rack (8 servers). The remaining hardware is the same for all configurations (though I'm not sure about the number of IB ports - it may be simpler for Oracle to ship identical IB switches, as well as easing the upgrade path). There will be upgrade kits available to enable customers to move from quarter to half to full rack variants at a later time.
Regular readers will notice that the processors are the rather
tasty 32nm Intel Westmere-EP (Gulftown) 56xx series chips which
I reckoned were the x86-64 middleware sweetspot
earlier this summer. However where I had anticipated WebLogic clusters
of maybe 2, 4, 8 or so servers, the Exalogic has 30 (before you start
daisy-chaining racks that is!)... one heck of a lot of computing power.
Now things start getting really interesting! You will be able to choose to run Oracle Enterprise Linux, Solaris x86, or an Oracle virtualisation product on your Exalogic. Mike Palmeter outlined three planned versions:
- V1 This is the release announced at OOW which will have the operating systems running on physical servers, either Oracle Linux 5.5 or Solaris 11 Express (due soon),
- V1.1 The subsequent release will be able to use OVM 3.0 to run virtual machines (either bare metal hypervisors or Solaris containers),
- V2 A future release will include advances such as Java RDMA to provide even higher performance access to the storage (and presumably other nodes?) etc.
Oracle say that the upgrade from each version will be software only and be fully automated.
Clearly V1.1 is the one I am most interested in - this is the first version where you will be able to use virtualisation in a fully supported manner. On a machine of this size I expect the use of virtual machines will be critical to be enable you to carve up the compute resources efficiently, essential for the "private cloud" or "utility" model of running servers. Oracle's Virtual Assembly Builder software should allow you to provision new middleware infrastructures much more easily and duplicate them very reliably. For example you might have a complete WebLogic/Coherence, SOA Suite, BPM, Web Center and Web Tier assembly built, tested and ready. You should then be able to just "drop" the entire assembly into a new environment on the Exalogic in one go... middleware nirvana in my book!
'Exalogic Elastic Cloud' Software
This is where you have to work hardest to establish what's under the covers. The "elastic cloud" software moniker is being used for a host of optimisations Oracle are building into WebLogic Server, JRockit (and presumably Hotspot), JRVE and the Oracle's optimized Linux kernel, JDBC and OVM - much of it appears to revolve around network I/O, and in particular, making best use of InfiniBand (e.g. by circumventing the TCP stack).
Having had a detailed discussion with Frits Hoogland in the OTN Mason Street Tent about Exadata it seems like Oracle are really onto something here, though I haven't really got all the layers and protocols straight enough in my own mind yet to feel confident enough to write about it. The Elastic Cloud term probably also covers extensions to the management software (monitoring, deploying and upgrading environments in bulk).
In the same way that the Exadata Storage Server has special features reserved only for use by the Database Machine (and vice-versa), the "Exalogic Elastic Cloud" software may only be available on Exalogic hardware.
As its name suggests, Exalogic is designed for WebLogic Server application servers, but of course will support other applications certified for the operating systems running on the x86-64 servers. WebLogic Server provides the foundation Java EE platform for a plethora of products from the SOA Suite through, ultimately, to Oracle's new enterprise applications. Therefore I think we can expect to see Exalogic and Exadata being sold as the best hardware on which to run your Fusion Applications.
I'm not in a position to comment on Exalogic performance other than, at the very least, the combination of so many Intel Westmere-EP chips (a full rack is 60 processors, 360 cores or 720 hyperthreads) and 40Gb/s InfiniBand means that it will have a collosal amount of processing throughput.
The Complete Exalogic Picture
If you've persevered enough to reach this point you may have been surprised to find that I decomposed Exalogic into its constituent parts before describing the big picture, in contrast to how it has generally been presented so far.
Oracle sees the appeal of Exalogic is as an integrated system, from the hardware upwards. Most of the technical decisions have been made by Oracle's engineering teams, saving the customer work and risk of sub-optimal configurations. I'm sure for CIOs and IT Directors of many companies this simplification will make Exalogic very attractive. An Exalogic server becomes a single line item on a Purchase Order, it is maintained by an Oracle field service department and it is operated by a single team in the customer's organisation (rather than a combination of storage, network, OS, and middleware administrators). Indeed such non-technical reasons may well in part account for the success so far of Exadata; if that's the case Oracle ought to be able to replicate that success with Exalogic.
The list price for a full Oracle Exalogic rack is expected to be US $1,075,000.
Incidentally, Oracle are comparing Exalogic with the IBM Power 795 server. Looking at its datasheet it too is a big beast, e.g. with up to 256 POWER7 processors running at 4GHz each with either single, 2 or 4 simultaneous threads per core. In my opinion I would say that Exalogic is a horizontally-scaled commodity server based design, whereas the Power 795 is a vertically-scaled mainframe-style design. Comparing the two is rather like comparing Godzilla to King Kong and asking which is best! In most respects a more similar comparison would be Exalogic versus a rack of Cisco UCS or HP BladeSystem Matrix, albeit both Cisco and HP solutions are rather more "assemble it yourself" than Oracle's pre-built solution.
ConclusionsExalogic is a significant announcement, and will probably keep industry analysts happy as it shows Oracle has derived benefit and developed significant new products as a result of their hardware engineering assets received from the Sun acquisition. It also demonstrates that Oracle is very serious about high-end pre-configured appliances - between Exadata and Exalogic Oracle can now supply a significant proportion of the compute power for many enterprise IT facilities (i.e. the mainframe as was).
Exalogic Elastic Cloud - A Brief Introduction (Mike Piech and Mike Palmeter) http://www.oracle.com/us/products/middleware/exalogic-wp-173449.pdf
Oracle OpenWorld 2010: (S317409)
Oracle Exalogic: Introducing an Optimized Compute Platform for Oracle Fusion Middleware (Mike Palmeter) http://www.eventreg.com/sb250/loginOD.jsp
For further information see: http://www.oracle.com/us/products/middleware/exalogic/index.html