Oracle Database Appliance - Initial Impressions

10 Dec 2012 by Simon Haslam (in Hardware)  |  Comments (1)

Although I noticed the Oracle Database Appliance (ODA) when it was launched last year I hadn't investigated it further until I attended the last session at UKOUG 2012 (ouch!) called Oracle Database Appliance Technical Deep Dive: Architecture and Internals, as delivered by Tammy Bednar from Oracle. This post is a summary of what I learnt from the presentation, a bit of reading from the Oracle website and Andy Colvin's blog (see References), plus some initial personal analysis of my own.

ODA (apparently pronounced to rhyme with Yoda) is hardware-wise:

• A 4U rack mount server chassis containing two blade-like compute units ("System Controllers"), i.e. like two 2U servers stacked on top of each other
• Each server then has:
• 2 Intel X5675 processors (i.e. 12 cores per host) and 96GB RAM
  
• a pair of internal onboard 1GbE NIC ports (used for the interconnect), with external connections for another 2 onboard NICs
  
• a quad port 1GbE NIC
• 2 10GbE PCI card (I can't see how this is used yet)
  
• ILOM with a NIC port
• a pair of local SATA disks for the operating system
  
• 2 dual ported SAS controllers for connection to the shared storage
  
• The chassis also has 24 directly attached 3.5" disk slots, populated with 4 SSD and 20 HDD ("high performance", i.e. 15k RPM). All disks are dual-ported and, by means of SAS expanders, are then visible to both hosts.
  

As its name suggests the two servers work as a standalone database cluster with all components appearing to be highly available if you run RAC databases.

Configuration is carried out by means of a Java wizard-style UI, called the "Oracle Database Appliance Manager Configurator," which creates a configuration file (onecommand for those familiar with Exa*). Note you can download this yourself to try at: http://www.oracle.com/technetwork/server-storage/engineered-systems/database-appliance/overview/index.html

In short, this prompts you for all the settings you need from domain names and IP addresses (host, VIP, SCAN, ILOM etc) through to a details of initial database to be created. Databases appear to be created using special DBCA templates and are basically small, medium, large etc and you can chose to have them as single instance (i.e. Database Restart - why would you though?), RAC One Node or full RAC. All RAID is done in software (i.e. in Linux or ASM) and the SSD is used purely for redo logs but there are options relating to ASM - whether to put aside more space for backup, whether to use normal (mirrored) or high (triple mirrored) redundance and how much ACFS space to set up (450GB max). A triple-mirrorred installation apparently gives you up to 4TB usuable space, double-mirrorred is 6TB. Any additional storage you need has to be provided by NFS (presumbaly Sun appliances) and will be accessed by dNFS.

You also have the option to configure Auto Service Request (ASR) to hook the system into MOS and raise SRs pre-emptively.

Patches for ODA are delivered as a single package, which contains firmware, Linux and database patches all together, though it does appear there is some control over when you apply each type of patch. Apparently the database patches are rolling (always?), though the OS ones may not be (I don't see why you can't patch one host at a time myself though). Currently ODA runs database 11.2 and OEL 5.* (UEK) - when it is upgraded to database 12 (an out of home upgrade) it sounds as though that could be rather more involved.

An interesting question was raised during the session about whether one-off patches were allowed (something many of us will be familiar with, especially important with severity 1 incidents) - apparently they are, but they will need to be rolled back before applying the next patchset (assuming the patch is in the next patchset - if not presumably you will need to get another one-off patch to apply afterwards).

Whilst ODA does have OEM Database Control pre-installed, curiously it doesn't have an OEM agent - you have to do install that yourself manually. To me it sounds likely that there will be an option for this soon so that it is simple to plug an ODA into your existing OEM Cloud Control system.

As with Oracle's other engineered systems you buy your licences separately to the ODA. You must use Enterprise Edition of the database with ODA, though when questioned as to why Standard Edition (which would include RAC in this configuration) wasn't allowed the answer wasn't very clear. If you have additional options (Spatial was mentioned) you can install those yourself, though I have to wonder whether this would be sensible as the further you deviate from standard the less you benefit from the 'factory' testing of the whole unit.

The ODA is priced at $50k list.

Analysis

Specification

My first obvious observation is that the X5675 (Westmere-EP) is not the latest generation of Intel server processors (Sandy Bridge-EP), some of which have been out for nearly a year now. I asked whether this specification was being updated: apparently a new revision of the ODA due out in the Spring. It wasn't stated but I suspect there's a fair chance the new processors will be the same as those in the Exadata X3 models, i.e. E5-2690 CPUs (8 core, 2.9GHz) and the memory will probably be upgraded to 128GB RAM per node. An obvious corrolary to this is that you will then have 2 x 16 core servers, i.e. 32 cores in total, requiring 16 Oracle Processor licences (if you are not licensing by Named User Plus). Of course you may chose to sub-license 24 cores like the previous model should you chose.

Whether the SSD is best used for redo logs is hard to be sure, and is likely to be application specific, though is probably a fair guess for many OLTP systems.

Licensing

Talking about licensing let's think about that for a moment. Let's assume you run RAC databases, fully populated, and license the ODA by processor (12 for the current model), then the list price (based on 3/12/12 price list) is:

• ODA hardware: $50k
  
• Oracle Database Enterprise Edition: 12*$47.5k
• RAC: 12*$23k
• Partitioning: 12*$11.5k
• OEM Diagnostics & Tuning Packs: 12*$10k
  

Note I am assuming that it would be highly desirable to have partitioning on a database of the size that would fit on these servers, plus the OEM packs are an obvious additional purchase (if you're buying a system where the configuration work has been done for you you're unlikely to want to do tuning by hand).

So this gives us a list price for a typical ODA of $50k for hardware and just over $1.1M for licences. Even if you allow for a good discount on the licences this probably still means the licences are an order of magnitude more expensive than the hardware.

You can see why people might be wondering about Standard Edition (SE) though - licensing the full ODA with SE (where you licence by socket not by core, i.e. 4 processor licences) at $17.5k each gives $70k list for the software, i.e. under a fifteenth of the price, albeit without partitioning or diagnostics like AWR.

Interestingly ODA does have a variable pricing model where you buy the hardware but only have to licence the number of processors you need, and can add licences later. I wonder whether this means you can also only licence RAC and, say, partitioning on a subset of cores (though you would need multiple Oracle homes to support that)? Another appeal of this might be if you have an "odd" number (say, 10) of licences already and don't want to buy any more to fit them onto a new server (Oracle doesn't recognise BIOS disabling of cores as a valid means of hard partitioning).

Note that ODA is a purely physical machine; if you are looking for the increased levels of agility a virtualised environment would typically offer you currently need to investigate the OEM 12c Cloud Management Pack for Oracle Database ($5k per Oracle Processor).

Backup, Recovery and Disaster Recovery

This would work the same as mainstream Oracle database practice, i.e. typically RMAN for backup/recovery and Data Guard for failover to a remote DR site.

Configuration

With ODA it sounds as though you end up with a very standard, OFA-style RAC cluster, running on Oracle Linux. It ought to be as easy to maintain as any other RAC system.

A couple of interesting points jump out around the 2*1GbE internal interconnect between nodes. Firstly these are wired together with cross-over cables (fibre according to Andy Colvin) instead of via a switch, which is something Oracle never used to support (see MOS). Secondly, I have heard Oracle recommending only 10GbE for the interconnect recently and clearly ODA will only reach 2GbE (even when allowing for active-active connections with 11.2 HAIP). Of course both of these restrictions can be stress tested by Oracle on standard ODA configurations but sometimes you wonder whether there's one rule for Oracle and another for the rest of us!

Conclusions

As soon as you start thinking about Enterprise Edition (EE) licensing you realise that the ODA makes quite a lot of sense since, even if it works out more expensive than, say, IBM, HP or Cisco (I haven't looked at their prices for this exercise), it is obvious that the hardware cost is not a very significant part of the whole solution. The exception to this is if you are considering a hardware refresh and already own the necessary enterprise licences (or have a ULA covering these products) in which case the licence cost drops out of the equation.

ODA's strengths are having a standard configuration deployed in an automated manner, needing minimal design decisions by the customer, and a consolidated patching regime. This in turn ought to reduce the time spent by customers working with Oracle Support and quite possibly lessen outages, especially at sites that aren't so proficient with RAC. I can see this being attractive if you have an Oracle database estate but not much RAC too, for example if you had active/passive database clusters protected by, say, Service Guard or Veritas, but wanted to move to RAC - the ODA does all the messy configuration work for you. Finally, it will also benefit from the Exa* model of everything being handled by the Oracle DBAs, i.e. minimising the political issues you sometimes see where storage, operating system and network are handled by autonomous teams.

ODA's achilles heel is that it is Enterprise Edition only so, if you can manage without EE features (e.g. by using a 3rd party product instead of Data Guard), then you will be able to build your own system much more cheaply (from 3rd party or even from an Oracle X4370M2 though it doesn't look like that's available stand-alone)... of course depending on how much value you place on the automatic configuration and patching.

The other consideration is how many of your production Oracle databases will fit in 4-6TB and how many peak IOPS you need compared what to ODA can serve up (I haven't read this anywhere yet - I'd guess 1500 or so). Of course you could easily have 2 or 3 ODAs but there is probably a point where moving to a more traditional arrangement of servers and centralised storage becomes more cost-effective and easier to manage. Alternatively from Oracle's view this is the point you'd be looking at an eighth or quarter rack of Exadata instead.

References

In researching this post I found that Andy Colvin from Enkitec has already written a couple of excellent articles on the ODA which go into further detail:

http://blog.oracle-ninja.com/2011/09/inside-the-oracle-database-appliance-part-1/

http://blog.oracle-ninja.com/2011/12/inside-the-oracle-database-appliance-%e2%80%93-part-2/