Admin Server Separation - Pros and Cons
16 Jun 2013 by Simon Haslam (in HA) | Comments (7)
|The majority of my Fusion Middleware work is focussed around
designing application infrastructure providing appropriate levels of
performance, resilience, flexibility and cost; one topic that comes up a
lot is how to deal with WebLogic admin servers.|
Personally I feel quite strongly that, for all but the simplest installations (or those where you have a tight installation package, such as Forms or EM 12c), the admin server should be run on a separate host (typically a virtual machine) from the managed servers. Curiously, this was one of the first conversations I had with Jacco Landlust maybe 4 or 5 year ago, and it turned out he had, independently, come to the same conclusion... maybe this was a good omen for our various colaborations since! Between us we have built perhaps a dozen Fusion Middleware infrastructures using this approach and my opinion hasn't changed.So, let me outline the considerations for separation (though this also strays into the debate over admin server high availability - HA - which deserves a post of its own):
Jacco Landlust & Simon Haslam co-presenting at UKOUG 2012
Firstly, the Admin Server for HA has to run active/passive. This is different from most services on managed servers which run active/active. Even when you look at singletons like JMS queues, or some services within OSB, you still have a relatively straightforward method within WebLogic to handle that. So architecturally I would say that the admin server is different to the managed servers.
I appreciate that some will dispute that Admin Server should be highly available. Yes it's true that the WebLogic management part of the admin server, strictly speaking, only needs to be running to make changes to the domain; in fact some purist Java EE sites only run the admin server for the duration of changes, shutting it down afterwards - the rest of the time all monitoring, restarts and so on, are handled by hand-written WLST scripts and/or JMX utilities. However in Oracle environments the OEM Fusion Middleware Control (FMC) is becoming increasingly important - you only have to see the functionality that Oracle adds to it in each release. The "old school" view for admin server protection is to copy round the domain somehow (re-mount the LUN, use a clustered filesystem, rsync or just plain FTP) and if the server hosting the admin server fails, along with the set of other more business critical managed servers, you "just" have to shuffle the VIP around, mount the LUN or copy files, and then start the admin server in its new location. Don't forget all this tinkering about is on one of your production managed server hosts and so very close to live services - if you only have two servers, on the last one. My view is that modern systems shouldn't need this kind of maintenance - even following a single hardware failure they should continue to "just work" without any expert intervention. For me this means high availability for the admin server too.
Then some people say the admin server footprint is very small so it doesn't matter where it runs. Well - yes and no. Firstly, anywhere you expect the admin server to run you need to leave spare resources for it - in memory that's currently around 1GB for 11g (WLS and FMC), but could well be closer to 1.5GB for 12c, plus a processor core or two (especially if you have SOA and don't want instance tracking via FMC to have an impact). Production Oracle licences are normally determined by cores, so if you are reserving a core or two for it that has an implied cost.
Another objection to the idea of admin server separation is that Oracle doesn't mention it in the documentation. In the EDGs for example they just say set up an administration VIP and refer you to the general CFC section of the HA Guide to think about what to do with it. Also Oracle employees don't talk about separating the admin server - possibly because no-one's standing back and questionning the conventional wisdom (developed maybe 5-10 years ago before widespread virtualisation), or maybe because they'd be accused of trying to increase the licensable footprint, I don't know. Anyway, there is a glimmer of hope - the newly architected "WebLogic running on Oracle Database Appliance" has the Administration Server running in its own VM.
Finally admin server separation offers a possible extra degree of security isolation: this is an argument I personally haven't used but one that could be relevant in very secure environments (e.g. those sites running products like Tripwire to lock down hosts). The admin server is pretty powerful - it can shut down any server in its domain, change the configuration, change the security providers and so on. A compromised host running a Managed Server could only really have an affect on itself which, whilst bad enough, could be self-contained. You could put separate administration hosts into a more highly trusted security zone than the managed servers, and restrict access to administrators. You can imagine an array of management products in such a zone: administrations servers, identity management consoles, enterprise manager OMSes etc.
Now let's look at the apparently biggest downside to separating the admin server onto its own server/Virtual Machine: licensing. You have to pay for a licence and support for the admin server (unless it's in a VMware VM in which case you're stuck paying for the whole host anyway), and this licence needs to cover the same products used in the domain (e.g. a SOA admin server will need WebLogic Suite plus SOA Suite). However you should be able to license the admin server by Named User Plus (NUP) minima, plus whatever you need for fail-over. On this basis separating the admin server for a cluster of 2 quad core x86-64 production servers adds 1/5 of a processor licence or 5%. Obviously for larger clusters this becomes even less significant. If you have a precisely sized system you could even argue that it saves you money as those cores used by the few administrators and licensed by NUP cost only 20% of the cores used by the managed servers. Note I have ignored hardware cost here because it's not usually very significant compared to the Oracle licence.
So, in summary, this is why I think you should separate the Admin Server for well-engineered, high availability OFM infrastructures:
- Architectural cleanliness
- Ease of managing admin server failover
- Isolation from, and risk reduction for, transactional servers
- Improved layer of security possible
- More efficient licensing of the managed servers
Jacco and I have been talking about separated admin servers at Oracle OpenWorld and elsewhere for at least a couple of years but very few Fusion Middleware administrators seem to consider it. I hope this post presents some convincing arguments and I'd welcome any comments, for or against!