Oracle Blog

Kedar Mhaswade's Weblog

Wednesday Jul 09, 2008

Disclaimer: I don't think this is not obvious (at least to some) and it (this observation) might have been made by many. I just reached it independently.

Javadoc is a great tool. It helps understand the basic hierarchy of an API and its public implementation. It's no replacement for looking into the source code, however. Sometimes, it might be misleading to draw conclusions based on what normal Javadoc shows, without looking into the code. Here is an example:

DAS: Domain Admin Server (One per domain) -- The process that controls the management of the entire domain.

NA: Node Agent -- Generally, one per box or Solaris container -- The process that controls the life cycle of server instances.

SI: Server Instance -- The real Java EE instances that run user applications in an enterprise.

Answer:

1. Background: The domain.xml controls the configuration. At every node-agent, there are also a few configuration files that are consulted by every NA. See NA section at docs.sun.com for details. Following are the points in time when the communication (for administration/management purpose) happens:

DAS communicates with each NA: Only when DAS needs to know NA's running status.

DAS communicates with each SI: When DAS needs to know SI's running status and when it needs to cascade the SI MBeans into the DAS's MBenServer.

NA communicates with DAS: During initial rendezvous (which may happen during creation of NA), synchronization of the NA itself and synchronization of each SI that NA is responsible for.

SI communicates with the DAS: Never, explicitly.

Thus, the communication is mainly driven by DAS. When the domain is created, the administration is configured to use an authentication realm named admin-realm. This realm points to what's called a FileRealm which is nothing but the implementation of a security realm implementation that uses admin-keyfile. If you see the domain's configuration, you'll find this file in config folder of that domain.

The communication happens over two channels. One is the HTTP channel and the other is RMI channel. For this purpose, there is a SynchronizationServlet and a System JMX Connector (standard in JDK 5) that is provided. Every DAS and SI, including the NA start a JMX RMI ConnectorServer that can be optionally configured to use transport layer security.

Every NA communicates with DAS multiple times, but the key points are of initial hand-shake and synchronization. The initial hand-shake is when NA makes DAS aware of its own existence and DAS correspondingly responds if it has the correct credentials. When the DAS is configured to have secure access (this is the default in enterprise profile domain), both the HTTP and JMX/RMI channels use Transport Layer Security with SSL/v3. Note that during the initial hand-shake, the DAS knows about NA's existence alone. DAS does not release the contents of the domain's repository during this phase. This happens over HTTP channel since creation of node-agent takes the DAS's admin-port (default: 4848) as an option.

After an NA is created, the most natural step is to start that NA. This is done by executing the asadmin start-node-agent command. Since this is the first-time startup of the NA, NA syncs up with the DAS. Note that startup of NA requires the correct credentials (admin user name and admin password) to be supplied. The DAS compares them against its own admin-keyfile and the communication succeeds only when this succeeds. The NA startup also requires the master password to be provided on the command line because in order to start, the NA has to be able to unlock the security store (e.g. keystore.jks) that it synced from the DAS. Note that master password is never put on the wire! It has to be provided at the time of both DAS startup and every NA startup. For advanced use cases, there is an unattended boot scenario that is handled by using the option --savemasterpassword which should be used with care.

The reason NA needs the master password is also to pass it on to the SI's it starts (as part of start-instance or start-cluster) so that these instances are able to unlock the security store to get the primary keys and certificates.

The NA always communicates with the DAS over JMX/RMI channel. Thus NA opens an RMI connection to the DAS where DAS is listening for RMI/JMX Connections. This is where the RMI Registry in DAS (default port 8686) comes into picture.

When the domain is created, it uses the self-signed certificate aliased s1as which is used for internal communication. This certificate is created anew every time a domain is created. The master password of a domain is what locks the server's keystore. In enterprise profile domain, NSS is used to manage the secure store, whereas in cluster profile domain, JKS manages the secure store. The semantics of the master password are unchanged in both the cases.

The Server Instances are synced with the DAS as part of either:

start-instance, or

start-cluster, or

start-node-agent --syncinstances procedure.

For this synchronization, they use the HTTP layer and communicate with the SynchronizationServlet that's listening for sync requests. This servlet is (of course) running in the DAS.

The server instances get the admin credentials from the node-agent process in a secure manner (using stdin). This also evident when you try to use the startserv script that's located in instance's bin folder.

The process of DAS communicating with the NA and SI's is identical in that it communicates with them over RMI/JMX in the other direction.

2. Transport Layer Security:

This is achieved when we enable the security-enabled flag on the admin-listener and jmx-connector named system on the DAS and server instances. Note that admin-listener (HTTP/S) is started only in the DAS. There is no admin-listener in server instances.

It's of course possible to use another CA-signed certificate for this purpose. It needs additional configuration after importing those certs in the store.

Monday Dec 17, 2007

This is of course a wacky idea and the one which search engines might/should jump on . I checked the working draft for HTML 5 and could not find it there.

The idea is about search engines aiding all the HTML documents that are about a certain topic and want to intersperse hyper-links to certain terms that appear elsewhere on the web. Of course, I, as an author of a document that references such concepts or terms, don't want to waste time on finding the exact web-page that demonstrates it. The intent of such a link is to provide more information about that term anyway. Then, why not have a facility like:

<a seref="Effective Java">Effective Java</a>

which will bring the page from your favorite search engine with "Effective Java" as your query string.

This is useful for the author because when the author does not really want to be specific to point to a known document, s/he can always make use of search engines which are supposed to provide more up-to-date information on a topic of general interest. More importantly, it examines if search relevance that search engines tout about is any good!

I told ya -- it is wacky , but I just implemented this in my previous post.

While I was getting used to IDEA IntelliJ IDE, one "feature" caught my attention. They have a way to create a "Singleton" by way of a pop-up menu. Here is the code that generates the class for this controversial and damned pattern (now that Google has a Singleton Detector ).

asadmin start-domain does not invoke the domain-folder/bin/startserv script anymore.

A user was using V1 in an interesting way. The assumption was asadmin start-domain will always invoke the startserv script.

Here is how:

Consider that I have a heterogeneous environment, with various libraries being used by my application. We have a complicated build environment and we finally make sure that all the libraries needed by the application are saved at a particular location say /libraries and is captured in an environment variable AS_LIBRARIES as:

export AS_LIBRARIES=/libraries/foo.jar:/libraries/bar.jar ...

Then we modified the startserv script as: -Dmylibraries=$AS_LIBRARIES ...

and finally we modified domain.xml as:

<java-config classpath-suffix="${mylibraries}" ....

/>

Now, this was clearly a hack and worked only because of peculiar way in which GlassFish V1 treats the system properties. This is clearly an area in GlassFish that needs improvement because of its over-dependence on (evil) System Properties. Here, however, the user had employed this to his advantage! In short, what he has done was:

Invented a way to pass an environment variable on the system whose value at a given point in time would be picked up by the domain dynamically, at the startup. That way, the user would just modify the value of the environment variable and restart the server to pick up the changes in the variable!

I liked the way they used it and I thought of helping him. Of course, there are better ways of doing this, but they had been having this for some time and apparently, this was the thing holding them back from upgrading to GlassFish V2.

I remembered suddenly, the GlassFish had an attribute named env-classpath-ignored which is set to true by default. This means, that whoever (user on Operating System) starts the application server, if s/he has an environment variable named CLASSPATH it is always ignored by the server by default. This works well in almost all cases, since CLASSPATH is being forgotten by the Java community.

I thought of suggesting this to him and instead of setting AS_LIBRARIES, if CLASSPATH is set as the environment variable in this user's environment, the problem would be solved.

The way the launcher deals with this is:

Look at the env-classpath-ignored and it it is set to false (non-default case), append its value to the -cp argument to the JVM.

Effectively place the jars pointed to by CLASSPATH into the application server JVM's System ClassLoader.

Lessons learned:
- Don't rely on private interfaces of a product. Hacks are OK, but look at the consequences.
- Consider using ${path.separator} instead of ':' or ';' if you want GlassFish to virtualize the operating platform for you. So, while setting path-like entries in domain.xml, always use ${path.separator}. GlassFish is designed to do the right thing.

Sunday Sep 30, 2007

It (Web 2.0?) has definitely started to change how we look at the Web. It is a collaboration of various technologies that is ultimately going to make the Internet a better place for everyone.

Translation is probably still a topic of active research.

I am trying to suggest an approach to translate TheAquarium into various languages so that developers around the globe can experience it and get benefited by GlassFish technologies.

Yahoo! Pipes is an interesting technology but I am not going to delve into it because there are various resources that help you get started with it. When I looked into it, the first thought that came to my mind was to use it for GlassFish and then I saw the translation module. It sort of guided me into defining the pipes as shown in attached graphic.

This shows the pipe that takes the English feed from feedburner (which is rather weird) and then translates the titles into French and creates the list.

I bet this technique can be exploited to translate the entire posts so that Global Developers are attracted to TheAquarium. We might not need the Language Buttons any more (although they serve a slightly different purpose).