Krishnan Subramanian

Java & SSL

Krishnan Subramanian — Thu, 28 Jul 2005 13:47:27 +0000

Java provides a whole slew of standardized APIs for everything under the Sun (pun intended) ranging from database access to security to protocols for remote access etc. Recently I was at a customer in Stockholm for 2 weeks – getting a testing/staging environment in place with a number of test and load scripts.

To put a long story short, a number of load test scripts I was writing required SSL communication and though I had written applications that communicated over SSL with pluggable authentication & authorization providers, I had never really gotten into SSL at the protocol level and decided to do that after my customer engagement.

SSL was originally developed and published by Netscape. SSL version 3.0 formed the basis of the Transport Layer Security (TLS) protocol version 1.0.

TLS 1.0 has 4 goals:

· Security (establish a secure connection between end points for communication)

· Interoperability (interoperate with different providers, implementations etc.)

· Extensible (new key & encryption methods can be added without creating a new protocol)

· Efficient (CPU & network utilization should be reduced)

The protocol stack looks something like this:

TLS Handshake | TLS Change Cipher spec | TLS Alert | HTTP
TLS Record Protocol
TCP
IP

The TLS Record Protocol encapsulates higher level protocols and has two basic properties:

· Connection is private – that is, keys for encryption are generated uniquely for each connection

· Connection is reliable – that is, tampering of data can be detected by integrity checks

It might be useful to understand how TLS operates. The essential sequences of steps for the handshake protocol are:

1. client sends a hello with security capabilities it (the client) knows about

2. server chooses the best capability from the client hello and responds with a chosen capability in its hello response (server hello)

3. server sends authentication information about itself and sends a key

4. server hello is done

5. client generates a key and sends it to server encoded with key sent by server in step 4

6. client finished

7. server finished

8. all subsequent communication between client and server switch to key generated in step 5

Asymmetric cryptography – also known as public key cryptography uses an encryption algorithm in which two keys are produced. One key is made public and the other is private. These keys are inverses of one another – that is, what one key encrypts, the other decrypts and vice-versa. And it is not computationally feasible/possible to derive the key inverse from one key. Examples of asymmetric algorithms include RSA, DSS etc.

In symmetric cryptography – also knows as secret key cryptography, the same key is used to encrypt and decrypt data.

Asymmetric cryptography is generally more computationally intensive compared to symmetric cryptography.

TLS 1.0 uses a combination of both asymmetric and symmetric cryptography. First, on the server side, a pair of public and private keys is generated. The private key is known only to the server while the public key can be distributed to clients. However, any party can supply a client with a public key thereby masquerading as a server. To solve this problem of public key distribution, certificates were introduced as a safe mechanism for an entity to pass on its public key to clients. The certificate is a digital equivalent of a passport and is issued by trusted organization usually called a certificate authority (e.g. Verisign, Thawte etc). The certificate authority will provide a certificate only upon proof of identity of the requester of the certificate. The certificate contains a number of fields including issuer (the certificate authority), the time period for which the certificate is valid, information about the entity that the certificate represents, the public key and the digital signature of the certificate authority that issued the certificate.

Thus, the recipient of the public key can examine the certificate and decide if it chooses to accept the public key.

To ensure data integrity TLS uses a cryptographic hash function. A hash is usually a fixed length of bits generated from a message. A cryptographic hash function is very similar to a checksum. While a checksum is used to check damages to data in transit, a cryptographic hash function is designed to check deliberate changes to data. Typically a cryptographic hash function will produce a large change to a resulting hash given a small change to the message. It is important to note that cryptographic hash does not require a key. Popular cryptographic hash functions include MD5, SHA-1 etc.

The TLS process begins by a TLS handshake as was explained previously. The details are:

1. Client sends a hello with the protocol it supports and the list of cipher suites it supports. The cipher suite contains information about the algorithm used for asymmetric cryptography (key exchange), symmetric cryptography and hash function (for data integrity and message authentication). The cipher suite will typically contain a list with the preferred suite on top. For example, one of the suites in the list might be: SSL_RSA_WITH_RC4_128_MD5. This indicates that the RSA algorithm is used for public-private keys (asymmetric cryptography), a 128 bit RC4 algorithm is used for secret keys (symmetric cryptography) and the MD5 cryptography hash function is used for data integrity and message authentication. The client hello message includes a list of compression algorithms if supported.

2. The server chooses the top cipher suite and protocol it knows about and responds with that information as a part of its server hello message.

3. The server sends its public key certificate to the client (server authentication)

4. The server completes its hello message

5. The client generates a pre-master secret, encrypts it using the public key from step 3 and sends it to the server. Since the private key is only known to the server, only the server will be able to decrypt the message.

6. Client sends a finish message – encrypted with the just negotiated algorithms and keys. The finish message includes the content of all previous handshake messages negotiated up until this message and its hash (Change Cipher Spec is done, but is not exactly a part of the handshake protocol and the finish is sent after the ChangeCipherSpec. Implies that communication will shift to using symmetric cryptography negotiated).

7. Server sends an equivalent finish message. The previous handshake messages included in the finish message with [its] hash prevent an eavesdropper from modifying any of the handshake messages (Change Cipher Spec is done, but is not exactly a part of the handshake protocol and the finish is sent after the ChangeCipherSpec. Implies that communication will shift to using symmetric cryptography negotiated).

8. From the pre-master key, the client and server arrive independently to the same master key value (symmetric cryptography) that is used for subsequent encryption of messages between the client and the server for that session.

On the Java side, Java provides an implementation of the TLS (version 1.0) and the SSL protocol (version 3.0) including encryption, server authentication and message integrity. The API is called the Java Secure Socket Extension (JSSE) and bundles an implementation of that API as well.

In Java, certificates have to be placed in a keystore - a Java Key Store to be precise. The keytool command line utility (bundled with the JDK) can generate test certificates or import existing certificates (issued certificates from a certificate authority). A keystore is standard mechanism to transport keys and certificates in a password protected file.

For example, to generate a RSA keypair, you would run:

$prompt> keytool -genkey -alias serverCert -keyalg RSA -keysize 1024 -validity 9999 -keystore server.ks

Enter keystore password: trustno1
What is your first and last name?
[Unknown]: Krishnan Subramanian
What is the name of your organizational unit?
[Unknown]: Professional Services
What is the name of your organization?
[Unknown]: Borland Software Corporation
What is the name of your City or Locality?
[Unknown]: Amstelveen
What is the name of your State or Province?
[Unknown]: North Holland
What is the two-letter country code for this unit?
[Unknown]: NL
Is CN=Krishnan Subramanian, OU=Professional Services, O=Borland Software Corporation, L=Amstelveen, ST=North Holland, C=NL correct?
[no]: y
Enter key password for
(RETURN if same as keystore password):

$prompt>

Initially when the keytool’s –genkey option is used, an RSA keypair is generated and the public key is wrapped in a self-signed certificate. Self-signed in the sense that the issuer of the certificate is the same as that of the entity for whom the keypair is being generated.

If you were to apply to a commercial certificate authority (e.g. Thawte, Verisign etc) you would generate a certificate signing request (CSR) using the keytool’s –certreq option on the above generate keystore and submit the output of the certificate signing request along with necessary documentation proving your organization’s identity among others. After verification of documents, the certificate authority will issue you a certificate that you can then import into the keystore.

However for testing purposes, it is overkill to apply to a certificate authority (fees for a certificate are usually a few hundred dollars). Remember that during the TLS handshake process, the server will send its certificate to the client. And it is up to the client to decide whether it will trust the certificate sent by the server. The trust is usually established by checking the issuing authority. There may be a certificate chain in which case, the client will navigate the certificate chain until it finds an issuing authority it will trust.

You can check the content of the keystore using the keytool’s –list option. For example,

$prompt>keytool -list -keystore server.ks
Enter keystore password: trustno1

Keystore type: jks
Keystore provider: SUN

Your keystore contains 1 entry

servercert, Jul 28, 2005, keyEntry,
Certificate fingerprint (MD5): 3E:4C:D2:5F:D7:1C:F1:BF:5C:C2:CE:20:17:52:94:69

The above entry displays one keyEntry (that is, a public/private key pair). It is important to note that the private key in the keystore is not accessible using the keytool utility.

For an SSL Server application, for example – a Socket based application; you would specify the location and password of the keystore as JVM parameters.

Here is a sample Socket Server application:

import java.io.*;
import java.net.ServerSocket;
import java.net.Socket;
import javax.net.ssl.SSLServerSocket;
import javax.net.ssl.SSLServerSocketFactory;

public class SocketServer {

  public static final boolean USE_SECURE = true;
  public static final int PORT = 6666;
  public static final String PASS_PHRASE = "knock knock";

  public static void main(String[] args) throws IOException {
    ServerSocket mServerSocket = null;
    if (USE_SECURE) {
      mServerSocket = getSecureSocket();
    }
    else {
      mServerSocket = getPlainSocket();
    }
    System.out.println("SocketServer listening . . . ");
    startListening(mServerSocket);

Share This | Email this page to a friend


Extending Container Managed Persistence (CMP) Types
Krishnan Subramanian — Wed, 22 Dec 2004 16:25:09 +0000
Entity beans represent an object view of data - usually stored in a relational database. The EJB 2.x specification does not limit the mapping of entity beans to a relational database although an RDBMS remains the most popular choice of supported persistence for most CMP engine implementations.
 Entity beans have been bashed - and continue to be bashed - for many reasons. The most common ones being performance (or the lack thereof) and the amount of necessary boiler-plate code to be written. The former is more of an implementation issue - that is, the responsibility of the Container vendor. And the latter is mostly due to the lack of proper tool support for treating EJBs as first class citizens. You can read some rantings concerning the latter at http://blogs.codegear.com/krish/archive/2004/11/12/1808.aspx
 But a majority of the entity bean bashing is quite naive and by people mostly misinformed about the problem space entity beans attempt to address and when/why they should be used. Many cite performance as a factor. If I were to state that a popular Open Source AppServer provides poor scalability and performance for entity beans, what would the reaction of the community be? Indifference? If I went further to state that for an out-of-the-box comparison (only JVM heap sizes adjusted) of this Open Source product and the Borland Enterprise Server (BES), the latter led by a margin of 20x (2000%) in terms of raw throughput (transactions per second) and 3x (300%) after extensive tuning of both products, would the reaction of the community still be one of indifference? And that a raw-SQL (using DAO from a session bean) version of the benchmark on the Open Source AppServer showed it still lagging behind in performance compared to the entity bean version of the same benchmark running on BES? I might publish the results of that benchmark if I get some response  
 But I digress.
 Most CMP 2.0 engines provide support for the following Java types that map to the specified SQL Types (these types are defined in the class java.sql.Types).
 Java Type                                 SQL Type
————————————————–
boolean/Boolean                           BIT
byte/Byte                                 TINYINT
char/Character                            CHAR(1)
double/Double                             DOUBLE
float/Float                               REAL
int/Integer                               INTEGER
long/Long                                 BIGINT
short/Short                               SMALLINT
String                                    VARCHAR
java.math.BigDecimal                      NUMERIC
byte[]                                    VARBINARY
java.sql.Date                             DATE
java.sql.Time                             TIME
java.sql.Timestamp                        TIMESTAMP
java.util.Date                            TIMESTAMP
java.io.Serializable                      VARBINARY
 In addition to these types, most Containers also provide support for LOBs (Large Objects) such as Oracle CLOBs (Character LOBs) and BLOBs (Binary LOBs). Many Containers also provide support for dependent objects, the fields of which are again are limited to the above listed types. Now assume that I have a type that is not listed or supported by the methods in the generic interfaces in the the java.sql package but rely on specific extensions provided by the database and the JDBC Driver. How do I map those to a CMP field?
 Let’s take a specific example. I was recently at a BES customer here in Europe in the first week of Dec 2004 (the weather there was 17C compared to the almost freezing weather in Amsterdam). I had to work with a team of 5; all of them very intelligent and with a great sense of humour. They were using Oracle 9i and were using an Oracle specific type - XMLTYPE as one of the columns in the database. This specific type is stored internally as a CLOB, but cannot be accessed directly as a CLOB by the JDBC APIs. To access this field, you would have to use oracle.jdbc.OracleResultSet specific methods along the following lines:
   // …. 
  oracle.jdbc.OracleResultSet oracleRs = (oracle.jdbc.OracleResultSet) resultSet;
  oracle.jdbc.OPAQUE value = oracleRs.getOPAQUE(index);
  if (!oracleRs.wasNull()) {
    String xmlString = oracle.jdbc.XMLType.createXML(value).getStringVal();
  }
 And some similar code to set/modify the value.
 The problem of course is that if you were using CMP, what type do you use as the CMP field? And more importantly, how do you map that field to the Oracle XMLTYPE in the database?
 Borland Enterprise Server provides a mechanism to do so. How this work in short is that whenever a particular column type (CMP field) matches some criteria you specify (either the CMP field type or CMP field name or both), the Container can delegate the accessing/mutating of this field to custom methods you implement. In all cases, there will be no code changes involved in the entity bean or its generated code or its persistence manager. BES provides the following abstract interface:
   package com.borland.ejb.pm;

  import java.rmi.RemoteException;

  public abstract interface JdbcAccesserFactory {
    JdbcGetter getGetter(Class cmpFieldType, String cmpFieldName) throws RemoteException;
    JdbcSetter getSetter(Class cmpFieldType, String cmpFieldName) throws RemoteException;
  }
 The implementation of this class has to create its implementation of the abstract class JdbcGetter and JdbcSetter. The custom implementations of JdbcGetter and JdbcSetter have to override methods to get or set the value using specific APIs. Methods in the JdbcGetter and JdbcSetter get passed the java.sql.PreparedStatement, java.sql.ResultSet and the value of the CMP field as parameters. From these, you can cast them to Oracle specific types and use Oracle JDBC Driver specific methods to get or set the value of the CMP field. To use our implementation of the JdbcAccesserFactory, an XML entry in the ejb-borland.xml (Table CMP property) instructs the EJB Container to use our class for accessing the value in the database.
 In the case of an XMLTYPE field in the Oracle database, the first observation might be to map things to a CMP field of type java.lang.String and then undertake the conversion of this String field to Oracle XMLTYPE via our custom JdbcGetter and JdbcSetter implementations. Mapping directly to a String field is possible, but requires some more code to ensure that other java.lang.String fields in the same entity bean do not get mapped to XMLTYPE as well. For example, I might have 10 java.lang.String CMP fields in the same entity bean, of which 9 of them map to an Oracle VARCHAR2 and only one maps to an XMLTYPE. Then I have to write extra code to ensure that only the java.lang.String CMP field that maps to an Oracle XMLTYPE should be handled by my JdbcAccesserFactory implementation.
 To get around these, for simplicity lets assume that I create a custom holder class along the lines of:
   package com.whatever.data;

  public class MyXmlStringHolder implements java.io.Serializable {
    private String xmlValue;
    public String getXmlValue() { return this.xmlValue; }
    public void setXmlValue(String newValue) {this.xmlValue = newValue; }
  }
 And the above class is the one I use as my CMP field for the column that has to map to an Oracle XMLTYPE. So my entity bean would look like:
   public abstract class OurEntityBean implements EntityBean {
    // skipped some methods …. 
    public abstract void setXmlMessage(MyXmlStringHolder sh);
  
    public abstract MyXmlStringHolder getXmlMessage();
    // here xmlMessage is a CMP field that maps to some
    // Oracle column of XMLTYPE
  } Now for the custom JdbcAccesserFactory implementation. This would be along the lines of:
 package com.borland.ejb.pm.oracle;
 import java.rmi.RemoteException;
 import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Types;
 import com.borland.ejb.pm.JdbcAccesserFactory;
import com.borland.ejb.pm.JdbcGetter;
import com.borland.ejb.pm.JdbcSetter;
import com.inprise.visitransact.jdbc2.PreparedStatementProxy;

import com.whatever.data.MyXmlStringHolder;
 import oracle.jdbc.OraclePreparedStatement;
import oracle.jdbc.OracleResultSet;
import oracle.sql.OPAQUE;
import oracle.xdb.XMLType;
 public class OracleXmlAccesser implements JdbcAccesserFactory {
  public OracleXmlAccesser() {
  }
   public JdbcGetter getGetter(Class fieldType, String fieldName) throws
      RemoteException {
    if (fieldType.getName().equals(MyXmlStringHolder.class.getName())) {
      return new XmlJdbcGetter();
    }
    return null;
  }
   public JdbcSetter getSetter(Class fieldType, String fieldName) throws
      RemoteException {
    if (fieldType.getName().equals(MyXmlStringHolder.class.getName())) {
      return new XmlJdbcSetter();
    }
    return null;
  }
   private class XmlJdbcGetter extends JdbcGetter {
    public Object get(ResultSet rs, int index, ClassLoader cl) throws
        SQLException {
      MyXmlStringHolder ret = new MyXmlStringHolder();
      OracleResultSet oracleRs = (OracleResultSet) rs;
      OPAQUE value = oracleRs.getOPAQUE(index);
      if (oracleRs.wasNull()) {
        return ret;
      } else {
        ret.setXmlValue(XMLType.createXML(value).getStringVal());
        return ret;
      }
    }
  }
   private class XmlJdbcSetter extends JdbcSetter {
    public void set(PreparedStatement ps, int index, Object object) throws
        SQLException {
      MyXmlStringHolder holder = (MyXmlStringHolder)object
      if (holder.getXmlValue() == null) {
        PreparedStatementProxy proxy = (PreparedStatementProxy) ps;
        OraclePreparedStatement ops = (OraclePreparedStatement) proxy.getPreparedStatement();
        ops.setNull(index, Types.VARCHAR);
      } else {
        PreparedStatementProxy proxy = (PreparedStatementProxy) ps;
        OraclePreparedStatement ops = (OraclePreparedStatement) proxy.getPreparedStatement();
        ops.setObject(index, XMLType.createXML(ps.getConnection(), holder.getXmlValue()));
      }
    }
     public void set(ResultSet rs, PreparedStatement ps, int index, Object object) throws SQLException {
      set(ps, index, object);
    }
  }
 }
 And the above class would be packaged as a library and present on the classpath of BES’s partition or deployed alongwith the EJB Module in a JAR or EAR. For an entity bean to use the above class to map its CMP field(s) to an Oracle XMLTYPE, you would need an entry in the ejb-borland.xml descriptor along the lines of:
 
 
   
    MyOracleTable
    
      MY_XML_COLUMN
      
        ignoreOnInsert
        java.lang.Boolean
        true
      
    
    
    
      jdbcAccesserFactory
      java.lang.String
      com.borland.ejb.pm.oracle.OracleXmlAccesser

And that’s about it!

-krish

Share This | Email this page to a friend

Concerns on the EJB 3.0 early draft

Krishnan Subramanian — Fri, 12 Nov 2004 00:51:53 +0000

I was going through the EJB 3.0 early draft specification - available at: http://java.sun.com/products/ejb/docs.html

And one of the things that struck me was that the specification has changed (yet again) with another incompatible conceptual leap. The move from EJB 1.1 to 2.0 was quite a big one and the move now from 2.x to 3.0 will be a bigger one from a J2EE Application Developer point of view. From an AppServer implementation (Borland Enterprise Server, WebLogic, WebSphere etc) point of view, it looks like the move from 1.1 to 2.0 was probably a bigger one than the one from 2.0 to 3.0. Of course, I’m not R&D and probably am not best qualified to comment.

However, having worked on numerous J2EE projects prior to joining Borland (~2 years), and working on customer assignments (both as a developer and a glorified trouble shooter the last 3+ years), I think I have seen enough of the good and bad to pass comment on the technology in general and its direction.

A striking thing about the draft specification is the introduction of fairly experimental technology. Like annotations. There is nothing wrong with annotations, except that most Java developers are not used to annotating their code. For example, a stateful session bean can be specified as:

@Target(TYPE) @Retention(RUNTIME)
public @interface Stateful {
String name() default "";
TransactionManagementType transactionManagement()
default CONTAINER;
}

I’m not convinced that a normal Java/J2EE developer is going to be comfortable annotating his/her code. It will probably take time getting used to (as with anything). My worry is that you normally would not change enterprise programming models to such an extent. This is not what enterprise software is about. XML descriptors are not out, but you could choose a middle path by reducing the number of required elements in the XML descriptors and having the EJB Container infer these/choose defaults based on (non-annotated) code.

The draft specification also looks like it is addressing a problem space it does not own. Namely: the lack of proper tools that manipulate EJBs as first-class citizens (components) rather than as disparate pieces (bean class, home interface, remotable/local interface, XML descriptors). This is probably why the specification requires a J2EE Application Developer to declare his bean class alongwith numerous annotations and then at deploy/runtime generate all the artifacts mentioned above (home interface, remotable/local interface, deployment descriptor settings, stubs, skeletons etc). The idea is not a bad one, except that I feel it is not the specification’s responsibility to do this. Would you embark on a enterprise scale ASP.NET project and use wordpad/notepad to write all of your code? JBuilder does a pretty good job of abstracting away all the disparate pieces and treating an EJB as a component in its EJB Designer. And you want to use a Java development tool that does at least that.

The draft specification also does not require a J2EE application developer to explicitly declare an interface as being a remotable one. Everything is a Plain Old Java Object (POJO). In my opinion, a bad idea. Often declaring an interface to be an explicitly remotable one is not necessarily a bad thing. Especially in a medium/large development team where many members may not realize that they’re manipulating an object reference as a local one when in reality all calls are being dispatched to a remote server. Location over-transparency? Other technologies require you to explicitly declare an interface as being remotable - example: RMI requires it, CORBA requires it, .NET Remoting requires it (afaik), so why move away from common practise?

A plain daft goal of the draft specification is: testability outside of the (EJB) Container. What does this mean? If we take a look at the most popular testing framework for Java code out there, it is junit. JUnit is such a basic framework that you start wondering how you’d test an enterprise application outside of a Container that provides all services essential for an enterprise class application. Err, I might as well say - I have a database driven application that I want to test without a database.

In the draft specification, it also seems like an entity bean is a data transfer object (or value object) is an entity bean. There is ‘detached’ model and a ‘managed’ model for entity beans when they are used by code in a (remote) client and code inside a container respectively. Often, applying basic modeling concepts tell you that you want to reduce coupling. But in this case, we’re encouraging client applications to access our domain/persistence model directly - albeit in a ‘detached’ mode with outstanding changes to be resolved back later (managed mode). I do not like this idea of blurring of lines between what consitutes your persistence object and what consitutes data objects (that are passed between tiers). And would you want to couple your client applications to your domain model?

On a related note, entity beans also expose a number of lifecycle methods to the developer. That is, apart from the EJB Container deciding when to read data from the database and when to write data (depending on transaction boundaries), the developer can choose to control these by calling the lifecycle methods explicitly. This might not be a good thing and might even encourage developers to start thinking about lower abstraction levels and mixing them up with business logic abstractions. Well, if we’re going to think and control many aspects we might as well go all the way and write our own EJB Container in the meantime These might also interfere with the different transaction commit options and caching decisions that can be specified for entity EJBs (pure deployment descriptor entries and therefore no touching of code required).

The draft specification also introduces the notion of Named Queries that can be specified as:

@NamedQuery(
name="findAllCustomersWithName",
queryString="SELECT c FROM Customer c WHERE c.name LIKE :custName"
)

Named queries could be problematic since they are declared by name and called by name. There are possibilities of developers mistyping things. And your application blows up at runtime. Oh wait, I want to test my application outside the Container, err umm uhhh!

-krish

Share This | Email this page to a friend

Re: ECO : Optimistic Locking

Krishnan Subramanian — Fri, 16 Jul 2004 15:12:59 +0000

First off, this post is in reply to a blog entry by Malcolm Groves at: http://blogs.codegear.com/malcolmgroves/archive/2004/06/28/659.aspx on Optimistic Locking done by ECO.

I first thought of replying on Malcolm’s blog site but then thought my entry might deserve a place of its own (presumptuous, but not vain I also come from the Java, J2EE world - and I have a deep respect for a lot of technologies in Delphi and Delphi.NET. Some of views on the topic might be bit off since I might not know how things work exactly (only plain ignorance ;). I have to admit that most of these issues have already been thrashed around in the J2EE world and Borland Enterprise Server provides a piece of functionality that is almost identical by name - but differing in its implementation. You will find a link to this functionality at the end of this post.

Malcolm’s entry stated how Optimistic Locking behaves when set to ModifiedMembers and when set to AllMembers. What caught my attention was that:

Two SQLs are sent to the database for every update to the database.
The isolation level for the above two SQLs themselves are seemingly undefined.

Firstly, it seems like sending two SQLs to the database for every update to the database is not a very scalable solution. One, it introduces more RPCs into the picture if the ECO Server-side application and database are on different machines. Two, it imposes a higher load on the database than necessary. That is, there are now more CPU cycles utilized at the database-tier. This could very quickly lead to the database-tier becoming a performance bottleneck which will then ripple up tiers to manifest itself as a scalability problem. This whole issue of two SQLs seems trivial till you actually consider high-concurrency OLTP-like applications where you want squeeze every little bit of performance out of your tiers since you are typically dealing with large amounts of data.

Secondly, the last point assumes that the two SQLs will run in proper isolation. What I mean by this - is not whether the two SQLs will run in the same transaction (which they must - else ECO is doing something horribly wrong); but what the isolation level will be of that transaction. Off the top of my head - I would say you atleast need a REPEATABLE READ isolation level (most database default to READ COMMITTED in which case you still have a problem). Imagine, under a high concurrency, T1 and T2 are two transactions:

    T1 performs a SELECT (prior to its UPDATE)
    T2 performs a SELECT (prior to its UPDATE)
    T1 assumes it can update the database and issues an UPDATE that succeeds
    T2 assumes it can also update the database and issues an UPDATE that also succeeds

I believe the above will occur if you select (pun intended) the default READ COMMITTED isolation level.

So, what is the solution you may well ask? Or more importantly is there a solution?

Yes.

The idea is to kill two birds with one stone. Or in this, to just use one SQL in lieu of two. And what kind of an SQL would that be? Let’s see. In ECO, after the first SQL is executed, it checks to see if the value returned by the database is different from those prior to edits/changes. This implies that ECO is holding this old value somewhere - whether it be an OptimisticLocking mode of ModifiedMembers or AllMembers. So, the smart thing for ECO to do would be to use that information in its WHERE clause. Let me give an example in the context of Malcolm’s example: Damien and I are working with a Person whose first name happens to be ‘Barney’.

Damien makes a change to this Person’s first name and calls him ‘Loony’ and while I make a change and decide to call him ‘Goony’. Assuming the problem in point two (above) does not exist, ECO will issue a SELECT when Damien’s transaction has to succeed. It will find that the database contains ‘Barney’ which is what Damien started out with in the first place. His transaction will succeed.

My transaction will then issue a SELECT, find that the person’s first name is now ‘Loony’ and not ‘Barney’ - which is not what I received in the first place (I received ‘Barney’) and so my transaction will fail. All well - assuming of course that problem in point 2 above does not exist.

The other way of doing things - maybe a bit more efficient - is to let ECO do a bit more work. Since ECO ‘knows’ its old values - that is, the values it receives when it issued the request to get the data to the user, it cannot be that hard for ECO to ‘diff’ these values with what the user had edited/changed. So, ECO should know what columns have actually changed. In this context of this example, it should know that the only change is one to the firstName column. And it also ‘knows‘ what the old value is (Barney) and new value (either Loony | Goony) is. And it could use this information to construct the WHERE clause.

    UPDATE Person
        SET firstName = ‘Loony’
        WHERE Person.BOLD_ID = 3 AND Person.firstName = ‘Barney’

And the other transaction would issue:

    UPDATE Person
        SET firstName = ‘Goony’
        WHERE Person.BOLD_ID = 3 AND Person.firstName = ‘Barney’

In this case, only one of the above two transactions will succeed since the WHERE will fail if it cannot find the given row. And in the case of an Optimistic Concurrency of AllFields, the WHERE clause would contain the values of all columns that the user first read in. So something along the lines of:

    UPDATE Person
        SET firstName = ‘Goony’
        WHERE Person.BOLD_ID = 3 AND Person.firstName = ‘Barney’ AND Person.lastName = ‘Slater’ AND Person.someField = ‘oldValue’ …

So, this approach:

Uses only one SQL
Is the ‘correct’ way. No ‘holes’ to be punched through it.

And also does not suffer from any potential scalability problem (side-effect

I’d be interested in hearing other’s opinions on this . . .

-krish

Borland Enterprise Server uses the same approach: http://info.borland.com/devsupport/bes/faq/all_versions/ejb/cmp.html#_how_optimisitic

Share This | Email this page to a friend

On Logging and logging in J2EE environments (and in Borland Enterprise Server)

Krishnan Subramanian — Fri, 16 Jul 2004 07:53:19 +0000

Almost every non-trivial application - be it J2EE or otherwise includes some kind of a logging functionality. More often than not, this functionality tends to be some home-grown solution.

Some might argue that popular debuggers (such as those provided with IDEs like JBuilder, Delphi, C#Builder etc) provide a better means of tracing through code. However, this argument assumes that:

Debuggers are always available
Debuggers can be used in the context of the runtime

Logging on other hand:

Provides information on the context of execution in the runtime
Is persistent - while debugging sessions are generally transient
Requires little or no configuration/setup from a user
Requires no specialized tools (apart from the logger)
Is more suited to tracing distributed and/or multi-threaded (i.e. high concurrency) applications

This post of mine does not preclude the use of debuggers. Debuggers have their place, and can be an invaluable tool to the programmer during development. Logging can be used during both development as well as deployment. Logging of course provides its own set of challenges:

Logging statements should be inserted only where appropriate. That is, where they provide information and not nonsensical information that provides little or no value.
Logging should not be overly verbose. Keep in mind the manner in which the application will be used in a deployment environment. If 100s or 1000s of concurrent users will be using the system and each action by every user generates a lot of logging information, then imagine the amount of information generated. In which case locating useful/relevant information becomes a challenge in itself.
Logging should be fast. That is, overhead should be minimal.

Till recently, I was content with using System.out.println(“ . . . “) and System.err.println(“ . . . “) when I had to quickly prototype things or write a proof-of-concept application for a customer. I had heard about log4j - a popular open source project from the Apache Group - though I had ignored that for a while - but not for long.

log4j has three main components:

Loggers
Appenders
Layouts

An application can make use of multiple loggers to log information. Each logger can use multiple appenders to service a logging request. An appender can be thought of as a simple output destination - such as the console, or a file, or a rolling file, or a socket, or an OS event log etc. Each appender can have a layout associated with it that controls how the logging request will be formatted. That said, it is not my intention for this post to be a manual on log4j - which this is quickly becoming

Borland Enterprise Server 6.0 (BES) uses log4j as its own underlying logging system. Now, that both simplifies and complicates things for the J2EE application you deploy in BES. It complicates things because:

log4j is already bundled with BES, and is used internally by BES partitions to log their own information and statistics.
The classloader scheme in BES will probably mean that the log4j library bundled with BES (under $BES/lib) will take precedence over log4j classes you may bundle with your J2EE JAR/WAR/EAR.
log4j Loggers you use within your application code will log to the partition’s rolling log file - which will now contain both BES partition internal information+statistics as well as logging information from your own application. This problem may be alleviated a little bit - since the partition’s logger uses the XMLLayout class to write to a rolling log file. Using the chainsaw browser, you can view/filter out information you do not need.
It assumes you are familiar with log4j

It simplifies things because:

log4j is already bundled with BES, and is used internally by BES partitions to log their own information and statistics.

You can reuse the XML configuration file that is used internally by that partition without starting from scratch.

The above point implies that logging statements from your J2EE Application can be redirected to another Appender. That is, the logging statements can be redirected to another destination (like a separate log file) instead of the one used by the partition.

The XML configuration file logConfiguration.xml used by BES’s partition lies under $BES/var/domains/base/configurations/[configuration_name]/mos/[partition_name]/adm/properties. The above configuration file is set via the JVM property log4j.configuration in $BES/bin/partition.config file and used by that partition.

This file contains two appenders called PRIMARY and FALLBACK. The PRIMARY Appender is a rolling file appender that appends to the file $BES/var/domains/base/configurations/[configuration_name]/mos/[partition_name]/adm/logs/partition_log.xml

The partition writes its logging requests to the above file (using the PRIMARY Appender) using an XMLLayout (by default). This also means that with little or no configuration, your J2EE Application code can issue log4j requests to the PRIMARY Appender which will then log that request in an XMLLayout to the partition_log.xml file.

So, the sample code below should work:

package com.krish.myapp.j2ee.ejb;

// imports skipped;

public class MySessionBean implements SessionBean {
private SessionContext sessionContext;
private static final Logger logger = Logger.getLogger(MySessionBean.class);

public void doSomeWork(String arg1, Integer arg2) {
    logger.debug(“In doSomeWork“);
    try {
      // do whatever it takes
      // to finish the work
      logger.info(“Transaction succeeded. Transaction Id is: “ + arg2)
    catch (Exception ex) {
      sessionContext.setRollbackOnly();
      logger.warn(“Transaction failed. Args passed were: “ + arg1 + “ “ + arg2, ex);
    }
}

// other EJB callback methods skipped for brevity
}

Of course, the problem now is that the log requests from our EJB now goes into the same partition_log.xml file that BES’s partition uses for its own logging requests. If you want the Application’s log requests to percolate to a different destination, we need to create a new Appender in the logConfiguration.xml file. Open up this file in your favourite XML Editor and add a new Appender along the lines of:

    <appender name="APPLICATION" class="org.apache.log4j.RollingFileAppender">
        <errorHandler class="org.apache.log4j.varia.FallbackErrorHandler">
            <root-ref/>
            <appender-ref ref="FALLBACK"/>
        <errorHandler>
        <param name="Threshold" value="ALL"/>
        <param name="File" value="adm/logs/application_log.xml"/>
        <param name="Append" value="true"/>
        <param name="ImmediateFlush" value="true"/>
        <param name="BufferedIO" value="false"/>
        <param name="BufferSize" value="8192"/>
        <param name="MaxBackupIndex" value="9"/>
        <param name="MaxFileSize" value="5MB"/>
        <layout class="org.apache.log4j.xml.XMLLayout">
            <param name="locationInfo" value="false"/>
        <layout>
    <appender>

    <category name="com.krish" additivity="false">
        <priority value="info" />
        <appender-ref ref="APPLICATION" />
    <category>

The above entry create a new Appender called APPLICATION that is utilized by all Loggers that inherit from “com.krish” such as the Logger in our EJB whose name is “com.krish.myapp.j2ee.ejb.MySessionBean”. The APPLICATION Appender appends all log requests to a new file called “application_log.xml“ that is created in the same directory as the partition’s log file.

Remember that log4j loggers automatically inherit settings from their parent loggers, else they inherit settings from the root logger. Also note the use of the additivity attribute value of false for the category element. If this is not set, additivity defaults to true, in which case the log statements you issue from your J2EE Application will flow to two different Appenders: the PRIMARY and the APPLICATION. In other words, the logging requests will appear in both the partition_log.xml file as well as the application_log.xml file. This would be a waste of resources and remember that our intention is minimize logging overhead.

Note that the category element also defines a priority value info. That is, only requests with a priority INFO or higher will be logged. In other words, DEBUG levels will be ignored.

log4j’s overhead is minimal, and the logging request itself determines if the request should actually be sent to the Appender(s). For example, if the logger.debug(“ . . . “) call were issued and the XML Configuration file allowed only INFO and higher levels, the overhead of the call to the logger.debug statement is minimal. However, this does not eliminate the overhead of parameter construction. For example:

public void doSomething(String arg1, java.sql.Connection conn) {
logger.debug(“doSomething called with argument: “ + arg1 + “ and using JDBC connection: “ + conn.toString());
}

Here, the JVM will have to construct the result of:

“doSomething called with argument: “ + arg1 + “ and using JDBC connection: “ + conn.toString()

prior to invoking the logger.debug() method. In JDK 1.4.2_04 (b05), the byte code generated looks like:

    3:   new     #10; //class StringBuffer
    6:   dup
    7:   invokespecial   #11; //Method java/lang/StringBuffer."":()V
    10: ldc     #16; //String doSomething called with argument:
    12: invokevirtual   #13; //Method java/lang/StringBuffer.append:(Ljava/lang/String;)Ljava/lang/StringBuffer;
    15: aload_1
    16: invokevirtual   #13; //Method java/lang/StringBuffer.append:(Ljava/lang/String;)Ljava/lang/StringBuffer;
    19: ldc     #17; //String and using JDBC connection:
    21: invokevirtual   #13; //Method java/lang/StringBuffer.append:(Ljava/lang/String;)Ljava/lang/StringBuffer;
    24: aload_0
    25: getfield        #7; //Field conn:Ljava/sql/Connection;
    28: invokevirtual   #18; //Method java/lang/Object.toString:()Ljava/lang/String;
    31: invokevirtual   #13; //Method java/lang/StringBuffer.append:(Ljava/lang/String;)Ljava/lang/StringBuffer;
    34: invokevirtual   #14; //Method java/lang/StringBuffer.toString:()Ljava/lang/String;

which is some processing overhead due to the construction of a StringBuffer and multiple calls to the StringBuffer.append method before returning the value via a call to the StringBuffer.toString method. In addition, there is an overhead of synchronization (monitor enter/monitor exit) since all the append methods in the StringBuffer class are synchronized. This overhead can be eliminated - if debugging is disabled - via a call to logger.isDebugEnabled() prior to the call to the logger.debug()

So, a slightly more optimal logging code might read:

  public void doSomeWork(String arg1, java.sql.Connection conn) {
    if (logger.isDebugEnabled()) {
      logger.debug(“doSomething called with argument: “ + arg1 + “ and using JDBC connection: “ + conn.toString());
    }
  }

If there are any opinions, feel free to share them with me.

Share This | Email this page to a friend

Brief Introduction

Krishnan Subramanian — Fri, 16 Jul 2004 03:39:16 +0000

I’m Krishnan - a consultant with Borland specializing in Java and J2EE. I joined Borland in mid-2001 and am based in Amsterdam, The Netherlands.

My job involves a fair bit of travel - primarily in Europe and an occassional foray into the Middle-East where I help Borland customers, partners and sales teams on technical issues. My work in Borland is a mix - ranging from providing educational and consulting services to delivering presentations, speaking at conferences and authoring technical papers.

The opinions expressed in the blog are mine and mine alone.

Share This | Email this page to a friend