Skip to content

Don’t call me, I’ll call you…

Tired of calling the DataSnap server to track down the status of the approval process started earlier? ;) What? What else the topic you thought it will be?

I am taking full advantage of the permission granted to talk about the new features of DataSnap 2010 (DS2010). In this post I will be talking about lightweight callbacks.

Lightweight callbacks can be passed as parameters for the server method based on the new type TDBXCallback delivered with DBXJSON unit. Why there? Because callback can exchange JSON values between server and client. It is the server that is now able to send over data wrapped though a JSON value and waits for a client response. Why lightweight? Because the callbacks ‘live‘ only as long as server method executes.

Your business process can now automatically use client side information to base its decisions on what path to follow next.

TDBXCallback requires one thing only: override its Execute method with the logic you want to be executed when server calls it. It is a class, so you can have fields in it with values specific to each instance. These stateful callbacks are managed by DS2010, i.e. they are freed after server method completes and they are executed within the same thread used to invoke that lengthy server method.

Callbacks on Server Methods

The only requirement a server method needs to fulfill is to have a TDBXCallback type parameter in order to communicate with the client during its execution. In the example below the LoginChatRoom procedure has one callback:


    procedure LoginChatRoom(const ChatRoom: string; const UserId: string; callback: TDBXCallback);

A server method can have as many callbacks as it wishes, they can be invoked in any order, but not in parallel since they use the same connection. Callbacks are synchronous, the server resumes execution only after the client responds to it.

Invoking the callback translates in invoking the execute method:


var
  msg: TJSONObject;
  response: TJSONValue;
  ...
begin
  msg := TJSONObject.Create;
  ...
  response := callback.Execute(msg);
  ...
  response.Free;
  ...
end;

The input parameter is freed by DS2010 but the callback return parameter is owned by the invoker; so it needs to be freed at some moment by the server method.

Invoking Server Methods with Callbacks

Client creates a callback by extending TDBXCallback class, like in the example below:


    type TChatCallback = class(TDBXCallback)
      private
        FVCLThread: TThread;
      public
        constructor Create(vclThread: TThread);
        function Execute(const Arg: TJSONValue): TJSONValue; override;
    end;

Usually the server methods with callbacks are lengthy and you may not want the client main thread being stuck while server executes. You also want to localize the callback logic inside the Execute method and avoid complex multi-thread handshake protocols. In the example above the VCL main thread is passed along and the Execute method uses it to update visual components:


function TChatClient.TChatCallback.Execute(const Arg: TJSONValue): TJSONValue;
var
  Data: TJSONValue;
begin
  Data := TJSONValue(Arg.Clone);
  TThread.Queue(FVCLThread, procedure begin
     ChatClient.LogMessage(Format('%s - %s: %s',
                  [TJSONObject(data).Get(0).JsonValue.Value,
                   TJSONObject(data).Get(1).JsonValue.Value,
                   TJSONObject(data).Get(2).JsonValue.Value]));
     Data.Free;
  end);

  Result := TJSONObject.Create;
  TJSONObject(Result).AddPair(TJSONPair.Create('status', TJSONTrue.Create));
end;

The callback queues the argument processing into the VCL thread and ends. The server resumes execution while the UI is updated in parallel.

The simplest way of invoking server methods is through the proxy code. Callback parameters are fully supported by the proxy code generators; the client side methods will have an identical signature as the server methods.

Once the callback class is created an instance can be passed along to the server method:


var
  conn: TSQLConnection;
  proxy: TChatServerMethodsClient;
  ...
begin
  ...
    conn.Open;
    proxy := TChatServerMethodsClient.Create(conn.DBXConnection);
    proxy.LoginChatRoom(ChatClient.ChatRoomEdit.Text,
                                 ChatClient.UserIdEdit.Text,
                                 TChatCallback.Create(FVCLThread));
  ...
end;

The callback instance is freed by the DS2010 for you.

Conclusion

Lightweight callbacks are easy to use. They are simple and offer a bi-directional way of communication between server and client. They can also increase the complexity of the code as they introduce new performance challenges, especially on the server side.

For more information please visit Embarcadero RAD Studio 2010.

PS
A chat sample code can be found in Code Central.

Tagged ,

JSON Types for Server Methods in DataSnap 2010

converter_1946

I am very happy I was granted permission to talk about the new features present in Delphi 2010 concerning database area.

DataSnap 2010 (DS2010) extended the list of server method parameter types with the full suite of JSON types.

It is now possible to pass a TJSONObject instance for example from a client process to a server as input or output parameter and receive a TJSONValue back as an output parameter.

The unit where JSON types are implemented is DBXJSON. In the paragraphs below will be talking about creating JSON objects, writing server methods with JSON parameters and using JSON objects to marshal in and out user types.

Creating JSON objects

JSON objects are regular objects created out of types defined in DBXJSON unit. One can create such objects either instantiating specific types or through parsing of a byte array representing a JSON value according to specs.

Types such as TJSONObject, TJSONArray, TJSONNumber, TJSONString, TJSONTrue, TJSONFalse, TJSONull are derived from TJSONValue and can be instantiated. TJSONPair is a type that can be only added to TJSONObject but it is not in itself a TJSONValue.

The code below creates a TJSONObject with a {"Hello":"World"} object:


  var
    obj: TJSONObject;
    ...
    begin
       ...
       obj := TJSONObject.Create;
       obj.AddPair(TJSONPair.Create('Hello', 'World'));
      ...

A JSON pair is formed by a string and a value. The TJSONPair has several constructors that will allow the creation of a pair less verbose. For example, in the example above is assumes that the value is actually a JSON string and creates the appropriate object.

A JSON value can be also created from a byte array using the two TJSONObject class functions ParseJSONValue. They both have similar signatures, one assumes that it has to parse the entire array to create a consistent JSON object while the second expects to use only a part of it for that.


class function ParseJSONValue(const Data: TBytes; const Offset: Integer): TJSONValue; overload; static;
class function ParseJSONValue(const Data: TBytes; const Offset: Integer; const Count: Integer): TJSONValue; overload; static;

By default a JSON object assumes ownership of its elements. Freeing the root object will cascade the release of all JSON pairs and/or values forming that particular instance. Each JSON object has an Owned property that provides ownership hints when an object is destructed. The default value, true, provides the information to an eventual container that the instance can be released together with the container itself. All objects with a false property will be spared assuming that they are owned by other resources (data stores, parameters, fields). This was done to ease the construction of complex objects based on existing ones but owned by separate entities (save memory and CPU).

Each object can be cloned through Clone function. This feature can be handy when server method input parameter need to be stored for longer than method execution itself.

ToString provides a Unicode JSON representation. This should not be used as input for the JSON parser as its content representation is not parser friendly; use ToBytes instead. The following code snippet will demonstrate how to obtain a standard byte representation of a JSON value:


SetLength(ToBytes, JSONValue.EstimatedByteSize);
SetLength(ToBytes, JSONString.ToBytes(ToBytes, 0));

Value property provides the Unicode representation for a TJSONString or a TJSONNumber but it has no use on other types.

Server method with JSON parameters

Using JSON parameter types for server methods is as natural as using them for any method or function; input, var, out and return parameters are accepted.

The following server method uses all possible combination:


function JSONPing(data: TJSONObject; var ok: TJSONValue): TJSONObject;

The DS2010 owns those parameters hence they will be destroyed immediately after method execution.
One cannot return an object that doesn’t own; here is where Clone method can be useful: if there is a need for any of the input parameters to be preserved outside the scope of the server method then it needs to be cloned.

It may be possible to keep bits and pieces of them through Owned, but be advised that you are doing it at your own risk.

Handling JSON objects over the client side

Clients can invoke server methods by using the proxy code generated for the server methods or directly using DBXConnection.

The proxy method signature is identical with the server method:


function JSONPing(data: TJSONObject; var ok: TJSONValue): TJSONObject;

The parameter life cycle is controlled by the proxy instance owner flag used for client creation:


constructor Create(ADBXConnection: TDBXConnection; AInstanceOwner: Boolean); overload;

When using the DBXConnection directly, the parameter setter function SetJSONValue accepts a Boolean flag that will control the eventual destruction of its instance upon execution.


FJSONPingCommand.Parameters[0].Value.SetJSONValue(data, true);

Marshaling user types into JSON objects

DBXJSONReflect unit provides the tools needed to marshal all user types into and from JSON values.

The marshaling technology is implemented by TTypeMarshaller class is based on traversing user objects and emitting specialized events for each action; an example would be when a new object is about to be visited OnTypeStart is called. The events implementation is done through a specialization of TConverter generic class.

The conversion is left at the latitude of the user; an user type can be marshaled into any representation such as XML or text or binary. Out of the box we provide JSON conversion through TJSONConverter class that can be coupled with TJSONMarshal class.

It is important to be able to restore the source content of the user object from the marshaled data. Due to the fact that some support classes can be quite complex and some types are not well supported by current RTTI runtime converters can be added. We currently support two categories:

  • Field: when the field is encountered on a user type the provided converter is used instead of the default one
  • Type: when a field of that type is encountered the converter is used instead of the default one

Each category has 4 types: convert the field value into another object, a string, an array of objects or an array of strings. For example a set or a record can be transformed on an array of strings.

For each user-defined converter a reverter is required to restore the field value. For example, an array of strings can be restored into a record’s content or a set based on field’s name or type.

To restore the user object the marshaled data are traversed this time only to generate a deep copy of the source data. Out of the box we provide TJSONUnMarshal that will traverse a JSON object and rebuild the user object.

We will provide a code snippet that will illustrate the marshaling of user objects. The user object types are:


  TAddress = record
    FStreet: String;
    FCity: String;
    FCode: String;
    FCountry: String;
    FDescription: TStringList;
  end;

  TPerson = class
  private
    FName: string;
    FHeight: integer;
    FAddress: TAddress;
    FSex: char;
    FRetired: boolean;
    FQualifications: TStringList;
    FChildren: array of TPerson;
    FNumbers: set of 1..10;
  public
    constructor Create;
    destructor Destroy; override;

    procedure AddChild(kid: TPerson);
  end;

We are using record, sets, string lists all combined to describe a TPerson with fields like name or children list.

For this particular example we will consider the FNumbers field transient and we will not marshal it in order to illustrate the warnings that are generated during the process.

The marshal classes used for the user objects and for the JSON equivalent form are declared and instantiated as such:


var
  m: TJSONMarshal;
  unm: TJSONUnMarshal;
  ...
begin
  m := TJSONMarshal.Create(TJSONConverter.Create);
  unm := TJSONUnMarshal.Create;
  ...

The JSON marshal instance (m) is created with the out-of-the box JSON converter. To restore the user object TJSONUnMarshal instance (unm) is part of the unit.

Dealing with special cases, such as string lists, arrays and records is done through three user defined converters:


  m.RegisterConverter(TPerson, 'FChildren', function(Data: TObject; Field: String): TListOfObjects
    var
      obj: TPerson;
      I: Integer;
    begin
      SetLength(Result, Length(TPerson(Data).FChildren));
      I := Low(Result);
      for obj in TPerson(Data).FChildren do
      begin
        Result[I] := obj;
        Inc(I);
      end;
    end);
  m.RegisterConverter(TStringList, function(Data: TObject): TListOfStrings
    var
     i, count: integer;
    begin
      count := TStringList(Data).Count;
      SetLength(Result, count);
      for I := 0 to count - 1 do
        Result[i] := TStringList(Data)[i];
    end);
  m.RegisterConverter(TPerson, 'FAddress', function(Data: TObject; Field: String): TListOfStrings
    var
      Person: TPerson;
      I: Integer;
      Count: Integer;
    begin
      Person := TPerson(Data);
      if Person.FAddress.FDescription <> nil then
        Count := Person.FAddress.FDescription.Count
      else
        Count := 0;
      SetLength(Result, Count + 4);
      Result[0] := Person.FAddress.FStreet;
      Result[1] := Person.FAddress.FCity;
      Result[2] := Person.FAddress.FCode;
      Result[3] := Person.FAddress.FCountry;
      for I := 0 to Count - 1 do
        Result[4+I] := Person.FAddress.FDescription[I];
    end);

The first converter is a field converter that is used in the context of marshaling a TPerson’s FChildren field. The list of children objects is returned as an array; these objects will be individually marshaled later.

The second converter is a type converter and it will be used for all TStringList fields; it simply returns an array of strings.

Finally, the last converter is again a field converter that deals with a record type; it transforms it into an array of strings with the record’s FDescription content is appended to it.

In a similar manner, the reverters are registered with the unm instance:


  unm.RegisterReverter(TPerson, 'FChildren', procedure(Data: TObject; Field: String; Args: TListOfObjects)
    var
      obj: TObject;
      I: Integer;
    begin
      SetLength(TPerson(Data).FChildren, Length(Args));
      I := Low(TPerson(Data).FChildren);
      for obj in Args do
      begin
        TPerson(Data).FChildren[I] := TPerson(obj);
        Inc(I);
      end
    end);
  unm.RegisterReverter(TStringList, function(Data: TListOfStrings): TObject
    var
      StrList: TStringList;
      Str: string;
    begin
      StrList := TStringList.Create;
      for Str in Data do
        StrList.Add(Str);
      Result := StrList;
    end);
  unm.RegisterReverter(TPerson, 'FAddress', procedure(Data: TObject; Field: String; Args: TListOfStrings)
    var
      Person: TPerson;
      I: Integer;
    begin
      Person := TPerson(Data);
      if Person.FAddress.FDescription <> nil then
        Person.FAddress.FDescription.Clear
      else if Length(Args) > 4 then
        Person.FAddress.FDescription := TStringList.Create;

      Person.FAddress.FStreet := Args[0];
      Person.FAddress.FCity := Args[1];
      Person.FAddress.FCode := Args[2];
      Person.FAddress.FCountry := args[3];
      for I := 4 to Length(Args) - 1 do
        Person.FAddress.FDescription.Add(Args[I]);
    end);

Let’s instantiate a couple of user objects and see what the outcome is:


var
  ...
  person, kid: TPerson;
  obj: TObject;
  ...
begin
  ...
  person := TPerson.Create;
  person.FName := 'John Doe';
  person.FHeight := 167;
  person.FSex := 'M';
  person.FRetired := false;
  person.FQualifications.Add('Delphi developer');
  person.FQualifications.Add('Chess player');
  person.FAddress.FStreet := '62 Peter St';
  person.FAddress.FCity := 'TO';
  person.FAddress.FCode := '1334566';
  person.FAddress.FDescription.Add('Driving directions: exit 84 on highway 66');
  person.FAddress.FDescription.Add('Entry code: 31415');

  kid := TPerson.Create;
  kid.FName := 'Jane Doe';
  person.AddChild(kid);

  v := m.Marshal(person);
  memo1.Lines.Add(v.ToString);

  obj := unm.Unmarshal(v);
  assert( obj is TPerson );
  assert( TPerson(obj).FAddress.FDescription  nil );
  assert( TPerson(obj).FAddress.FDescription.Count > 0 );
  assert( TPerson(obj).FAddress.FDescription[0] = 'Driving directions: exit 84 on highway 66', 'Description');
  ...
end;

The output is displayed into the form’s memo1 component. For convenience and ease of use we formatted a little and presented it below:


{"type":"Converter.TPerson",
 "id":1,
 "fields":{"FName":"John Doe",
           "FHeight":167,
           "FAddress":["62 Peter St",
                       "TO",
                       "1334566",
                       "",
                       "Driving directions: exit 84 on highway 66",
                       "Entry code: 31415"],
           "FSex":"M",
           "FRetired":false,
           "FQualifications":["Delphi developer",
                              "Chess player"],
           "FChildren":[{"type":"Converter.TPerson",
                         "id":2,
                         "fields":{"FName":"Jane Doe",
                                   "FHeight":0,
                                   "FAddress":["","","",""],
                                   "FSex":"",
                                   "FRetired":false,
                                   "FQualifications":[],
                                   "FChildren":[]}}]}}

Transient fields will be signaled as warnings. You can check if the JSON representation contains 100% of the user data by checking the HasWarnings function of the marshaller instance.


  ...
  if m.HasWarnings then
    for I := 0 to Length(m.Warnings) - 1 do
      memo1.Lines.Add('Transient: ' + m.Warnings[I].FieldName);
  ...

Conclusion

JSON introduction to DataSnap 2010 opens this technology to

  • User Objects: server side data can now be accessed by client processes
  • Business Logic: server side rules interact with client side through callbacks
  • Third party applications: Web based applications based on JavaScript can smoothly interact with server methods.

We illustrated how aspects of the goals above can be implemented using JSON types. They are easy to handle, extensible and they can be used to stream structured data to third party applications.

For more information please visit Embarcadero RAD Studio 2010

JSON reflection code can be seen here.

Tagged , ,

Bad Behavior has blocked 17 access attempts in the last 7 days.