Cloning ClientDataSet Cursors

Last winter I got around to writing a book about Delphi's ClientDataSet component. This book was published at the end of March, and is now available worldwide (there's a link at the end of this article).

This post covers an interesting advanced feature of ClientDataSets called cloned cursors. Chapter 11 of my book provides an in-depth discussion of cloning ClientDataSet cursors, but I have refrained from repeating any material from that chapter. Instead, this is a fresh article that I hope adds additional value above and beyond the book's discussion. In addition, this article includes a look of several valuable new features introduced in some of the latest versions of Delphi, including anonymous methods and anonymous threads.

Overview of Cloned ClientDataSet Cursors

Cloning can mean very different things, when it comes to software. In some cases, such as in Delphi's TJSONObject (JavaScript Object Notation Object) and .NET's DataSet, a clone is a complete and separate copy of the original instance.

Cloned ClientDataSet cursors are different. A cloned ClientDataSet cursor is a new reference to an existing ClientDataSet's internal storage.

That internal storage has two parts: Data and Delta. Data is an OLEVariant that holds a ClientDataSet's data, in its current state, which includes any changes that have been made to that data since the data was loaded into the ClientDataSet. Delta is also an OLEVariant, and it contains the change cache. The change cache is a description of just the changes that have been made to the data since it was loaded (so long as the change cache is being maintained, which is what happens when a ClientDataSet's LogChanges property is set to True, the default value).

That both Data and Delta are OLEVariants is important, in that OLEVariants are reference counted data structures. Importantly, when the last reference to an OLEVariant goes out of scope, it is removed from memory automatically, and this behavior plays an important role in ClientDataSet cloned cursors.

Cloning a cursor involves two ClientDataSets, at a minimum, but can include many more than that. One of these ClientDataSets is the source ClientDataSet, and it must be active. When a ClientDataSet is active, it has a reference to a Data and Delta. Even then, Data and/or Delta may be empty, but they exist and can hold data. For example, if they are both empty to begin with, adding a record to the source ClientDataSet will add one record to Data, as well as one to Delta. Data reflects the current state of the ClientDataSet's content with one record, and Delta reflects that one record was added.

The second ClientDataSet in a cloning operation is a ClientDataSet that will receive the cloned cursor, and I will call this ClientDataSet the target. When a source ClientDataSet's cursor is cloned, the target ClientDataSet's Data and Delta properties refer to the Data and Delta properties of the cloned (source) ClientDataSet.

From that point on, and changes to Data (including index creation) and Delta are reflected in both the clone (target) and the original (source). That does not mean, however, that these ClientDataSets will be exactly the same. Although both will refer to the same Data and Delta, there are many other properties that are independent. Specifically, each ClientDataSet referring to a shared Data and Delta will maintain their own values in the AppServer, Filter, Filtered, IndexName (or IndexFieldNames), MasterFields, MasterSource, OnFilterRecord, ProviderName, ReadOnly, RecNo (current record), and RemoteServer properties.

You clone a ClientDataSet's cursor by calling CloneCursor from the target ClientDataSet. This method has the following signature:

procedure CloneCursor(Source :TCustomClientDataSet; 
  Reset: Boolean; KeepSettings: Boolean = False);

You pass at least two parameters to CloneCursor. The first argument is a reference to the source ClientDataSet whose cursor you want to clone.

You use the second and third parameters to indicate whether the clone should adopt view-specific properties of the ClientDataSet it is cloning. If Reset is False and KeepSettings is True, the receiving ClientDataSet will adopt the AppServer, Filter, Filtered, IndexName (or IndexFieldNames), MasterFields, MasterSource, OnFilterRecord, ProviderName, ReadOnly, and RemoteServer properties of the source ClientDataSet. If you pass True in the second parameter (in which case you should pass False, the default, in the third parameter), the values of these properties in the clone will be reset to their default values.

Once a ClientDataSet has cloned another ClientDataSet, the two ClientDataSets will continue to refer to a common Data and Delta, until one of them is closed or clones a different ClientDataSet. Importantly, their properties will remain independent, regardless of the values of the second and third parameters of CloneCursor. From that point on, each can use a different filter, range, current record, and so forth, and therefore may reflect radically different views of the common data store.

There is one exception to the above statement. If you clone a ClientDataSet that is filtered (Filter is True and Filtered is a Boolean expression limiting which records are displayed), and you pass a value of False (no reset) in the second parameter of CloneCursor, the receiving ClientDataSet will also employ the filter and it cannot be removed. Under this situation, you can apply an additional filter to the clone, but you can never get back to a completely unfiltered view. This appears to be a bug. As a result, I recommend that you always pass True in the second parameter and manually apply properties from the source ClientDataSet to the target.

Once a source ClientDataSet is cloned, the target and the source are equal owners of the Data and Delta. Any one of them can post new changes to Data, and can undo changes in Delta.

When one of these ClientDataSets performs one of these actions, the result is immediately apparent in both. Furthermore, closing one of these ClientDataSets has no effect on Data and Delta, unless it is the last ClientDataSet that refers to Data and Delta, in which case Data and Delta will be released.

As implied earlier in this post, cloning is not necessarily about two ClientDataSets. It is perfectly possible to clone a source ClientDataSet, and then use the target as the source of yet another clone, and so on. Once again, all ClientDataSets that point to the same Data and Delta are equal. In other words, a source ClientDataSet has no special privileges with respect to the Data and Delta.

A Simple Example

The sample project SimpleClone demonstrates some basic elements of a cloned cursor. The main form of this application is shown in Figure 1.

Figure 1. The main form of the SimpleClone project

This project contains a ClientDataSet, named ClientDataSet1, which is opened when the project runs and displayed in the upper DBGrid. When you click the button labeled Clone ClientDataSet1, ClientDataSet2 is cloned from ClientDataSet1, and displayed in the lower DBGrid.

The following is the code that appears on the OnClick event handler of the button labeled Clone ClientDataSet1.

procedure TForm1.CloneButtonClick(Sender: TObject);
begin
  if ClientDataSet1.Active then
  begin
    ClientDataSet2.CloneCursor(ClientDataSet1, False, True); 
    CloneButton.Enabled := False;
    CloseButton.Enabled := True;
  end
  else
    raise Exception.Create('ClientDataSet1 not active. Cannot clone');
end;

Once cloning is complete, the views of these ClientDataSets are independent, in that they can have different sort orders and different current records, as shown in the Figure 2.

Figure 2. Two ClientDataSets share Data and Delta

If you now close ClientDataSet1, by clicking the button labeled Close ClientDataSet1, the ClientDataSet2 will remain active, and can continue working with the data by editing it and saving it to a file, or pointing to a properly configured DataSetProvider, after which it could call ApplyUpdates to apply its changes to an underlying database.

A More Complex Example

One technique that I've found particularly useful is to use cloned cursors to permit concurrent access to Data and Delta from two or more threads. So long as Data and Delta are treated as readonly, each thread can read Data (and Delta), perform its task, and then terminate.

Here is an example. You might create a Windows service that exposes an Internet Direct (Indy) TIdTCPServer component. This component can listen for requests from TIdTCPClient components. Each client connection is accepted on a different thread, and each client might request one or more records from a common ClientDataSet. Importantly, each client request is distinct from every other client request.

The thread on the Windows service that receives a client request can clone a cursor onto a globally available ClientDataSet, retrieve the data associated with that request and return it to the client, and then free the clone before terminating. So long as no code needs to make changes to Data or Delta, the ClientDataSet is thread safe.

If changes did need to be made to the shared data, the addition of a synchronization object, such as a TMultiReadExclusiveWriteSynchronizer could be used to create a thread-safe architecture for permitting updates.

I considered creating a somewhat simple example of a Windows service that returned data from a shared ClientDataSet to client applications using Indy socket components. However, doing so would raise a great many issues that I cannot address adequately in this post.

Instead, I decided to demonstrate the essential methods that can be used to retrieve specific subsets of data from a shared ClientDataSet in a multi-threaded environment. Fortunately, this demonstration also makes use of a couple of handy features that have been introduced into Delphi in the last several releases: Anonymous methods (Delphi 2009) and anonymous threads (Delphi XE).

Here's the setup. I created an application that needs to create physical files that contain only a subset of data from a shared ClientDataSet. These physical files are written by worker threads that retrieve a subset of data from the shared ClientDataSet, write that data to a file, and then terminate. These worker threads perform their task concurrently.

The main form of this application is shown Figure 3. Here you can see a button that will initiate the file creation process. Also shown is a listbox that contains the customer numbers for those customer's whose orders records will be written to individual files, one file per customer.

Figure 3. The main form of the CDSCloneToFile project

The code that extracts the data for a particular customer from the shared ClientDataSet is shown in the ReturnRecordsFromCDS method. ReturnRecordsFromCDS is called by a worker thread which passes to this method the customer number whose orders are contained in the shared ClientDataSet named ClientDataSet1.

function TDataModule1.ReturnRecordsFromCDS(CustNo: Integer): TClientDataSet;
var
  CDS: TClientDataSet;
  DSP: TDataSetProvider;
begin
  //Create the Result and a temporary ClientDataSet
  Result := TClientDataSet.Create(nil);
  CDS := TClientDataSet.Create(nil);
  try
    //Create the DataSetProvider and hook it up
    DSP := TDataSetProvider.Create(nil);
    DSP.DataSet := CDS;
    Result.SetProvider(DSP);
    try
      CDS.CloneCursor(ClientDataSet1, True);
      CDS.IndexName := 'CustNoIdx';
      CDS.SetRange([custno], [custno]);
      //The following populates the Result CDS
      //with just the orders for a particular customer
      Result.Open;
    finally
      DSP.Free;
    end;
  finally
    CDS.Free;
  end;
end;

ReturnRecordsFromCDS works by using a DataSetProvider and a ClientDataSet combination to retrieve only a range of records from a cloned cursor. These records, once returned to the thread that called ReturnRecordsFromCDS, are then written to a file using the returned ClientDataSet's SaveToFile method.

But this method also points out some interesting elements of a ClientDataSet on which a range is set. Namely, if you set a range on a ClientDataSet and then call SaveToFile, all of the records of the ClientDataSet are saved, not just those in the range.

To get around this, ReturnRecordsFromCDS creates a new ClientDataSet (Result), and associates it with a DataSetProvider (DSP). This combination is used to populate Result with a set of records from a DataSet (the DataSet to which the DataSetProvider's DataSet property is associated). This DataSet is a ClientDataSet (CDS) that is cloned from ClientDataSet1.

Once cloned, an index is set on CDS (this index is a feature of Data obtained as a result of the cloning process) and then a range is set. After that, calling Open on Result draws only those records from CDS in the range. Result is then returned to the calling method.

The fun stuff, if you will, can be found in the method that calls ReturnRecordsFromCDS. This method, named GenerateXMLFileForCustomer, is passed the customer number whose orders data needs to be written to a file. It performs its work using an anonymous thread. GenerateXMLFileForCustomer is shown in the following code listing.

procedure TForm1.GenerateXMLFileForCustomer(CustNo: Integer);
var
  AnonThread: TThread;
begin
  //Create the anonymous thread to run an anonymous method
  AnonThread := TThread.
     CreateAnonymousThread(
       procedure
       var
            CDS: TClientDataSet;
            FileToWrite: String;
       begin
         CDS := DataModule1.ReturnRecordsFromCDS(CustNo);
         try
           FileToWrite := FAppDir + '\' + IntToStr(CustNo) + '.xml';
           if FileExists(FileToWrite)then
                DeleteFile(FileToWrite);
                CDS.SaveToFile(FileToWrite);
           finally
             //Free the ReturnRecordsFromCDS Result
             CDS.Free;
           end;
       end);
  //Start the anonymous thread
  AnonThread.Start;
end;

An anonymous thread is created by passing a reference to a procedure to the TThread class method named CreateAnonymousThread. CreateAnonymousThread returns a TAnonymousThread (a TThread descendant), which you then execute by calling its Start method. Once the anonymous thread has completed the execution of the procedure, it terminates and frees itself.

In the case of GenerateXMLFileForCustomer, the procedure passed to CreateAnonymousThread is an anonymous method. An anonymous method is an unnamed method whose implementation appears inline in your code.

Importantly, an anonymous method closes over the local variables (including formal parameters) of the method in which it appears, maintaining the context of these values even after the method in which the anonymous method is defined has exited. This is called a closure, and it permits each execution of this anonymous method to maintain the value passed in the CustNo formal parameter.

Each call to GenerateXMLFileForCustomer creates a different worker thread that runs the anonymous method. Each anonymous method calls the ReturnRecordsFromCDS method, which returns a new ClientDataSet that holds the orders for the given customer. After ensuring that the target file does not already exists (it will be deleted if it does), the anonymous methods writes the contents of this ClientDataSet to the appropriately named file, after which it frees the ClientDataSet created by the call to ReturnRecordsFromCDS.

GenerateXMLFileForCustomer is called from the OnClick event handler of the button labeled Create Files for Customers. The following is the code that appears on that event handler:

procedure TForm1.CreateFilesButtonClick(Sender: TObject);
var
  i: Integer;
begin
  for i := 0 to ListBox1.Count - 1 do
    GenerateXMLFileForCustomer(StrToInt(ListBox1.Items[i]));
end;

Turning back our attention to the Windows service/client example I mentioned earlier, GenerateXMLFileForCustomer would be implemented on the OnExecute event handler of the TIdTCPServer component. Since that method executes in a different thread for each client connection, the use of the anonymous method would be unnecessary. All the OnExecute event handler would need to do is to read the customer number passed by the client, execute the ReturnRecordsFromCDS method passing to it this customer number, and then return the XMLData property of the returned ClientDataSet to the client application, after which the returned ClientDataSet would be freed.

Alternatively, the returned ClientDataSet could be returned to the requesting client using a stream. Whether text or binary data is returned to the client, the service might also encrypt and/or compress the data before returning it, in which case the client must know to decrypt and/or decompress the data it receives.

Summary

When you clone a ClientDataSet cursor, you permit an additional ClientDataSet to refer to the Data and Delta properties of an existing ClientDataSet. While all ClientDataSets that share the same cursor share the same Data and Delta, they do not share any other properties, permitting each ClientDataSet to maintain an independent view of the data.

For more information about Delphi in Depth: ClientDataSets, as well as for links for purchasing the book, please visit http://www.JensenDataSystems.com/cdsbook.

Copyright © 2011 Cary Jensen. All Rights Reserved

Author’s Note: This article originally appeared in the SDN Magazine # 109, published on May 5th, 2011.  For information about the Software Development Network, please visit http://www.sdn.nl/.

Creating a TThread in Delphi that can Self-Terminate on Timeout

Author's Note. When I originally wrote this post, I described a technique by which a thread forcibly terminated itself by calling the TerminateThread function of the Windows API. Part of my goal was to share a technique that I haven't seen described before, which also made me somewhat suspicious that the technique might be wrong minded.

Though the code seemed to work alright, I invited comments from readers if they knew otherwise. The first comments to this post stated very clearly that this technique is not an acceptable one. In short, unless there is a very good reason (and there are few of those), you should not call the TerminateThread function.

Still, the original concept, that of a self-terminating thread, is an attractive one. However, instead of forcibly terminating a thread, the code presented in this updated post simply attempts to terminate the thread through its normal mechanism, by calling the thread's Terminate method. While this might not terminate the thread quickly (or ever if it is truly deadlocked), there is much less of a downside compared to the use of TerminateThread.

I considered removing this post. On the other hand, I think the comments are very good, well considered, and educational. In addition, I have modified the code presented here to remove the potentially harmful call to TerminateThread, and replaced it with a less forceful, yet still useful alternative. The following is the modified posting.

What can you do when a thread becomes unresponsive? Better yet, how can you create an instance of Delphi's TThread class that terminates itself if it becomes unreasonably slow?

This is the question that I had to address recently. In short, I was creating a Delphi application that used an Internet Direct (Indy) TIdTcpClient to make calls to a RESTful Web service. In doing this, I considered the sage advise of my friend and fellow Delphi Developer Days presenter Marco Cantù.

Marco is fond of pointing out that you should make remote calls to Web services or other Internet resources from a thread, permitting your user interface to remain responsive even if the service to which you are communicating becomes slow or worse.

There is a problem, however. Terminating a thread by killing it outright is serious business, and can easily lead to memory leaks and other similarly nasty side effects. For example, resources that your thread has allocated may end up abandoned upon the termination of the thread. And, as pointed out in some of the comments to the original post, can go so far as to undermine your entire process.

Since my original issue was associated with communicating with a RESTful Web service using a connection object (an Indy socket client in this case), I reflected on the technique I've used in JavaScript to make asynchronous calls to the same service. In that code I created a timer that was canceled once the XMLHttpRequest object (XHR) callback triggered. If the timer expired before the asynchronous request returned, the timer canceled the request. On the other hand, if the callback was executed, the timer was canceled.

So, here for your consideration is a simple TThread descendant that calls its own Terminate method (from the primary thread of execution) if it does not terminate on its own after a specified timeout. I follow this unit with some observations and final comments.

unit TimeoutThreadu;

// No guarantees or warranties are expressed or implied concerning
// the applicability of techniques or code included in this example.
// If you wish to use techniques or code included in this example,
// it is your responsibility to test and certify any code or
// techniques design adopted as a result of this project.

interface

uses
  Classes, Windows, ExtCtrls, SysUtils, SyncObjs;

type
  TTimeoutThread = class(TThread)
  strict private
    { Private declarations }
    FTimer: TTimer;
    FTimeout: Boolean;
    procedure OnTimer(Sender: TObject);
  protected
    procedure Execute; override;
  public
    constructor Create(CreateSuspended: Boolean;
      Timeout: Integer); overload;
    destructor Destroy; override;
  end;

implementation

uses mainformu;
{ TTimeoutThread }

constructor TTimeoutThread.Create(
  CreateSuspended: Boolean; Timeout: Integer);
begin
  FTimer := TTimer.Create(nil);
  FTimer.Interval := timeout;
  FTimer.OnTimer := OnTimer;
  Self.FreeOnTerminate := True;
  FTimeout := True;
  Self.Create(CreateSuspended);
end;

destructor TTimeoutThread.Destroy;
begin
  //Note that a destructor is called even if
  //an exception is raised in the constructor.
  //That is why FTimeout is set to True after
  //the thread's resources have been created.
  if FTimeout then
  begin
    FTimer.Enabled := False;
    FTimer.Free;
 end;
  inherited;
end;

procedure TTimeoutThread.Execute;
begin
  if FTimeout then
    FTimer.Enabled := True;
  while True do
  begin
    //Your thread's code goes here
    //Simulate work
    sleep(2000);
    if Self.Terminated then exit;
  end;
end;

procedure TTimeoutThread.OnTimer(Sender: TObject);
begin
  //This code executes in the primary
  //thread of execution. As a result, it will
  //execute even if the thread is currently blocked.
  //Note, however, if the thread is blocked, it will
  //not actually terminate until it is no longer blocked.
  //Furthermore, calling a thread's Terminate method does
  //not actually terminate the thread. It only set the thread's
  //Terminated property to True. If your thread is designed
  //to run continuously until it detects its termination,
  //it is the responsibility of the code in the Execute method
  //to test the Terminated property, and to exit gracefully
  //once it finds that Terminated has been set to True.
  FTimer.Enabled := False;
  Self.Terminate;
end;

end.

This thread can be created as a self-terminating thread using code as simple as the following:

with TTimeoutThread.Create(True, 10000) do
  Start; //call Resume with Delphi 2007 or earlier

To begin with, this thread can be run without or without a timeout. If you call the inherited constructor, it does not timeout.

When you call the overloaded constructor, you pass the timeout, in milliseconds, in the second parameter. In that case, the first statement in the Execute method (the one method of the thread that runs in a worker thread) initiates the Timer.

If the thread terminates normally, and it was created with a timeout, it terminates the timer. And, if the timer expires at about the same time as the thread is terminating, there is no harm. Calling Terminate on a terminated thread is not a problem, and setting a timer's Enabled property to False when it is already set to False is likewise not a problem.

But permit me to mention a limitation. This thread is freed on termination. If you need a thread to stick around after it has terminated, this one is not for you. On the other hand, you can create an OnTerminate event handler, and persist any information that the thread has collected from there.

Once again, this post has been modified. The original code included a call to the Windows API function TerminateThread from within a synchronized block of code in the OnTimer event handler. I asked for input regarding this approach, and the response was universal: it was the wrong thing to do. Please enjoy the thoughtful comments submitted by readers.

Creating Custom Compiler Errors in Delphi

Unlike my normal posts, this one is short, and it describes an interesting Delphi solution that I want to share with you.

Here is the issue that I was faced with, along with the solution. I work with a number of different clients on a wide range of technologies (though it seems as though most of my work these days is Internet-related). Furthermore, I use many different versions of Delphi. Though it is not truly relevant to the technique that I want to share today, I run each version of Delphi in a separate virtual machine. If you are curious about why I do this, please feel free to read Developing in a Virtual World and Creating a More Manageable Development Environment

Getting to the point, here is the issue I faced. I have a project that I originally wrote using Delphi 2007, and in which I used Internet Direct (Indy) components that employ SSL (secure socket layer). I have since migrated that project to a more recent version of Delphi, and in the process deployed newer versions of the ssleay32.dll and libeay32.dll DLLs. These newer DLLs are incompatible with the version of Indy I have installed in Delphi 2007.

This might not sound like there is a problem, but there is. Specifically, now that I have moved to a later version of the SSL libraries, I should never compile and deploy that application using Delphi 2007. Remember, as a consultant I use many different versions of Delphi, and for the particular client for whom I built this application, we are using no less than three different versions of Delphi. Since I don't work on these projects everyday, it would be easy for me to check out this application from version control and try to update it using the wrong version of Delphi.

What I needed was some mechanism to prevent me from attempting to compile and deploy this particular application in Delphi 2007 (or earlier). What if, I thought, I could instruct the compiler to generate a fatal error upon attempting to compile the project in Delphi 2007, but not from a later version. Oh, sure, I could kludge a solution by including some reference to a symbol that was introduced in Delphi 2009, but what I really wanted was to be able to generate a meaningful compiler error, such as "This project must be compiled in Delphi 2009 or later" instead of a relatively meaningless

[DCC Error] Unit1.pas(28): E2003 Undeclared identifier: 'TBalloonHint'

Fortunately, there is an elegant solution, but I was unaware of it at the time. So, in my quest to find the correct solution, I turned to the Gods of Software, which is to say that I posted by inquiry on StackOverflow. My original post was answered by several people (Delphi folks that you'll likely know), and I had my answer.

So, here is the compiler directive that I added to my project, which I adapted from the answers provided on StackOverflow:

{$IF CompilerVersion <= 19.0} // Delphi 2007 = 19.0
  {$MESSAGE Error 'This project must be compiled in Delphi 2009 or later'}
{$IFEND}

And do you know what? This past week, when I was onsite with the client for whom I built the application, I encountered the following compiler error.

Thank goodness my application is now smarter than me.

Backward Compatibility using Defalt Parameters

When a function, procedure, or method requires at least one parameter, the final (right-most) parameters in the parameter list can be declared to have a default value. When the parameters declared to have a default value are omitted during the invocation of the subroutine, the default values are assigned to the corresponding formal parameters.

While this feature is interesting by itself, there is another use for this feature that permits you to extend an existing method without breaking older versions.

The best way to explain this trick is with an example. Imaging that you have a method you call to display an error message on your page. This method takes a single string parameter, and assigns that value to one or more Label controls (or some other display element).

Here is an example of what the implementation of that method might look like:

procedure TForm1.SetMessageText(Value: String);
begin
  MessageLabel1.Caption := Value;
  MessageLabel2.Caption := Value;
end;

Let's assume that these Label controls use a red foreground color, so that when a non-empty string is assigned the message stands out, and looks like an error message. Assigning an empty string to the method sets the Labels's caption property to an empty string, effectively turning off the message.

So, whenever you need to display an error message, you simply call this method, using something similar to the following:

SetMessageText('You must enter a starting date before continuing');

Now image that at some later time you realize that you want to be able to display non-error messages using these same Labels. These non-error messages should use a black font for the Labels, and not a red one.

Instead of create a new method, you can simply extend the existing method easily by adding a new parameter. Importantly, in order to avoid having to go back and modify your existing calls to SetMessageText, this second parameter will use a default value to direct the method to use a red font when this parameter is absent.

Here is how the updated method implementation might look:

procedure TForm1.SetMessageText(Value: String; IsError: Boolean = True);
begin
  if IsError then
  begin
    MessageLabel1.Font.Color := clRed;
    MessageLabel2.Font.Color := clRed;
  end
  else
  begin
    MessageLabel1.Font.Color := clBlack; 
    MessageLabel2.Font.Color := clBlack;
  end;
  MessageLabel1.Caption := Value;
  MessageLabel2.Caption := Value;
end;

Now, when you use a call something like the following, your message will be display in a black font, indicating that this is an informational message:

SetMessageText('The time is ' + DateTimeToStr(Now), False);

Any call that omits the second parameter, or calls that include True in the second parameter, will display the message in a red font.

There are several restrictions concerning the use of default parameters. These are:

• The specified default values must be represented by a constant expression
• Parameters that are object types, variants, records, or static arrays cannot have default values
• Parameters that are class, class reference, dynamic arrays, and procedural types can only have nil as their default value

Advantage AdsQuery and RecNo

At the time of this writing, Marco Cantù and I have completed the US part of our 2011 Delphi Developer Days tour, and are beginning the European part at the end of next week (yes, there are a few seats available in each city. If you are interested in attending, you must act soon. Otherwise, you’ll have to wait until next year). One of the benefits of Delphi Developer Days is that we limit registration to just over 30 people in each city, which ensures that we have a sufficient number of attendees to stimulate discussion and networking, but few enough that every attendee has the opportunity to talk with Marco and me on a one-on-one basis.

At our Baltimore stop in April, one of the attendees approached me with a problem that he encountered during his attempt to convert his commercial application from using the Borland Database Engine (BDE) to the Advantage Database Server (ADS). (This is the most logical and easiest migration for BDE developers, in my opinion).

In his tests, the AdsQuery (the ADS equivalent to the BDE Query) did not treat the RecNo property the same as other DataSets. In all other DataSets, he noted, RecNo reports the position of the current record within the current DataSet with respect to the order of the records in the DataSet. For example, in a ClientDataSet, a Query, or a SQLDataSet, RecNo will be equal to 1 when you are on the first record of the DataSet, which is the default when you first open the DataSet, regardless of which index you’ve select (ClientDataSet) or SQL ORDER BY clause (Query or SQLDataSet) you are using.

“There must be a way around this,” he conceded, “since the Advantage Data Architect can report the correct record position for records in queries that include in an ORDER BY clause.”

I put together a simple test between sessions, and confirmed his observation that RecNo, at least using the default settings of the AdsQuery, did in fact return the position of that record within the underlying table, and not the position of the record in the current record order. As a result, when an AdsQuery that employs an ORDER BY clause is first opened, RecNo for the current record (the first record in the view) might 55, or 1,002,837, or whatever position that first record occupies in the underlying table, regardless of the order of that record’s position in the result set.

RecNo has an additional use, as well. RecNo can be used to navigate a DataSet (at least in DataSets that support navigation). Specifically, if you set RecNo to 1, the DataSet navigates to record 1 in the current sort order. However, with the AdsQuery in my test, setting RecNo to 1 navigated to the first record in the underlying table, which is not necessary the first record in the indexed or sorted order of the result set.

I had not seen this problem before, and I didn’t solve the problem on the spot. However, I did say that I would try to understand the issue better, and report my results to the Advantage team if I could not find an acceptable solution.

I’ve been very busy, but I finally found time to put together a demonstration of the issue. It confirmed that in the default configuration, the AdsQuery.RecNo property is associated with a record’s true position in the underlying table. However, after some poking around, I found two properties that change this behavior. Specifically, when you set an AdsQuery’s Sequenced property to True, and its SequenceLevel property to slExact, the AdsQuery treats RecNo the same as all other DataSets I tested.

I imagine that there is a performance explanation for the default values of the AdsQuery properties. Specifically, so long as you don’t need to use the RecNo property to determine the view-specific position of a record in a result set, and do not need to use it to perform view-specific navigation, the default property values result in faster query performance. When the standard DataSet.RecNo behavior is required, however, the extra overhead is worth the benefit. So, if you are using Advantage, and need the AdsQuery.RecNo property to behave the DataSet way, all you need to do it tweak its properties a bit.

I am sharing this story for two reasons. First, I hope that this post helps the next Advantage user who runs into this problem. Second, it demonstrates added value that Marco and I bring to Delphi Developer Days. I can’t promise that we can solve every problem for you, but we are there to listen, and we do our best to offer suggestions for solutions.

Live Templates Revisited

While code templates have been in Delphi since Delphi 4, they seem simple compared to the new live templates feature first introduced in Delphi 2006. Live templates provide self-describing, intelligent code insertion, and interactive navigation to the variable parts of the template. If you are not using them now, it's time to take another look.

Consider the inserted template shown in the following figure. This template was created by executing the forb template, which inserts a for loop with a begin..end block.

Notice that initially the I (the loop iteration variable) is highlighted. This permits you to immediately edit the variable name, if you like. For example, many Delphi developers use a lowercase variable named i as their loop iterator (even though Delphi itself is case insensitive).

You will also notice that the I, as well as the 0 and List.Count appear as bordered text. These bordered text areas represent the sync points of the live template. After changing I to i, press Tab or Shift-Tab to navigate between the sync points of the template. As you navigate, you will notice that the hints change to describe the role of the currently selected point.

The following figure shows the same live template shown in the preceding figure after you have navigated to the third variable part.



Many live templates, such as the forb template, also support intelligent code insertion, or active scripting. With active scripting, code executes when you insert or interact with the template, conditionally performing one or more tasks for you. In the case of the forb template, if the function in which you insert this template does not already contain a declaration for the loop iterator variable, one will be inserted for you once you press the Tab key after completing all sync points. The following code segment depicts how a previously empty method might look after the execution of the active script associated with the forb live template.

Invoking Live Templates

Delphi provides you with a large number of standard live templates for each of its language personalities (Delphi, C++, XML, and so forth). These templates can be invoked either on demand or automatically, depending on how the live template is defined as well as how the Delphi environment is configured.

Regardless of a live template's definition, you can invoke a live template on demand by positioning your cursor within the editor where you want to place the live template and press Ctrl-J. Delphi responds by displaying the Insert Template menu, shown in the following figure. Select the template you want to insert from this menu and press Enter.



Similarly, you can begin typing the first few characters of the template shortcut name before pressing Ctrl-J. If only one template's shortcut name matches the characters you've typed, that template is selected and inserted, without the Insert Template menu being displayed. If two or more template shortcut names match what you've typed, the Insert Template menu appears, though only the matching template names are shown, in which case you then select which of the displayed templates you want and press Enter.

Automatic invocation is available when a live template and Delphi are both configured for it. For those live templates that include the invoke="auto" attribute in the template element of the template definition, auto completion is available when the Code templates completion and Auto complete checkboxes are checked on the Code Insight table of the Options dialog box. You display this dialog box by select Tools
Options from Delphi's main menu.



With automatic live template invocation, live templates are inserted when you type the name of a template shortcut and press Tab, Spacebar, or Ctrl-J. For example, if you type forb and press Tab with code templates completion enabled, the forb live template is instantly inserted into your code at the position of your cursor.
Live templates also show up in the code completion menu, which you can invoke by pressing Ctrl-Spacebar. Templates always appear at the end of this list.

You can also invoke live templates from the Templates pane, shown in the following figure. To display the Templates pane, select View | Templates from Delphi's main menu.



Each of the standard templates that ship with Delphi are defined by an XML file. You can edit this XML if you want to change the template's content or behavior. To edit one of the standard templates, select the template in the Template pane and then click Edit from the Template pane's toolbar. The following figure shows the case live template in the code editor.

The case live template is a good example to learn from, since it includes point, hint, text, description, author, and script elements. Furthermore, this is a surround template, which means that a code segment that you select before executing this template will result in the selected block of code being enclosed in the try block.

The Parts of a Live Template

The root element of a live template is the codetemplate element. The codetemplate element two attributes, xmlns and version, which define an XML namespace and a version, respectively. The codetemplate element has a single child element, named template.

The template element can contain four attributes. The name attribute is required, and must define a unique name for the template for the specified language.

The remaining attributes are optional. Use surround="true" to define a surround template, one that can surround a block of code. Use invoke= to define the type of invocation for the template. Valid values for invoke are "auto," "manual," and "none" (the default).

Set invoke to auto or manual to insert the template by typing its name and pressing a key. For auto templates you can press Space or Tab, and for manual templates you can press Tab. With any invocation type, you can type the name of the template and insert it using Ctrl-J, or you can press Ctrl-Spacebar and select the template from the displayed list.

The final attribute, inplace="true", defines an expansion template. It is unclear what an expansion template is, since it does not appear to be used.

The point, description, and author elements are child elements of template, and they are optional. When one or more point elements are used, they define custom sync points in your template code. Each sync point can include a text element, defining the default value, and a hint element, which sets a hint for the point.

A point, as well as the code element, can also define a script attribute, which references an active script. Active scripts are described later in this post.

The description element provides the description of the template that appears in the popup menus and the Templates dialog box. The author element identifies the template's author.

The final element, the code element, describes the code that the template inserts. This element has several attributes, as well as a CDATA section. For a Delphi template, set the language attribute to Delphi. Also, define the delimiter that identifies the custom and predefined sync points. Since the $ character, which is the default delimiter, has special meaning in Delphi, most developers define the | character (the vertical bar) as the delimiter.

The code itself appears in the CDATA section, between the [ and the ]]> characters. This code can include literal characters, hard returns, and sync points, both custom and predefined. In the code segment shown in the case live template, the |expression| and |cases| strings represent the position of the corresponding sync points in the live template. You can see these points defined earlier in the template definition.

In addition to your custom sync points, there are three predefined points. These are |surround|, |*|, and |end|. If your code template is a surround template, you include the |surround| point to identify where the selected block will appear, relative to the rest of the template. When |*| appears in your template, it specifies that what following on that line should be indented by the block indent amount defined on you Options dialog box. You can include as many |*| predefined points in your live template as you need.

Finally, the |end| predefined point identifies where your cursor will appear after the live template has been inserted. Like the |surround| point, there can be only one |end| point in your live template.

Custom Live Templates

While the standard live templates that ship with Delphi are extremely useful, you will be delighted to discover that you can create your own, custom live templates.

To create a custom live template, click New from the Templates pane's toolbar. Alternatively, you can select the Template Wizard from the Object Repository. To do this, select File | New | Other from Delphi's main menu. Then, select the Template Wizard from the Other Files node of the Object Repository and select Ok.

The new template appears as an XML document in the code editor, as shown in the following figure. As you inspect this figure, you will notice that the new template is displaying a hint. This is because the new custom live template is created by a live template. Not only does this reveal the incredible power of live templates, but the hints that are provide greatly simplify the process of creating your own new, live template.


 To complete your custom live template, fill in the various variable parts of the template, and add your own elements, as needed, to define the features you want in your template.

In addition to the hints that the Live Template template provides you with, you should also examine the standard live templates that ship with Delphi for insight and tips about writing your own, first-rate custom live templates.

Surround Templates

Surround templates are a special subset of live templates that permit you to quickly enclose, or surround, a selected block of code. Examples of surround templates include forb, try, procedure, and comment live templates.

For example, imagine that you want to enclose the following code selection in a try ... except block in a Delphi project.



With the desired code selected, select the tryf live template from the Templates pane, and then click Execute. Alternatively, select the block of code to surround, right-click in the editor, select Surround from the displayed context menu, and then select tryf.

Notice also that the code block that you selected has been indented as well, making the resulting code more readable. You are now ready to insert a call to the Free method of the TStringList in the finally block, ensuring its destruction once you are through with it.

If you create your own custom surround templates, those will automatically appear in the Surround context menu. Specifically, the Surround context menu includes any custom live templates that you've created for the current language personality where the surround attribute is set to true in the