Monday, September 26, 2011

Creating a TThread in Delphi that can Self-Terminate on Timeout

Author's Note. When I originally wrote this post, I described a technique by which a thread forcibly terminated itself by calling the TerminateThread function of the Windows API. Part of my goal was to share a technique that I haven't seen described before, which also made me somewhat suspicious that the technique might be wrong minded.

Though the code seemed to work alright, I invited comments from readers if they knew otherwise. The first comments to this post stated very clearly that this technique is not an acceptable one. In short, unless there is a very good reason (and there are few of those), you should not call the TerminateThread function.

Still, the original concept, that of a self-terminating thread, is an attractive one. However, instead of forcibly terminating a thread, the code presented in this updated post simply attempts to terminate the thread through its normal mechanism, by calling the thread's Terminate method. While this might not terminate the thread quickly (or ever if it is truly deadlocked), there is much less of a downside compared to the use of TerminateThread.

I considered removing this post. On the other hand, I think the comments are very good, well considered, and educational. In addition, I have modified the code presented here to remove the potentially harmful call to TerminateThread, and replaced it with a less forceful, yet still useful alternative. The following is the modified posting.

What can you do when a thread becomes unresponsive? Better yet, how can you create an instance of Delphi's TThread class that terminates itself if it becomes unreasonably slow?

This is the question that I had to address recently. In short, I was creating a Delphi application that used an Internet Direct (Indy) TIdTcpClient to make calls to a RESTful Web service. In doing this, I considered the sage advise of my friend and fellow Delphi Developer Days presenter Marco Cantù.

Marco is fond of pointing out that you should make remote calls to Web services or other Internet resources from a thread, permitting your user interface to remain responsive even if the service to which you are communicating becomes slow or worse.

There is a problem, however. Terminating a thread by killing it outright is serious business, and can easily lead to memory leaks and other similarly nasty side effects. For example, resources that your thread has allocated may end up abandoned upon the termination of the thread. And, as pointed out in some of the comments to the original post, can go so far as to undermine your entire process.

Since my original issue was associated with communicating with a RESTful Web service using a connection object (an Indy socket client in this case), I reflected on the technique I've used in JavaScript to make asynchronous calls to the same service. In that code I created a timer that was canceled once the XMLHttpRequest object (XHR) callback triggered. If the timer expired before the asynchronous request returned, the timer canceled the request. On the other hand, if the callback was executed, the timer was canceled.

So, here for your consideration is a simple TThread descendant that calls its own Terminate method (from the primary thread of execution) if it does not terminate on its own after a specified timeout. I follow this unit with some observations and final comments.

unit TimeoutThreadu;

// No guarantees or warranties are expressed or implied concerning
// the applicability of techniques or code included in this example.
// If you wish to use techniques or code included in this example,
// it is your responsibility to test and certify any code or
// techniques design adopted as a result of this project.

interface

uses
  Classes, Windows, ExtCtrls, SysUtils, SyncObjs;

type
  TTimeoutThread = class(TThread)
  strict private
    { Private declarations }
    FTimer: TTimer;
    FTimeout: Boolean;
    procedure OnTimer(Sender: TObject);
  protected
    procedure Execute; override;
  public
    constructor Create(CreateSuspended: Boolean;
      Timeout: Integer); overload;
    destructor Destroy; override;
  end;

implementation

uses mainformu;
{ TTimeoutThread }

constructor TTimeoutThread.Create(
  CreateSuspended: Boolean; Timeout: Integer);
begin
  FTimer := TTimer.Create(nil);
  FTimer.Interval := timeout;
  FTimer.OnTimer := OnTimer;
  Self.FreeOnTerminate := True;
  FTimeout := True;
  Self.Create(CreateSuspended);
end;

destructor TTimeoutThread.Destroy;
begin
  //Note that a destructor is called even if
  //an exception is raised in the constructor.
  //That is why FTimeout is set to True after
  //the thread's resources have been created.
  if FTimeout then
  begin
    FTimer.Enabled := False;
    FTimer.Free;
 end;
  inherited;
end;

procedure TTimeoutThread.Execute;
begin
  if FTimeout then
    FTimer.Enabled := True;
  while True do
  begin
    //Your thread's code goes here
    //Simulate work
    sleep(2000);
    if Self.Terminated then exit;
  end;
end;

procedure TTimeoutThread.OnTimer(Sender: TObject);
begin
  //This code executes in the primary
  //thread of execution. As a result, it will
  //execute even if the thread is currently blocked.
  //Note, however, if the thread is blocked, it will
  //not actually terminate until it is no longer blocked.
  //Furthermore, calling a thread's Terminate method does
  //not actually terminate the thread. It only set the thread's
  //Terminated property to True. If your thread is designed
  //to run continuously until it detects its termination,
  //it is the responsibility of the code in the Execute method
  //to test the Terminated property, and to exit gracefully
  //once it finds that Terminated has been set to True.
  FTimer.Enabled := False;
  Self.Terminate;
end;

end.

This thread can be created as a self-terminating thread using code as simple as the following:

with TTimeoutThread.Create(True, 10000) do
  Start; //call Resume with Delphi 2007 or earlier

To begin with, this thread can be run without or without a timeout. If you call the inherited constructor, it does not timeout.

When you call the overloaded constructor, you pass the timeout, in milliseconds, in the second parameter. In that case, the first statement in the Execute method (the one method of the thread that runs in a worker thread) initiates the Timer.

If the thread terminates normally, and it was created with a timeout, it terminates the timer. And, if the timer expires at about the same time as the thread is terminating, there is no harm. Calling Terminate on a terminated thread is not a problem, and setting a timer's Enabled property to False when it is already set to False is likewise not a problem.

But permit me to mention a limitation. This thread is freed on termination. If you need a thread to stick around after it has terminated, this one is not for you. On the other hand, you can create an OnTerminate event handler, and persist any information that the thread has collected from there.

Once again, this post has been modified. The original code included a call to the Windows API function TerminateThread from within a synchronized block of code in the OnTimer event handler. I asked for input regarding this approach, and the response was universal: it was the wrong thing to do. Please enjoy the thoughtful comments submitted by readers.

13 comments:

  1. Killing a thread will leak 1 Mb (default) of thread stack.

    Kill 100 threads and you'll leak 100 Mb (minimum).

    These leaks are not reported by Delphi tools, because they are leaks of virtual memory. Delphi tools watch only for Delphi heap, not other resources.

    For this reason - it's better to raise exception in this thread (Get/SetThreadContext). Raising exception will allow thread to shutdown nicely, but still interrupts current operation. Unfortunately, you can't cancel kernel call.

    Another issue - if you happen to terminate thread (or raise an exception) right when thread is inside memory manager's function - you can very well say goodbuy to your process. Memory manager's internal state will be corrupted, so future memory allocations will fail. And you can't do much without allocating memory. Though raising exception can behave better, if MM use try/finally.

    Surely, this is extremely rare race condition. But it CAN happen.

    Personally, I think it's best to restart your application right after you had terminated hanged thread (if you want to play insanely safe - you can even replace memory manager, just in case). Better yet - let potentially dangerously operation run in child process. You can safely kill child process - there are no similar issues with it.

    Apart from thread killing, I would consider using async operations and just cancelling them, letting thread shutdown nicely.

    One more approach - ask thread to terminate and then just forget about it (set reference to nil). You can start a new operation - all right. When "terminated" thead wake up (and it WILL eventually wake up) - it will see "terminated" state, so it will ignore results and just shuts down (including deleting thread object). I think it's the best (sync) method for one-process application.

    But in most cases, terminating thread will play "good enough". Your application will work OK, only it can hang/crash in very-very-very rare cases.

    Or may be you should just reduce operation's timeouts...

    ReplyDelete
  2. Gunsmoker: Thank you for your detailed comments. You point out many nice alternatives to terminating a thread, and your warnings about thread termination certainly suggest that the technique I have described in an unacceptable one.

    I am particularly concerned that I have not run across any discussion about the 1MB leak per terminated thread in any of the Windows documentation. How can I confirm that this is actually happening?

    I look forward to more input.

    ReplyDelete
  3. GunSmoker's comments are bang on. Don't call TerminateThread. Regarding the leaking of the stack, just read the documentation of TerminateThread. It leaks on Winver<6, i.e. 2k, XP, 2003 server. But not on Vista up.

    ReplyDelete
  4. Since the original problem is Indy related, you shoud use an Indy solution instead of hacking a dangerous TThread solution. Indy has its own timeouts available. They are set to Infinite by default, but you can assign whatever values you need. Also, you can abort a blocked Indy socket operation by simply disconnecting the socket from a different thread context than the one that is blocked.ou shoud use an Indy solution instead of hacking a dangerous TThread solution. Indy has its own timeouts available. They are set to Infinite by default, but you can assign whatever values you need. Also, you can abort a blocked Indy socket operation by simply disconnecting the socket from a different thread context than the one that is blocked.

    ReplyDelete
  5. Remy: I agree that an Indy-specific solution is best, but that was just the initial inspiration for a more general solution to unresponsive threads. This code could easily be adapted to disconnect the Indy component from the OnTimer event, instead of calling TerminateThread. The OnTimer event handler executes on a thread context different from that of the TThread itself.

    ReplyDelete
  6. > How can I confirm that this is actually happening?

    It's said right in the description of TerminateThread :)

    > The target thread's initial stack is not freed, causing a resource leak.

    As David pointed out - the behaviour was changed in Windows Vista. But it's still a valid point.

    BTW, documentation also lists "interrupting important function" issue in more details - not just MM, but it also speaks about kernel32 calls, etc.

    Similar issue: http://blogs.msdn.com/b/oldnewthing/archive/2003/12/09/55988.aspx

    Still, TerminateThread behaves "reasonable well". It's good as last resort measure, but it should be avoided as general solution.

    ReplyDelete
  7. When I am dealing with threads(luckily, not so often) I usually use windows messages to interact with the form.

    What happens is: Thread sends messages to the form, the form fetches information from thread's properties/methods.

    I generally prefer PostMessage to SendMessage .

    This way, I can just send a timeout to the form which will in turn terminate the thread.

    But that's just me.

    A

    ReplyDelete
  8. How can I kill threads on TCP Indy Server to avoid the "Terminate Thread Timeout" ???

    ReplyDelete
  9. I really liked your idea, but I can not work with timers because the lack of precision. I made some adjustments to work with TickCount windows.


    unit UTimeoutThread;

    interface

    uses
    Classes, Windows, ExtCtrls, SysUtils, SyncObjs;

    type
    TTimeoutThread = class(TThread)
    private
    FTimerStart: Cardinal;
    FTimerLimit: Cardinal;
    FTimeout: Boolean;
    protected
    procedure Execute; override;
    public
    constructor Create(CreateSuspended: Boolean; Timeout: Cardinal); overload;
    end;

    implementation

    { TTimeoutThread }
    constructor TTimeoutThread.Create(CreateSuspended: Boolean; Timeout: Cardinal);
    begin
    FTimerStart:=GetTickCount;
    FTimerLimit:=Timeout;
    FreeOnTerminate:=True;
    FTimeout:=True;
    inherited Create(CreateSuspended);
    end;

    procedure TTimeoutThread.Execute;
    var
    FTimerNow:Cardinal;
    begin
    FTimerNow:=GetTickCount;
    while not(Terminated) and ((FTimerNow-FTimerStart)<FTimerLimit) do begin
    //Your thread's code goes here
    //Simulate work
    sleep(25);
    FTimerNow:=GetTickCount;
    end;
    end;

    end.

    ReplyDelete
  10. Sooo much room for havock here. Mixing a TTimer with a thread instance that is self freeing literally allows for a queued wm_timer message to cause a cause into a freed object's call and random crashes that will make no sense.

    Worse, you need the "unresponsive" thread to actually still be processing enough to get the terminated signal.

    So, basically, you risk random crashes from rush conditions and probably never achieve what you are actually after in the first place.

    If your thread is responsive enough to check the terminated signal, it is responsive enough to time itself with GetTickCount. If it isn't responsive because it is blocked, nothing here will help except true outside tracking and TerminateThread.

    If you need better control beyond that, consider spinning your threads out to an outside process that CAN be terminated a little more efficently.

    ReplyDelete
  11. Hello guys,

    i have a more general question. Lets say i have a MyThread class derived from TThread. In the main program i have a procedure (a button onclick event) where i define and create an instance of MyThread locally. The same procedure then starts the thread. Now i want to have another procedure (another button's onclick event) which starts certain methods of the running thread. In particular, i want to call something like a "MyThread.StopCalculation" prodecure which leads to a fast end of the calculations in MyThread.execute.

    I know how to get access to the running thread via "TThread.CurrentThread" and its ThreadID. However, the result of CurrentThread is always of the type TThread and not of the type MyThread, so that i have no access to the methods (e.g. the StopCalculation method). Is there any trick to do this or do i need to define instances of threads always as global variables if i want to use the threads methods within different procedures?

    ReplyDelete
    Replies
    1. There are two general ways to do this. The easier, assuming that your thread class is in scope of the code, is to hard-cast the TThread returned by TThread.CurrentThread to your custom thread class: TMyThread(TThread.CurrentThread).

      A second way is to implement an interface in your thread class, say IMyThread. That interface must define all methods and properties you want to access (such as StopCalculation). Then, you can do something like the following to call methods of your thread

      if Supports(TThread.CurrentThread, IMyThread) then
      (TThread.CurrentThread as IMyThread).StopCalculation;

      I hope this helps.

      Delete
  12. OK, just seeing this, and agree with your and the commenter's assessments. I wanted to make a point about Marco's thoughts on always using threads for IP communications.

    In general, he's exactly right. But in certain applications, you need to strictly limit the number of those threads. A fair amount of IP-connected equipment out there, especially a lot of industrial equipment and IOT devices, have strict limits on the number of allowed concurrent connections - for example, most of the AnyBus industrial-protocol converters only allow 4 concurrent connections on the IP side. A common practice with such devices is to have a polling or event-response thread that deals with receiving data from the device, and other threads that will open another brief connection to send data/commands. This is bad practice because it uses two or more connections on a device where connections are a limited resource. Wherever possible, use only one connection thread, and use a thread-safe output queue or other technique to share that connection as needed.

    ReplyDelete