MV* with Lazarus: between Presenter and ViewModel

The MVC conundrum

Sooner or later a programmer will get in touch with the acronym MVC (Model-View-Controller). Despite its ubiquitous presence in discussions or articles about code design, there's few comprehensive examples of using this pattern with Delphi / Lazarus. Most of the examples are just "one form application" that does not show how to organize a large scale application. There's not even a common pattern between them, some have reference to the controller in the view while others do the opposite.

This is not an object pascal exclusive issue. Other languages have the same problem and the reason is simple: the MVC as was designed to Smalltalk decades ago does not fits naturally in the modern, event driven, GUI architecture. The MVP (Model-View-Presenter) and its variations Passive View and Supervising Controller updates the pattern to match the requirements of today user interfaces. There's also Presentation Model and its most famous deviation MVVM (Model-View-ViewModel).

Meeting the presentation layer

All in all, the objective of all these patterns are to separate the presentation from the business layer thus facilitating the code maintenance. The difference lies in the responsibility of each presentation layer component and how they interact with the model (business object). In order to improve the architecture of my Lazarus projects, i found that the MVP is the most doable to be used with object pascal. There are good examples of MVP/PassiveView with Delphi that could be easily adapted but, in my opinion, is overkill and counterproductive to define read and write properties for each GUI element.

I have forms as simple as seem below

procedure TAppConfigViewForm.FormShow(Sender: TObject);
begin
  BaseURLEdit.Text := Config.BaseURL;
end;

procedure TAppConfigViewForm.SaveButtonClick(Sender: TObject);
begin
  Config.BaseURL := BaseURLEdit.Text;
  Config.Save;
end;

Having to define a view and a presenter interfaces and implement a presenter to such a simple view is a no-no to me. On the other hand, in complexes views, handling the GUI logic in a separated component is worth the work.

With these in mind, i defined an interface (IPresentation) to abstract how a view (TForm) is configured and show. To use just reference one by a string id, call SetProperties to set published properties and ShowModal to show it.

  
var
  Presentation: IPresentation;

Presentation := PresentationManager['myview'];
Presentation.SetProperties(['ConfigProp', FConfig]).ShowModal;

The presentations are registered to a specialized IoC container through two overloaded methods:

  
IPresentationManager = interface
  procedure Register(const PresentationName: String; ViewClass: TFormClass);
  procedure Register(const PresentationName: String; PresenterClass: TPresenterClass);
end;

Both has a PresentationName argument that will identify the presentation. The first overload accepts a TFormClass, the view is instantiated directly and there's no presenter. The second, accepts a PresenterClass that will be responsible to show the view.

This is how the presenter and view classes looks:

//presenter 
interface

  TNutritionEvaluationPresenter = class(TBasePresenter)
  public
    function ShowModal: TModalResult; override;
    function CanImportPreviousEvaluation: Boolean;
    procedure ImportPreviousEvaluation;
    procedure SaveEvaluation;
    property EvaluationData: TJSONObject read GetEvaluationData;
  end;

implementation

uses
  NutritionEvaluationView;


function TNutritionEvaluationPresenter.ShowModal: TModalResult;
var
  View: TNutritionEvaluationViewForm;
begin
  View := TNutritionEvaluationViewForm.Create(nil);
  try
    View.Presenter := Self;
    Result := View.ShowModal;
  finally
    View.Destroy;
  end;
end;

//view
interface

uses
  NutritionEvaluationPresenter;


  TNutritionEvaluationViewForm = class(TForm)
  [..]
  published
    property Presenter: TNutritionEvaluationPresenter read FPresenter write SetPresenter;
  end;

procedure TNutritionEvaluationViewForm.ImportPreviousLabelClick(Sender: TObject);
begin
  FPresenter.ImportPreviousEvaluation;
end;

procedure TNutritionEvaluationViewForm.SaveButtonClick(Sender: TObject);
begin
  FPresenter.SaveEvaluation;
end;

procedure TNutritionEvaluationViewForm.FormShow(Sender: TObject);
begin
  ImportPreviousLabel.Visible := FPresenter.CanImportPreviousEvaluation;
  //update GUI with evaluation data
end;

The Presenter here is acting more like a ViewModel (expose data, state, operations to view) than a true presenter. It works fine but with serious caveats:

• The view and the presenter know each other which defeats the purpose of independent implementations. Also is not possible to hold a view reference in presenter interface (circular unit reference)
• The TForm presenter property must be set manually (subject to forget)
• Registering a TForm class that expects a presenter directly will crash since there'll be no presenter

Interfaces and conventions to the rescue

I was not not really satisfied with the above approach, so reworked the code and got the following design:

• The presentation register method now has three arguments: name, view class and presenter class (optional). When the presenter class is not defined, the view is instantiated directly
• The view (TForm) is show by the internal code. No need to the presenter do it.
• If a presenter class is specified, the view class must define a published property named Presenter. An error is throw if the property does not exists or if is of an incompatible type
• The presenter property can be declared as a interface also, allowing to completely decouple the presenter from the view implementations
• There's the possibility to bind a view instance to a presenter property. Not implemented since, until now I did not need.

So much talk. The current code can be found here  and a example how I use it here.

Thoughts about application architeture with Lazarus

The Delphi books of my days (or why i'm not guilt of my application's poor design)

As most of Lazarus developers, i started to code in Delphi (in fact i learned computer programming with turbo pascal) and to get most of the tool i read some books, i bought three or four and read part of others in bookstores. This is supposed to be a good practice when learning a new technology.

The problem, noticed by me only years later, is the lack of teaching of good application design like separation of concerns (view, business, persistence layers) and how to achieve them with Delphi. Most of the books focused in the visual aspect (how to create a good looking form, reports etc) and how to setup datasets and the db aware controls. The closer to a good practice advice was putting datasets and datasources in data modules instead of forms.

We can't even blame the book authors. The Delphi's greatest selling point was (is?) the Rapid Application Development (RAD) features.

Recipes for a bulky spaghetti

In early days, when developing my applications, i was a diligent student: i put database logic in data modules and designed the forms as specified in the books. But, as all developers that created applications with more than three forms knows, things started to get hard to evolve and maintain.

Keeping the database components in data modules did not help much. You end with shared dataset states, and all problems that comes with it, across different parts of applications.

Below is a data module's snapshot of my first big application (still in production, by the way).

It could be even worse if i had not started to use a TDataset factory in the middle of development 

In the end, the project has code like:

  // a form to select a profile
  DataCenter.PrescriptionProfilesDataset.Open;
  with TLoadPrescriptionProfileForm.Create(AOwner) do
  try
    Result := ShowModal;
    if Result = mrYes then
      DataCenter.LoadPrescriptionProfile;
  finally
    DataCenter.PrescriptionProfilesDataset.Close;
    Destroy;
  end;
  
  //snippet of DataCenter.LoadPrescriptionProfile (copy the selected profile to PrescriptionItemsDataset)
  with PrescriptionItemsDataset do
  begin
    DisableControls;
    try
      FilterPrescriptionProfileItems(PrescriptionProfilesDataset.FieldByName('Id').AsInteger);
      while not PrescriptionProfileItemsDataset.Eof do
      begin
        NewMedication := PrescriptionProfileItemsDataset.FieldByName('Medication').AsString;
        if Lookup('Medication', NewMedication, 'Id') = Null then
        begin
          Append;
          FieldByName('PatientId').AsInteger := PatientsDatasetId.AsInteger;
          FieldByName('Medication').AsString := NewMedication;
          FieldByName('Dosage').AsString := PrescriptionProfileItemsDataset.FieldByName('Dosage').AsString;
          [..]
          Post;
        end;
        PrescriptionProfileItemsDataset.Next;
      end;
      ApplyUpdates;
      PrescriptionProfileItemsDataset.Close;
    finally
      EnableControls;
    end;
  end;

Its not necessary to be a software architect guru to know that this is unmanageable

Eating the pasta with business objects and inversion of control

In the projects that succeeded the first one, most of the data related code is encapsulated in business objects. The data module does not contain TDataset instances anymore, it's responsible only to act as a TDataset factory and to implement some specific data action. To work with dataset it's necessary just reference one from a key which leads to code like the below:

  FWeightHistoryDataset := DataModule.GetQuery(Self, 'weighthistory');
  FWeightHistoryDataset.ParamByName('prontuaryid').AsInteger := FId;
  FWeightHistoryDataset.Open;

This fixes the shared state issue since each dataset has a clear, limited scope. But does not solve the  business objects dependency of a global instance (DataModule), which  makes testing harder.

In the project that i'm starting, i solved the dependency to the global instance by using the service locator pattern through the IoC Container i cited in a previous post. I defined a resource factory service that is resolved as soon as the business object is created, opening the doors to setup testing environments in a clear manner.

All done?

Not yet. The business logic is contained in specific classes, there's no shared state across application and no hardcoded global dependency but the view layer (forms) is still (dis)organized  in the classic way with each TForm calling and being called by other ones directly. This problem, and the solutions i'm working, will be the subject to a future post.

Generic cross data report with lazreport

In the lazreport repository there's a demo app showing how to create a cross data report. It uses two instances of TfrUserDataset: one for the master (row) data and one for the cross (column) data. At first glance the component does not provide another way to build such reports. A deeper look shows the contrary. Here are the steps to build a cross data report with arbitrary number of rows and columns.

WARNING: to follow this guide is necessary basic lazreport knowledge.

Prepare the report

Nothing special here

• Create an empty report


• Add a Master Data band


• Add a Cross Data band


• Add a Text Object inside the Cross Data


• In the Text Object put a variable named value: [value]

Add handler to retrieve the value

Those familiar with lazreport will have no problems:

procedure TForm1.frReport1GetValue(const ParName: String; var ParValue: Variant);
begin
  if ParName = 'value' then
    ParValue := IntToStr(FRow) + ' - ' + IntToStr(FCol);
end; 

Just the column and row indexes for demonstration purpose. Using together with matrix like data structures the retrieve of actual data is straightforward.

Set the number of rows and columns

The most attentive developers will notice that no dataset (even the virtual dataset) was linked to each band. In fact running the report at this stage will lead to a blank page.
If the number of columns and rows are previously know just set the Virtual Dataset option for each band. This is not an optimum solution since not always we have that info. Here's how to set the Virtual Dataset record count at runtime:

procedure TForm1.frReport1BeginDoc;
var
  BandView: TfrBandView;
begin
  BandView := frReport1.FindObject('MasterData1') as TfrBandView;
  BandView.DataSet := '9';
  BandView := frReport1.FindObject('CrossData1') as TfrBandView;
  BandView.DataSet := '2';
end;

This will create a report with nine rows and two columns. Yep, you read right: lazreport store the number of the records of band's Virtual Dataset in an string field, the same field that store the name of an associated TfrDataset.
WARNING: don't look at lazreport source. It may scare the faint hearted ;-)

Track the row and column positions

The tricky part. The first thing to do is add two integer fields (FRow and FCol) to the Form/Data Module containing the TfrReport instance.

To get the column add an event to OnPrintColumn, and store the ColNo parameter:

procedure TForm1.frReport1PrintColumn(ColNo: Integer; var ColWidth: Integer);
begin
  FCol := ColNo;
end;

Notice that the Lazarus IDE will create an event declaration with the second parameter named Width. This will not compile with {$mode ObjFpc}. Renaming it to ColWidth will make the compiler happy.

There's not an event that pass the current row position. The first try is to hook into the OnBeginBand

procedure TForm1.frReport1BeginBand(Band: TfrBand);
begin
  Inc(FRow);
end;

Running the report with this will show wrong row indexes because it will increment in all bands not only the Master/Row band. The fix is easy:

procedure TForm1.frReport1BeginBand(Band: TfrBand);
begin
  if Band.Typ = btMasterData then
    Inc(FRow);
end;

It's done, add Data Header and Cross Header bands, glue with actual data and the generic cross data report is done. The sample project.

Make a generic control behaves like a "DropDown window"

Some controls, like the dropdown list of a combo box, disappears as soon as focus is lost. In web applications / pages this concept is expanded further by allowing form controls inside the drop down window.

To make a generic LCL control works like those web widgets, basically is necessary to hide it when the focus is lost. If the drop down control is a form this can be accomplished as simple as setting the BorderStyle to bsNone and using the Deactivate handler to hide itself:

procedure TMyForm.FormDeactivate(Sender: TObject);
begin
 Hide;
end;

For TFrame is possible to put an instance of it in an temp TForm configured as above. For other control classes, created at design time and with a parent already assigned, this may work but is not desired.

The alternative is to detect when the focus has changed and then check if the focused control is outside the "drop down" control. Unfortunately, AFAIK, there's no way in VCL/LCL to detect globally when the focus changed. Well, in fact there's an event that just do that: Screen.OnActiveControlChange. The drawback of using this event each time a "drop down" control is used is that will override a previously set event handler. Fortunately, LCL provides an alternative to set multiple handlers through Screen.AddHandlerActiveControlChanged.

So, for a TPanel descendant we would use something like to add remove the handler when the visible state is changed:

procedure TMyPanel.VisibleChanged;
begin
 if Visible then
   Screen.AddHandlerActiveControlChanged(@FocusChangeHandler)
 else
   Screen.RemoveHandlerActiveControlChanged(@FocusChangeHandler);
end;

In the handler code check if the focused control is itself or a child. If not hide:

procedure TMyPanel.FocusChangeHandler(Sender: TObject; LastControl: TControl);
var
 AControl: TControl;
begin
 AControl := Screen.ActiveControl;
 if (AControl <> Self) and not IsParentOf(AControl) then
   Visible := False;
end;

Yep! When the focus goes to outside of the "drop down" control it will automatically hide itself.

But...

Sometimes clicking outside of the control will not change the focus so it will not hide like desired. The solution is to detect user inputs (mouse click) globally. Setting OnMouse* events for all form controls is a no-no for obvious reasons. Using Application.OnUserInput event is an idea but has the same drawback of Screen.OnActiveControlChange. Similar problem, similar solution: is possible to set multiple handlers through Application.AddOnUserInputHandler.

The updated VisibleChanged code:

procedure TMyPanel.VisibleChanged;
begin
 if Visible then
 begin
   Screen.AddHandlerActiveControlChanged(@FocusChangeHandler);
   Application.AddOnUserInputHandler(@UserInputHandler);
 end
 else
 begin
   Screen.RemoveHandlerActiveControlChanged(@FocusChangeHandler);
   Application.RemoveOnUserInputHandler(@UserInputHandler);
 end;
end;

And the input handler that checks if the control where mouse is over is outside or not:

procedure TMyPanel.UserInputHandler(Sender: TObject; Msg: Cardinal);
var
 AControl: TControl;
begin
 case Msg of
   LM_LBUTTONDOWN, LM_LBUTTONDBLCLK, LM_RBUTTONDOWN, LM_RBUTTONDBLCLK,
   LM_MBUTTONDOWN, LM_MBUTTONDBLCLK, LM_XBUTTONDOWN, LM_XBUTTONDBLCLK:
   begin
     AControl := Application.MouseControl;
     if (AControl <> Self) and not IsParentOf(AControl) then
       Visible := False;
   end;
 end;
end;

Relatively simple but doing that for each control would be annoying so i wrote a component that takes care of it (among other few details). Enjoy.

The cost of accessing object fields (part 1)

The common sense make us believe that adding more code and/or more variables leads to bigger programs. Looking at the generated code of one example in the previous post, the addition of one variable made the executable smaller. This occurs because fpc is smart enough to reuse registers (in this case eax).

This week, while fixing one Lazarus bug i noticed the following pattern in the generated code of method TDBEdit.DataChange:

 movl 12(%ebx),%eax
 movl 24(%eax),%eax

Basically this is the code to access FDataLink.Field property (the first instruction get the FDataLink address and the second get the Field address). So what would happen if this field was "buffered" in a TField local variable?

Before:

procedure TDBEdit.DataChange(Sender: TObject);
begin
  if FDataLink.Field <> nil then begin
    Alignment := FDataLink.Field.Alignment; 
  [..]

After:

procedure TDBEdit.DataChange(Sender: TObject);
var
  DataLinkField: TField;
begin
  DataLinkField := FDataLink.Field;
  if DataLinkField <> nil then begin
    Alignment := DataLinkField.Alignment; 
  [..]

This simple change lead to these differences.

As expected the code became smaller but two things surprised me:

There's no increase in the temporary memory allocated

The variable assignment did cost nothing (not even one instruction)

The above test was done with a "clone" of TDBEdit.DataChange in a test project. To make sure there are no confounding factors i also tested with the original code to confirm the differences. Notice that in this case, although the code is also smaller, the addition of the variable increase the temporary memory allocated as well the variable assignment requires one extra instruction. Bad.

But there was one last hope: compile LCL with -O2 option (i assumed that LCL was already compiled with that optimization turned on). Seems that my assumption was wrong. The -O2 option did the trick: the same result as before.

In the next post i will play with a few more scenarios.

And remember: don't forget to put -O2 in LCL build options when doing a release, it makes difference.

The discover of RTTI

I never got much interest, or knowledge, by Delphi/fpc RTTI. But the recent fuzz about the new RTTI features introduced in Delphi 2010 raised my curiosity even if the fpc provides only the old style RTTI. 

The opportunity to learn, and use, RTTI came when i figured the possibility to enhance/clear some of my code.

To show a generic TForm i created a very imaginative simple function:

function ShowForm(FormClass: TFormClass; Owner: TWinControl): TModalResult;
var
  Form: TForm;
begin
  Form := FormClass.Create(Owner);
  try
    Result := Form.ShowModal;
  finally
    Form.Destroy;
  end;
end;

This little function works nice and save some boilerplate code but it's limited to forms that don't need to initialize a property before is called since i don't know class type before hand.

As stated before, messages can be used to notify with arbitrary information any TControl, so i created a variant of the ShowForm that takes two ordinal (LPARAM and WPARAM) parameters and send a CM_INIT message to the created TForm instance. The TForm descendant would need to add a CM_INIT message handler and interpret the TLMessage parameter.

function ShowForm(FormClass: TFormClass; Owner: TWinControl; WData: WPARAM = 0; LData: LPARAM = 0): TModalResult;
var
  Form: TForm;
begin
  Form := FormClass.Create(Owner);
  try
    Form.Perform(CM_INIT, WData, LData);
    Result := Form.ShowModal;
  finally
    Form.Destroy;
  end;
end;

So to set to true the MyBool field of a TForm descendant i would call:

ShowForm(TMyForm, nil, 1);

And in the CM_INIT handler:

procedure TMyForm.CMInit(var Msg: TLMessage);
begin
  MyBool := (Msg.lParam = 1);
end;

It worked nice for simple variables like a integer or boolean, but things started to look clumsy when i needed to pass a variable of string or TObject type. Furthermore there's the limitation of restricted number of variables and the danger of the need to assume the meaning of the Msg(TLMessage) fields

There's where the RTTI ability to set arbitrary properties came in hand. All i needed to do is publish a property in the TForm descendant and set it through RTTI functions. This way i got type safety, unlimited number of parameters/variables and clearer code.

The new ShowForm interface:

function ShowForm(FormClass: TFormClass; Owner: TWinControl; FormProperties: array of const): TModalResult;

FormProperties is a array of const where the even items are the property names and the odd items, the property values.

What about RTTI? Pretty simple and direct:

//stripped code (no type check, no array of const parsing)
uses typinfo;
[..]
  ClassInfo := Form.ClassInfo;
  PropInfo := GetPropInfo(ClassInfo, PropertyName);
  case PropInfo^.PropType^.Kind of
    tkAString, tkSString:
      SetStrProp(Form, PropInfo, StrPropertyValue);
    tkInteger:
      SetOrdProp(Form, PropInfo, IntPropertyValue);
    tkBool:
      SetOrdProp(Form, PropInfo, Integer(BoolPropertyValue));
  end; 
[..]

Now to set MyBool property of TMyForm i do:

ShowForm(TMyForm, nil, ['MyBool', True]);

A lot clearer no? ;-)

Draw Rotated Text

Current version of Lazarus provides the ability to draws text in an arbitrary rotation angle. Although is needed just to set the TFont.Orientation property to configure the feature, the position of the draw text will change according to the angle. So, to make things easier, i wrote a routine that draws a rotated text centered in a given Rect.

If someone needs something similar:

type
  TRotateType = (rtNone, rtCounterClockWise, rtClockWise, rtFlip);

procedure DrawRotateText(Canvas: TCanvas; const R: TRect; 
  const Text: String; RotateType: TRotateType);
var
  TextExtent: TSize;
  SavedOrientation: Integer;
begin
  SavedOrientation := Canvas.Font.Orientation;
  TextExtent := Canvas.TextExtent(Text);
  case RotateType of
    rtNone:
    begin
      Canvas.Font.Orientation := 0;
      Canvas.TextOut((R.Right - R.Left - TextExtent.cx) div 2,
        (R.Bottom - R.Left - TextExtent.cy) div 2, Text);
    end;
    rtCounterClockWise:
    begin
      Canvas.Font.Orientation := 900;
      Canvas.TextOut((R.Right - R.Left - TextExtent.cy) div 2,
        (R.Bottom - R.Left + TextExtent.cx) div 2, Text);
    end;
    rtFlip:
    begin
      Canvas.Font.Orientation := 1800;
      Canvas.TextOut((R.Right - R.Left + TextExtent.cx) div 2,
        (R.Bottom - R.Left + TextExtent.cy) div 2, Text);
    end;
    rtClockWise:
    begin
      Canvas.Font.Orientation := -900;
      Canvas.TextOut((R.Right - R.Left + TextExtent.cy) div 2,
        (R.Bottom - R.Left - TextExtent.cx) div 2, Text);
    end;
  end;
  Canvas.Font.Orientation := SavedOrientation;
end;

Finally...

... the VirtualTreeView port is released.

The release notes.

Convert database files from Ansi to UTF-8

While porting old Delphi projects that uses paradox files i faced the problem that the data was stored in an ANSI code page (the current Lazarus version expects UTF-8 encoded strings).

So i wrote an simple application that can convert the encoding of database files (sqlite3, dbf, paradox).

That program can be useful for more than legacy code: after converting a spreadsheet data to a sqlite3 file using Sqlite Data Wizard. The resulted file was in ANSI and there was no option (AFAIK) to convert directly to UTF-8.

I put it here in hope that can be used by someone else.

Using messages to notify events

Lately, i've been using frames extensively in my projects and recently i faced a little problem while working with them: how to notify the parent control (often a TForm) changes in the state of the frame controls/data?

I tried some approaches:

1) Add an event handler to the TDatasource.OnDataChange event.
This has the disadvantage of not being a generic solution since not always the frame has a TDataSource. Also this event is fired in changes to all fields of the respective TDataset, and often only a few fields need to be monitored, leading to some overhead. Yet is not possible to set the event at design time due to bug 14947.

2)Create handlers for events of child controls.
Works by setting in the parent form handlers for events, e.g. OnChange, of child controls of the frame. It has the drawback of needing more than one event handler in the case where is necessary to monitor more than one control/field.
Another problem is that such event handlers can be cleared by the IDE due to bug 14835. Not to say that sometimes the changes in the frame are not propagated to frame instances. In this case the workaround i found to sync the frames was to remove and add the frame again, loosing all properties set in the frame parent control.

So, i decided to implement a specific event for the frame. The first approach i thought was adding a TNotifyEvent property to the frame but this has almost the same drawbacks of approach 2. At this point the idea of using messages came in my mind.

AFAIK messages are used mostly in LCL and seldom in user projects so i had doubts if would work. Here is what i did:

1) Declare a custom message id constant
I declared the following constant in a shared unit:

LM_CHILDDATACHANGED = LM_USER + 1;

2) Create a method in the frame to send the message
I created a method SendDataChangeMsg with the following code:

var
Form: TCustomForm;
begin
Form := GetFirstParentForm(Self);
if Form <> nil then
Form.Perform(LM_CHILDDATACHANGED, 0, 0);
end;

The code is pretty straight. It get the parent form and then send the message through Perform method

3) Hook the events of the controls that should be monitored
Mostly OnChange events. Just called SendDataChangeMsg inside them

4) Add a property ValidData
This property checks if the data in the monitored controls are valid

5) Intercept the message at the frame parent (TForm)
In the TForm i put the frame, i created a method to intercept the message declared as follow:

procedure ChildDataChanged(var Msg: TLMessage); message
LM_CHILDDATACHANGED;

I'm using that to allow saving or not the data so i put a code to enable/disable the save button:

SaveButton.Enabled := Frame.ValidData;

That's all. It's working fine for me. In the end i got a pretty clear notify system. Now i can remove/add the frame as necessary without the need to reconnect the event handlers every time.

Qt and Lazarus: a nice surprise (again)

Long, long time ago i wrote how the Lazarus Qt interface impressed me by its functionality. Yesterday, after updating my Lazarus working copy to do some debugging in Linux, i rebuilt the IDE as usual. But the IDE looked a little different, most notably the source editor font. I've got sometime to figure that the IDE was compiled using the Qt interface. I use the gtk2 interface but somehow i left the LCL build configured to Qt, so the unexpected result.

Again, the Qt interface surprised me. The IDE ran fine with good looking and very responsive except by bugs 15101 and 15103. Unfortunately, i use a lot the features that these bugs affect and i will stay with gtk2 for now. But as soon as these bugs are fixed i will give a try to Qt again.

Status of Virtual Treeview port

It has been more than one year after the last blog update about the Virtual Treeview (VTV) port and some people may be asking what happened with it.

The port is far from dead. In fact, it is fully working under Win32, Gtk1/2 and Qt since, at least, the previous six months. I was just waiting to the Lazarus 0.9.26 release to do an official release. Lazarus 0.9.26 is out, so where is the new VTV?

One of the main features of Lazarus 0.9.26 is the Unicode support for Win32 interface that, in your turn, uses UTF8 encoding. Currently VTV supports Unicode by using UTF-16/WideString and was working fine. After the LCL Unicode switch some encoding conversion problems appeared when iterating with strings returned by databases or LCL controls, so i decided to anticipate the migration to UTF-8 before doing a release.

This task is not trivial and i'll start to work on it only after December, so don't expect an release soon.

This has some advantages:

• There will be no need to further string types changes (WideString -> String) in applications created with the released component
• It will be faster since WideString is known to be slower specially under Win32

Anyway if someone is interested in testing the component check the instructions on how to use the svn version here. Be aware that a change from WideString to String type will be necessary in the long run.

LCL, Gtk2, Pango and Cairo

Introduction

Lately, in the Lazarus mail list, has been a lot of discussion about the performance of LCL/TextOut under Gtk2 and many (erroneous) arguments used derives from the lack of proper knowledge, including from myself. So in this article, i'll try to clear the things a bit.

When Gtk2 was released back in 2002, Pango was one of the Gtk2 main new features. Pango provides support to render Unicode text (encoded in utf8) with advanced layouts. But i was also one of the reasons of the degraded performance of Gtk2 when compared with its predecessor.

Pango has a flexible design allowing to use different backends (Cairo, Xft, Win32, X) to render text. Until version 2.6 Gtk2 used the Xft backend that was replaced, starting from version 2.8, by cairo (a 2D vector drawing library) as the default renderer. At that time the performance dropped even more, but after some work in pango and in cairo, the things got better.

So, the first point to take in consideration when evaluating pango performance is the version of pango and cairo. More on this later.

Testing LCL

In order to get an accurate diagnostic, i wrote an test application that fills an entire window with text using TextOut and compared the results (output quality and time to draw) of the Linux widgetsets (Gtk1, Gtk2, Qt).

Here is the output:




The time to draw an entire screen (average times of 15 iterations):

gtk1: 24ms
gtk2: 150ms
qt: 92ms

The gtk1 widgetset is really fast but it has two drawbacks: the font quality is low and the screen flickers while updating.
The gtk2 widgetset is the slowest loosing to qt by 60%, but in other hand has the sharpest font draw. An important point is that when double buffer is disabled (LCL disables it by default), the screen flickers just like gtk1, but if double buffer is enabled there's no flicker and the screen is updated instantaneous.
The qt widgetset is in the midterm both in quality and in speed. Not sure why qt text looks blurred: if is a configuration or the default font is not so good. There's also no screen flicker since qt is all double buffered.

Gtk2 dissected (almost)

Pango allows more than one engine to be used and there's also the options to draw directly using cairo or the old gdk functions (that use the X11 bitmap fonts). So i wrote an application using direct calls to the gtk2 api. It draws text with default Gtk2 (Pango/Gdk), Pango/Xft, Xft, Cairo, Gdk/X11.

The output (Cairo is equal to Gtk2 and Gdk/X11 is equal to LCL/Gtk1):



The time results:

Gtk2 (Pango/Gdk): 130ms
Cairo: 90ms
Xft: 70ms
Gdk/X11: 12ms
Pango/Xft: 110ms

Gtk2: very close to LCL/Gtk2
Cairo: the same quality of Pango/Gdk (I would be very surprised if was different ;-)) but significantly faster
Xft: Almost two time faster then Pango/Gdk but with lower quality output (An configuration issue?)
Gdk/X11: basically the same output of gtk1. Again really faster but without the screen flicker of gtk1.
Pango/Xft: faster than Pango/Gdk, slower than Xft. Out of option since is not working at all: it draws always at 0,0 coordinates.

Alternatives to pango?

From this tests, using directly Cairo to draw text seems to be a good option to replace Pango: the same quality and faster. But is not that easy. With direct calls to Cairo is needed to do all sort of text position (bidi, alignment) and text styles (underline) manually. Also it would be necessary to do the font selection and loading logic manually in a system specific way,i.e., is necessary different code to Linux/Win32/MacOSX.

Another option is using Xft directly. Aside from the different/worse output, it would be necessary to change the widgetset to retrieve the XftDraw handle for each drawable which is a complex task principally for double buffered controls. All in a system specific way.

Gdk/X11: the quality of output and lack of Unicode support makes a no-no option. At least for me.

Conclusion

Is there a direct/easy/faster replacement for Pango under LCL/Gtk2? No.

Here is necessary to make another question: there's a real need to replace it? Like shown earlier, default Pango is really slow compared to other alternatives, but, principally when double buffer is enabled, there's no visible glitches or general system slowdown. Is also necessary to take in account the increase of code size and complexity in LCL/Gtk2 side to make such change.

Notes

• The test applications can be found here and here. Is necessary the package chronolog;
• The test were conducted in a Celeron 1.4, 512MB, with an intel integrated video running Ubuntu 8.04;
• Upgrading from Ubuntu 7.10 to 8.04 leaded to an significant speed in all widgetsets (Gtk2 250 > 150 / Gtk1 50 > 24ms);
• I also tested drawing with low level Pango api. I'll post the results later;
• Win32 took impressive 5ms in the same test Not to be considered since the win32 test was done in another (more powerful) machine;
  
• A point that is not directly related to text draw but affects performance of LCL/Gtk2 is the fact that the LCL/Gtk2 test application calls 4 times FcFontSort, while a Gtk2 pure application calls only once. This is an expensive call that takes almost 20% of the application time.
  

Reduce memory usage of LCL

To test the conclusions of the last post i modified the field order of some LCL classes to group together Boolean fields.

Here's the return value of the InstanceSize property:

TButton
before: 874 bytes
after: 829 bytes (saved 24, 16 and 5 bytes in TControl, TWinControl and TCustomButton respectively)

TMenuItem
before: 152 bytes
after: 136 bytes

In an application with 100 controls and 20 menu items you save 2400 (considering only TControl memory) and 320 bytes respectively.

Maybe be this is negligible in computers with 1GB or more of RAM, but for mobile platforms it makes a difference.

The patch is here. Have fun!

What Time Is It?

How many cairo clocks the world needs?

I don't think we have sufficient!

Strange days...

Since day 1 of Lazarus, the gtk1 widgetset is the default, and reference, interface under Linux.

Today, after one of my applications crashed using gtk1, i tried it with the Qt4 interface: it worked flawlessly!!.

The Qt interface, given the recent work, is in the way to become the standard widgetset under Linux. Wait and see.

Not so far ...

Before than i think i got VTV to work under Windows (In fact is working since, at least, two weeks).

Is almost everything there: Header, OLE drag and drop (and VCL too), Unicode, Bidi. Alpha Blend is still missing. There are also some graphic glitches with the more advanced examples.

Here's a sample:

Traps of Delphi to Lazarus conversion

There's only a thing worse than a bug difficult to resolve: a bug difficult to find. As an example, i was getting a weird bug in the drawing of VTV. The colors were being painted wrongly and only after a few hours i figured what was going on. See the code below:

with Canvas do
begin
[..]
Brush.Color:=Color;
[..]
end;


Apparently nothing wrong. But the Devil resides in the details. Under Delphi there's no TCanvas.Color while there's in LCL. The result of this code, in Delphi, is that Brush.Color is assigned to TControl.Color. In LCL, Brush.Color is assigned to TCanvas.Color (which in your turn points back to Brush.Color!!).

You can't even say that there's a bug in LCL or in the program. Is just one of traps found while converting Delphi code.

The same occurs with TCanvas.Width/Height (does not exist in Delphi) and TBitmap.Width/Height when used inside compound "with" statements.

Children of the port

A working new VirtualTreeview is far from reality but there are already some results.

Since this is really a complex software, a lot of debugging will be necessary so i needed to extend the Multilog component/package. I just released a new version. But one of the functions i added was the ability to log memory areas and i needed a component to display it. This way i ended porting ATBinHex (in fact is a grandchild of the port ;-)).

As a bonus i implemented TOleStream that was missing from Delphi.

You can find them at https://luipack.bountysource.com/

Ports of Delphi components

Last week i started to port two Delphi components: the already know VirtualTreeView and ATBinHex a Hex viewer. At the moment VirtualTreeView is compilable but does not work and ATBinHex works under Windows only. Both code can be get in LuiPack, a project i started to centralize and help keep track the components i develop.

Before someone would ask: "Why do a new port of VirtualTreeView if already exists one?", here are the answers:

• The original port of VirtualTreeView (VTV) was based in version 4.0.17 from December 2003. Currently we are in version 4.5.1. A lot of changes happened after that. One option was to see what changed and than backport to the lazarus version. But two problems arise: At the time of 4.0.17 the VTV code was not in a version control system making hard to know what changed. Furthemore, the chances that applying these changes would conflict with the modifications done in lazarus port are high.
  So this time, the VTV code is in a svn server being easy to see what changed and than backport small modifications to lazarus if appropriate.
• Some features were removed in the port process (CheckBox images, AlphaBlending, Bidi support) and other bugs introduced. To completelly fix them and add the missing features would require deep knowledge of how VTV works and the success was not guaranteed. So i believe the work to do a new port would be only a more hard than to fix it.
• In that time LCL/fpc was far from what is now. This allow to do a easier port without the need to remove features. So, instead of removing features the idea is to fix the LCL, where appropriate or redo with LCL/fpc functions.

PS: Don't expect a working VTV soon, because of the decision to not remove features will lead to a long road.