ClientDataSet cannot update fields with null value (II)

25.4.05

Before continue the second bug report, I want to share some information about TClientDataSet.

From A ClientDataSet in Every Database Application by Cary Jensen, ClientDataSet has 2 data stores. One is delta. It represents the change log that all changes made to Data. Specifically, for each record that was inserted or deleted from Data, there resides a corresponding record in Delta. For modified records it is slightly different. The change log contains two records for each record modified in Data. One of these is a duplicate of the record that was originally modified. The second contains the field-by-field changes made to the original record.

If inserts a record, the TClientDataSet.Delta data packet will be as:
<?xml version="1.0" standalone="yes" ?>
<DATAPACKET Version="2.0">
<METADATA>
<FIELDS>
<FIELD attrname="ID" fieldtype="string" WIDTH="10" />
<FIELD attrname="NAME" fieldtype="string" WIDTH="10" />
<FIELD attrname="AGE" fieldtype="i4" />
</FIELDS>
<PARAMS DATASET_DELTA="1" />
</METADATA>
<ROWDATA>
<ROW RowState="1" ID="S001" NAME="PETER" AGE="10" />
<ROW RowState="4" ID="S002" NAME="JOHN" AGE="20" />
</ROWDATA>
</DATAPACKET>
If deletes the record, the TClientDataSet.Delta data packet will be as:

<?xml version="1.0" standalone="yes" ?>
<DATAPACKET Version="2.0">
<METADATA>
<FIELDS>
<FIELD attrname="ID" fieldtype="string" WIDTH="10" />
<FIELD attrname="NAME" fieldtype="string" WIDTH="10" />
<FIELD attrname="AGE" fieldtype="i4" />
</FIELDS>
<PARAMS DATASET_DELTA="1" />
</METADATA>
<ROWDATA>
<ROW RowState="2" ID="S001" NAME="PETER" AGE="10" />
</ROWDATA>
</DATAPACKET>
If updates the record by setting age = 20, the TClientDataSet.Delta data packet will be as:
<?xml version="1.0" standalone="yes" ?>
<DATAPACKET Version="2.0">
<METADATA>
<FIELDS>
<FIELD attrname="ID" fieldtype="string" WIDTH="10" />
<FIELD attrname="NAME" fieldtype="string" WIDTH="10" />
<FIELD attrname="AGE" fieldtype="i4" />
</FIELDS>
<PARAMS DATASET_DELTA="1" />
</METADATA>
<ROWDATA>
<ROW RowState="1" ID="S001" NAME="PETER" AGE="10" />
<ROW RowState="8" AGE="20" />
</ROWDATA>
</DATAPACKET>

You may notice that a additional row of data in the data packet. The first row is the orginal contents of the record while the second one is the change(s) we made to the data. Passing the orginal contents of row is because:
1. primary key(s) may not be available in the second row
2. the record may be updated by others while we edit the record

In the second row of data packet, you can find only the changes made to data is included. RowState is an indicator telling MIDAS server what to do with the values in that row, like insert, delete or update. Well, It's time back to my second bug report. What happens if updates the record by setting AGE = Null? The delta data packet is like this:
<?xml version="1.0" standalone="yes" ?>
<DATAPACKET Version="2.0">
<METADATA>
<FIELDS>
<FIELD attrname="ID" fieldtype="string" WIDTH="10" />
<FIELD attrname="NAME" fieldtype="string" WIDTH="10" />
<FIELD attrname="AGE" fieldtype="i4" />
</FIELDS>
<PARAMS DATASET_DELTA="1" />
</METADATA>
<ROWDATA>
<ROW RowState="1" ID="S001" NAME="PETER" AGE="10" />
<ROW RowState="8" />
</ROWDATA>
</DATAPACKET>
RowState tells us the record is modified but NO information about the change on AGE field is listed in second row. This is because XML does not support null attribute value. That's why I say ClientDataSet cannot update fields with null value. Two bug reports in Borland Quality Centre has been raised about this issue in 2002 and 2004 but they are still open :(
Report#617
Report#8137

I do not know how other developers avoid this issue in their application but I know the users are quite unhappy.


Comments:
Well.. gloria... there is indeed a fast solution for this, a little bit dirty... but fast...
Why don't you set the value to zero ?.. Maybe this would affect in some places at your code if you check the null value for the field... but this is the fastest solution that it comes me to mind right now...
God knows I disklike completely dirty solutions... but if your clients are pressing... you can apply this solution now and after if you may find a solution you can just make a loop to the database and change all zero ages to null...
I wish this helps.

I had asked the component users for this approach ... but customer is *always right*
Besides, string and numeric fields can avoid this problem by setting value = empty string or zero, but how about datetime fields? Some controls show 1899/12/30 if datetime value is zero.

Now I also take a dirty solution :P
In myCustomDataSet, a List of fields with null value is prepared in DoBeforeApplyUpdates and is passed to myCustomDSProvider. Then a data massage is preformed in myCustomDSProvider.InternalApplyUpdates with this list before sending modified data to host.

Well if you have a need, there is one of my components named "TDBARMultiValueLabeledEdit" that would not show 1899/12/30 #;.DD just blank... if you need it ... just tell me... I send it to you (full source)

Now is 9pm... but still working at office :(
~~~ very very hungry ~~~

Albert, Thanks your suggestion. But I cannot use yours as my company already has her own set of controls. But I may ask you many many Qs about control development later...
Develop a visial control is a difficult (FOR ME!)

Today I told my manager how I solved the clientdataset problem. He said it's dirty but no objection...hehee... maybe because he knew nothing about Delphi

9pm... oh dear... now you are a Developer :-) !!!.
Ok. No problem (about Component development). Just tell me. As you probably now, I am just around the corner. #;.D

Hello my friends...

I have this same problem...

I find solution for this problem...

Hi,
I have reviewed to Borland Q's and find that this problem has solution:

If 'newValue' is unassigned this means the record is not updated.
If 'newValue' is null this means the record is cleared (value = null).
If 'newValue' contains a value this is the new value.


You can use TDatasetProvider and it's events to deal with custom SQL updates in that manner.

JK

Dear Gloria.

I have read your article.
Im actually working with BCB6 but Im coming from the Flex/Flash world and .NET. So the best way for me to design an application is a multitier one.
Im using the library IBPP for Firebird and I would like to translate the property Delta by myself into SQL instructions as I did in the past with the DeltaPackets in flash.

How can I access to the XML described in your article? What I get in a TClientDataSet->Delta is non XML data.

Thanks in dvance.

Did you know that you are so beautiful? ;)

David

Hi Gloria,

I don't know if it's still relevant to you, but I post it here because many people looking for a solution to the same problem find your blog via Google.

You are right in stating that the XML of the Delta lacks information for fields that have been set to null. So rather than looking at the XML of the Delta, you can look at the delta itself. I mean the rows of the clientdataset to which the Delta was assigned (like CDSDelta.Data := CDSOriginal.Delta;).

But how do you get to the RowState, like it was in the XMLDelta? At first I could not find it. I was looking for ROWSTATE in the Delphi sources, but couldn't find anything.

I found out that the rowstate is the same as the UpdateStatus of the Delta dataset. So you can do something like:

CDSDelta.First;
while not CDSDelta.Eof do
begin
case CDSDelta.UpdateStatus of
usDelete:
GenerateDeleteSQL(CDSDelta);
usInserted:
GenerateInsertSQL(CDSDelt);
usModified:
GenerateUpdateSQL(CDSDelta); // Perhaps using the previous line with unmodified data.
end;
CDSDelta.Next;
end;

If you need the delta as XML to send it from a client to server for example, you can just generate an XML with all values including null updates based on this information.

Kind regards.

Dbx is being discontinued in 2015 and to this day I have not figured out a solution to this problem .. Does anyone found solution? Thank you .

Post a Comment

<< Home