Re: [Jdbm-general] Buffering Log File

Kris Leite wrote:
> I have a suggestion for JDBM that appears to improve the speed of
> writing out to the log file. The log file does not use any buffering so
> writeObject is sending data one character at a time to the log file. By
> buffering, there appears to be a about 20% improvement in the log file
> operations (Win/XP on 3GHZ Intel platform). General throughput
> will vary depending on how many commit()'s are used.
Yeah, buffering should improve the performance. Not creating new streams
all the time should improve it even more. You may want to try and create
custom OOS/OIS streams that override the class descriptor methods as
well, and have them be implemented by simply writing the class name
instead of the entire class descriptor. That should bring down the size
of the log file a bit.
/Rickard

Kris Leite wrote:
> I have a suggestion for JDBM that appears to improve the speed of
> writing out to the log file. The log file does not use any buffering so
> writeObject is sending data one character at a time to the log file. By
> buffering, there appears to be a about 20% improvement in the log file
> operations (Win/XP on 3GHZ Intel platform). General throughput
> will vary depending on how many commit()'s are used.
Yeah, buffering should improve the performance. Not creating new streams
all the time should improve it even more. You may want to try and create
custom OOS/OIS streams that override the class descriptor methods as
well, and have them be implemented by simply writing the class name
instead of the entire class descriptor. That should bring down the size
of the log file a bit.
/Rickard

Kris Leite wrote:
> >Not creating new streams all the time should improve it even more.
>
> Hmm, if you are implying that using the "reset()" of the
> ObjectOutputStream.
> Then there was a slight improvement by replacing line #317:
> oos = new ObjectOutputStream(fos);
> with:
> oos.reset();
>
> If you are suggesting removing it completely, that has a side effect
> of GC not working since OOS/OIS keep references to the stored
> object until the stream is closed or reset.
.. unless you use OOS.writeUnshared() which will not keep references.
/Rickard

Hi Rickard,
>.. unless you use OOS.writeUnshared() which will not keep references.
Good point. Had not tried that new 1.4+ feature. I put it in and found
it does improve throughput by another 7% to 9%. There is a noticable
reduction of CPU usage also. Don't have good numbers but it is around
10% or more. I implemented this change by replacing line #309
from:
oos.writeObject(txns[curTxn]);
to:
oos.writeUnshared(txns[curTxn]);
And removing Lines #315 to 317.
The only drawback to this change is that it requires JRE 1.4+. For me,
it is just fine. Don't know if that is acceptable for JDBM 0.20+ release.
Thanks,
Kris
Rickard Öberg wrote:
> Kris Leite wrote:
>
>> >Not creating new streams all the time should improve it even more.
>>
>> Hmm, if you are implying that using the "reset()" of the
>> ObjectOutputStream.
>> Then there was a slight improvement by replacing line #317:
>> oos = new ObjectOutputStream(fos);
>> with:
>> oos.reset();
>>
>> If you are suggesting removing it completely, that has a side effect
>> of GC not working since OOS/OIS keep references to the stored
>> object until the stream is closed or reset.
>
>
> .. unless you use OOS.writeUnshared() which will not keep references.
>
> /Rickard
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by BEA Weblogic Workshop
> FREE Java Enterprise J2EE developer tools!
> Get your free copy of BEA WebLogic Workshop 8.1 today.
> http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
> _______________________________________________
> Jdbm-general mailing list
> Jdbm-general@...
> https://lists.sourceforge.net/lists/listinfo/jdbm-general
>
> .
>