On Wed, Feb 08 2006, Hans Hübner wrote:
2006/2/7, Kamen TOMOV ktomov@web.de:
On Tue, Feb 07 2006, Hans Hübner wrote:
.. I will propably work on database maintenance and operations so that BKNR based systems can be run for a long time without much nursery.
What kind of nursery do you mean?
The snapshotting mechanism currently eats disk space without bounds,
It seems to me it eats as much as necessary - no more no less. Or am I missing something?
If you regularily snapshot from a cron job, all snapshots will be kept on disk and need to be manually moved to offline media. Depending on the transaction rate, it may be neccessary to snapshot often, which will use a lot of disk space. It would be good if the Datastore would provide an automatic mechanism that purges older snapshots. This could be done on a simple (only keep N snapshots) or a more complex basis (keep the last three snapshots, one for each week, one for each month or some such).
Also, a restore tool is needed that helps selecting a snapshot to restore. I have been bitten many times while selecting an older snapshot to restore and selecting a too-early one. Instead of manually having to select a certain snapshot, one should be able to specify a date and time (or manually created markpoint) to restore to and have the environment select the right snapshot file and rollforward location.
It would also be good if a snapshot could be automatically created as soon as a certain restore time limit has been reached. This would require adding up the execution time of restored and newly executed transactions and scheduling a snapshot only if it is required. Together with the automatic selection of a snapshot to restore, this would make the database system much easier to use, and it would also help with the disk growth problem.
I see. These would probably be useful tools.
I was thinking about the general purpose of the snapshotting mechanism. It seems to me that it exists solely because the hardware can not meet the demands of the applications only in terms of store-objects restore times. For not so demanding applications I would rather do without snapshotting. The prevalence model allows tracking the history of the changes of the data.
As a framework user I would rather not rely on snapshotting to allow changes of my store-object based objects. This is something that I would fix in the framework.
In my opinion the main functionality must not depend on the snapshotting mechanism, only the other way around.
and the blob garbage collection is not automated.
What exactly do you mean? If we delete blob wouldn't the memory be cleaned?
Yes. Blob files cannot be deleted when the blob object is deleted because they may be needed again, should the user decide to restore an earlier snapshot. An integrated snapsht archival function would be able to move deleted blobs offline together with the corresponding snapshot file and transaction log. At the moment, there is the (bknr.datastore:delete-orphaned-blob-files) function which can be called to get rid of stray cats.
I meant the memory of the blob object, not the file. I suppose this memory is garbage collected, right?
As to the disk memory I was surprised to find out (some time ago) that the files are not deleted on deleting the blob object. Now I understand your reasons. The integrated snapshot archival function that you suggest sounds reasonable. Perhaps once we have it, we could consider deleting the associated file on blob deletion.
Regards,