UnQLite Users Forum

Cursor strange behaviour - deleting records?

append delete RicDias

Hello,

I've been doing some tests with unqlite and found a strange behaviour with cursors.
Here's the simple test file.

% C
#include <stdio.h>
#include <stdlib.h>
#include "unqlite.h"

unqlite *pDb;
int rc;

int getSize(){
	unqlite_kv_cursor *pCursor;
	rc = unqlite_kv_cursor_init(pDb,&pCursor);

	int size = 0;
	/* Point to the first record */
	for( unqlite_kv_cursor_first_entry(pCursor) ; unqlite_kv_cursor_valid_entry(pCursor) ; unqlite_kv_cursor_next_entry(pCursor) ){
		size++;
	}
	return size;
}

int main(int argc, char** argv) {

	unqlite_open(&pDb,"test.db", 0x00000004);


	if(argc == 1) {
		fprintf(stderr,"Size: %d\n",getSize());
		fprintf(stderr,"Size: %d\n",getSize());
		fprintf(stderr,"Size: %d\n",getSize());
	}else{
		int i = 0;
		for(; i < 1000 ; ++i ){
			unsigned int zKey = unqlite_util_random_num(pDb);
			int zData = 2*i;

			// Perform the insertion
			unqlite_kv_store(pDb,&zKey,sizeof(zKey),&zData,sizeof(zData));
		}

		unqlite_commit(pDb);
	}

	fprintf(stderr,"Final size: %d\n",getSize());

	unqlite_close(pDb);

	return EXIT_SUCCESS;
}
%

here comes the strange part:
$ ./unqlitetest 1
Final size: 1000
$ ./unqlitetest 2
Final size: 2000
$ ./unqlitetest
Size: 2000
Size: 1991
Size: 1991
Final size: 1991

Why do I get the right size in the first call and wrong in the others?
The worst part is that if I store all the keys in the first call and try to get their values after that, it fails to find them in the storage. This means something is being deleted in the process.

Is there anything that I'm doing wrong??
Any help would be appreciated.

Best regards,
Ricardo Dias

Reply RSS

Replies

append delete #1. flanhard

Hi Ricardo,

This is an append operation, so in the first call, the database does not exist, it is created and the data is stored normally while in the late calls, the database is already on disk with new keys (might be same) appended in so do not expect the same db size.

append delete #2. RicDias

Hi flanhard,

I'm sorry but I did not quite understand your answer. You might not have understood the problem correctly, so I'll try to elaborate. :)

Between the calls nothing is changed (nothing is removed nor added). How can I iterate from the first to last record twice sequentially and count a different number of elements?

Shortly, I add 2000 elements, close and when I re-open my db, I iterate from first to last element twice and the result is: In the first call, I count 2000, but in the second I count less than that. I don't think this is the expected result... :/

Best regards,
Ricardo

append delete #3. RicDias

Bump!

I'm interested in using unqlite, but with this bug it doesnt fit my requirements.
Can anyone say if this is being investigated and if a fix will be released?

Best regards,
Ricardo

append delete #4. chm

Hi Ricardo,

Of course, we're investigating you issue and will update our status accordingly. Again, thanks for you report.

append delete #5. Yura

Seems to be an instance of https://github.com/symisc/unqlite/issues/3

append delete #6. chm

@Yura

Yes absolutely, the patch was already merged with the main trunk in github and will be available here shortly.

append delete #7. lbn

I downloaded the patched version of unqlite from github. I then re-ran the above mentioned test case of Ricardo's. And I still got the bad results. When I read through the entire file using the cursors routines, the first time through works fine, but when I read through it again without closing the database first, it still misses records. Just like Ricardo described above.
A serious bug and problem.
Until I can't count on retrieving all the data in an unqlite database, I won't be able to use this package - although it is exactly what I'm looking for.

append delete #8. chm

We've found the problem and was fixed for the Vedis library, since it share the same codebase as UnQLite. It will be merged here pretty shortly.

This is the issue: https://github.com/symisc/vedis/issues/3

append delete #9. dcv

It looks like "Yuras" merged the fix for this bug into the unqlite repository about four months ago, so is the amalgamation file "unqlite.c" going to be updated to include this patch?

I understand the original authors are busy with other projects now, but it appears that the updated "unqlite.c" file is not available on github, so I'm not sure how a third party can submit a patched version of the amalgamation file.

append delete #10. chm

OK, the amalgamation C file, has been added to the github repo: https://github.com/symisc/unqlite/tree/master

append delete #11. shiva

Hello . I have already merged the latest update from here https://github.com/symisc/unqlite/tree/master.

In the following function I am trying to delete the entries by itearting the cursor. But, not all entries are deleted and its giving different results when I use the cursor in different directions.

1. Creating a database to store 5 elemnets with different keys and values
2. Close the database, so that the contents are flushed to the disk. (without closing the the order of storage is reversed when trying to access these elements)
3. Open the database in Write mode
4. When traversed the database to delete the entries,
case 1:
for( unqlite_kv_cursor_first_entry(pCur_parts_list) ; unqlite_kv_cursor_valid_entry(pCur_parts_list) ; unqlite_kv_cursor_next_entry(pCur_parts_list))

using this only 3 dentries were delated(it skips every alternate entry and deletes (entry 1, 3, 5 are delated. But, entry 2 and 4 stays )

using this,
case 2:
for( unqlite_kv_cursor_last_entry(pCur_parts_list) ; unqlite_kv_cursor_valid_entry(pCur_parts_list) ; unqlite_kv_cursor_prev_entry(pCur_parts_list))

only the last entry is deleted. Rest all stays.

%.c

-----------------------------------------------------------------------------------------------------------

int Persistency()
{
int rc,i,pos;
const char *applcns[] = {"first", "second", "third", "fourth", "fifth"};

/* Open our database */
rc = unqlite_open(&pDb_parts_list,"Partitions_List" ,UNQLITE_OPEN_CREATE);
if( rc != UNQLITE_OK ){
Fatal(0,"Out of memory");
return 0;
}
else
puts("Partitions_List created/opened");

for(i = 0; i< 5 ; i++)
{
random_no = rand();
printf("\n %d Random No=>",random_no);
rc = unqlite_kv_append(pDb_parts_list,applcns[i],-1,&random_no, sizeof(random_no));
if( rc != UNQLITE_OK ){
/* Insertion fail, extract database error log and exit */
Fatal(pDb_parts_list,0);
}
else
printf("\n %s Partiton added \n",applcns[i]);
}
unqlite_close(pDb_parts_list);

rc = unqlite_open(&pDb_parts_list,"Partitions_List" ,UNQLITE_OPEN_READWRITE);
if( rc != UNQLITE_OK ){
Fatal(0,"Out of memory");
return 0;
}
else
puts("Partitions_List created/opened");

/* Allocate a new cursor instance */
rc = unqlite_kv_cursor_init(pDb_parts_list,&pCur_parts_list);
if( rc != UNQLITE_OK ){
Fatal(0,"Out of memory");
return 0;
}


//Remove element from Partitions List Database
for( unqlite_kv_cursor_first_entry(pCur_parts_list) ; unqlite_kv_cursor_valid_entry(pCur_parts_list) ; unqlite_kv_cursor_next_entry(pCur_parts_list))
{
rc = unqlite_kv_cursor_delete_entry(pCur_parts_list);
puts("\n Entry deleted from Partitions List Database");
}

unqlite_kv_cursor_release(pDb_parts_list,pCur_parts_list);
unqlite_close(pDb_parts_list);

return 1;
}

output

-----------------------------------------------------------------------------------------------------------

case 1: ./out/persservdemo
Partitions_List created/opened

1804289383 Random No=>
first Partiton added

846930886 Random No=>
second Partiton added

1681692777 Random No=>
third Partiton added

1714636915 Random No=>
fourth Partiton added

1957747793 Random No=>
fifth Partiton added
Partitions_List created/opened

Entry deleted from Partitions List Database

Entry deleted from Partitions List Database

Entry deleted from Partitions List Database

case 2:
./out/persservdemo
Partitions_List created/opened

1804289383 Random No=>
first Partiton added

846930886 Random No=>
second Partiton added

1681692777 Random No=>
third Partiton added

1714636915 Random No=>
fourth Partiton added

1957747793 Random No=>
fifth Partiton added
Partitions_List created/opened

Entry deleted from Partitions List Database

append delete #12. Charles

It does not appear that this issue is fixed in UnQLite 1.1.6.

https://github.com/symisc/unqlite/issues/41

Reply

(Leave this as-is, it’s a trap!)

There is no need to “register”, just enter the same name + password of your choice every time.

Pro tip: Use markup to add links, quotes and more.

Your friendly neighbourhood moderators: chm_at_symisc, devel_at_symisc