UnQLite Users Forum

Memory consumption again

append delete Staon

Hi,

I like the library and I would like to use it in my next project as a data backend. But it seems that the library consumes too much memory. Maybe I'm doing something wrong.

This is my test:
1) to create a hash table (unqlite_open),
2) 1000000 times insert pairs of approximately 30 B length (unqlite_kv_store). Commit the table after each 10000 insertions (unqlite_commit).
3) Close the table (unqlite_close).

Thus I inserted approximately 30 MB of data. The resulted file is approximately 90 MB long (it's OK). However, the application consumes more than 260 MB of RAM!

I tried to set the UNQLITE_CONFIG_MAX_PAGE_CACHE but there was no effect. And the value is probably ignored by the library as well as I can see in the source codes.

Invocation of the unqlite_lib_shutdown after the insertions cannot help because our platform (QNX4) doesn't return allocated memory back to the system.

I used the 1.1.6 version and I can observe the same behavior at Linux too (thus it's probably not an error caused by my port to QNX). Is this the usual behavior or do I something wrong?

Thank you for your response.

Reply RSS

Replies

append delete #1. devel

Can you submit any chunk of code to reproduce the scenario? Thx

append delete #2. Staon

OK, the code is exactly what I wrote above:

int main(
int argc_,
char* argv_[]) {
unqlite* db_; /* Database handle */
int info_;
int i_;

info_ = unqlite_open(&db_, "mem.uql", UNQLITE_OPEN_CREATE);
if(info_ != UNQLITE_OK) {
exit(-1);
}

/* Store some records */
for(i_ = 0; i_ < 1000000; ++i_) {
/* -- store value */
info_ = unqlite_kv_store(db_, &i_, sizeof(i_), "01234567890123456789012345", 26);
if(info_ != UNQLITE_OK) {
exit(-1);
}

if(((i_ + 1) % 10000) == 0) {
info_ = unqlite_commit(db_);
if(info_ != UNQLITE_OK) {
exit(-1);
}
}
}

/* -- now the process consumes (I think so) too much memory */

info_ = unqlite_close(db_);
if(info_ != UNQLITE_OK) {
exit(-1);
}

unqlite_lib_shutdown();
return 0;
}

Reply

(Leave this as-is, it’s a trap!)

There is no need to “register”, just enter the same name + password of your choice every time.

Pro tip: Use markup to add links, quotes and more.

Your friendly neighbourhood moderators: chm_at_symisc, devel_at_symisc