[C++ MFC] in memory databases

giugio · #1 05-Feb-2008, 06:51

I would create a Information system application on a civil/structural project written in C++ MFC. I have a list of beam, frame and shell objects (in some collections) with a lot of properties I would to select and update this collections on the fly.

I read some on in-memory databases what are the goal of these databases?

Is that a good solution for these databases?

p.s. the project is in MFC and I can't use managed code (.net) or linq.

#2 05-Feb-2008, 16:01

Quote:

Originally Posted by giugio

Hy.
I would create a Information system application on a civil/structural project written in c++ mfc.
I have a list of beam ,frame and shell objects(in some collections) with a lot of properties i would to select and update this collections on the fly.
I read some on in-memory databases what are the goal of these databases?
is that a good solution for these databases?
ps. the project is in mfc and i cant use managed code(.net)or linq
Thanks.

The goals are (as I understand them) to provide a common API to a very fast (speed of memory accesses and related instruction execution times for relevant code segments) database. The benefit is especially helpful when objects are constantly undergoing changes as a result of activity, such as interaction with the user resulting in substantial changes in numerous objects.

One of the detractions from their use is that a "burp" in power usually means they are hosed, whereas with file-based systems, this is somewhat less likely depending on the design of the database and its related commit and/or transactional implementation. Also, configuration criteria may affect resistance to "external issues," such as cache size settings, etc. Some of these settings are implicit by default as a result of "setting up" the database for "some number of users." For example, cache sizes and commit change event timing may increase as a result of a system set up for a larger number of users than a smaller number of users in order to keep expected disk activity at a point acceptable with the limitations of standard hardware constraints.

In a situation where a single user or single program is interacting with the database, I'd generally recommend using a traditional embedded (file-based) database as opposed to an "in memory" database. If you feel that you want to explore the world of in memory databases, you may want to also consider using an object-oriented database. A lot of it depends on what you're doing with your objects and what you want to accept in terms of "pain" in moving to any database system from whatever you are doing/using now.

You can use a simple, traditional embedded database and point its "database file" at an area of memory mapped into your process space as a file as a very easy way to get started with an "in memory" database system that can also be used as a traditional file-based system, if deemed appropriate for your needs.

I'd probably start with the file-based DB and "just see" if it is fast enough for my needs. Chances are it will be good enough for just about any engineering application where objects are not graphical. That is, where "image" rendering is desired and pixel data is constantly being modified. The embedded file-based choice also gives you the ability to easily read from a file-based database, set up a memory mapped file, essentially clone the file-based DB into the memory mapped file and then "use database db_mmapped" in your SQL statement...effectively switching you on-the-fly to the memory mapped DB during application runtime. Periods of inactivity or on explicit commits, you can merge the memory mapped version with the file-based version as often as deemed appropriate for your application or users' needs.

I recommend taking a look at MySQL for more details on using their embedded C API. Of course, someone has written a C++ wrapper, if that is of help to you.

One thing that you may want to consider doing is making a rough estimate of the total number of objects and the total number of properties that can be changed during runtime. Then try to apply some kind of estimate as to how often those properties will change based on the following sets:

Set1: Very often/continuously changing
Set2: Frequently changing with relatively long periods of inactivity
Set3: Moderately change, may or may not change very often
Set4: Infrequently changes
Set5: Very rarely changes

Try to assign a number of objects to each set and then the approximate number of properties in each object so that you have the following breakdown (example shown below):

Set1: 45 Objects 390 Properties
Set2: 185 Objects 17500 Properties
Set3: 600 Objects 90000 Properties
Set4: 90 Objects 6800 Properties
Set5: 130 Objects 8600 Properties

Whatever the distribution is, then you can make some "educated guesses" as to what level of safety versus execution speed you can accept in your database choice.

Something else that may be more practical is to create COM (or COM+) objects (or ActiveX objects) that utilize a file and/or memory mapped file based embedded database for storing data/state information. Using "real C++," this is not incredibly difficult, assuming that your classes are properly designed and include "archiving operators" (the MFC-ish term for streaming operators) where necessary.

...if not, well, that may very well be part of the "pain" associated with your database options!

:davis:

giugio · #3 06-Feb-2008, 01:50

Tanks Davis.
The goal for me may be to use a query language such sql for the interrogations.
WIth sql or simil sql i can compose sql string runtime and execute it.
The sql string can change and composed by the interaction of the user with the program and this is very hard to do without sql.
i search something simil linq in c++ mfc.
Exist?
You talk me about an object oriented in memory db ,what ?
What free or open source library or db you advce me?
Thanks a lot.

#4 06-Feb-2008, 06:10

Quote:

Originally Posted by giugio

Tanks Davis.
The goal for me may be to use a query language such sql for the interrogations.
WIth sql or simil sql i can compose sql string runtime and execute it.
The sql string can change and composed by the interaction of the user with the program and this is very hard to do without sql.
i search something simil linq in c++ mfc.
Exist?
You talk me about an object oriented in memory db ,what ?
What free or open source library or db you advce me?
Thanks a lot.

Try Firebird (free) or MySQL (free for non-commerical use) for an RDBMS.

www.firebirdsql.org

www.mysql.com

ODBMSes are a bit more challenging. There are some that are "free for development, but must pay to release" and then there are some that are not OMG conformant that are free, but really centered around UNIX rather than Windoze...but can probably be built on Windoze with some effort.

Check out: lambda.uta.edu

...for LDB which is built atop of SHORE. Many ODBMS are built with Java bindings while relatively few exist in open space for C++. Most of the C++ products are commercial. Check out PSE (and its cousin PSE Pro) as these may still be available from Object Store:

www.dei.isep.ipp.pt

...is dated in the 1996/1997 timeframe, which is about the last time that I used PSE on a project. It doesn't seem to matter how often I tell "managers" that an ODBMS would be a "better choice," they lean toward their areas of comfort and go with RDBMS products. It makes sense for the 80%...I guess.

PSE is now only available as PSE Pro from Object Store, who was bought by Progress several years ago:

www.progress.com

If you don't know OQL or if you are concerned about the licensing issues, you may want to default to Firebird or MySQL.

Something else for you to consider is using SQLite, which is a lightweight SQL engine designed for embedding into applications. It is very quick and easy to use. I suppose that somebody has a C++ wrapper around its C API, too--not that it is needed for what you're doing, it may just be more convenient to use.

www.sqlite.org

:davis:

giugio · #5 07-Feb-2008, 02:30

Thanks Davis.
So there isn't Something similar to Linq for c++ mfc?
I find this :
www.ddj.com/database/184401205?pgno=1
that should be done for me, but I can't understand how download this library!

giugio · #6 08-Feb-2008, 03:55

sorry , but i have some problem with the precedent post.
i find this:http://www.ddj.com/database/184401205?pgno=1
(white for 15 s).
That work fine for me.
But I dont understand the "shared memory" and the SCCollection class.
Is possible insert a record in the in memory db and retrieve it from the collection?
Thanks.

giugio · #7 11-Feb-2008, 07:15

I read that with ado can use disconnected recordset and can sort/filter the data in memory,what are differences from the ado(not .net) disconnected recordset and the in memory dbs?
And i can't find a in memory db for the vc++ !
i find "timesten" by oracle but i don't understand if is free:
http://www.oracle.com/technology/software/products/timesten/index.html
ps.thanks a lot davis

#8 11-Feb-2008, 09:10

Quote:

Originally Posted by giugio

I read that with ado can use disconnected recordset and can sort/filter the data in memory,what are differences from the ado(not .net) disconnected recordset and the in memory dbs?
And i can't find a in memory db for the vc++ !
i find "timesten" by oracle but i don't understand if is free:
www.oracle.com
ps.thanks a lot davis

Since I'm a UNIX guy, I think that you'd have better luck in the www.gidforums.com.

Why not use sqlite and embed it into your app? Try just loading it from a disk file--you *may* find that the disk based file access is plenty fast enough for your needs.

Here is a quick example:

1st, create a database using the "DOS" utility

Code:

C:\src\sqlite>sqlite3 demo.db
SQLite version 3.5.6
Enter ".help" for instructions
sqlite> create table tbl_objects( id integer primary key autoincrement, name varchar(20) unique );
sqlite> insert into tbl_objects values( NULL, "Box" );
sqlite> insert into tbl_objects values( NULL, "Car" );
sqlite> insert into tbl_objects values( NULL, "Plane" );
sqlite> insert into tbl_objects values( NULL, "Shape" );
sqlite> insert into tbl_objects values( NULL, "Tool" );
sqlite> insert into tbl_objects values( NULL, "Model" );
sqlite> insert into tbl_objects values( NULL, "Vector" );
sqlite> insert into tbl_objects values( NULL, "Angle" );
sqlite> select * from tbl_objects order by name;
8|Angle
1|Box
2|Car
6|Model
3|Plane
4|Shape
5|Tool
7|Vector

2nd, write the functionality you need into your app (from the SQLite web site)

CPP / C++ / C Code:

#include <stdio.h>
#include <sqlite3.h>

static int callback(void *NotUsed, int argc, char **argv, char **azColName){
  int i;
  for(i=0; i<argc; i++){
    printf("%s = %s\n", azColName[i], argv[i] ? argv[i] : "NULL");
  }
  printf("\n");
  return 0;
}

int main(int argc, char **argv){
  sqlite3 *db;
  char *zErrMsg = 0;
  int rc;

  if( argc!=3 ){
    fprintf(stderr, "Usage: %s DATABASE SQL-STATEMENT\n", argv[0]);
    exit(1);
  }
  rc = sqlite3_open(argv[1], &db);
  if( rc ){
    fprintf(stderr, "Can't open database: %s\n", sqlite3_errmsg(db));
    sqlite3_close(db);
    exit(1);
  }
  rc = sqlite3_exec(db, argv[2], callback, 0, &zErrMsg);
  if( rc!=SQLITE_OK ){
    fprintf(stderr, "SQL error: %s\n", zErrMsg);
    sqlite3_free(zErrMsg);
  }
  sqlite3_close(db);
  return 0;
}

3rd, Execute the app

Output:

Code:

sqlitedemo demo.db "select * from tbl_objects order by name"
id = 8
name = Angle

id = 1
name = Box

id = 2
name = Car

id = 6
name = Model

id = 3
name = Plane

id = 4
name = Shape

id = 5
name = Tool

id = 7
name = Vector

:davis:

giugio · #9 12-Feb-2008, 02:39

Thanks Davis ,i try Sqllite and c++ :is really fantastic.
Fine.
I ask you the last item:
I have 3 collection of principal object used by my application:
1)point
2)frame
3)shell
This collections are rendered by my application with opengl(the opengl data is the collecton itself)and have a lot of properties for mathematics Calculations.
My problem is that i would to match queries(select insert and update) on the objects of the collectons and for some reasons i can't replace this object collection structures.
for this i have to:
1)insert a copy of the objects in the collection in the sqllite db whith the same ids of the object in the collections
2)match queries on the db and...
3)return all the ids of the matching records affected by the queries
4)apply the changes on the objects of the collections whith the same ids of the returned ids

There is a best solution for do this?
Thanks.

giugio · #10 21-Feb-2008, 08:09

nobody reply?

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Rate This Thread
Rate This Thread:

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Memory leak when nothing is happening... How can I even debug this ?	Algar	MS Visual C++ / MFC Forum	10	19-Nov-2007 07:17
Hard drive/CPU Diagnoses Issues	binarybug	Computer Hardware Forum	1	22-Jan-2007 19:23
Pipeline freeze simulation	darklightred	C++ Forum	6	27-Jul-2006 19:37
Pointer Usage in C++: Beginner to Advanced	varunhome	C++ Forum	0	19-Aug-2005 09:25
[Tutorial] Pointers in C (Part I)	Stack Overflow	C Programming Language	1	08-Apr-2005 18:35

[C++ MFC] in memory databases

Re: [c++ mfc]in memory databases

Re: [C++ MFC] in memory databases

Re: [C++ MFC] in memory databases

Re: [C++ MFC] in memory databases

Re: [C++ MFC] in memory databases

Re: [C++ MFC] in memory databases

Re: [C++ MFC] in memory databases

Re: [C++ MFC] in memory databases

Re: [C++ MFC] in memory databases