Need help for write data into multiple files

sitha · #1 18-Nov-2006, 02:55

Hi Everybody,
I want to write data into multiple files say for example, I should write "HAI" into files like.......
1agent.dat, 2agent.dat, 3agent.dat and so on...

please look at my following coding,

Main part of the program as,

CPP / C++ / C Code:

#define NMB_MCS 10

using namespace std;

int main( )
{
  fstream out("agent.dat",ios::out);
  string  file;
  char    filename[1000];
  void   RenameFile(int);
  string int2string(int);

    for(int i=0 ; i<(int) NMB_MCS ; i++)  {
          
       RenameFile( i );
      

    [color="Red"]file = int2string( i ) + "\\agent.dat";[/color]
   int j;
   [color="Red"]for( j = 0; file[j] != '\0'; j++ ) filename[j] = file[j];
   filename[j]='\0';[/color]
  	if( !out  ) cerr << "file not created at i = " << i << "\n";
    else        out  << "HAI" << "\n";
	
  }
  out.close();
}

Here I included two funtions. there are,
first is change intger to string

CPP / C++ / C Code:

string int2string(int i)
{
#if   (NMB_MCS ==10)
int	   dec    = 10;
int    nmx    = 2;
string result = "00";
#elif (NMB_MCS == 1000)
int	   dec    = 1000;
int    nmx    = 4;
string result = "0000";
#elif (NMB_MCS == 10000)
int    dec    = 10000;
int    nmx    = 5;
string result = "00000";
#endif
int    id;

  for( int n = 0; n < nmx; n++ )
  {
    id = i/dec;
	result[n] += id;
	i -= dec*id;
	dec /= 10;
  }return result;
}

second is for rename the file,

CPP / C++ / C Code:

void      RenameFile( int i )
{
//  string mv;
  char   mv_command[1000];
  string int2string(int);

  string mv = "rename agent agent" + int2string(i);
  int j;
  for(j=0;mv[j]!='\0';j++) mv_command[j]=mv[j];
  mv_command[j]='\0';
  system(mv_command);
}

I hope in my two functions there is no problem but problem in main program specially marked by red lines. I hope you could understand my reqirement or purpose.
I am confused very much how to handle this problem.
Pls help me according my purpose, how should I write my main program.

Thanks lot in advance.

Regards,
Sitha.

amad1337 · #2 18-Nov-2006, 05:03

The only thing i could find is in the intostring function, wheres the statement

result[n] += id;

is not valid, restult varibale is string type and you trying to asign a int to it.

Try this link

http://www.experts-exchange.com/Prog..._20670737.html

To find some info on how to convert int to string.

davekw7x · #3 18-Nov-2006, 08:03

Quote:

Originally Posted by sitha

Hi Everybody,
I want to write data into multiple files say for example, I should write "HAI" into files like.......
1agent.dat, 2agent.dat, 3agent.dat and so on...

Here's what your program seems to be trying to do (Here I disregard issues of programming practices with std::string objects; I am just trying to follow your logic into the actual program):

1. Opens a file named "agent.dat" for writing (So it's a brand new, empty file. If there was previously a file named "agent.dat", it's destroyed.)

2. Calls a function named RenameFile that executes the following system call:

"rename agent agent00"

Note that if there is no file named "agent", the system call does nothing except create an error message.

3. Creates a string whose value is "00agent.dat"

4. Repeats steps 2 and 3 except instead of "00", it uses "01", "02", "03", ..., 10.

5. Closes the (still empty) file "agent.dat".

Note that after the first time through the loop, if there had been a file named "agent", it has been renamed to "agent00", so all subsequent calls to RenameFile() result in a system call that is guaranteed to fail.

Now, I respectfully suggest that you write down a sequence of steps that you really mean to execute (in English or other non-programming language) and then see what you need to do in your program to implement the requirements.

Then try to write the code.

Regards,

Dave

sitha · #4 18-Nov-2006, 08:38

Dear Dave,
Yeah I understood my logic is too bad for creating a multible files.

Is there anyway to create multible files?

For every loop a function should create a files.
eg,

Code:

      for( int i; i<100; i++) {

         ofstrem output( i  +agent.dat, ios::out)
      
    }

for corresponding i value if my program create files my problem will be over.

Pls help me for this regard.

Thanks a lot.

Regards,
Sitha.

davekw7x · #5 18-Nov-2006, 09:25

Quote:

Originally Posted by sitha

Dear Dave,
Yeah I understood my logic is too bad for creating a multible files.

Is there anyway to create multible files?

If you are going to have more than one file open at a time, you need to have as many ofstream objects as there are files.

One way is to make an array of ofstreams:

CPP / C++ / C Code:

    ofstream outfile[NMB_MCS];
    string basename("agent.dat");

You don't need to rename anything (so lose the RenameFile() function --- it's pretty ugly anyhow). Just create a new name for each ofstream and open the file:

CPP / C++ / C Code:

    int i, number_files;
    for (i = 0; i < NMB_MCS; i++) {
        outname = int2string(i) + basename;
        outfile[i].open(outname.c_str());
        if (!outfile[i]) { // bail out if there was a problem
            cout << "Can't open file " << outname << " for writing." << endl;
            return 0; // or you may want to implement some recovery strategy
        }
    }

Then use each one as you need to. For example:

CPP / C++ / C Code:

    int j;
.
.
.
    for (j = 0; j < NMB_MCS; j++) {
        outfile[j] << "This is file number " << j << endl;
        outfile[j] << "End of text" << endl;
    }

And, to close them all:

CPP / C++ / C Code:

    for (j = 0; j < NMB_MCS; j++) {
        outfile[j].close();
    }

Note that all operating systems have some limit on the number of files that are allowed to be open at any one time, so this may not work of you need 10000 files. (There may or may be an implementation limit for the compiler also. It's not always easy to find out what the limits are.)

If you only need one file open at a time (like if you are going to write some particular thing to each file), then a single ofstream will serve. Just make a loop that repeatedly does the following:

1. Generate a new file name, using a base name and the current loop counter value.
2. Open the ofstream with that file name.
3. Write the information, or do whatever you need to do with that file.
4. Close the ofstream.

Regards,

Dave

davekw7x · #6 18-Nov-2006, 10:08

Quote:

Originally Posted by amad1337

The only thing i could find is in the intostring function, wheres the statement

result[n] += id;

is not valid, restult varibale is string type and you trying to asign a int to it.

The string was initialized to a number of zero chars, so result[n] is the char '0'. Adding an int whose value is greater than or equal to zero and less than or equal to nine gives the corresponding char. That is, an integer value of zero gives '0', an integer value of one gives '1', etc. (Sneaky, huh?)

I might have put a some extra protection against using the routine with an invalid value for NMB_MCS, but the routine appears to work OK in this program.

Actually I (probably) would have done it some other way entirely, but the point is that this routine is not creating problems in the given program.

Furthermore it's a learning experience to look at things from a slightly different point of view. That's the main reason I keep coming back: I always learn something here (always).

Regards,

Dave

sitha · #7 19-Nov-2006, 02:09

Hi,
Thank you so much for your kind help. Now it is working fine.
Sure I want to deal with around 10000 of files and each file size may be around 200kb, not only for writing but also for reading I want to deal with 10000 of files.

Pls look at my following code for reading of files and write into a temporary file so called driftdata.

CPP / C++ / C Code:

int readFiles(char * dirname)
{
   DIR* dirp;
   string buff;
   struct dirent* dp;
   dirp = opendir(dirname);
   
   if ( !dirp )   {
      cout << "Error: failure opening directory" << endl;
      exit(1);
    } 
      
   errno = 0;
 
   ofstream output("driftdata");
   
   while ( dp = readdir(dirp) )  {            // read directory
	   if(dp !=NULL) {		   
         if(strstr(dp->d_name, ".dat"))  {    // to check dat files
	                 
             ifstream input(dp->d_name);      // to read files
                
                [color="Red"]while( getline(input, buff) ) {  // read data in files
                   for( int i=0;  i<4; i++) { 
	                getline(input, buff);
	                } 
	                istringstream tmp(buff);	      
	      
                     cout << buff << endl;
                     output << buff << endl;
                  
            }  [/color]
         }
     }
      
      if ( errno ){
         cout << "Error: readdir() failure!" << endl;
         exit(1);
       }    
   }
   output.close();
   closedir( dirp );

}

Here pls tell me how to improve my coding as you suggested that at a time open a file and read data and close it. then open second file and so on.

Also I want to read say first line, fifth line, 10th line like that I should skip 4 lines for reading. I did this in the red color coding. but it didnt work well and it produced blank lines as well in the output file.

Pls help me for these both case.

davekw7x · #8 19-Nov-2006, 09:03

Quote:

Originally Posted by sitha

code for reading of files and write into a temporary file so called driftdata.
.
.
.
you suggested that at a time open a file and read data and close it. then open second file and so on.

So, you need to close the stream inside the loop before opening it for the next file.

Quote:

Originally Posted by sitha

Also I want to read say first line, fifth line, 10th line like that I should skip 4 lines for reading.

Then use a variable named lineno (or some such thing). Initialize it to zero each time you open a new file and increment it each time you read a line. Make some logic inside the loop that tells the program which lines to write.

For example if you want to write line 1, 5, 9, 13, ... (Do the first one then skip 3 and write then skip 3 and write,...) the logic is ((lineno % 4) == 1). If the logic is more complicated, then do whatever you need:

CPP / C++ / C Code:

    while ((dp = readdir(dirp)) != NULL) { // Get next file name from directory
        if (strstr(dp->d_name, ".dat")) {  // see if file name is "*.dat"
            ifstream input(dp->d_name);    // Yes: Open the file
            int lineno = 0;
            while (getline(input, buff)) {
                if ((++lineno % 4) == 1) { // write line number 1, 5, 9, etc.
                    cout << "Writing line " << lineno << " from "
                        << dp->d_name << endl;
                    output << buff << endl;
                }
            }
            output.close();
        }
    }

Note the cout << statement that I put in for debugging purposes. Start by testing a directory with a small number of small files. Make absolutely sure that the program handles this case exactly the way you have in mind. (Look in the output file to make sure that the lines have been copied correctly.) Then you can omit or change the debugging printout to suit your needs.

For example with a directory with two data files

Code:

Writing line 1 from y.dat
Writing line 5 from y.dat
Writing line 9 from y.dat
Writing line 1 from x.dat
Writing line 5 from x.dat
Writing line 9 from x.dat
Writing line 13 from x.dat
Writing line 17 from x.dat
Writing line 21 from x.dat
Writing line 25 from x.dat
Writing line 29 from x.dat
Writing line 33 from x.dat
Writing line 37 from x.dat
Writing line 41 from x.dat
Writing line 45 from x.dat
Writing line 49 from x.dat
Writing line 53 from x.dat
Writing line 57 from x.dat

The first file had 11 lines; the second had 58.

Regards,

Dave

sitha · #9 19-Nov-2006, 20:05

Hi Dave,
Thank you so much for your helping. What coding you have writen here is really I expected to do.
Actually I am a Civil Engineer working on numerical analysis and GIS stuff. But my porgram knowledge is very limited So Pls suggest me how can I improve my skill in programing?
And pls tell me how can I use Object Oriented technique for this type of programing.

Regards,
Sitha.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Rate This Thread
Rate This Thread:

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[Include] Doubly-linked List	dsmith	C Programming Language	6	14-Apr-2006 13:12
Strange C++ code memory leakage problem	gaoanyu	C++ Forum	7	04-Nov-2005 08:09
Need help reading data from files	blitzy	C Programming Language	5	15-Mar-2005 19:07
[CONTEST?]Data Structure Test	dsmith	C Programming Language	2	06-Jun-2004 15:13
using vector or array	oshiotse	C++ Forum	4	16-Apr-2004 10:59

Need help for write data into multiple files

Re: Need help for write data into multiple files

Re: Need help for write data into multiple files

Re: Need help for write data into multiple files

Re: Need help for write data into multiple files

Re: Need help for write data into multiple files

Re: Need help for write data into multiple files

Re: Need help for write data into multiple files

Re: Need help for write data into multiple files