Reading and writing binary files in certain format

Dream86 · #1 02-Aug-2004, 10:16

Hi,
I am trying to write a program that reads a binary file (an image with a 512B header) and writes in into another binary file( an image file with a new header-1024B). The program is suppose to change the header of the input file. The parameters of the input file (old header format) is different from the parameters of the output file (new header format). Both formats have some parameters in common. I initialized the other parametes that are not in common between the two formats to a nonsense value( I chose -1 since it is not an accetable value for the parametes of the image header). then according to the order of the parameters in the new format, data is either being read from the old format or from the nonsense initialized values. Then the image data is beign read and written exactly as they are in the inputfile. The problem that I have now is that the output file is not stable. when i run the program on a file for different times, i get output files with different sizes, sometimes even the output file is generated but the size of it is 0. I hope the way I explained it make sense. Does anyone know what might be the problem?
Thank you

davekw7x · #2 02-Aug-2004, 15:46

What are you reading and writing: bytes, or what? Are you reading and writing integers with binary files, or what?

Show some code; give us some clues.

Dave

Dream86 · #3 02-Aug-2004, 20:47

Sorry for not being clear.
I am reading and writing raw data. The input file is a raw data file output of a microscope. The file has a 512 bytes header that includes the information about the image like size, data, color, and etc. The output file again is a binary file. The output file should have a header of 1024 bytes plus the image bytes. so the program is reading the bytes from the input files and write them in different order in the output file (i.e. the program should change the header of the input file). But since the output file is 512 B larger than the input file, 512 bytes are being read from a class that includes the extra variables of the output file initialized to -1. All the bytes are being read as longs.

CPP / C++ / C Code:

  ic_img_hdr imageIn;    // new format

   oldFormatParams imageOut;   //old format

   //initializing the vars to a nonsense value
   imageOut.numSection = -1;
   imageOut.zStart = -1;
   //continue like this until all the vars are initialized...

   ifstream inFile(argv[1], ios::in | ios::binary)

    inFile.seekg(16, ios::beg);
    inFile.read( (char*) &imageIn.totalx, sizeof(long) );  //1
    inFile.seekg(20, ios::beg);
    inFile.read( (char*) &imageIn.totaly, sizeof(long) );  //2
    inFile.seekg(24, ios::beg);
    inFile.read((char*) &imageIn.bpp, sizeof(long) );     //4
    inFile.seekg(8, ios::beg);
    inFile.read( (char*) &imageIn.startx, sizeof(long) );  //5
    //continue like this untill the first 512 bytes of the input file is read in order

    char* outFileName[100];
    strcpy( (char*)outFileName, argv[1] );
    strcat( (char*)outFileName, "Formated.bin");

    ofstream outFile( (char*)outFileName, ios::out | ios::binary);

    outFile.write( (char*) &imageIn.totalx, sizeof(long) );                    //1
    outFile.write( (char*) &imageIn.totaly, sizeof(long) );                    //2
    outFile.write( (char*) &imageOut.numSection, sizeof(long) );               //3
    outFile.write( (char*) &imageIn.xspc, sizeof(long) );                      //11
    outFile.write( (char*) &imageIn.yspc, sizeof(long) );                      //12
    outFile.write( (char*) &imageIn.zspc, sizeof(long) );                      //13
    outFile.write( (char*) &imageOut.alpha, sizeof(long) );                    //14
    outFile.write( (char*) &imageOut.beta, sizeof(long) );                     //15
    outFile.write( (char*) &imageOut.gamma, sizeof(long) );                    //16
    //continue like this until all bytes(those read from the inputfile and 
   // those vars initialized to -1  are being written

   //writing the image data

  unsigned long imageInSize;
                
   inFile.seekg(16, ios::beg);
  //inFile >> imageIn.totalx;
  inFile.read( (char*) &imageIn.totalx, sizeof(long) );

  //inFile.seekg(20, ios::beg);
  //inFile >> imageIn.totaly;
  inFile.read( (char*) &imageIn.totaly, sizeof(long) );

  imageInSize = imageIn.totalx * imageIn.totaly;

  inFile.seekg(24, ios::beg);
  inFile.read( (char*) &imageIn.bpp, sizeof(long) );

  if (imageIn.bpp == 8 )
  {
    char* buffer = new char[imageInSize*2];
    inFile.seekg(512, ios::beg);
    inFile.read(buffer, imageInSize*2);
    
    outFile.write( (char*) &buffer[0], imageInSize*2);
   
    delete[] buffer;
  }

  else if (imageIn.bpp == 16)
  {
    unsigned short* buffer = new unsigned short[imageInSize*2];
    inFile.seekg(512, ios::beg);
    inFile.read(buffer, imageInSize*2);
   
    outFile.write( (char*) &buffer[0], imageInSize*2);
    delete[] buffer;
  }

davekw7x · #4 02-Aug-2004, 23:02

I think I understand what you have in mind.

A couple of questions:

this isn't right:

CPP / C++ / C Code:

 else if (imageIn.bpp == 16)
{
unsigned short* buffer = new unsigned short[imageInSize*2];
inFile.seekg(512, ios::beg);
inFile.read(buffer, imageInSize*2);

outFile.write( (char*) &buffer[0], imageInSize*2);
delete[] buffer;
}

It shouldn't even compile since the first argument to inFile.read should be a pointer to a char, not a pointer to an unsigned short.

In either case (imageIn.bpp == 8, or imagiIn.bpp == 16) you are trying to read 2*imageIn.totalx*imageIn.totaly bytes. Is this correct? Are you sure that you have read the .totalx and .totaly quantities correctly? (Use cout<< to show the values you are actually reading). Are you actually reading and writing that many bytes?

Dave

Dream86 · #5 03-Aug-2004, 11:24

in the case (bpp == 8 ) , the number of bits per pixel of the image is 8 and in the other case, there are 16 bits per pixel. so when there is one byte per pixel, i am reading it to a char array which is one byte per cell. When there is 2 bytes per pixel, i am reading it to a short array which is 2 bytes per pixel. so each array cell represents a pixel data of the image. I had another program before this that only read and write the image data of a raw data file. This part of the code is from that program. I do not ger compiler error though. If i change the unsigned short* buffer to a char * buffer (in the case where bpp == 16), i do not get the correct image (i can open the output image using NIH program and look at the output image). I do not know if there is other way of doing it using char* in both cases?

thanks

davekw7x · #6 03-Aug-2004, 12:42

So, if imageInSize is the number of pixels, the following tries to read two times the number of pixels, since each pixel takes one byte:

CPP / C++ / C Code:

 if (imageIn.bpp == 8 )
{
char* buffer = new char[imageInSize*2];
inFile.seekg(512, ios::beg);
inFile.read(buffer, imageInSize*2);

outFile.write( (char*) &buffer[0], imageInSize*2);

delete[] buffer;
}

Is that what you meant to do?

Dave

davekw7x · #7 03-Aug-2004, 13:05

The following, from your original post, should not compile, assuming you are using the C++ standard <fstream> (what compiler did you use)?

CPP / C++ / C Code:

 else if (imageIn.bpp == 16)
{
unsigned short* buffer = new unsigned short[imageInSize*2];
inFile.seekg(512, ios::beg);
inFile.read(buffer, imageInSize*2);

outFile.write( (char*) &buffer[0], imageInSize*2);
delete[] buffer;
}

The inFile.read() must have a (char *) first argument. you could use

CPP / C++ / C Code:

inFile.read((char *)buffer, imageInSize*2);

or (using the style that you used in the outFile.write statement):

CPP / C++ / C Code:

inFile.read((char *)&buffer[0], imageInSize*2);

Note that, since there are imageInSize pixels, you can allocate an array of imageInSize unsigned shorts, since each element of the array holds one pixel. The .read and .write statements always count bytes, so 2*imageInSize is the correct number of bytes to read and write for this case.

(It does no harm to allocate more than you need, but is unnecessary and is misleading for future code maintainers.)

Dave

Dream86 · #8 05-Aug-2004, 20:12

I use the gnu g++ compiler. As you said, i casted the buffer array in read() (although i was not getting any compiler error for it) :

CPP / C++ / C Code:


inFile.read((char*)buffer, imageInSize*2);

Also, I fixed the problem that you mentioned for the case when bpp == 8 (the image size).

Thank you very much, but I am still having the same problem. The output file is not stable. But in most cases i get an output file with 0 size or size that is much less than the reasonable one.

The strange thing that happens is that the size of the output file changes when I run the program on another file!! I do not know how this can be possible. For example, say I run the program on FileOne, say the output file is 2891776. Then i run the program on FileTwo. When I do the ls -lt command or ls -s , the size of the FileOneOutPut has changed to 0 and and the size of the FileTwoOutput is less than what it is suppose to be. may be is it something with Linux that i am missing here??

Thank you

davekw7x · #9 05-Aug-2004, 21:16

Without seeing your entire program, I can only suggest that you make sure you are reading all parameters correctly. Specifically use cout<< to see what values you are using for imagein.totalx and imagein.totaly, since these affect the size of the output file. Use cout<< to see what the total number of bytes is supposed to be. Etc., etc.

Running the program on a second input file obviously shouldn't affect the contents of the previous output. It's not a Linux thing, it's a program thing. (If you had bad hardware or a corrupted operating system or utility program, I would expect more drastic misbehaviour than just this program behaving badly.)

General debugging hints:

Sprinkle cout<< throughout the program to make sure it's doing what you had in mind. Are there ways that the program can determine exactly how many bytes are read and written? Stuff like that.

Regards,

Dave

Dream86 · #10 06-Aug-2004, 10:35

I used cout in different palces to check the read and write. Everything seems to be fine...I do not know how to check the total number of bytes being read and written by the program?

This is suppose to be an easy read and write operations. I think the part that reads and writes the image data is fine because i use that part separately to read and write only the image data and i get the correct output. Also i use that part in another program that combines the images of different files in a directory without the header part , and that seems to be working as well.

I have spent so much time on this and i am really fraustrated since i do not know what is going wrong. Here is my entire code. I really appreciate if you please take a look at it and let me know what you think.

CPP / C++ / C Code:


#include <fstream.h> 
#include <stdlib.h>  
#include <string.h>   
#include "hdr_s.h"
#include "Format.h" 



int main(int argc, char* argv[])
{

   if (argc !=2)
   {
     cerr << endl << "*Enter the following in the command prompt:" << endl;
     cerr << "        Format.exe  NameOfRawDataFile" << endl << endl;
     exit(1);
   }

   ic_img_hdr imageIn;    // new format

   oldFormatParams imageOut;   //old format

   //initializing the vars to a nonsense value
   imageOut.numSection = -1;
   imageOut.zStart = -1;
   imageOut.xInterval = -1;
   imageOut.yInterval = -1;
   imageOut.zInterval = -1;
   imageOut.alpha = -1;
   imageOut.beta = -1;
   imageOut.gamma = -1;
   imageOut.mapC = -1;
   imageOut.mapR = -1;
   imageOut.mapS = -1;
   imageOut.minDensity = -1;
   imageOut.maxDensity = -1;
   imageOut.meanDensity = -1;
   imageOut.ISPG = -1;
   imageOut.symOperatorBytes = -1;
   imageOut.extraSpaceOne = -1;
   imageOut.extraSpaceTwo = -1;
   imageOut.extraSpaceThree = -1;
   imageOut.extraSpaceFour = -1;
   imageOut.extraSpaceFive = -1;
   imageOut.extraSpaceSix = -1;
   imageOut.extraSpaceSeven = -1;
   imageOut.extraSpaceEight = -1;
   imageOut.zResFactor = -1;
   imageOut.extHdrSiz = -1;
   imageOut.lambdaOffsetOne = -1;
   imageOut.lambdaOffsetTwo = -1;
   imageOut.lambdaOffsetThree = -1;
   imageOut.lambdaOffsetFour = -1;
   imageOut.lambdaOffsetFive = -1;
   imageOut.lambdaOffsetSix = -1;
   imageOut.datTypeFlag = -1;
   imageOut.datValOne = -1;
   imageOut.datValTwo= -1;
   imageOut.origTiltOne = -1;
   imageOut.origTiltTwo = -1;
   imageOut.origTiltThree = -1;
   imageOut.curTiltOne = -1;
   imageOut.curTiltTwo = -1;
   imageOut.curTiltThree = -1;
   imageOut.lambdaInfoOne = -1;
   imageOut.lambdaInfoTwo = -1;
   imageOut.lambdaInfoThree = -1;
   imageOut.zOrigin = -1;
   imageOut.xOrigin = -1;
   imageOut.yOrigin = -1;
   imageOut.numOfUsedLabel = -1;

 
   ifstream inFile(argv[1], ios::in | ios::binary);

   if (!inFile)  
   {
     cerr << endl << "File could not be opened." << endl;
     cerr << "Make sure the name of the file is correct and try again." << endl;
     cerr << endl;   
     exit(1);           }

    inFile.seekg(16, ios::beg);
    inFile.read( (char*) &imageIn.totalx, sizeof(long) );  //1
    cout << "totalx read: " << imageIn.totalx << endl;
    inFile.seekg(20, ios::beg);
    inFile.read( (char*) &imageIn.totaly, sizeof(long) );  //2
    cout << "totaly read: " << imageIn.totaly << endl;
    inFile.seekg(24, ios::beg);
    inFile.read((char*) &imageIn.bpp, sizeof(long) );     //4
    inFile.seekg(8, ios::beg);
    inFile.read( (char*) &imageIn.startx, sizeof(long) );  //5
    inFile.seekg(12, ios::beg);
    inFile.read( (char*) &imageIn.starty, sizeof(long) );  //6
    inFile.seekg(500, ios::beg);
    inFile.read( (char*) &imageIn.xspc, sizeof(float) );  //11
    inFile.read( (char*) &imageIn.yspc, sizeof(float) );  //12
    inFile.read( (char*) &imageIn.zspc, sizeof(float) );  //13
    inFile.seekg(332, ios::beg);
    inFile.read( (char*) &imageIn.label, sizeof(char)*144 );  //56-256
    cout << "label: " << imageIn.label << endl;
    inFile.seekg(4, ios::beg);
    
 
// write the header into a bin file(raw data) in the correct(old) format

    char* outFileName[100];
    strcpy( (char*)outFileName, argv[1] );
    strcat( (char*)outFileName, "Formated.bin");

    ofstream outFile( (char*)outFileName, ios::out | ios::binary);
    if (!outFile)
    {
      cerr << endl << "File could not be opened." << endl;
      cerr << "Make sure the name of the file is correct and try again." << endl;
      exit(1);
    }

    outFile.write( (char*) &imageIn.totalx, sizeof(long) );                    //1
    cout << "totalx written: " <<imageIn.totalx << endl;
    outFile.write( (char*) &imageIn.totaly, sizeof(long) );                    //2
    cout << "totaly written: " << imageIn.totaly << endl;
    outFile.write( (char*) &imageOut.numSection, sizeof(long) );               //3
    cout << "numSection: " << imageOut.numSection << endl;
    if (imageIn.bpp == 8)                                                      //4
    {
      imageOut.mode = 1;
    }
    else if(imageIn.bpp == 16)
    {
      imageOut.mode = 2;
    }
    outFile.write( (char*) &imageOut.mode, sizeof(long) );                     //4
    cout << "mode: " << imageOut.mode << endl;
    outFile.write( (char*) &imageIn.startx, sizeof(long) );                    //5
    outFile.write( (char*) &imageIn.starty, sizeof(long) );                    //6     
    outFile.write( (char*) &imageOut.zStart, sizeof(long) );                   //7  
    outFile.write( (char*) &imageOut.xInterval, sizeof(long) );                //8
    outFile.write( (char*) &imageOut.yInterval, sizeof(long) );                //9
    outFile.write( (char*) &imageOut.zInterval, sizeof(long) );                //10
    outFile.write( (char*) &imageIn.xspc, sizeof(long) );                      //11
    outFile.write( (char*) &imageIn.yspc, sizeof(long) );                      //12
    outFile.write( (char*) &imageIn.zspc, sizeof(long) );                      //13
    cout << "zspc: " << imageIn.zspc << endl;
    outFile.write( (char*) &imageOut.alpha, sizeof(long) );                    //14
    outFile.write( (char*) &imageOut.beta, sizeof(long) );                     //15
    outFile.write( (char*) &imageOut.gamma, sizeof(long) );                    //16
    outFile.write( (char*) &imageOut.mapC, sizeof(long) );                     //17
    outFile.write( (char*) &imageOut.mapR, sizeof(long) );                     //18
    outFile.write( (char*) &imageOut.mapS, sizeof(long) );                     //19
    outFile.write( (char*) &imageOut.minDensity, sizeof(long) );               //20
    outFile.write( (char*) &imageOut.maxDensity, sizeo(long) );               //21
    cout << "maxDensity: " << imageOut.maxDensity << endl;
    outFile.write( (char*) &imageOut.meanDensity, sizeof(long) );           //22
    outFile.write( (char*) &imageOut.ISPG, sizeof(long) );                     //23
    outFile.write( (char*) &imageOut.symOperatorBytes, sizeof(long) );     //24
    outFile.write( (char*) &imageOut.extraSpaceOne, sizeof(long) );        //25
    outFile.write( (char*) &imageOut.extraSpaceTwo, sizeof(long) );        //26
    outFile.write( (char*) &imageOut.extraSpaceThree, sizeof(long) );          //27
    outFile.write( (char*) &imageOut.extraSpaceFour, sizeof(long) );           //28
    outFile.write( (char*) &imageOut.extraSpaceFive, sizeof(long) );           //29
    outFile.write( (char*) &imageOut.extraSpaceSix, sizeof(long) );            //30
    cout << "extraSpaceSix: " << imageOut.extraSpaceSix << endl;
    outFile.write( (char*) &imageOut.extraSpaceSeven, sizeof(long) );          //31
    outFile.write( (char*) &imageOut.extraSpaceEight, sizeof(long) );          //32
    outFile.write( (char*) &imageOut.zResFactor, sizeof(long) );               //33
    outFile.write( (char*) &imageOut.extHdrSiz, sizeof(long) );                //34
    cout << "extHdrSiz: " << imageOut.extHdrSiz <<  endl;
    outFile.write( (char*) &imageOut.lambdaOffsetOne, sizeof(long) );          //35
    outFile.write( (char*) &imageOut.lambdaOffsetTwo, sizeof(long) );          //36
    outFile.write( (char*) &imageOut.lambdaOffsetThree, sizeof(long) );        //37
    outFile.write( (char*) &imageOut.lambdaOffsetFour, sizeof(long) );         //38
    outFile.write( (char*) &imageOut.lambdaOffsetFive, sizeof(long) );         //39
    outFile.write( (char*) &imageOut.lambdaOffsetSix, sizeof(long) );          //40
    outFile.write( (char*) &imageOut.datTypeFlag, sizeof(long) );              //41
    outFile.write( (char*) &imageOut.datValOne, sizeof(long) );                //42
    outFile.write( (char*) &imageOut.datValTwo, sizeof(long) );                //43
    outFile.write( (char*) &imageOut.origTiltOne, sizeof(long) );              //44
    outFile.write( (char*) &imageOut.origTiltTwo, sizeof(long) );              //45
    outFile.write( (char*) &imageOut.origTiltThree, sizeof(long) );            //46
    outFile.write( (char*) &imageOut.curTiltOne, sizeof(long) );               //47
    cout << "curTiltOne: " << imageOut.curTiltOne << endl;
    outFile.write( (char*) &imageOut.curTiltTwo, sizeof(long) );               //48
    outFile.write( (char*) &imageOut.curTiltThree, sizeof(long) );             //49
    outFile.write( (char*) &imageOut.lambdaInfoOne, sizeof(long) );            //50
    outFile.write( (char*) &imageOut.lambdaInfoTwo, sizeof(long) );            //51
    outFile.write( (char*) &imageOut.lambdaInfoThree, sizeof(long) );          //52
    outFile.write( (char*) &imageOut.zOrigin, sizeof(long) );                  //53
    outFile.write( (char*) &imageOut.xOrigin, sizeof(long) );                  //54
    outFile.write( (char*) &imageOut.yOrigin, sizeof(long) );                  //55
    outFile.write( (char*) &imageOut.numOfUsedLabel, sizeof(long) );           //56
    cout << "numOfUsedLabel: " << imageOut.numOfUsedLabel << endl;
     outFile.write( (char*) &imageIn.label, sizeof(char)*144 );             //57-256
    
    cout << "label written: " << imageIn.label << endl;


//writing the image data

  unsigned long imageInSize;
                
  imageInSize = imageIn.totalx * imageIn.totaly;

  inFile.seekg(24, ios::beg);
  inFile.read( (char*) &imageIn.bpp, sizeof(long) );
  cout << "bpp read: " << imageIn.bpp << endl;

 
  if (imageIn.bpp == 8)
  {
    char* buffer = new char[imageInSize];
    inFile.seekg(512, ios::beg);
    inFile.read((char*)buffer, imageInSize);
    
    outFile.write( (char*) &buffer[0], imageInSize);

    delete[] buffer;
  }

  else if (imageIn.bpp == 16)
  {
    unsigned short* buffer = new unsigned short[imageInSize*2];
    inFile.seekg(512, ios::beg);
    inFile.read((char*)buffer, imageInSize*2);
   
    outFile.write( (char*) &buffer[0], imageInSize*2);
    cout << "0: " <<  buffer[0] << endl;
    cout << "1: " << buffer[1] << endl;
    cout << "20: "  << buffer[20] << endl;
    cout << "100: " << buffer[100] << endl;
    cout << "134: " << buffer[134] <<endl;
    cout << "200: " << buffer[200] <<endl;
    cout << "340: " << buffer[340] << endl;
    cout << "500: " << buffer[500] << endl;
    cout << "900: " << buffer[900] << endl;
    cout << "2000: " << buffer[2000] << endl;
    cout << "1000000: " << buffer[1000000] << endl;
    cout << "1447679: " << buffer[1447679] << endl;
    cout << "1447680: " << buffer[1447680] << endl;
  
    delete[] buffer;
  }

    return 0;
}

Thank you very much

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Rate This Thread
Rate This Thread:

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
reading and writing to a file in C++	mgp6q	C++ Forum	17	02-Mar-2004 13:42
[Tutorial] Standard I/O	aaroncohn	C Programming Language	20	27-Feb-2004 22:07