Memory Block Size

wayne--scales · #1 17-Apr-2007, 19:46

I am writing a program trying to return the size of a memory block in C-Free on windows. I have been using the malloc_size() method and including the stdlib.h header file. When I try to run my program in minix-3 (which I have to do) malloc_size is undefined.

Isn't malloc_size in the standard minix library? If not does anyone know any way around this? Is there another way to get the size of a memory block?

Thanks.

Howard_L · #2 23-Apr-2007, 13:35

Boy, malloc() must be topic of the week!
There are 2 or 3 threads showing which have given me a much better understanding of malloc that you might want to give a read.
Especially '2D arrays:dynamic allocation and freeing'.

I googled malloc_size and a man page comes up from Apple Developement (which interestingly is a BSD man(3) page) which is for malloc() and has malloc_size as well . (Also a lot of other people seem to be asking the same q as you!)

The definition there is:

Quote:

size_t malloc_size(void *ptr);
The malloc_size() function returns the size of the memory block that
backs the allocation pointed to by ptr. The memory block size is always
at least as large as the allocation it backs, and may be larger.

So if it is not available in your minix environment, I would think you could get this value from an object after it has been allocated memory by malloc().

I am new at this but I've attempted an example:

CPP / C++ / C Code:

/* malloc_size.c
   not sure if I'm doing this right, 
   but it has the output I expected until I mallocated chars! 
   malloc() allocates  machine word size (int) as minimum ??) 
*/

#include <stdio.h>
#include <stdlib.h>

int main()
{
  int x1, x2;
  //int *array[];

  printf("How many  ints do you want to store in array1? ");
  scanf("%d", &x1);

  printf("How many chars do you want to store in array2? ");
  scanf("%d", &x2);

  int *array1[x1];
  *array1 = (int *)(malloc(x1 * sizeof(int)));

  /* check to ensure there was memory for malloc() to allocate */
  if(array1 == NULL)
  {
    printf("malloc could not allocate %d bytes of memory for array1 \nNotify Homeland Security Immediately \n ",  (x1 * sizeof(int)) );
    return 1;
  }
  printf(" x1 * sizeof(int): %2d \n", (x1 * sizeof(int)) );
  printf("   sizeof(array1): %2d \n", sizeof(array1));

  /* Always free mallocated memory. (I think I'm doing it correctly) */
  free(*array1);

  /* I'm trying to see what happens after using free()  */
  printf("\narray1 has been free'd (I think) \n");
  printf(" x1 * sizeof(int): %2d \n", (x1 * sizeof(int)) );
  printf("   sizeof(array1): %2d \n", sizeof(array1));

  /* Now mallocate a char array */
  char *array2[x2];
  *array2 = (char *)(malloc(x2 * sizeof(char)));

  /* check to ensure there was memory for malloc() to allocate */
  if(array2 == NULL)
  {
    printf("malloc could not allocate %d bytes of memory for array \n ",  (x2 * sizeof(char)) );
    return 1;
  }
  printf("x2 * sizeof(char): %2d \n", (x2 * sizeof(char)) );
  printf("   sizeof(array2): %2d  what????\n", sizeof(array2));

  /* Now see if array is still there after free() and another malloc() */
  printf(" x1 * sizeof(int): %2d \n", (x1 * sizeof(int)) );
  printf("   sizeof(array1): %2d \n", sizeof(array1));

  /* Now to free mallocated memory of array2. */
  free(*array2);

  return 0;
}
/*  OUTPUT:
How many  ints do you want to store in the array? 2
How many chars do you want to store in the array? 2
 x1 * sizeof(int):  8
   sizeof(array1):  8

array1 has been free'd (I think)
 x1 * sizeof(int):  8
   sizeof(array1):  8
x2 * sizeof(char):  2
   sizeof(array2):  8    <-what ...2 of size int ????
 x1 * sizeof(int):  8
   sizeof(array1):  8
*/

Well, I don't know if that's any help.
I seem to be asking twice as many questions as I might hope to answer...

Now, I read that in my nifty new old K&R (p186) that:

Quote:

In malloc the requested size in characters is rounded up to the proper number of header-sized units; the block that will be allocated contains one more unit, for the header itself, and this is the value recorded in the size field of the header. The pointer returned by malloc points at the free space, not at the header itself

.....and goes on to talk about how this memory need not be contiguous, oh my, so I was going to say that you might be able to add 1 "unit" to come up with actual memory used, but if it happens to be non- contiguous, each piece may have this additional unit.... which might be what malloc_size might return. And on another look at the definition above the words "size that backs the allocation" take on more meaning to me now... So after all this I have to say I don't know? Dave???!

At any rate, you might try comparing the size of a malloc 'd object with what you can get using the malloc_size function you have available on your Windows machine. See what you get... Sorry.
++Howard;

davekw7x · #3 23-Apr-2007, 14:42

Quote:

Originally Posted by Howard_L

ys:dynamic allocation and freeing'.

I googled malloc_size and a man page comes up from Apple Developement (which interestingly is a BSD man(3) page) which is for malloc() and has malloc_size as well .
.
.
.So after all this I have to say I don't know? Dave???!

But you had a lot of fun and maybe learned a thing or two by looking, right? (That's the reason I keep coming here: I always learn something ---always.)

Here's the thing: malloc() is a standard library function. That is to say: the C Language Standard defines its existence and its required behavior.

Here is the entire description of malloc() from the C99 standard:

Quote:

Originally Posted by ISO/IEC 9899:1999

Synopsis

CPP / C++ / C Code:

#include <stdlib.h>
void *malloc(size_t size);

Description

The malloc function allocates space for an object whose size is specified by size and whose value is indeterminate.
Returns
The malloc function returns either a null pointer or a pointer to the allocated space.

That's it! Little details like a variable named malloc_size and big details like how malloc() is supposed to do its thing are left to implementations (that is: to the compiler and library writers).

In the appendix on "unspecified behavior", the Standard specifically mentions portability issues concerning:

Quote:

— The order and contiguity of storage allocated by successive calls to the calloc,
malloc, and realloc functions

In other words, successive calls may or may not give contiguous blocks of memory, and later calls may have lower addresses than earlier (or vice-versa --- the behavior is explicitly unspecified). Compiler and library implementers are free to do it any way that they bloomin' well please. (And they can change implementation details any time they want to.)

Bottom line: anything that you discover (or guess) by experimenting with a particular compiler won't necessarily be valid with other compillers. Not even with other versions of the same compiler unless it is specifically documented for that particular version of that particular compiler.

If it is open-source, than you may be able to look at the source code (lots of luck on that!!!) to find answers. Sometimes there is specific documentation (other than source code) about compiler internals, and, maybe even library internals.

Good hunting!

Regards,

Dave

Footnote: One other little detail: Not all compilers try equally hard to comply with any of the Standards. So, even if a Standard gives some requirement, any particular compiler may not do it that way (and may or may not bother to document any variances). However, if the Standard specifically mentions that certain behavior is "unspecified" or "undefined" or otherwise "unguaranteeed", then all compiler writers are off the hook. They can do just about anything (within the Standard's definition of "unspecified" or "undefined" behavior or results) and still claim compliance.

Howard_L · #4 23-Apr-2007, 19:25

Quote:

But you had a lot of fun and maybe learned a thing or two by looking, right?

You betcha! Learned more than a couple, and more to come. I WOULD like to be able to come up with answers instead of questions though.... This is a great place you guys have here. Lots to see and do at a level I can understand.

Quote:

They can do just about anything (within the Standard's definition of "unspecified" or "undefined" behavior or results) and still claim compliance.

Well then, I guess it's understandable that malloc() allocated int size spaces for my char array? Hey, plenty of room to move. hmmm I guess knowing that, and if that happens on any machine that might run my .exe, I could take that into consideration when I call malloc. Heck, if I allocated for 30 - 50 length strings 5000 times, that space would add up. Might try (strlen*6)/10*sizeof(char) or something. Ahh, and Maybe before doing that I could find out what malloc_size is for a char and adjust accordingly!

It would be interesting to hear what the malloc_size was supposed to be used for in the original post and what his DOES return compared to what the size(mallocated object) is.

And yes, I'm havin fun, thanks,
++Howard;

Howard_L · #5 24-Apr-2007, 13:47

Going over the program I posted earlier in this thread I found that I hadn't really created what I thought I did. The idea was to allocate an array of ints or chars and look at the sizeof(array). I think I created an array of pointers intead! So my printout makes sense in that respect since a pointer is stored in size int:

CPP / C++ / C Code:

How many  ints do you want to store in array1? 10
How many chars do you want to store in array2? 10
 x1 * sizeof(int): 40
   sizeof(array1): 40
  sizeof(*array1):  4
x2 * sizeof(char): 10
   sizeof(array2): 40  what????
  sizeof(*array2):  4

Ha! yes indeed, for when I tried to fill it with ints:
for(x=0; x < (x1); x++)
  { array1[x] = x; }

I got this:  Error: need explicit cast to convert  from: int  to  : int *
A cast:  (void*)x ...took care of that 
I note that I can assign one or two more places than are allocated,
any more and program fails 'illegal operation'  
...that's good, some safety is in place for when I mess up.

So I went back and saw that I should declare and allocate like this:

CPP / C++ / C Code:

/* Method 1  mallocate an int array */
/*  int *array1[x1];
  *array1 = (int *)(malloc(x1 * sizeof(int)));     */

/* Method 2  mallocate an int array */
  int *array1;
  array1 = (int *)(malloc(x1 * sizeof(int)));

Which after doing I found that I could not look at the 'object', , just at what the pointer was pointing at. In this case array1[0] and array2[0]:

CPP / C++ / C Code:

How many  ints do you want to store in array1? 10
How many chars do you want to store in array2? 10
 x1 * sizeof(int): 40
   sizeof(array1):  4
  sizeof(*array1):  4
x2 * sizeof(char): 10
   sizeof(array2):  4  
  sizeof(*array2):  1

So now I'm wondering how I can get a look at the entire object size?
I'm thinking I need to declare it differently. Perhaps (*)array1 ? ...no success thusfar...
Or, I guess just have to do the math since I don't have a malloc_size handy...

On another matter with the program I previously posted:
On my old k6-2 laptop in 98 with a bunch of stuff open
using method2, malloc would max out at about 225000000 - 225500000 chars , , and I noticed that when I increase ints, I have to reduce chars accordingly. My ints don't seem to get freed.
So, I don't think my: free(array1); ...is working correctly.

Along with that, using method 1, interestingly, I max out at 259800 - 259900 (259826+-) but it's not my malloc error message, instead it's an alert window. I was thinking I was hitting a limit on the number of pointers available.
The number is mysteryously close to a 256xxx or an ffff ffff kind of thing.
But the alert says : (only included the top section)
"MALLOC_SIZE1 caused a stack fault in module MALLOC_SIZE1.EXE at 0167:004095c2."
It's not a deal, I was just curious about it.

Anyhow if anyone might have any thoughts on these issues, especially the free() at this point, I'd be interested to hear.
Thanks, ++Howard;

davekw7x · #6 24-Apr-2007, 15:02

Quote:

Originally Posted by Howard_L

Going over the program I posted earlier in this thread I found that I hadn't really created what I thought I did. The idea was to allocate an array of ints or chars and look at the sizeof(array).

If you have allocated an array with a declaration like int x[20], you can use the sizeof() operator to determine the size of the array.

CPP / C++ / C Code:

#include <stdio.h>

int main()
{
  int x[100];     /* an array of ints               */
  double y[100];  /* an array of doubles            */
  double *z[100]; /* an array of pointers to double */
  printf("sizeof(x) = %d, sizeof(x[0]) = %d\n", sizeof(x), sizeof(x[0]));
  printf("Number of elements in x = %d\n\n", sizeof(x)/ sizeof(x[0]));

  printf("sizeof(y) = %d, sizeof(y[0]) = %d\n", sizeof(y), sizeof(y[0]));
  printf("Number of elements in y = %d\n\n", sizeof(y)/ sizeof(y[0]));

  printf("sizeof(z) = %d, sizeof(z[0]) = %d\n", sizeof(z), sizeof(z[0]));
  printf("Number of elements in z = %d\n\n", sizeof(z)/ sizeof(z[0]));
  return 0;
}

Output on my 32-bit Linux system with GNU gcc:

Code:

sizeof(x) = 400, sizeof(x[0]) = 4
Number of elements in x = 100

sizeof(y) = 800, sizeof(y[0]) = 8
Number of elements in y = 100

sizeof(z) = 400, sizeof(z[0]) = 4
Number of elements in z = 100

The size of a pointer is four bytes, as is the size of an int. The size of a double is eight bytes.

Output on my 64-bit Linux system with GNU gcc:

Code:

sizeof(x) = 400, sizeof(x[0]) = 4
Number of elements in x = 100

sizeof(y) = 800, sizeof(y[0]) = 8
Number of elements in y = 100

sizeof(z) = 800, sizeof(z[0]) = 8
Number of elements in z = 100

Note that the size of a pointer is eight bytes, but an int is still four bytes. (The exact sizes of variables, including pointers, are not specified in any C or C++ Standard.) Also note that values of the expressions involving the "sizeof" operator in the examples are constants. The values of the expressions are known at compile time.

Now if you create an "array" with malloc, there is no way (no way) that a standard C program can know the size of the block of memory that was actually created for its use. Your program knows how many bytes it requested and that's how many it was granted, and that is the number that it can legally use. Period. Full stop.

Consider the following:

CPP / C++ / C Code:

      int size;
      double *pd;

      size = 100;
      pd = malloc(size * sizeof(double));
      if (pd == NULL) {
          printf("Couldn't allocate %d bytes for pd\n", size * sizeof(double));
          return 0;
      }
      printf("Allocated %d bytes for pd\n", size * sizeof(double));
      printf("sizeof(pd)  = %d\n", sizeof(pd));
      printf("sizeof(*pd) = %d\n", sizeof(*pd));

Output from the 32-bit system:

Code:

Allocated 800 bytes for pd
sizeof(pd)  = 4
sizeof(*pd) = 8

The first sizeof() value tells the size of a pointer to double. The second one tells the size of a double. Again, the sizeof() operator in this example is something whose result is constant and is known at compile time (and in this case has absolutely nothing to do with the amount of memory that is in the block whose address is the value of the pointer).

Quote:

Originally Posted by Howard_L

On another matter with the program I previously posted:
On my old k6-2 laptop in 98 with a bunch of stuff open
using method2, malloc would max out at about 225000000 - 225500000 chars , , and I noticed that when I increase ints, I have to reduce chars accordingly. My ints don't seem to get freed.

If you free them, they are freed. That means that you can't legally use that particular block of memory any more.

Depending on what else is going on at the time, if you have allocated everything that your program will be allowed (by the operating system) to allocated, it is just possible that if you free them you might not be able to allocate the same number of bytes again.

1. There is a certain amount of overhead associated with blocks of memory obtained from malloc. Allocating many small blocks may actually cause more bytes of memory to be reserved for your program than allocating one large block. This is one of those "implementation" details, and not all malloc() library functions behave the same way, but is "typical", for cases that I have personally investigated. (In other words Your Mileage May Vary.)

2. There is no inherent requirement for garbage collection in the standard library function malloc() and it is quite possible that if you allocate and free a number of blocks that you might end up with an allocatable pool of memory that has enough bytes for your next call to malloc(), but no single block has enough contiguous bytes to satisfy the request.

The standard absolutely requires that the bytes be contiguous, since that is a requirement to be able to treat the contents of the block as an array. Due to this shortcoming (lack of specified garbage collection in C), there have been a number of "garbage collection" functions (and C++ classes) that have been published to do the deed.

3. If you have returned your bytes to the operating system's memory pool (by using free()), some other process may have claimed them in the interval between the time that you freed them and the time that you requested them again. This behavior depends on the operating system, and is even more beyond the control or knowledge of the lowly C programmer than any of the implementation-dependent characteristics of the library function.

In other words: There is an absolute limit on the size of the virtual address space for any given CPU. This represents an upper bound on the total memory for all processes that are running at a given time. A particular compiler (and linker and loader) might have some lower limit for a program. Operating system settings (we used to call it Job Control Language) may set limits on the amount of memory that the operating system will make available to a given class of program.

In other words: A general-purpose operating system (Windows, Linux) actually represents a non-deterministic system! Programmers can develop a sense of reasonable expectations and, with good programming practices, can protect against lots of stuff, but if you really have a need, in today's program, for several gigabytes of array storage to accomplish a mission-critical task, you probably need a little more control over the operating environment than stock versions of commercial operating systems are likely to provide.

In fewer words: Your Mileage May Vary.

A final note: malloc() is a standard library function, and we are discussing the Standards-imposed requirements of malloc(). There aren't any, other than the couple of sentences that I quoted earlier.

Library function writers for popular compiler distributions seem to favor speed first, memory overhead second, and garbage collection never. I, personally, would not argue with those decisions. If I have special needs, I will take care of them with special functions.

Regards,

Dave

Howard_L · #7 24-Apr-2007, 15:50

Whew,, that's a print... Thanks for taking the time.

re: sizeof(mallocated array)
So we use the math method on known values...

re: free()
I guess in situations like this I just need to be aware of the behavior and what I might expect to find (and look out for) the next time I approach the limits.

Gosh,, and I'm supposed to be working on console manipulation..... sidetracked AGAIN!
Thanks, ++Howard;

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Rate This Thread
Rate This Thread:

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Hard drive/CPU Diagnoses Issues	binarybug	Computer Hardware Forum	1	22-Jan-2007 20:23
Strange C++ code memory leakage problem	gaoanyu	C++ Forum	7	04-Nov-2005 09:09
Pointer Usage in C++: Beginner to Advanced	varunhome	C++ Forum	0	19-Aug-2005 10:25
[Tutorial] Pointers in C (Part I)	Stack Overflow	C Programming Language	1	08-Apr-2005 19:35
Having a problem	Chuckles	Computer Hardware Forum	19	13-Sep-2004 13:17

Memory Block Size

Re: Memory Block Size

Re: Memory Block Size

Re: Memory Block Size

Re: Memory Block Size

Re: Memory Block Size

Re: Memory Block Size