Such as? I'm interested to know what challenges you see that make static arrays a bad thing.

I disagree.

After programming for over 30 years I rarely have problems with static arrays. And I rarely use dynamic arrays. I feel they cause other challenges.

This topic is definitely is a decision of choice and experience. Your experience is different than mine, and I'm asking learn other options and reasoning -- it's not a challenge.

__________________

Cow: You're a lawyer too?
Mooseblood (mosquito): Ma'am, I was already a bloodsucking parasite. All I needed was a briefcase!

The major differences are that (smart) dynamically allocated buffers usually do not exhibit conditions of array bounds overflow and take away the responsibility of ensuring array boundaries are always "carefully observed."

Contrarily, static allocation and "dumb" dynamic allocation of arrays are easily overflowed without proper observance. In fact, many (if not most) security exploits spring from exceeding array boundaries in "good ol' C code."

The use of dynamically allocated arrays, by itself, does not ensure that the problem magically goes away. In fact, deallocating the memory becomes an issue and memory leaks are much more likely with "dumb" arrays. Smart arrays that are self-managed are the preferred way to avoid both types of pitfalls, though even they have some important limitations that can be negatively exploited, though, it is much easier to limit the exploitation to (usually) a temporary nuisance of memory allocation. In server environments, even this can be problematic due to the potential number of clients performing the same (assumed vicious) interaction.

There is no perfect cure. What becomes apparent is that using the simplest thing that works and is most easily managed and is reasonably portable is often the proper choice. Again, all memory allocations are going to be heavily dependent on the experience of the programmer and the nature of the programming task and surely of the programming language used and the resource constraints of the system/operating system deployed.

Where possible using "smart" dynamically allocating/deallocating arrays is preferrable given an even playing field of commonality among desktop and server systems.

Another--less often touched upon--issue is that of stack size management. Statically allocated arrays increase stack size requirements. This is typically not a "serious" problem, though it can be depending on the amount of data allocated and the processing work necessary. Allocating on the heap reduces stack requirements. Even it is not without a double edge. Heap fragmentation and/or corruption is another potential "feature" of dynamic allocation, though often negated with modern operating systems.

Some kinds of systems and programs are rather obviously not conducive to static allocations. For example, a digital A/V animation and rendering application would likely need very large buffer allocations. These kinds of allocation would likely not make very good sense if allocated on the stack.

Stack allocations tend to be less well-virtualized than heap allocations, though obviously operating system/features dependent.

Stack allocations tend to be difficult to postpone until "needed," especially in pure C where all variables must be declared before being used. This can dramatically increase application startup time, though it doesn't necessarily do so.

Static allocations tend to be win or lose scenarios where you either have the memory available to you and the program runs as expected, or you don't and the program fails to properly load. Granted, this is largely negated by modern operating systems and the ability to virtualize memory and entire processes, but even in a non-extreme case, it is easy to see how a heap allocation-based program can offer alternatives and/or feedback options based on a failed allocation whereas the stack allocation will just blindly fail.

You'll rarely find a stack allocation being used as a "shared memory" object between two processes. Some kinds of memory usages tend to naturally be more heap-oriented.

Additionally, heap-based allocations can be finely controlled by the operating system such that the physical page mapping is performed on (any number of criteria) a "as used" basis. Certain kinds of lazy allocations are possible that are not with conventional stacks.

Static arrays lead to program designs that fail to grow well. They also venture into (arguable) territory that suggests that using #defines to size the array is somehow a good thing. Modern editors negate the problem of associating such with their real values, but even editing the value to increase the size of the array offers another set of problems. Statically declared array sizes also lend themselves to being abused by programmers, who write the following:


This (or even when 20 is "poundefined") leads to a "more maintenance required" programming methodology than simply using dynamically allocated "smart" arrays.

However, statically allocated arrays do not require the headaches required to use "dumb" dynamically allocated arrays, such as always having to tell "everyone" how large they are--or the problems that come with getting some calculation wrong and trying to trace it down in a complex system.

Smart arrays are the way to go when system speed and resources are of the typical PC variety. Smart arrays will tell you their size, manage their memory requirements, lend themselves to generic programming, work with common algorithms, happily free themselves (and what they contain) when out of scope, be type independent, et cetera.

In my initial reply, I didn't qualify dynamic with smart. Even when using "dumb" arrays, there are times that heap allocation is the only reasonable choice. I can easily see how you (or anyone) would consider them better avoided, especially for novice programmers.

In some systems, heap allocations can take much longer to perform, access and use than stack allocations. A lot of it depends on the physical memory layout of the system and the system itself. Again, all largely mitigated by modern PC hardware/OSes.

Smart "containers" are one of the reasons why C++ is rapidly deposing C in many programming environments, particularly in heavily resourced PC systems. Also, stack-based buffers can not easily suit a role where they suffice for generic programming tasks. A good example is one of a string buffer. How large will the string buffer need to be for all of the strings that it may encounter? Even algorithms that create segmented buffers of various sizes and fill them until the need is managed results in code that is significantly more complex and more difficult to maintain than just using a smart buffer. Defensive programming practices suggest not fixing size, but handling the potential for change that exists in any buffer used...and handling that change in ways that make sense and provide increased levels of feedback when things go wrong so that problems are more readily and timely resolved.


:davis:

Grrr... Since you moved the thread, this isn't a 'beginners' topic anymore.
Everything is situational.

If I, as a developer, can control the size of an array, static is fine. I know all about it.

BUT.. often (read, ALWAYS),
in what I do, I cannot control the size. My arrays are generally communications events, be it over a COM port, or over ethernet, or other's, it doesn't matter.

I need to dynamically allocate all arrays, be it a struct (older DDE Stuff with perfectly functional legacy code) or class (which, by nature of my business are dynamically loaded).

Then again, except for the legacy code, I pretend not to know about arrays and use mostly std::vector.

I respect you, WaltP, and have followed your advice often, but there are cases that a static <construct of any sort> just won't do. Would you define a static <construct> to retrieve a rowset from a SQL table?

I need to do the above. Additionally, I need to decide the most efficient way to communicate with external hardware....
Example: MyUser want's to read a block of 32 bit integers, based on a 'tagname', going from 'tagname0' to 'tagname155' (and they don't even know the internal represntation). My hardware can send 256 bits at a time. I need to parse the user request (and I don't know it in advance) into packets the hardware can deal with, but return to the user an entire block. Partial updates are not acceptable.

Then too, there is the case of (yes, even I hate them) *.INI files which I parse to allow my users to add functionality that they require, but not everyone does. Should I limit my users to selecting only 'X' number of updates in 'Y' time? or should I let them choose to retrieve 'Z' words of information in whatever time it takes?
I actually do let the users modify the XML files to do whatever they want, no matter how long it takes... should I say 'Sorry, I will only support 10 data-request entries'?

Perhaps, I am wrong, but dealing mostly with either embedded or Windows systems, I feel I would be more than arrogant to demand a block of memory to handle my ABSOLUTE WORST CASE SCENARIO on startup.

If a user exceeds the system limit, I will warn them about page-swap problems, or that they need more RAM. But that should not be a waste of resources that aren't needed.

IMHO, of course

I moved this thread because this 'discussion' doesn't belong in the help thread. That doesn't negate it being a beginners topic. It just doesn't clutter up someone's thread.


Absolutely.


Thank you.
I never said nor meant to imply dynamic arrays should be avoided at all costs. There are definite places they should be used, and must be used. I simply disagreed that they should mostly be used. Which is how I interpreted the statement
Maybe I misinterpreted, but to me this meant "use them whenever possible", not "use them when necessary". The actual meaning is probably between the two.

All I'm saying is I disagree with the idea of creating a dynamic array simply because you can. If you know the maximum array you need, define it. If you don't know how much space is needed, allocate dynamically.

If you know you need 1000, and may need up to 1500, define static.
If you normally need 1000, but at times you will need 5000 or more, even in rare cases, allocate dynamically.
If you normally need 1000, but at times you will need at most 5000, even in rare cases, choose. Either way will work, but based on specific program requirements/definition/design, one way may make more sense.

Linked lists, many communications systems, directory manipulators, editors -- all pretty much need/require dynamic buffers.


I have not yet absorbed your post davis. But you mention smart allocation and dumb allocation. Please define. I want to be sure I understand your terms.

__________________

Cow: You're a lawyer too?
Mooseblood (mosquito): Ma'am, I was already a bloodsucking parasite. All I needed was a briefcase!

Well. I am not an expert programmer or something.

But I feel that Correctly designed dynamic memory allocation is extremely useful that static ones.

Just my 2 Rupees...
Regards,
Paramesh.

__________________

Don't walk in front of me, I may not follow.
Don't walk behind me, I may not lead.
Just walk beside me and be my friend.

More correctly, I should have said: "Create smart dynamically allocated storage when given the option." I tend to use "smart" storage so often that I sometimes forget that dynamic doesn't necessary suggest smart.

What I meant by what I meant to say (grin) is that if you have the option, then use it whenever possible. That "option" is exceptionally situational dependent in just about every imagined coding challenge there is...but it is also somewhat notably language dependent. (For semantics, WaltP, I use dependant when it is a noun and dependent when it is an adjective.)

By its nature, storage is fleeting, in that, the closet is always too small, as is the hard drive, as is the toy box, as is the garage, as is the attic, etc..etc.

The statement "the only constant is change" always seems to apply to storage of all types. If you allocate a buffer for 32 bytes, you're going to want to stuff more into it soon enough. If 32 bytes are all you're ever going to need, fine, the option is clear...there is no other real option, right? However, if you're going to accept anything that is likely to vary in length, a fixed sized buffer creates more code than it solves, whether the buffer is statically or dynamically allocated. "Growable" buffers tend to be dynamically allocated due to their nature and the nature of stack space. What happens when you pass the address of a statically allocated buffer into a call to realloc?


..but it compiles quietly even with maximum warnings...well quietly except for warning me that I'm using C++ style comments.

Obviously, the responsibility is left to the programmer to properly use the tools provided in the Standard C Library.

If one considers the code to fully implement a doubly linked list, it can get fairly involved. Now, imagine the code needed to ensure that the list can grow automatically every time a new node/element is added to its tail. This is why it is fairly rare that C programmers spend the energy needed to build "smart" dynamically allocating storage...even if there are plenty of free implementations of dynamically growable doubly linked lists out there available for downloading. Why? Because it isn't part of the standard library? ...news flash, neither is their code even though it uses the std c lib.

Anyway, there is an adversion to using "not created here" code, but that is beside the point.


What are you going to do, in C, using the standard C library, to allocate dynamically if you don't know how much space is needed? What memory allocation function doesn't require a size to be passed to it? ...whether static or dynamic allocation is used? ...any of the built-in types in single quantities including unitialized pointer types. You can't get through "Hello, World!" without exceeding those requirements...though one can argue that static string allocations do not explicitly require the size, you DO have to type some number of characters, though.

Anyway, WaltP, I don't think that that is "All I'm saying" in so many words. I don't mean to be rude or confrontational, but your words were:


This tends to sound more like they are better avoided because of the other challenges they cause.

Also, we should probably set a limit on the number of programming years to those relevant to the language(s) being discussed unless programming languages prior to C are important to the content of the discussion...and unless you mean to say that you've been programming in C since 1976 (or earlier), some 2 years before K&R (first edition) was published.



...define the static storage as 1500, right?



Dumb allocation is:


Smart allocation is that which is growable without user interaction (no need to realloc) by simply adding more units of the type to the "container." It is smart because most implementations grow in a "smart" way, unlike most programmers who alloc 1000 bytes and then realloc another 1000 and then again, until they get to where the demand is satisfied. Of course, how many times do we see allocations of "just whatever" size (like 100 or 500) for some unknown length buffer? Also, anybody ever wonder what the value of BUFSIZ is?

Well, it is supposedly tuned for the particular system...and it is usually some multiple of the architecture page size. However, how often do you find people using it in their code? Don't modern architectures allocate a minimum of a page at a time anyway? For that reason, is there ever any reason to allocate less than a page?

Smart allocation isn't just about knowing the hardware and the operating system, but also in how smart the algorithm for reallocation (growing) is. There are many techniques employed and a common theme is threshold-based growth as demand warrants.

Rather than trying to write "the next" smart buffer, most programmers should simply use one. So, how many do?

The STL vector is an excellent example of a smart buffer.

That option isn't always available, is it?


:davis: