GIDForums  

Go Back   GIDForums > Computer Programming Forums > C++ Forum
User Name
Password
Register FAQ Members List Calendar Search Today's Posts Mark Forums Read

 
 
Thread Tools Search this Thread Rate Thread
  #1  
Old 06-Jul-2008, 07:24
Lancet Lancet is offline
New Member
 
Join Date: Jul 2008
Posts: 11
Lancet is on a distinguished road

Storing long string constants inside a program


Hi (first post ),

I am working on a project where I have to store really big string constants (JavaScript snippets) inside my application. At the moment this looks very ugly as one could imagine. Is there a way I can create plain-text and put that into strings preprocessor wise (or the like)?
In the end there should be a single executable (well dll actually (but that shouldn't matter)) so putting this strings inside some text-file or database that loads runtime is not possible.
As I can only think that this problem should occur more often I am kind of hesitant to ask but my gOogle-Skilz have proven to be useless in this matter and searching the forum didn't yield anything either.
So if someone has an idea how to do this or some search-words or that kind of information I would be happy to get advice.

By the way; I have to use Visual Studio 2005 for this.
Are resource files the way to go?
Hmm...

Thanks for your time.

PS: The smiley in this post is for the sole purpose of enhancing the post's cuteness I am thus in breach with "guideline 10" of "Posting Requests for Help"
I accept severe punishment.
  #2  
Old 06-Jul-2008, 08:50
davekw7x davekw7x is offline
Outstanding Member
 
Join Date: Feb 2004
Location: Left Coast, USA
Posts: 5,217
davekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to behold

Re: storing long string constants inside a program


Quote:
Originally Posted by Lancet
...At the moment this looks very ugly as one could imagine....
Personally, I try not to spend too much time imagining how ugly things could become. See Footnote.

What's wrong with something like the following?
It doesn't use any Windows-specific or Visual Studio-specific program constructs. It's standard C++ (works for standard C also). So, unless you need to make it easy to incorporate different strings for different languages, you don't need anything fancy.

CPP / C++ / C Code:
#include <iostream>

using namespace std;

int main()
{
  const char * usage_message = 
  " remezz(): Parks-McCellan (Remez exchange) linear phase FIR filter design\n\n"
  " Usages:\n"
  " [h,err]=remezz(N,F,A)\n"
  " [h,err]=remezz(N,F,A,'hilbert')\n"
  " [h,err]=remezz(N,F,A,'differentiator')\n"
  " [h,err]=remezz(N,F,A,W)\n"
  " [h,err]=remezz(N,F,A,W,'hilbert')\n"
  " [h,err]=remezz(N,F,A,W,'differentiator')\n\n"
  " N: FIR filter length\n"
  " F: Vector of frequency band edges (0-1) in pairs.\n"
  "    e.g. F=[0 0.5 0.7 1] specifies two bands 0-0.5 and 0.7-1.\n"
  "    0.5-0.7 is considered as a transition band (don't care).\n"
  " A: Vector of the desired amplitude at each band edge.\n"
  "    Should be same length as F. e.g. A=[1 1 0 0] specifies 0-0.5 have\n"
  "    gain of 1, and 0.7-0.1 have gain of 0.\n"
  " W: Vector of weighting function for the ripple ratio between each band.\n"
  "    One value for each band. (The length is one half that of A.)\n"
  " 'hilbert', 'differentiator': specifies either a Hilbert transformer or\n"
  "    differentiator. (Default is a passband filter.)\n";

  cout << usage_message << endl;

  return 0;
  }
Adjacent string literals (optionally separated by whitespace) are automatically concatenated into a single string.

Regards,

Dave

Footnote: Here is one of those web pages that you don't even have to open in your browser. The URL says it all:
ugly-is-as-ugly-does on screaminblackmonkeyspit
  #3  
Old 06-Jul-2008, 09:37
Lancet Lancet is offline
New Member
 
Join Date: Jul 2008
Posts: 11
Lancet is on a distinguished road

Re: storing long string constants inside a program


Hello Dave,

Quote:
It doesn't use any Windows-specific or Visual Studio-specific program constructs.

I usually compile with g++ and cross-platform is very important for me but this project will stay "very most likely" on Windows. (of course I'd prefer a portable solution)

Here is what I do at the moment (does not much differ from yours):

CPP / C++ / C Code:
const char* get_first_level_nodes_count = "\
some_global_object.println('get_first_level_nodes_count');\
//more code ...\
";

The problem is, that I will get really long strings and many of them.
Parts of these strings will surely change so I have to figure out some sort of sorting and categorizing scheme to keep my sanity.

If I don't find an other way I will write myself something to produce this code from the original text (escaping quotation marks and the like one the way too).
Doing this by hand would drive me mad

So, any other ideas?
  #4  
Old 06-Jul-2008, 09:45
Lancet Lancet is offline
New Member
 
Join Date: Jul 2008
Posts: 11
Lancet is on a distinguished road

Re: Storing long string constants inside a program


I think that the image is a well done illustration of the positive aspects of a despicable person of your choosing. *me having Mitläufer-mentality*
  #5  
Old 06-Jul-2008, 15:40
davekw7x davekw7x is offline
Outstanding Member
 
Join Date: Feb 2004
Location: Left Coast, USA
Posts: 5,217
davekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to behold

Re: storing long string constants inside a program


Quote:
Originally Posted by Lancet
...


I usually compile with g++ ...

Here is what I do at the moment...
would drive me mad...
ideas?

I don't know exactly what you want to do, so I'll do something that is interesting to me. Maybe it will help you; maybe not...

I would write a program that would read lines of text and create a program that would consist of a function that returns a string consisting of those lines of text. (Kind of like being a resource compiler.)

Suppose I have the following text file:

Code:
The quick brown fox etc... A line with some "quote" marks; Ttthhat's all folks

To make a string literal of these, I would need to use backslash characters to escape the quote marks, and I would probably put a '\n' character at the end of the lines inside the string.

So the output file might look like this:
CPP / C++ / C Code:
// lines.cpp
#include <iostream>
#include <string>
using namespace std;
string lines()
{
    string retstr(
        "The quick brown fox etc...\n"
        "A line with some \"quote\" marks;\n"
        "Ttthhat's all folks\n"
    );
    return retstr;
}

Here's a simple program that processes lines of text like this and creates the C++ function file as its output:

CPP / C++ / C Code:
//
// makelines.cpp
//
// Read a file consisting of lines of text
// The lines are used to create a function
// that returns a string consisting of those
// lines all concatenated together.
//
// For simple illustration, quote marks on
// a line are escaped properly in the string
// literal. For this program, a newline is 
// inserted in the string at the end of each 
// input file line.
//
// Input file and output file names are hard-coded
// here; obviously they could be mad to be
// command-line arguments, or whatever...
//
// davekw7x
// July, 2008
//
#include <iostream>
#include <fstream>
#include <string>

using namespace std;

string process_line(const string &);

int main()
{
    char *inname = "lines.txt";
    char *outname = "lines.cpp";

    char *preamble = 
        "// lines.cpp\n"
        "#include <iostream>\n"
        "#include <string>\n"
        "using namespace std;\n"
        "string lines()\n"
        "{\n"
        "string retstr(\n";

    char *postamble = 
        ");\n"
        "return retstr;\n"
        "}\n";
    string line;
    ifstream infile(inname);
    if (!infile) {
        cerr << "Can't open file " << inname << " for reading." << endl;
        exit(EXIT_FAILURE);
    }
    ofstream outfile(outname);
    if (!outfile) {
        cout << "Can't open file " << outname << " for writing." << endl;
        exit(EXIT_FAILURE);
    }

    int linecount = 0;
    outfile << preamble;
    while (getline(infile, line)) {
        ++linecount;
        cout << linecount << ": " << line << endl;
        outfile << process_line(line) << endl;
    }
    outfile << postamble;
    cout << "Total lines read = " << linecount << endl;
    return 0;
}


string process_line(const string & instring)
{
    string retstring("\"");
    for (unsigned i = 0; i < instring.length(); i++) {
        if (instring[i] == '"') {
            retstring += '\\';
        }
        retstring += instring[i];
    }
    retstring += "\\n\"";
    return retstring;
}

Of course, you might need something more elaborate than simply escaping the quote characters, but I hope you get the idea.

Here's the actual output (I didn't bother trying to indent it properly, as is typical for machine-generated program files):
CPP / C++ / C Code:
// lines.cpp
#include <iostream>
#include <string>
using namespace std;
string lines()
{
string retstr(
"The quick brown fox etc...\n"
"A line with some \"quote\" marks;\n"
"Ttthhat's all folks\n"
);
return retstr;
}

Now, here's a test program for using the function defined in lines.cpp:

CPP / C++ / C Code:
// test.cpp
#include <iostream>
#include <string>

#include "lines.h"

using namespace std;

int main()
{
    string stuff = lines();
    cout << "Here's the stuff:" << endl << endl;
    cout << "----------start stuff------------" << endl;
    cout << stuff;
    cout << "-----------end stuff-------------" << endl;
    cout << endl;
    cout << "How did it look?" << endl;
    return 0;
}

And, here's "lines.h"
CPP / C++ / C Code:
#ifndef LINES_H__
#define LINES_H__
#include <string>
std::string lines();
#endif

And here's the output of the test program when I compiled it together with lines.cpp:
Code:
Here's the stuff: ----------start stuff------------ The quick brown fox etc... A line with some "quote" marks; Ttthhat's all folks -----------end stuff------------- How did it look?

Here's a Makefile for GNU make and gcc on my cygwin Windows XP platform, so that everything is generated automatically. (Also used on my Linux system by following the comments about $(EXEEXT):
Code:
# #GNU Makefile for Linux or Windows with cygwin gcc # CXX = g++ CXXFLAGS= -Wall -W -pedantic LD = g++ OBJEXT = .o #For Windows, comment out the next line #EXEEXT = #For Linux, comment out the next line EXEEXT = .exe TARGET = test OBJECTS = test$(OBJEXT) lines$(OBJEXT) HEADERS = lines.h $(TARGET)$(EXEEXT): $(OBJECTS) $(LD) $+ -o$(TARGET) $(TARGET)$(OBJEXT): test.cpp $(HEADERS) lines$(OBJEXT): lines.cpp # #The "makelines" program processes lines.txt to create lines.cpp # makelines$(EXEEXT): makelines.cpp $(CXX) $(CXXFLAGS) $< -o $@ # # Here's the rule to create lines.cpp # lines.cpp: lines.txt makelines$(EXEEXT) ./makelines @echo @echo "*** The makelines program has created the file 'lines.cpp'*** " @echo clean: rm -f $(TARGET)$(EXEEXT) makelines$(EXEEXT) lines.cpp \ $(OBJECTS) makelines$(EXEEXT)

When I did "make clean" and then "make", here's what I saw:
Code:
g++ -Wall -W -pedantic -c -o test.o test.cpp g++ -Wall -W -pedantic makelines.cpp -o makelines.exe ./makelines 1: The quick brown fox etc... 2: A line with some "quote" marks; 3: Ttthhat's all folks Total lines read = 3 *** The makelines program has created the file 'lines.cpp'*** g++ -Wall -W -pedantic -c -o lines.o lines.cpp g++ test.o lines.o -otest

This created lines.cpp and test.exe

Regards,

Dave
  #6  
Old 07-Jul-2008, 05:18
Lancet Lancet is offline
New Member
 
Join Date: Jul 2008
Posts: 11
Lancet is on a distinguished road

Re: Storing long string constants inside a program


Thank you for your great work Dave,

Quote:
I don't know exactly what you want to do, so I'll do something that is interesting to me. Maybe it will help you; maybe not...

You did exactly what I had in mind (except that I'll maybe write it in ruby).

I am still curious if there is a well accepted way of getting a long text into a string constant without putting the text altogether to the rest of the source-code...

I really appreciate extensive reply.

Regards,

Rob
  #7  
Old 07-Jul-2008, 09:11
davekw7x davekw7x is offline
Outstanding Member
 
Join Date: Feb 2004
Location: Left Coast, USA
Posts: 5,217
davekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to behold

Re: Storing long string constants inside a program


Quote:
Originally Posted by Lancet
...well accepted way of getting a long text into a string constant...

There are about a million ways (maybe more). I don't know whether any of which of them is/are well-accepted. I personally accept things that I like. I like most things that work and are easily maintainable and that don't use proprietary or other closed-source tools. Sometimes I am forced to accept things whether I like them or not. So I accept them: maybe gracefully, maybe not.

Let's think this through. Here are a few comments showing my thought processes so far:

1. A string constant must be known to the compiler at compile time. This is consistent with your desire not to have the executable target program read in the stuff at execution time.

2. At compile time, the compiler only knows what you tell it.

3. The way that you tell the compiler something is to put it into source code somewhere. If the stuff you need is in a separate file (as in my example, where the C++ source for the string stuff was in a file named "lines.cpp"), it may be compiled separately and the result linked together with the main(), but it still must be some kind of source code recognizable by that compiler.

4. At link time, when the executable target program is created, the other stuff (object files) could have come from any kind of source file for which your compiler or other tool can compile into a form that can be combined with the object code for main(). For example, the GNU Octave program links a bunch of C++ and C code together with matrix manipulation and other math functions that are written in Fortran. So the other stuff doesn't have to originate as C++.

4. Windows compiler suites (like Visual Studio) allow resource code, such as stuff to define string constants, to be in some other format (not C++ or C# other high-level language code), but a separate tool (the Resource Compiler) converts it to something that can be linked together with the other stuff (or put into a static library file or dll).

Implementation details of the Resource Compiler are unknown to me. That is: I don't know (and I don't particularly care) how it does it since it is closed source. There are open-source tools that do the equivalent, but I haven't looked at them since I haven't needed anything more elaborate than my simple example.


5. My little example uses the C++ compiler to create a program that converts the string constant "resource file," lines.txt, into a standard C++ source file for a function. That file, lines.cpp can be compiled and its function will be linked into the target executable with the same C++ compiler without using any proprietary or other non-C++ tools.

6. My example is a real simple task, and it is easily automated using only the GNU make utility. Slightly more "interesting" stuff might be handled as shell scripts, possibly in conjunction with sed, awk, perl, etc., bit I stick with standard C/C++ and the make utility until I get really, really tired of it and decide that I need something more elaborate. After all I already have a Makefile for any but the most trivial projects. Like C++ itself, I think I use something like 1% of the power of the make utility (maybe less), and if I need to do something that I haven't considered before, I try to learn more about the tools that I already use. If I were a perl wonk, maybe I would do it in perl. I hope that you get the idea.

7. Go back over my comments 1-6 with a view towards answering your question about ways of doing the deed (or possibly restating it in a way that we can go from here).

Regards,

Dave
  #8  
Old 07-Jul-2008, 10:53
Lancet Lancet is offline
New Member
 
Join Date: Jul 2008
Posts: 11
Lancet is on a distinguished road

Re: Storing long string constants inside a program


You just opened my eyes ,

especially point 5 and 6 made it clear for me, that I should automate the procedure of creating the C++ strings. Normally I have an aversion against machine-generated source-code but I guess this should be ok...

I'll check back with my results when I am done.

Regards,

Rob
  #9  
Old 07-Jul-2008, 13:38
davekw7x davekw7x is offline
Outstanding Member
 
Join Date: Feb 2004
Location: Left Coast, USA
Posts: 5,217
davekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to beholddavekw7x is a splendid one to behold

Re: Storing long string constants inside a program


Quote:
Originally Posted by Lancet
...Normally I have an aversion against machine-generated source-code ...

The trick is that after the development and debugging, you don't really want to look at the machine-generated C++. Always deal with the input "lines.txt" file (or whatever), whose format you have designed for your convenience. The fact that you will be looking at "lines.cpp" from time to time (when questions and/or bugs raise their ugly heads) means that you should at least put newlines at the end of each line rather than have one big long line, as I have seen in certain machine-generated code. Stuff like that. Heck, you could even put in a comment or two. here and there.

The problem is that if you hand the project off to someone else, they are likely to edit "lines.cpp" directly instead of "lines.txt," thereby royally screwing the pooch for repeatability and project maintainability. That's why you make the entire process automatic instead of telling the end user to manually recompile and/or and re-apply the conversion program whenever the strings are to be altered.


Bottom line: In my experience, regardless of the excellence and quantity of external documentation, the only thing that you can count on surviving is the source code. Make it easy on yourself and future maintainers by putting everything possible (and making it as clear as possible) in the source code (including documentation in and of the Makefile.)


Regards,

Dave
  #10  
Old 08-Jul-2008, 03:27
Lancet Lancet is offline
New Member
 
Join Date: Jul 2008
Posts: 11
Lancet is on a distinguished road

Re: Storing long string constants inside a program


Ok,

Here is what I have done till now:

Each string I want to use is in a separate file.
I create a header file with a map<file_name,string_in_file>

This is the Ruby script I use to do that:
txt2string.rb:
Code:
#!/usr/bin/env ruby require 'rubygems' require 'pp' require 'choice' require 'fey_read_file' Program_name = 'txt2string' Program_version = '2008-07-07 1709' Choice.options do header '' header 'Specific options:' option :input_files, :required => true do short '-i' long '--input_files *LOCATION' desc 'the location of the input files (to create string constants of)' #cast String end option :output_file do short '-o' long '--output_file LOCATION' desc 'the location of the output file' cast String #default 'tmp.h' end separator '' separator 'Common options: ' option :help do long '--help' desc 'Show this message' end option :version do short '-v' long '--version' desc 'Show version' action do puts "#{Program_name} #{Program_version}" exit end end end params = Choice.choices import_string_section = '' params.input_files.each{|file_name| puts file_name data = Fey.read_file(file_name) # TODO check if plain ascii (or convert to ascii) data.gsub!(/\\/,"\\\\"); # replace every backslash with two backslashes data.gsub!(/\n/,"\\n"); # replace new lines with new line characters data.gsub!(/\"/,"\\\""); # replace double quotation marks with with escaped versions of them # TODO some more modifications # add one entry import_string_section += "is[\"#{file_name}\"] = \"#{data}\";\n" # entries look like this: # is["file_name1"] = "value1"; # is["file_name2"] = "value2"; } # prepare output text (adding import_string_section) Output_text = "// // this file is machine generated source-code // #include <map> #include <string> std::map<std::string,const char*>& get_import_strings(); void init_import_strings(); static std::map<std::string,const char*>* import_strings = 0; std::map<std::string,const char*>& get_import_strings(){ if(import_strings == 0){ init_import_strings(); } return *import_strings; } void init_import_strings(){ import_strings = new std::map<std::string,const char*>; std::map<std::string,const char*>& is = *import_strings; #{import_string_section} } " if(params.output_file) # write to file File.open(params.output_file,'w') {|file| file.write(Output_text)} else # write to standard out puts Output_text end

fey_read_file.rb:
Code:
require 'open-uri' module Fey # uses open-uri and on failure File.open (so 'http://' and '\\server'(windows smb share) work) def Fey.read_file path file = nil data = nil puts ' opening file' begin puts ' trying open-uri' file = open(path) rescue puts ' open-uri failed' file = nil end if !file begin puts ' trying File.open' file = File.open(path,'r') rescue puts ' File.open failed' file = nil end end if file puts ' OK' begin puts ' reading data from file' data = file.read puts ' OK' rescue puts ' WARNING - could not read data' end file.close else puts ' WARNING - could not open file' end return data end end

If I call this script like this:
$ ./txt2string.rb -i file_name1 file_name2 -o header_name.h
I get this output file:
header_name.h
CPP / C++ / C Code:
// 
// this file is machine generated source-code
//

#include <map>
#include <string>

std::map<std::string,const char*>& get_import_strings();
void init_import_strings();

static std::map<std::string,const char*>* import_strings = 0;

std::map<std::string,const char*>& get_import_strings(){
	if(import_strings == 0){
		init_import_strings();
	}	
	return *import_strings;
}

void init_import_strings(){
	import_strings = new std::map<std::string,const char*>;
	std::map<std::string,const char*>& is = *import_strings;
	is["file_name1"] = "text_in_file1";
	is["file_name2"] = "text_in_file2";
}


Which I can use in main.cpp like this:
CPP / C++ / C Code:
#include <iostream>

#include "header_name.h"

using namespace std;

int main(int argc,char* argv[]){
	map<std::string,const char*> strings = get_import_strings();
	string search_key = "file_name2";
	if(strings.find(search_key) != strings.end()){
		cout << strings[search_key] << endl;
	}
	
	//map<std::string,const char*>::iterator i = strings.begin();
	//const char* c = (*i).second;

	cout << (*(strings.begin())).second << endl;
	return 0;
}

Now I only have to add this to my Visual Studio environment... (what great fun... )

Any suggestions for improvements are very welcome.
 
 

Recent GIDBlogProblems with the Navy (Chiefs) by crystalattice

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Memory leak when nothing is happening... How can I even debug this ? Algar MS Visual C++ / MFC Forum 10 19-Nov-2007 08:17
Two-Tier data dissemination code installation problem nidhibansal1984 Computer Software Forum - Linux 6 16-Sep-2007 11:13
Compiled program does not execute Serpentine MS Visual C++ / MFC Forum 1 23-Jan-2007 10:41
Help with String Program limergal C++ Forum 6 03-Dec-2006 15:47
help with survey program shaffer C++ Forum 5 01-Dec-2006 09:51

Network Sites: GIDNetwork · GIDWebHosts · GIDSearch · Learning Journal by J de Silva, The

All times are GMT -6. The time now is 10:34.


vBulletin, Copyright © 2000 - 2009, Jelsoft Enterprises Ltd.