FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian User

 
 
LinkBack Thread Tools
 
Old 05-07-2008, 12:11 AM
Rich Healey
 
Default C++ programming: keeping count of data items read from file

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Mark Allums wrote:
> H.S. wrote:
>> Hello,
>>
>> In a C++ program I am reading a data file for later processing and
>> computations. While reading that data file, I want to keep track of
>> data items (doubles) read.
>>
>> The data file is just a text file with N lines with C doubles in each
>> line (N and C are known a priori). For now, I just read from the file
>> stream in to a 2D array variable by reading each double at a time.
>>
>> Now I am trying to introduce some sanity checking into this reading
>> block. Here is what I am trying to do:
>> 1. Verify how many doubles I have read in each line. Must be C. If
>> they are not C, then the input file is corrupt.
>> 2. Verify that the total number of data items are NxC. This is simple,
>> I just keep a track of how many numbers I have read.
>>
>> So, how do I go about doing (1) above? I was thinking of somehow
>> checking if I have reached the end of line somehow (EOL?) but haven't
>> found a method to do so. All I have found is EOF.
>>
>> thanks,
>> ->HS
>>
>>
>
> Not directly helpful, but some suggestions:
>
> 1. You might want to learn PERL or Python or Ruby, and do it there.
FWIW, this is very easy to do in Python.
PSFWIW: Satan uses Ruby.
> 2. If it has to be C++, learn enough PERL to write a filter for the data
> file, and transform it so that it has one double per line.
> 3. Debug the data generator /in situ/ with a good debugger, and bypass
> the need to do the sanity checking.
> 4. Find a good C++ reference, and use it. There are several.
>
> Slightly more helpful:
>
> 1. Read one line at a time in as a string, then operate on the string.
> 2. C++ has the ability to do everything that C does in a low level way,
> but why? Use the C++ way, or use the C way:
>
> #include <cstdio>
> #include <iostream>
> . . .
> using namespace std;
> . . .
> ios::sync_with stdio();
> . . .
> int blah = fscanf(somefile,"%f %f %f %f
", d1,d2,d3,d4);
> if (blah != correctvalue)
> {
> dosomething();
> closefiles();
> cout << "error in data file
";
> exit(1);
> }
> . . .
> // etc.
>
>
> (The ios::sync_with_stdio(); line may differ slightly on different C++
> implementations. I haven't used it in a while. May be spelled synch_.
> Too lazy to look it up.
>
> The fscanf line may just be wrong. I quit writing C programs years ago.
> Too old, memory failing.)
>
>
>
Rich
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIIPOtLeTfO4yBSAcRAkvQAJ4v8ZhzTnupPjRRpkAQai XxTzOpHgCgiHnv
YmSOypexOxGn2ttCLl/YpaI=
=vbdY
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-07-2008, 01:56 AM
"Jordi Gutiérrez Hermoso"
 
Default C++ programming: keeping count of data items read from file

On 06/05/2008, H.S. <hs.samix@gmail.com> wrote:
> APRACK FORTRAN library needs the input data in a 2D array (the arrays need
> to be arranged in column-major format). But, to answer your query, I don't
> *have* to read it in an array, I could read it in a list and then copy it to
> an array before I call ARPACK routines.

Btw, C++ provides std::valarray precisely with Fortran-like matrices
in mind. However, it seems this is the black sheep of the C++ stdlib,
since it seems C++ implementations are unable to optimise
std::valarray as much as intended (I've never really understood the
details), and I've never seen any project use std::valarray at all.
:-/

But anyways, valarrays store data precisely in the numeric formats
that Fortran expects.

On 06/05/2008, Mark Allums <mark@allums.com> wrote:
> (And is also an example of something that is wrong with the C++ standard
> library, when you need the c_str() member of string so often to get any real
> useful work done. Kind of defeats the purpose of having string in the first
> place.)

Yeah, that's because historically the stream classes were written
before the string class. :-/

That's scheduled to be fixed in C++0x. I wish the GNU implementation
of C++ already fixed it, since they do have other fixes (e.g. defining
at() for std::map which is technically non-standard but completely
reasonable).

On 06/05/2008, Mark Allums <mark@allums.com> wrote:
> But I would rather statically link to stdio than to iostream

Although it's true that sometimes the C++ Hello World seems bloated to
the C Hello World, the difference becomes negligible in any project of
considerable size beyond Hello World. Embedded devices may be a
different thing, and I understand that it's allowable and encouraged
to write the hackiest code in embedded devices, even liberally
sprinkling your C code with ASM.

So I hear, so I hear...

- Jordi G. H.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-07-2008, 04:29 PM
"Kevin B. McCarty"
 
Default C++ programming: keeping count of data items read from file

"H.S." wrote:

> Michael Marsh wrote:

>> Can you read full lines out into, eg, a stringstream, and parse your
>> doubles out of that? You'd hit an EOF at the end of each line in that
>> case. I'm not sure how you'd get stream out line-at-a-time, though
>> there may be a stream operator that sets the appropriate behavior.
>
> Yup, that could be done by getting a line till the end of "
" character and then parsing the line. I was just wondering if there was any other way (was trying to avoid parsing).

std::getline() ? I think there is a version that outputs the result as
a string, from which you can then create an istringstream.

You can read numbers from an istringstream the same way as from cin with
">>" operators, so the amount of parsing should be minimal (as long as
the file is not corrupt).

best regards,

--
Kevin B. McCarty <kmccarty@gmail.com>
WWW: http://www.starplot.org/
WWW: http://people.debian.org/~kmccarty/
GPG: public key ID 4F83C751


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-08-2008, 07:10 PM
"H.S."
 
Default C++ programming: keeping count of data items read from file

James Allsopp wrote:

hi,
Try something like this, just add some pointers;
scan is just a simple object and l is a class vector.
HTH
jim

int nearest::readdata(std::string s, std::vector<scan> & l)
{
//read in scuba core list
std::ifstream input(s.c_str());
std::string temp, pos, x ,y;
char * t;
std::cout <<"Reading " << s <<std::endl;
while(!getline(input,temp).eof())
{
scan n;
std::stringstream s(temp);
s >> n.name;
s >> x;
s >> y;
n.glon=strtod(x.c_str(),&t);
n.glat=strtod(y.c_str(),&t);
l.push_back(n);
}
input.close();
return 0;
}



Thanks for your pointer. For future references, here is what I have
working so far:

///////////////////////////////////////////////////////

#include <iostream>
#include <fstream>
#include <sstream>
#include <string>
#include <cstdlib>
#include <iomanip>


/* read 2D data from a file which has each a data vector on each line.
The function allocates memory at the pointer supplied and returns 0 if
the reading went well. Partial reading of the data (first few numbers in
each line and first few lines in the file) is also supported.


post: Dynamic memory has been allocated and needs to be cleared when not
needed.
return: Retuns 0 if files is successfully read otherwise a non-zero is
returned.

*/
int iReadDataFromFile(std::string strFN,//file which holds the data
double **dppData,//address of pointer to save data in
int iR,//read these many top dimensions
int iC /*read these many top data lines*/
)
{
int r,c;
int iCtr=0;
double x;

iCtr = 0;//keep count of items read
int iLN=0;//initialize the line number counter
std::string str;

//open the file to read in the data
std::ifstream isData(strFN.c_str(),std::ios::in);
if (!isData) {
std::cerr << "ERR: opening file " << strFN << std::endl;
return -2;
}

std::clog << "Reading 2D data from file '" << strFN << "'." <<
std::endl;


*dppData = new double [iR * iC];//allocat memory
double *dpData= *dppData;//make a local variable (synonym); easier
to use


#ifdef VERBOSE_READ_DATA_FILE
std::clog << "Data read: " << std::endl;
std::clog << std::setprecision(5);
std::clog << std::setiosflags(std::ios::scientific);
#endif

//now do the reading
for (c = 0; c < iC; c++,iLN++) {//for first iC lines in the file
if ( ! (std::getline(isData, str)) ) {//if line reading went bad
std::cerr << "ERROR: Insufficient data lines in " << strFN
<< ". Expected " << iC << " lines but got only "
<< iLN+1 << std::endl;
isData.close();//don't need this anymore
return -1;
}
else {//line was read okay
std::stringstream ss(str);
for (r = 0; r < iR; r++){//for first iR nums in line
if ( !(ss >> x)){//reading the stream failed
std::cerr << std::endl <<
"ERROR while reading item " << r+1 <<
" on line " << iLN + 1 <<
" in file " << strFN << std::endl;
isData.close();//don't need this anymore
return -1;
}
else {//reading the stream went okay
dpData[c + r*iC] = x;//choose col- or row-major here
iCtr++;
#ifdef VERBOSE_READ_DATA_FILE
std::clog << dpData[c + r*iC] << " ";
#endif
}
}//end for first iR nums in line
#ifdef VERBOSE_READ_DATA_FILE
std::clog << std::endl;
#endif
}//end else line was read okay
}//end for first iC lines in the file

#ifdef VERBOSE_READ_DATA_FILE
std::clog << std::resetiosflags(std::ios::scientific);
std::clog << std::setprecision(0);
#endif

isData.close();//don't need this anymore
return 0;
}




int main(int argc, char * argv[]) {


std::string strFileName("data.dat");

double *dpData=NULL;

iReadDataFromFile("data.dat",&dpData,3,9);
delete [] dpData;

return 0;
}


///////////////////////////////////////////////////////


I compiled it using either of the following to command depending on if I
wanted to see what was being read:

$> g++ -g -DVERBOSE_READ_DATA_FILE -ansi -Wall readdata.cc -o readdata
$> g++ -g -ansi -Wall readdata.cc -o readdata


I am sure the above routine can be further improved. One way to make it
more C++-like is to use exceptions to see if something went wrong with
getline or while reading a number from the string stream.


regards,
->HS



--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-09-2008, 01:16 AM
"s. keeling"
 
Default C++ programming: keeping count of data items read from file

Ron Johnson <ron.l.johnson@cox.net>:
>
> On 05/06/08 13:25, H.S. wrote:
> > Ron Johnson wrote:
> >>
> >> Is this a binary file or a text file?
> >
> > hmm. Text. I made it clear in the original post.
>
> Sorry. It just seems (to an old C programmer) that this is pretty
> simple problem, unless there's some tricky detail that you aren't
> telling us.

That's exactly what I was thinking looking at the problem. No offence
meant to the OP, but this sounds like a trivial problem for perl. We
must be missing something.

Man, does C++ produce ugly, obtuse code (no offence meant to C++ code
posters; thanks), and this from a perl programmer.

OT, but isn't it nice to know they have tornados in VA too[*]? :-)

[*] Sorry d-u; inside joke. No offence ...
--
Any technology distinguishable from magic is insufficiently advanced.
(*) http://blinkynet.net/comp/uip5.html Linux Counter #80292
- - http://www.faqs.org/rfcs/rfc1855.html Please, don't Cc: me.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-09-2008, 01:48 AM
"Jordi Gutiérrez Hermoso"
 
Default C++ programming: keeping count of data items read from file

On 08/05/2008, s. keeling <keeling@nucleus.com> wrote:
> Ron Johnson <ron.l.johnson@cox.net>:
>
> >
> > On 05/06/08 13:25, H.S. wrote:
> > > Ron Johnson wrote:
> > >>
> > >> Is this a binary file or a text file?
> > >
> > > hmm. Text. I made it clear in the original post.
> >
> > Sorry. It just seems (to an old C programmer) that this is pretty
> > simple problem, unless there's some tricky detail that you aren't
> > telling us.
>
>
> That's exactly what I was thinking looking at the problem. No offence
> meant to the OP, but this sounds like a trivial problem for perl. We
> must be missing something.

Perl is an interpreted language. This makes it unsuitable for serious
number crunching.

Don't get me wrong. Octave and SciPy are interpreted languages and ok
for testing out ideas or profiling, but once push gets to shove,
you're going to want a compiled language with the best possible
efficiency, and only C++ seems like a reasonable modern alternative.
Fortran is still favoured amongst the numerical community for similar
reasons.

- Jordi G. H.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-09-2008, 02:32 AM
"H.S."
 
Default C++ programming: keeping count of data items read from file

s. keeling wrote:

Ron Johnson <ron.l.johnson@cox.net>:

Sorry. It just seems (to an old C programmer) that this is pretty
simple problem, unless there's some tricky detail that you aren't
telling us.


That's exactly what I was thinking looking at the problem. No offence


None taken.


meant to the OP, but this sounds like a trivial problem for perl. We
must be missing something.


Yes. It is the scale of the problem.



Man, does C++ produce ugly, obtuse code (no offence meant to C++ code
posters; thanks), and this from a perl programmer.


The problem may be quite trivial in the languages you mention as far as
the mechanics of reading of lines from a file are concerned. However,
the experiments under question (for which I am writing the code) involve
reading large number of files which may be huge in size (thousands of
lines and many numbers on each line in each file). For that, I very much
doubt languages like Perl or Python are going to outperform C or C++ in
efficiency.


Had I been sure to get the same efficiency with Perl or bash or Python
(awk, sed, cut, and so on ... ), I would have taken that route.


Like they say, right tool for the right job ....

->HS


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-09-2008, 12:55 PM
Ron Johnson
 
Default C++ programming: keeping count of data items read from file

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/08/08 21:32, H.S. wrote:
> s. keeling wrote:
[snip]
>> Man, does C++ produce ugly, obtuse code (no offence meant to C++ code
>> posters; thanks), and this from a perl programmer.
>
> The problem may be quite trivial in the languages you mention as far as
> the mechanics of reading of lines from a file are concerned. However,
> the experiments under question (for which I am writing the code) involve
> reading large number of files which may be huge in size (thousands of
> lines and many numbers on each line in each file). For that, I very much
> doubt languages like Perl or Python are going to outperform C or C++ in
> efficiency.

Most of the time will be taken up by disk IO, and (in Python, at
least) the IO routines are written in C.

So if all this particular app does is read in text and write out
binary, Python (and Perl, if it's IO routines are written in C)
would have been the better choice.

> Had I been sure to get the same efficiency with Perl or bash or Python
> (awk, sed, cut, and so on ... ), I would have taken that route.
>
> Like they say, right tool for the right job ....

But with all the time taken to figure out how to write the program,
the Perl/python program would have completed long before the C++
app, even if it took 10x longer.

Remember, there's developer scale as well as execution scale.

- --
Ron Johnson, Jr.
Jefferson LA USA

We want... a Shrubbery!!
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFIJEmuS9HxQb37XmcRAhT1AJ42FTJHFrpshoyQDLejbt wudZD2JACfYlo2
tFPMSPbSIFFbDNFSfVF/ALI=
=wu2p
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-09-2008, 01:19 PM
"Jordi Gutiérrez Hermoso"
 
Default C++ programming: keeping count of data items read from file

On 09/05/2008, Ron Johnson <ron.l.johnson@cox.net> wrote:
> Remember, there's developer scale as well as execution scale.

The execution scale is all that matters to us, since a wrong algorithm
or language can easily exacerbate execution times by orders of
magnitude, while developer time hardly ever suffers from several
orders of magnitude.

You're not going to convince a numericist to give up compiled
languages. :-) Give it up.

- Jordi G. H.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 05-09-2008, 01:37 PM
Ron Johnson
 
Default C++ programming: keeping count of data items read from file

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/09/08 08:19, Jordi Gutiérrez Hermoso wrote:
> On 09/05/2008, Ron Johnson <ron.l.johnson@cox.net> wrote:
>> Remember, there's developer scale as well as execution scale.
>
> The execution scale is all that matters to us, since a wrong algorithm
> or language can easily exacerbate execution times by orders of
> magnitude, while developer time hardly ever suffers from several
> orders of magnitude.

Then write your apps in FORTRAN. (But then, you aren't the OP...)

> You're not going to convince a numericist to give up compiled
> languages. :-) Give it up.

I'm not going to try. Well, not much... Since all the functions
are already written in a compiled language, what you're really doing
is using Python as "stitching".

You really ought to take a look at NumPy, and compare it to whatever
you write in C/C++. In addition to the standard C->Python bindings,
NumPy also has interfaces to your existing FORTRAN and C++ libraries.

http://www.scipy.org/Cookbook/BuildingArrays
http://numpy.scipy.org//

- --
Ron Johnson, Jr.
Jefferson LA USA

We want... a Shrubbery!!
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFIJFOwS9HxQb37XmcRAhLPAJ0UI8GlQxitylk0CY6SLP YxhQhEUgCfcQT6
NhH5eVX+SUm/DRC2eZqECHw=
=GOMn
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 

Thread Tools




All times are GMT. The time now is 10:23 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org