StreamTokenizer, data records, indexing/ newline trouble
Hi,
I'm attempting to parse data records from a text file,
using StreamTokenizer without success so far.
Can someone help spot problems in following SCCSE or
perhaps suggest an alternate approach?
Thanks,
Jeff Higgins
Each data record contains four comma separated quoted string fields,
the quoted string fields may contain any printable character except double
quote. They may also contain newlines. They may also be empty or null.
Each data record begins on a new line.
Sample input:
"11","Title 11","USA","Title 11 description contains no newlines"
"12","Title 12",,"Title 12 description contains no newlines"
"123","Title 123","USA","Title 123 description contains"
two newlines"
"1234","Title 1234","CAN",
Desired output:
11 Title 11 USA Title 11 description contains no newlines
12 Title 12 null Title 12 description contains no newlines
123 Title 123 USA Title 123 description contains
two newlines
1234 Title 1234 CAN null
SCCSE and sample input/output follow;
import java.io.*;
import java.util.*;
public class FieldTokenizer {
public static void main(String args[])
{
if (args.length == 0) {
System.err.println("missing input filename");
System.exit(1);
}
ArrayList<Field> list = new ArrayList<Field>();
try {
FileReader fr = new FileReader(args[0]);
BufferedReader br = new BufferedReader(fr);
StreamTokenizer st = new StreamTokenizer(br);
st.resetSyntax();
st.eolIsSignificant(false);
st.quoteChar('"');
st.whitespaceChars(',', ',');
int type;
while ((type = st.nextToken()) != StreamTokenizer.TT_EOF) {
Field dummy = new Field();
for(int i = 0; i < 4; i++){
if(i == 0){
dummy.code = st.sval;
st.nextToken();
}
else if(i == 1){
dummy.title = st.sval;
st.nextToken();
}
else if(i == 2){
dummy.country = st.sval;
st.nextToken();
}
else{
dummy.description = st.sval;
st.nextToken();
}
}
list.add(dummy);
}
br.close();
}
catch (IOException e) {
System.err.println(e);
}
for(Field f : list){
System.out.println(f.code + " " + f.title +
" " + f.country + " " + f.description);
}
}
static class Field {
String code;
String title;
String country;
String description;
}
}
for input:
"11","Title 11","USA","Title 11 description contains no newlines"
"12","Title 12","USA","Title 12 description contains no newlines"
"123","Title 12","USA","Title 123 description contains no newlines"
"1234","Title 123","CAN","Title 1234 description contains no newlines"
"12345","Title 1234","MEX","Title 12345 description contains no newlines"
produces output:
11 Title 11 USA Title 11 description contains no newlines
null 12 Title 12 USA
null null 123 Title 12
Title 123 description contains no newlines null null 1234
CAN Title 1234 description contains no newlines null null
Title 1234 MEX Title 12345 description contains no newlines null
------------------------------------------------------------------
for input:
"11","Title 11","USA","Title 11 description contains no newlines"
"12","Title 12","USA","Title 12 description contains
two newlines"
"123","Title 12","USA","Title 123 description contains no newlines"
"1234","Title 123","CAN","Title 1234 description contains no newlines"
"12345","Title 1234","MEX","Title 12345 description contains no newlines"
produces output:
11 Title 11 USA Title 11 description contains no newlines
null 12 Title 12 USA
null null null null
null null null null
null null null null
null null null
Title 12 USA Title 123 description contains no newlines null
Title 123 CAN Title 1234 description contains no newlines null
Title 1234 MEX Title 12345 description contains no newlines null
--------------------------------------------------------------------