Uh, this is really HTTP not HTML, but let me pitch in with one place where
there's a really good reason for an exception. In the Open Text Web Index,
(http://www.opentext.com:8080) the job of analyzing a query, finding matching
pages, and generating readable summaries is quite tricky, and has to be done
very fast (and getting faster all the time). If we had to do all this,
stash the results (which are highly unpredictable in size) in a buffer or
file, compute the content-length, and then send it, there would be two bad side
effects:
1. We'd have to do more processing, which would slow the whole thing down, and
2. lots of people have browsers that show results as they show up. We
make their lives pleasant by emitting the header when we start processing
the query, the matching-page totals when we know them, and the
result summary as we compute it.
Luke is right that the consumer has no way of telling that we haven't just gone
into an infinite loop; but in this case, the price of self-defence against
idiots is too high.
(We had an amusing bug in the early life of the index which caused our
httpd to tell all clients that the content-length was in all cases 241;
the reactions of the browsers, when they received more (or less) data were
instructive and amusing).
Cheers, Tim Bray, Open Text Corporation (tbray@opentext.com)