« March 26, 2005 | Main | March 30, 2005 »

Sunday, March 27, 2005

Linux: Under Pressure

This long week-end I've been running an experiment on the main Fourmilab server, enabling negotiated compression of outbound text files. The idea is simple; if the browser (or other user agent, such as a search engine's Web crawler) indicates it understands compressed files in its request headers, the Fourmilab Web server will dynamically compress the file and send it in compressed form. This, along with interim logging of compression, is accomplished with the following directives in the Apache server configuration file: 
    AddOutputFilterByType DEFLATE text/html text/plain text/xml
and: 
    DeflateFilterNote Input instream
    DeflateFilterNote Output outstream
    DeflateFilterNote Ratio ratio

    LogFormat '"%r" %{outstream}n/%{instream}n (%{ratio}n%%)' deflate
    CustomLog /server/log/httpd/deflate_log deflate
This is considered a conservative configuration, but in fact the degree of compression it achieves on my site is almost precisely the same as a far more complex set of exclusion rules, since almost every file whose type doesn't match the DEFLATE list is already compressed.

The log entries produced by the CustomLog definition look like the following: 

"GET /yoursky HTTP/1.1" 232/320 (72%)
"GET /earthview/learth.map?114,91 HTTP/1.1" 295/409 (72%)
"GET /earthview/vplanet.html HTTP/1.1" 2619/6506 (40%)
"GET /robots.txt HTTP/1.0" 497/1349 (36%)
"GET /webtools/base64/ HTTP/1.1" 2517/5641 (44%)
"GET /skyscrsv/ HTTP/1.1" 4041/12112 (33%)
"GET /autofile/www/section2_83_6.html HTTP/1.1" 1990/5433 (36%)
"GET /earthview/vplanet.html HTTP/1.1" 2619/6506 (40%)
Note that the last three fields in each record give the compressed length, the original length, and the ratio of the compressed length to the original length, which isn't the usual "percent compression" you're used to; here, a smaller value indicates better compression.

Posted at 01:06 Permalink