[OT] LWP::UserAgent and the TE: header

Nicholas Clark nick at ccl4.org
Fri Jun 20 11:51:15 BST 2008

So, by default, LWP::UserAgent sends headers including:

TE: deflate,gzip;q=0.3
Connection: TE, close
User-Agent: lwp-request/2.08

Both of these are redundant when talking between two of our internal servers.
I can see how to get rid of the User-Agent - pass an option agent=>"" to
LPW::UserAgent->new(). (And even better, RPC::JSON will let me do that)

What I can't see is how to eliminate TE:
The Transfer Encoding header is automatically added when Compress::Zlib is
around. It is. But it isn't doing anything useful here. So I'd like to get
rid of it.

Problem is that there is an option to turn it off in the Net::HTTP constructor:

=item $s = Net::HTTP->new( %options )


  SendTE:          Initial send_te attribute_value

(and there's actually also "HTTPVersion" so we could set that to 1.0)

but LWP::UserAgent creates its protocol object like this:

$protocol = eval { LWP::Protocol::create($scheme, $self) };

which is this method in LWP::Protocol:

sub create
    my($scheme, $ua) = @_;
    my $impclass = LWP::Protocol::implementor($scheme) or
	Carp::croak("Protocol scheme '$scheme' is not supported");

    # hand-off to scheme specific implementation sub-class
    my $protocol = $impclass->new($scheme, $ua);

    return $protocol;

which *only* allows us to pass in the User-Agent string (and the scheme).
So I can't see a way to tweak the protocol object, either at construction
or call time, to change these things.

Is there a way, or would it involve dropping the use of LWP::UserAgent and
building the entire protocol conversation from lower level LWP modules?

Nicholas Clark

More information about the london.pm mailing list