Faking User Agent with Wget
Written by Steve Lake
From http://www.raiden.net
Posted on: Jan 27, 2010 at 07:49am
Section: Tutorials
Printer Friendly Version
Legacy URL
A good portion of the Linux community who are familiar with the command line are also very familiar with a powerful tool known as Wget. It’s sort of an FTP/web download program for grabbing files off the web. Combined with other programs like screen, it’s invaluable for a wide number of things. But I bet those of you who use it didn’t know that you cold also use it to fake the user agent information, did you?
Well, you can, and the trick is amazingly simple, and yet both hugely practical, and life saving in some cases. Case in point, there’s a couple sites I go to where I need to download some stuff, but the site prevents downloads, save with their own proprietary software, or a specific browser. (Internet Explorer for example.)
A simple way around this is to use Wget’s header information feature. Browsers and many 3rd party programs use a system called “User Agent”, which is little more than a simple, short text string sent by the program to identify itself to the server. Case in point, this is the user agent string for apt-get (Ubuntu APT-HTTP/1.3) and this is the string for Firefox 3.5. (Mozilla/5.0 (X11; U; OpenBSD i386; en-US; rv:1.9.1) Gecko/20090702 Firefox/3.5) (A good list of most known user agent strings is available here)
To activate the user agent string in wget, you will need to type “-U” followed by the user agent string encapsulated in quotes. Here’s an example using the Apple Quicktime 7.6.2 user agent (for those of you who like to download and watch Apple trailers):
wget -U “QuickTime/7.6.2 (qtver=7.6.2;os=Windows NT 5.1Service Pack 3)”
After that, you simply include the rest of your command line, such as file to be downloaded, and any other pertinent information. It’s surprising how few people actually know and use this trick. But with more sites excluding browsing and/or downloading by “unapproved” browsers or download agents, the user agent string is becoming an important part of thwarting those efforts, while continuing to maintain your freedom to choose what method you use to download or surf with.
I hope this helps. And give it a try sometime on an otherwise difficult site. You might be surprised at how useful it turns out to be.
