Downloading using wget via a proxy server

Sadly i am unable to update fedora 9 via the proxy server in our college and hence am still stuck using Firefox 3 Beta 5. I had some problems saving the pdf files once they opened in evince in the web browser and hence decided to use wget for the downloads.

Downloading using wget :

//First set the http_proxy variable to the necessary address

export http_proxy=’http://ur.proxy.server:port’

//and then in the wget command line execute with an option –proxy=on

wget –proxy=on http://whatuwant

u can use other options as –proxy-user and –proxy-password for password protected proxies

u can switch of the proxy with –proxy=off

Jai Gurudev

EDIT: Once this proxy variable is set all your apps which require proxy server settings can run from the CLI. After this i was able to update my system via yum after doing a bit moving…. Theres a GUI too for this .

In Fedora : Places => Preferences => Network .

Creating a mirror of a website in Linux.

GNUs wget command line program is a very popular for downloading single files from a server. It is much more powerful than that and offers some really cool features.One of them is the mirror feature. Suppose you want to mirror a website say: http://www.abc.com . In its very basic form you can use it as follows

$ wget -m http://www.abc.com

However , this can be troublesome as the links on the mirrored website will be pointing to the actual links and not on the relative links. to fix this add the option -k to the command to fix this as follows:

$ wget -mk http://www.abc.com

Another issue is of bandwidth. You are going to put some strain on the remote server if your planning to mirror a website directly.Hence one of the ways in which you can purposefully slow down your download is by using -w option as follows:

$ wget -mk -w 20 http://www.abc.com

This will delay the requests to the server by 20 seconds.add the suffix m for delay in minutes,d for delay in days.

Rsync is equally good in mirroring websites … but you need ssh access on the remote server. By using wget u can mirror the public files on the remote server.