Bypassing a corporate Web filter when using the command line

or ‘How to bypass a corporate Web filter and download YouTube videos via the command line’

One of the offices where I work uses a Web filter to block access to certain sites, such as YouTube. However, sometimes it is necessary to view blocked Web sites for work purposes. For example, these days a lot of companies or individuals post product reviews on YouTube that are useful for work purposes. In such cases I have used Tor to access the blocked sites in a Web browser such as Firefox, Chrome, Konqueror etc. See my post How to install and use Tor for anonymous browsing or to access country-restricted content from another country for details of how to set up and use Tor with a Web browser.

But sometimes I need to access blocked Web sites from the command line. For example, today I needed to download a YouTube video for work purposes, and I wanted to use youtube-dl to do it. The solution was simple…

First I launched vidalia and polipo as explained in the above-mentioned post on Tor, then I launched another Konsole/Terminal window and entered the commands shown below:

$ # First find out what resolutions are available for the video I want to download:
$ youtube-dl -F https://www.youtube.com/watch?v=T3Rr4CUoTSQ
Setting language
T3Rr4CUoTSQ: Downloading webpage
T3Rr4CUoTSQ: Downloading video info webpage
T3Rr4CUoTSQ: Extracting video information
[info] Available formats for T3Rr4CUoTSQ:
format code extension resolution note
140 m4a audio only DASH audio , audio@128k (worst)
160 mp4 192p DASH video
133 mp4 240p DASH video
134 mp4 360p DASH video
135 mp4 480p DASH video
136 mp4 720p DASH video
17 3gp 176x144
36 3gp 320x240
5 flv 400x240
43 webm 640x360
18 mp4 640x360
22 mp4 1280x720 (best)
$ # Now try to download the video at the resolution I want:
$ youtube-dl -f 22 -o Clevo_W230ST_overview.flv https://www.youtube.com/watch?v=T3Rr4CUoTSQ
Setting language
T3Rr4CUoTSQ: Downloading webpage
T3Rr4CUoTSQ: Downloading video info webpage
T3Rr4CUoTSQ: Extracting video information
ERROR: unable to download video data: HTTP Error 403: Forbidden

As you can see above, the corporate Web filter blocked youtube-dl from downloading the video.

So I informed the shell session about the local HTTP proxy (polipo) running on my laptop, by assigning and exporting the environment variable http_proxy using the following syntax:

export http_proxy=http://server-ip:port/

which in my case meant the following (refer to my article on Tor):

$ export http_proxy=http://127.0.0.1:8123/

and then I was able to download the video from YouTube despite the corporate Web filter:

$ youtube-dl -f 22 -o Clevo_W230ST_overview.flv https://www.youtube.com/watch?v=T3Rr4CUoTSQ
Setting language
T3Rr4CUoTSQ: Downloading webpage
T3Rr4CUoTSQ: Downloading video info webpage
T3Rr4CUoTSQ: Extracting video information
[download] Destination: Clevo_W230ST_overview.flv
[download] 100% of 100.23MiB in 05:50
$

Useful Reference: How To Use Proxy Server To Access Internet at Shell Prompt With http_proxy Variable

About these ads

About Fitzcarraldo
A Linux user with an interest in all things technical.

3 Responses to Bypassing a corporate Web filter when using the command line

  1. great article. just found your blog and love it.
    can you write on how to use youtube-dl on android terminal emulator.
    i ‘ve got python for android and sl4a installed, but don’t know how to install youtube-dl.
    Thanks.

  2. Vasco says:

    If you have a ssh server to connect to:
    ssh -fND some_port user@server
    can be used instead of Tor.

    • Vasco says:

      In this case, the protocol is socks5 instead of http
      export http_proxy=socks5://127.0.0.1:some_port

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 51 other followers

%d bloggers like this: