Kurt McKee

lessons learned in production

Hey there! This article was written in 2007.

It might not have aged well for any number of reasons, so keep that in mind when reading (or clicking outgoing links!).

Dealing with hotlinking

Posted 21 April 2007 in bandwidth-theft, hotlinking, prevention, and web

Hotlinking

Using content hosted on another person's site, usually in the form of an image or audio file.

One of my friends recently informed me that someone was hotlinking an image of a 2-year-old family member on a Myspace profile. He was not pleased, and swapped the image out with an enormous image of the Firefox logo (roughly 7,000 by 6,500 pixels large). Despite destroying the page layout, the image was never removed. After several months he got fed up and replaced the image again, this time with a jarringly explicit image.

Do not hotlink to other people's content.

Nobody likes paying for bandwidth theft. But how can hotlinking be defeated? I've seen several options:

Cookie checking

When the user visits a webpage on exemplo.com, a cookie is set. If an image is requested, exemplo.com checks for the existence of the cookie. If the user has that cookie, it's likely that they're visiting a webpage on exemplo.com that contains the image being requested. If the user doesn't have that cookie, it's likely that the image is located on linktheif.com.

Referer checking

When content is requested on the internet, most browsers automatically send a bit of information called the `Referer` header. If you were to click on the Wikipedia link at the top of this article, Wikipedia would know that you're coming to Wikipedia from kurtmckee.org. In much the same way, if an image is requested from exemplo.com, in most cases exemplo.com will know whether the image is on a webpage on exemplo.com or linktheif.com.

Rolling URLs

Each time a user visits exemplo.com, the image URL changes. The URL will expire after a certain amount of time, such as five minutes. As an example, if the user visits the webpage on July 18th at 1:37p, the image URL might be `/07181337/happy.jpg`. When the image is requested, exemplo.com can easily check whether the image has been requested within a certain amount of time.

The problem with all of these methods is that they assume that the content can only validly be requested from exemplo.com, which isn't the case nowadays. Some users (and even some mobile internet providers!) automatically block the Referer header, for instance. Further, feeds have freed content to be viewed in many, many places and at any time. As such, the user is unlikely to have the correct token (a cookie or referer header) and will probably not be viewing the content within the window of opportunity created by rolling URLs.

Automated content blocking will lock out legitimate users.

I discussed the problem with Allan, and he suggested that, rather than blocking the content, the content could still be delivered but be flagged for administrator review. Perhaps after a rolling URL is visited 20 times after its expiration, it is flagged for review. Perhaps after a Referer pointing to an external site has requested the content 20 times, the content could be flagged for review and possible content blocking.

For my own purposes, I would probably employ Referer checking. For a site that hosts wallpaper downloads or similarly desirable images, some combination of rolling URLs and Referer checking would probably be a better solution.