Using the .htaccess File & .Htaccess Blocking
Using the .htaccess File
Web designers often ask how to handle redirects or to password protect directories. The .htaccess file can do these things and more. For this article Miraz Jordan has collated various tips mentioned on the Wisewomen mailing list, and from several other sources.
What the .htaccess file can do
- If you’re reorganising your site and moving pages around, you can use the .htaccess file to redirect visitors from the old page to the new one.
- Another function of the .htaccess file is to allow you to serve up pages which include PHP or Server Side Includes (SSI) but whose file name still uses the .htm or .html extension.
- Allow or prevent directory browsing.
- Because the server should check the .htaccess file before anything is delivered to the client, you can use it to password protect parts of your site.
- You can also block various bots with the .htaccess file — for example, you can keep some spammers out, or prevent search engine spiders from indexing your images folder.
You can read the definitive information on .htaccess files at Apache.org.
Reveal hidden .htaccess files
The filename for the .htaccess file begins with a . (dot). This causes it to be hidden on many Operating Systems. You may have trouble finding or working with such hidden files.
On the server
Set your FTP software to show files beginning with a dot, or access the file through your server’s Control Panel — File Manager.
Some FTP software, such as Interarchy, allow you to edit files directly on the server. Select a file and choose Listing menu — Edit with. To get Interarchy to display files whose name begins with a dot visit Preferences — transfers and uncheck Ignore .files.
On Mac OS X
If you download the file to a Mac running OS X you will have trouble finding it as the Mac hides files whose filenames begin with a dot. You can edit hidden files on a Mac though, provided you can find them.
A standard Finder search can find hidden files but you may find a tool such as Tinkertool or Pathfinder useful. Set the preferences to show hidden files. Be careful not to move, delete or edit any other hidden files unless you know what you’re doing as otherwise you can break things.
On Windows
[Thanks to Susan from the WW list for this information and screen shot.]
If you download the file to a computer running Windows you will have trouble finding it as Windows hides files whose filenames begin with a dot. You can edit hidden files on Windows though, provided you can find them.
- Open File Explorer
- Go to Tools — Folder Options… and click on the tab “View”.
- Make sure that the option “Show hidden files and folder” is checked.
Be careful not to move, delete or edit any other hidden files unless you know what you’re doing as otherwise you can break things.
Create a new .htaccess file
Use a plain text editor such as Notepad (not Word) or TextEdit to create a document called htaccess.txt on your computer. Don’t add the dot at the start of the filename or it may become invisible. Upload that file to your server, then rename it to .htaccess. Make sure you add the . (dot) at the start of the file name and remove the .txt extension. Be sure to upload it in ASCII format, not Binary.
FTP software such as Interarchy may allow you to directly create a new file on the Server. See the Listing menu — Create File.
The .htaccess file can go in the root directory and it will then also affect all directories below it. Each other directory may also have its own .htaccess file.
Redirects
Let’s say you’ve moved a file or directory, or both: www.example.com/training/test.html is now located at www.example.com/learning/newtest.html. You want visitors to end up at the correct page, even if they use the old address.
Open the .htaccess file and enter this on one single line:
redirectpermanent /training/test.html http://www.example.com/learning/newtest.html
Note that this is search engine-friendly, too. Search engines will change the links in their index to the new link on the basis of the redirectpermanent directive. More info: httpd.apache.org/docs/mod/mod_alias.html#redirectperm.
Parse PHP in .html files
Perhaps you have have been learning PHP and want to include some commands in existing html files. The books will tell you to rename those files with a .php extension. Rather than renaming all your files you can use the .htaccess file to tell the server to allow html files to include php. More info: www.desilva.biz/php/phpinhtml.html.
Allow SSI in .html files
Tip provided by Deb from the WW list and rewritten by Miraz.
You may be on a server that requires files to end in .shtml for Server Side Includes. Here is a tip if you do not wish to use the .shtml extension, or if you have added Server Side Includes to existing .htm or .html files. Add the following to your .htaccess file:
AddType text/html .shtml .shtm .htm .html
AddHandler server-parsed .shtml .shtm .htm .html
You can add whichever extensions are relevant.
Files which must be parsed by the server before being displayed may not load as quickly as standard pages. If you use this code in your.htaccess file, the server will parse all .html and .htm pages, including those that do not contain any SSI includes. This could significantly slow the loading of pages which do not use the includes. Be cautious if your pages hold extensive graphics. [Deb mentioned she had not seen slower load times.]
Allow or prevent directory browsing
Tip provided by Deb and rewritten by Miraz.
A good way to increase security on your site involves the .htaccess file. You can override server settings to allow or prevent directory listing.
Prevent directory browsing
Suppose you have a directory which doesn’t have a default file (index.html), such as a folder of images, for example. A visitor may enter an address ending with a / and see a list of all the files in the directory.
You can prevent directory browsing by adding this line to your .htaccess file:
IndexIgnore */*
Allow directory browsing
There may be times when you want or need to allow visitors to browse a directory. For example, you may need to allow access to files in a directory for downloading purposes on a server that is configured to not allow it.
Many servers are configured so that visitors cannot browse directories. In that case visitors will not see the contents of the directory but will instead get an error message.
You can override the servers settings and allow directory browsing with this line:
Options +Indexes
Password protection
Tip provided by Sheila from the WW list and rewritten by Miraz.
You can password protect individual files with .htaccess. It’s usually done directory-wide with <Directory> but you can use <Files> to specify a single file:
<Files secret_file.html>
AuthType Basic
AuthName "Team Page"
AuthUserFile path_to_password_file
Require user username
</Files>
This only protects the single page; all the files that it is linked to are not protected. More detailed information: httpd.apache.org/docs-2.0/howto/auth.html.
Block various bots
[Stephanie of Glenfinnan Web Hosting supplied this information which was rewritten and amplified by Miraz with reference to: www.webmasterworld.com/forum13/687.htm. Stephanie said: You can block various bots with the .htaccess file, Gini and someone else [from the WiseWomen list] posted their list of spam and spider bots [6 Kb txt file] awhile back
.]
Spambots frequently visit our sites for various nefarious purposes. You can block them like this:
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^.*Ants.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*attach.*$ [OR]
[Many more similar lines …]
RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [OR]
RewriteRule /* http://www.fbi.gov [L,R]
The last line sends them off to the FBI, but you could use any URL you wish. Alternatively you could just send them to a standard error page:
RewriteRule ^.* - [F]
Summary
The .htaccess file is a very powerful file which can keep visitors away or send them elsewhere, protect pages and directories with a password, allow you to include PHP and SSI within pages which have a .html extension and prevent or allow directory browsing. Handle the .htaccess file with care but use this information as a starting point for further exploration.
Be sure to always test your site after making changes to the .htaccess file, and have fun experimenting.
User Blocking And Access Prevention Using .htaccess Files
The htaccess file is a wonder of website coding. It can do a great deal of things that we might otherwise be dependent on our host providers to do, in many cases charging admin fees to add the appropriate features to our accounts or to set it up in the first place. Unfortunately, though, the use of htaccess files is not always permitted by host providers because done incorrectly it can cause a security risk. Not only is this a problem for your site but it can also be a problem for other sites hosted on the same server. That said it is not impossible to find a host that offers the ability to use htaccess files and if you want to use htaccess files then you should look for a host that allows it.
Blocking Users
We already covered the use of htaccess to password protect pages and to use custom error pages but these are only two uses of the htaccess file. It is also possible to block users based on their IP addresses, IP ranges, referring domain, and block bots or automated software from accessing your site or folders. While the uses of blocking specific IP addresses are relatively limited the htaccess file does prevent an incredibly simple but effective method of doing just that should the need arise.
Blocking Users By IP Address
Blocking users by IP address or IP range are both very simple things to achieve and only require a couple of htaccess commands to be added to the appropriate htaccess file.
To block a single IP address add the following lines:
order allow,deny
deny from 111.22.3.4
allow from all
Blocking Users By IP Range
Because the commands are read and actioned from the top down it is important to close the block by using the allow function. Without doing this it can cause the htaccess script to keep running, slowing your site and your server. The code above would only block an individual with the IP address 111.22.3.4. However, the following code would block a range of IP addresses as we will discuss;
order allow,deny
deny from 111.22.3.
allow from all
Combining IP Blocking Methods
Instead of blocking an individual user this would block all users with an IP address beginning in 111.22.3 and allow all other users access to the folder. It is possible to combine these two methods in order to block a single user and a range of IP addresses using the following as an example:
order allow,deny
deny from 111.22.3.4
deny from 222.33.4.
allow from all
Blocking Users From Multiple Domains
Blocking users according to IP address or IP range is easy as you can see from these examples. A slightly more complex procedure, although still relatively easy, is to block users depending on the referrer that sent them. For example it is possible to block every user that visits your site from a link on the website www.blockreferrer.com and www.blockreferrer2.com by adding the following code to your htaccess file in the usual manner.
RewriteEngine on
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} blockreferrer\.com [NC,OR]
RewriteCond %{HTTP_REFERER} blockreferrer2\.com
RewriteRule .* – [F]
Obviously this blocks all users from visiting your site having followed a link from either of these two domains. This also includes sub-domains such as mail.blockreferrer.com or example.blockreferrer.com.
Blocking Users From A Single Domain
To block users visiting your website from a single referrer simply remove the second RewriteCond line (in this case the blockreferrer2\.com) line and remove the “OR” from the end of the first line. In our example your amended file would now read:
RewriteEngine on
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} blockreferrer\.com [NC]
RewriteRule .* – [F]
Blocking Bots And Offline Scripts
As well as blocking users it is also possible and often beneficial in terms of bandwidth or traffic resource to block bad bots and site scrapers. Below we provide the code to do this but obviously you will need to find the names of the bots to block.
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^BadBot1 [OR]
RewriteCond %{HTTP_USER_AGENT} ^BadBot2
RewriteRule ^.* – [F,L]
In the above examples any users that are blocked because of the code you enter will be redirected to your 403 error page. Because they aren’t taking up your valuable bandwidth this can improve the speed of your website loading time for your other, genuine users. It may also prevent your server from completely failing, or crashing, thanks to an overload. By blocking bad bots it is also possible to prevent email harvesting software from taking your email address and sending you regular spam. Again, this is not only irritating but causes your mailbox usage to increase dramatically.
Securing Your htaccess File – Stop Prying Eyes
The process of preventing your htaccess file from being read is an incredibly easy one that requires four short lines of code to be placed into the htaccess file itself. The code required is as follows:
<Files .htaccess>
order allow,deny
deny from all
</Files>
This is another deny function similar to those discussed earlier in the article but the first line tells htaccess that you wish to block all users (deny from all) from accessing the .htaccess file (files .htaccess).
What Is Hotlinking?
Other methods exist, through the use of the htaccess file, to prevent the unwarranted usage of your bandwidth. Hot linking is an activity undertaken by people wanting to display images that appear on your website without saving and uploading the images to their own server. Leeching on your bandwidth in this way means that every time the image is downloaded to a user’s browser you essentially pay the price in bandwidth and potentially in money.
Preventing Hotlinking With A Denial
There are two methods to do this. One is a simple prevention of hot linking, essentially banning all domains except your own from showing this content and instead displaying a broken image icon. The other enables you to display an alternative image whenever hot linking is detected. While this may still use your bandwidth initially, the perpetrator of the hot linking is unlikely to continue displaying the image once he or she realizes that it isn’t what they were intending.
rewriteEngine on
rewriteCond %{HTTP_REFERER} !^$
rewriteCond %{HTTP_REFERER} !^http://(www\.)?your-website.com/.*$ [NC]
rewriteRule \.(gif|jpg|js)$ – [F]
Replacing Images In Hotlinks
This version displays the broken link icon or a similar icon. The last line is used to determine the types of file that this works for. In this case gif, jpg, and js files cannot be hotlinked from outside the your-website.com domain. In order to replace an image and display an alternative image, use the following code in your htaccess file:
RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?your-website.com/.*$ [NC]
RewriteRule \.(gif|jpg)$ http://www.your-website.com/nohotlink.jpg [R,L]
Again, you need to replace the your-website.com with your own domain and the image path on the bottom line with the path to the replacement image.
Preventing The Display Of All Your Directory Contents
Preventing access to an index of images or files is a good idea for the sake of site security and often because you simply don’t want people to browse through the images on your site out of context. Many hosts provide this as a matter of course, with all indexes being invisible to visitors. However, this may not be the case with the host you are currently using. As long as you do, instead, have access to your htaccess file then it needn’t be a problem and is in fact one of the easiest htaccess commands to include.
IndexIgnore *
Adding this to the htaccess file in any directory will prevent the index for that directory, and all subdirectories, from being displayed. This means that placing it in the root folder of your site will prevent all indexes from being displayed.
Limiting Directory Viewing
By adding a limiter after the wild card symbol “*”it is possible to specify the types of file you wish to restrict within the index.
IndexIgnore *.jpg
Enabling The Display Of Directory Contents
This means that any .jpg files within that directory and subdirectories will not be listed when the directory listing is shown. Sometimes you may want to display the contents of a particular subdirectory but not the parent directory. Include either of the two above examples in the directory itself. In the subdirectory that you wish to display include an htaccess file with the following command:
Options +Indexes
Conclusion
For many people it is a surprise to learn that the simple htaccess file has so many uses but we have really only scratched on the surface of its potential. As well as these simple commands it is possible to command redirects, add MIME types, and enable Server Side Includes (SSI). If you want a greater degree of control over your own website than you currently have then htaccess provides an excellent way to go about this.
The most important thing is to check that your host allows you to amend or edit your htaccess file, or even to add one in the first place. Used incorrectly they can cause a security threat to your website, the server, and other websites hosted on your server. However it is also possible to reduce the amount of spam you receive, cut down your bandwidth usage, prevent others from accessing your site, and much more. Htaccess commands are refreshingly simply to include even for the complete beginner.
