From Gentoo Linux Wiki

[edit] Introduction

This document describes how to install the netfilter and iptables kernel modules and configure them to successfully thwart brute-force SSH attacks.

Rules from http://www.debian-administration.org/articles/187 have been used in this article.

[edit] Installation

Enable the following kernel modules:

[*] Networking support 
    Networking options  ---> 
    [*] Network packet filtering framework
       [ ]   Network packet filtering debugging
       [*]   Advanced netfilter configuration
             Core Netfilter Configuration --->
             <M> Netfilter connection tracking
             <M> Netfilter Xtables support
             <M> "state" match support

             IP: Netfilter Configuration --->
             <M> IPv4 connection tracking support 
             <M> IP tables support 
             <M> recent match support
             <M> Packet filtering

Be sure to recompile your kernel and install the modules:

gmake; gmake modules_install

Add the following to /etc/modules.autoload.d/kernel-2.6:

nf_conntrack
nf_conntrack_ipv4
xt_state
ipt_recent
ip_tables
iptable_filter
xt_tcpudp

Emerge IP Tables:

emerge iptables

rc-update add iptables default

[edit] Deploy

Load the modules:

echo nf_conntrack nf_conntrack_ipv4 xt_state ipt_recent \
ip_tables iptable_filter xt_tcpudp | xargs -n1 modprobe

Load the rules (Note the ethernet interface, eth0 in this case, may need to be changed to match your configuration):

iptables -A INPUT -i eth0 -p tcp -m tcp --dport 22 -m state \
--state NEW -m recent --update --seconds 30 --hitcount 1    \
--name ssh_attempt --rsource -j DROP 

iptables -A INPUT -i eth0 -p tcp -m tcp --dport 22 -m state \
--state NEW -m recent --set --name ssh_attempt --rsource 

These rules will allow one ssh connection per remote host every 30 seconds and drop subsequent attempts, which is sufficient to cause a timeout in automated ssh bruteforce attack scripts. It's also generous enough for the average user.

[edit] Configuration

Set SAVE_ON_STOP to no in /etc/conf.d/iptables to prevent accidentally wiping out your rules:

 # Save state on stopping iptables
 SAVE_ON_STOP="no"

Save the current IP Tables rules that were manually set during the Test step above:

 /etc/init.d/iptables save

Start the IP Tables service:

 /etc/init.d/iptables start