By default, a linux machine will drop traffic that’s not destined for an IP bound locally. If you’re planning on receiving traffic not ultimately destined for you (ie: you’re behaving as a router along the path to the ultimate destination), then you’ll need to flip a couple flags in /etc/sysctl.conf to allow traffic forwarding:

net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1

Reload sysctl:

sysctl -p

Add your NAT rule to the iptables prerouting chain in the nat table. For example, maybe you’re plugged into both a public and private network, and you want to forward traffic to your public IP on tcp/80 to the web server on your private network. Note that this particular step is only necessary if the public network needs to be able to reach the webserver on our private network. You can skip the PREROUTING chain altogether if you only want to allow egress traffic from the servers on your private network, and block all ingress.

When the webserver on 10.0.0.12 tries to respond to the public address, that’s going to be problematic. Since the client originally connected to us and not the webserver, we can’t have the webserver being visible in the response – this would cause a TCP RESET. The return traffic will definitely hit us though, ’cause we’re that webserver’s default gateway. As such, we can correct this by masquerading as the source of this routed traffic.

Make sure we allow the relevant traffic as well. Since we’re routing port 80 to the webserver, we’ll want port 80 open on the INPUT chain. Since the webserver sends all its traffic through us, we need to allow all of it out via our FORWARD chain – unless we’re doing egress filtering here, in which case you’d glorify your FORWARD chain.

Oh and one more thing. If you want to NAT traffic that’s originating from this router server in the first place, then you need to add your rules to the OUTPUT chain in the nat table. For example, say we’ve setup a SOCKS proxy on port 8080 and we want to force all our outgoing HTTP traffic to use that SOCKS proxy. Here’s how you’d do that:

This can be really useful in cases where an application is trying to connect to a remote resource, but that resource has changed IPs and it’s going to take you 3 weeks to correct every single instance of the old IP within your code. Instead of being offline for those three weeks, throw up a NAT rule so attempts to reach the old IP are NAT’d to the new IP instead:

Here’s how a sender verification callout works. Let’s say I’m an innocent mail server, and some jerk is trying to send me a message. There’s a very strict procedure for how mail servers talk to each other. Here’s an example, with the sender’s (1.1.1.1) stuff in red, the receiver’s (2.2.2.2) stuff in blue, and an explanation of each part in black:[Establish TCP connection to receiver]220 server.receiving.com SMTP Sendmail Fri, 05 Mar 2010 15:47:18 -0600
220-We do not authorize the use of this system to transport unsolicited,
220 and/or bulk e-mail.
This is the receiver saying hi. If the receiver is broken, or if the IP address of the sender is considered “bad” for some reason, then the receiver won’t send a “220”. The “220” is a status code meaning “Hi let’s talk”.EHLO server.sending.com
The sending server introduces itself. “EHLO” is part of SMTP protocol. “server.sending.com” is the hostname of the server sending the message. The receiver can now run sender verification on the sending server’s hostname. This is not a “sender verification callout” – it’s just “sender verification”.250-server.receiving.com Hello server.sending.com [1.1.1.1], pleased to meet you
250-PIPELINING
250-SIZE 25000000
250-8BITMIME
250 HELP
Sender verification passed. Receiving server tells the sender some basic guidelines about how it likes to communicate.MAIL FROM: <[email protected]> SIZE=1405
Sender tells the receiver what mail account created the message. The receiver can now run a sender verification callout.250 Ok
Callout must have checked out. Sender’s mail address checks out, and transaction can proceed.RCPT TO:<[email protected]>
Sender reports the email address to which it’s trying to send.250 Ok
Receiver acknowledges that it will, in fact, accept mail addressed to that account.DATA
Sender says it wants to send the actual message now.354 Start mail input; end with <CRLF>.<CRLF>
Receiver says to go ahead, and says that after the entire message has been sent, indicate that the message is done by sending a period (.) on a line by itself.To: [email protected]
Subject: Test
Hey dude, those lines up there are headers, and this crap right here is the message body.
.
The message. Headers come first. The period by itself indicates there’s no more to the message.250 Ok: queued as 1234567890
Receiver indicates it got the message, and stuck it in the mail queue.QUIT
Sender says goodbye.221 Bye
Receiver says goodbye.

So that’s how mail servers talk to each other when everything goes smoothly. I kind of skimmed over the actual sender verification, and the sender verification callout. So, using the above exchange as an example, here’s what happens behind the scenes.

Sender Verification: The receiving server knows the IP address of the sending server as soon as the TCP connection is established (obviously – there are two IPs in the conversation, and one’s itself). Once the sending server sends the “EHLO” line, then the receiver knows the hostname of the sending server. Sender verification is when the receiving server performs a reverse DNS lookup on the IP address of the sender, and makes sure the reverse DNS response matches the hostname in the “EHLO” line.

You can check the reverse DNS of the sender’s IP like so:root@server[~]# host 1.1.1.1
1.1.1.1.in-addr.arpa domain name pointer server.sending.com.
root@lucient[~]#

Since reverse DNS for the IP is “server.sending.com”, and the EHLO line reported the hostname “server.sending.com”, sender verification passes.

Sender Verification Callout: Sender verification related to the hostname of the sending server. The callout relates to the specific mail account sending the message. In the “MAIL FROM:” line, the sender is claiming that an email account “[email protected]” exists, and that it created this message. The receiver performs a callout to make sure this account does in fact exist. To do this, the receiver pretends like it’s going to send an email to that account, and if it’s able to send a message, then the callout passes. The important thing to note here is that the receiver doesn’t necessarily connect back to the sending server. Here’s what the receiver does to perform a callout:

So if I try to send a message to “[email protected]”, with the server at “5.5.5.5” take the message? If so, then this is probably a valid account. If not, then it’s probably not a valid email account, and someone could very likely be trying to send me spam. Let’s find out:Connected to server.mailhost.com (5.5.5.5).
220 server.mailhost.com
EHLO server.receiving.com
250-server.mailhost.com Hello server.receiving.com [2.2.2.2], pleased to meet you
250-PIPELINING
250-SIZE 25000000
250-8BITMIME
250 HELP
MAIL FROM: [email protected]
250 Ok
RCPT TO: [email protected]
250 Ok

Apparently “[email protected]” is a valid email address. That means the sending account passes sender verification callout. At this point “server.receiving.com” disconnects from 5.5.5.5. Notice how the callout didn’t actually send a message – it just made sure that it’s possible to send mail to the address.

Now, if the account “[email protected]” is fake, then the DNS lookups for “sending.com” or “server.mailhost.com” might have failed, or a connection to 5.5.5.5 port 25 (SMTP) may have timed out, or the server at 5.5.5.5 may have just refused to accept a message for that account, with an error message like this:550 No such domain at this location ([email protected])

There are any number of ways the callout could have failed. What you need to know is that unless everything goes smoothly during the callout, the original message from server.sending.com (1.1.1.1) will be rejected by server.receiving.com (2.2.2.2) on the basis of “sender verification callout” or “sender verify callout”.

Moral of the story: Don’t send mail *from* an email address unless you can send mail *to* that same address.

Buy a domain.Any registrar will do.
If someone else is going to host the DNS for your domain, skip to step 4. Otherwise, continue from here. The next few steps are how you make sure the registrar knows your server exists as a nameserver. Until the registrar is aware of your server’s existence, no domains can tell the registrar that your server hosts their DNS.

Create nameservers. You do this at the same registrar where you bought your domain. Login to your account at your registrar. This can be tricky, because registrars can’t seem to agree on what to call this. I would say you want to “register nameservers”. GoDaddy calls it creating “host records”. I don’t know what other registrars call it, because I use GoDaddy. The main idea is that you want to pick names (we’ll use ns1 and ns2.rootmypc.net as an example) and associate those names with IP addresses. You should have control over the DNS server configuration at those IPs. Here’s an example of registered nameservers:
ns1.rootmypc.net - 1.1.1.1
ns2.rootmypc.net - 2.2.2.2

Now that the registrar has the IPs for these nameservers hard-coded, it solves the chicken/egg problem. More about this later.

Create the “glue”. All that means is that you need to create an A record for each of the nameservers you just registered. Wherever DNS is hosted for “rootmypc.net”, go there and make sure these records exist in the zone file for “rootmypc.net”:
ns1 IN A 1.1.1.1
ns2 IN A 2.2.2.2

Note: You just created/registered a couple nameservers. They’re not doing anything yet, but they exist, which means now we can tell stuff to start using them.

Tell your registrar the nameservers that will be hosting DNS for your domain. Let’s say you want to use 1.1.1.1 and 2.2.2.2 (registered as ns1 and ns2.rootmypc.net) as nameservers for your domain. Go to the registrar where you bought your domain and set your authoritative nameservers to ns1 and ns2.rootmypc.net. You don’t need to give it IPs for this, because the guy who created these nameservers (maybe even YOU, if you followed the steps above) told his registrar the IPs tied to those names. You just use the names. How you do this will be different depending on your registrar.

Create the zone file for your domain. You just told your registrar that ns1 and ns2.rootmypc.net are hosting DNS for your domain. Those names point to specific IPs (1.1.1.1 and 2.2.2.2). You need to go to the server(s) at those IPs and create a zone file for your domain. The leg work is done at this point, and the zone file is the raw data that you actually care about. Here’s an example of your zone file:
$TTL 3600
@ 3600 IN SOA ns1.domain.com. admin.domain.com. (
2010030400 ; serial, todays date+todays
3600 ; refresh, seconds
30m ; retry, seconds
3600000 ; expire, seconds
3600 ) ; minimum, seconds
domain.com. IN A 5.5.5.5
www IN CNAME domain.com.

You could even copy/paste that into your zone file if you want, but obviously you’ll want to put your own domain in place of domain.com, and you need to replace 5.5.5.5 with the IP address where your web site exists.

Don’t forget your NS records. Yes, the registrar knows what nameservers your domain uses, and yes that should get the job done most of the time. However, your zone file should contain records indicating what nameservers your domain uses, and these records should agree with what you told your registrar. Still using ns1 and ns2.rootmypc.net as examples, here’s what you would need to add to your zone file:
domain.com IN NS ns1.rootmypc.net.
domain.com IN NS ns2.rootmypc.net.

Update the serial number after making changes, and “rndc reload”, or restart the service, or whatever.

RAID is not the same as having backups, people. With a RAID-5, you can handle a *single drive* failure with no downtime, but if multiple drives fail, or the raid controller fails, or if you’re rooted and all your data gets deleted, then you’re screwed. Stop thinking that RAID makes you immune to data loss. If you’re paying for RAID, pay for 1 more standalone drive to hold your backups too. To do anything less is just irresponsible.

If you think your RAID is safe, check out the stats here:http://blog.kj.stillabower.net/?p=93
While pondering those stats, be sure to remember that they do not consider the possibility of a hardware failure of the RAID card itself.

Despite the fact that it’s really not all that difficult, no one understands DNS. As such, it’s probably a good thing that you don’t need to understand DNS to properly configure reverse DNS for your server.
So you’ve got a server up on an IP (1.2.3.4, for example) and now you need to set reverse DNS for that IP. Does reverse DNS for an IP have to match the hostname that resolves to that IP? No. That’s dumb. If that was the case, virtual hosting would be impossible.

Virtual hosting: Multiple web sites on the same IP address. A vast majority of web sites on the internet are hosted virtually on a shared IP.

So what should you use for your Reverse DNS? You should use the hostname advertised in your mail server’s 220 HELO banner. Why? Because it’s necessary to help your server pass spam checks.

Why is it important that these match? A server sending spam may report its hostname as “server.rootmypc.net” so that the recipient of the spam sees “server.rootmypc.net” in the message headers and blames the wrong person for sending that spam. You can’t fake the IP of a sending mail server, and as such you can’t really fake reverse DNS either. By ensuring reverse DNS matches a mail server’s hostname before accepting mail from that server, you prevent masquerading hostnames.

A lot of Windows services love to create log files (IIS for example), but when it comes to log rotation in Windows, the functionality is glaringly absent. In linux it’s as easy as “tmpwatch –mtime 24″. In Windows there actually is a solution, but it’s a bit more complicated.

That’s probably not going to reach the destination, however. The problem with this is that it sends from user@hostname. For example, [email protected] or [email protected] This is almost never a valid address, so the message is going to fail the sender verification check.

To correct this, you need to add at least a “-f” argument to your sendmail call. For example:

Now what if you don’t have a mail server on localhost? Unfortunately the sendmail binary has no option to use a mail relay. It’s possible to edit sendmail.cf and configure all mail to be sent via a remote mail relay (SMART_HOST, as sendmail calls it). That sucks though. Why not just use telnet? Here’s the bare minimum to send a message with telnet:

You might have to adjust the sleep values depending on the speed of your connection. The resolver settings at your remote mail relay could also affect the necessary sleep times by quite a bit, but that’s another post.

Of course, that takes advantage of the built-in linux firewall. We didn’t use the built-in Windows firewall in the last post, so maybe it’s unfair to take advantage of a firewall on linux. As such, here’s how to block the IP on linux in the same manner we blocked it on Windows (by adjusting the routing tables):

There’s probably a better way to do this – if you know of one, please comment on this post. I have a pretty high opinion of myself, but I’m not so stubborn as to think I already know the best way to do everything.

Firewall options for a Windows server are kind of sparse. I’ve heard good things about ZoneAlarm, but haven’t used it myself. I tend to believe that any OS worth running should provide the ability to blacklist an IP. Windows Firewall is nice for closing off a few ports, but realistically it’s not a competent firewall. It has no capacity to whitelist or blacklist an IP. Don’t start ranting at me about how you can change the scope an open port so only certain subnets can access a server – that’s not a blacklist or a whitelist, it’s port-filtering.

So, assume your server’s Remote Desktop service is getting brute-forced by 10.0.0.5. How do you block the attacker? You could find and install some third-party software, but that’s annoying. You could enable Windows Firewall and open port 3389, then change the scope to only allow your office network, but then what if you want to connect from home, or your DHCP lease runs out, or you forget to open a different important port (53/tcp for example, which would break all your domain keys and SPF records)? Face it- Windows Firewall is good at what it does, but it doesn’t do enough to be useful. Here’s a method that will effectively blacklist that pesky 10.0.0.5 attacker:

First get some leg-work out of the way. You’ll only need to do this the first time:
Open up “Add Hardware” from the Control Panel.
Tell the www.yzzerdd that you’ve already connected the hardware.
Select “Add a new hardware device” from the bottom of the list.
Install manually by selecting from a list.
Network adapters.
Microsoft -> Microsoft Loopback Adapter

Installed? Good. You’ll never have to do that again.

Now we get into the steps you need to take for every IP you want to block.

Open Network Connections from your Control Panel. There will be a new one there with the device name “Microsoft Loopback Adapter”. You can change its name from “Local Area Connection 4″ to something meaningful (Loopback, perhaps?) if you’d like.

Note: During the next paragraph, Windows might insist you need to reboot for changes to take effect. Windows is lying.

Open it up and set its IP to 10.0.0.5 and the netmask to 255.255.255.0. Leave everything else blank. Yes this will block 255 addresses, and not just 10.0.0.5. To block only 10.0.0.5, we’d have to use the netmask 255.255.255.255, and Windows won’t let you do that. Go ahead and try it. I’ll wait.

Find the folder (“key”) that contains a REG_MULTI_SZ with the name “IPAddress” and the value “10.0.0.5”. In this folder (“key”), you’ll need to edit the REG_MULTI_SZ with the name “SubnetMask”, and change “255.255.255.0” to “255.255.255.255”.

Close regedit before you break something.

Open Network Connections from Control Panel. Right-click on “Loopback” (you did change the name, right?) and select Disable. Right-click again and select Enable. Now you’re done. Open up a command prompt and type “ipconfig” if you want verification that the subnet mask is now 255.255.255.255. While you’re there, run “netstat -ano” and marvel at the lack of connections to RDP from 10.0.0.5. Congratulations, you “blacklisted” an IP in Windows.

Why does this work?

For the system at 10.0.0.5 to attempt a login to RDP on your server, it needs to first establish a TCP connection. That means they send a SYN to port 3389, you send a SYNACK back to them, then they send an ACK, at which point a connection comes into existence. By binding the IP locally, your server creates a new route for traffic destined to 10.0.0.5. Traffic to this IP used to fall on your default gateway, and go out to the internet. Now traffic to the IP goes to your loopback adapter.

So, 10.0.0.5 sends you a SYN. Your public adapter receives this SYN and sends a SYNACK to 10.0.0.5, which gets caught by the new route and this SYNACK goes to your loopback adapter.

Your loopback adapter receives a SYNACK for which it sent no SYN, which means someone is violating TCP/IP protocol. Instead of building a connection, your loopback adapter sends a RESET back to your public adapter. The loopback adapter now washes its hands of this nonsense.

The public adapter receives a RESET, but was expecting an ACK. This means something, somewhere, went horribly wrong. The public adapter, having failed to establish a connection, forgets any of this ever happened.

The server at 10.0.0.5, having sent a SYN and never receiving anything in response, eventually gives up. When the attacker notices all these TCP timeouts, he/she assumes you have blocked their IP on a firewall.

In conclusion…

Is it perfect? No. Is it better than being hammered by a DDOS or brute-force attack? Obviously.

Granted, some of these are probably SSH or Putty hot-keys. I’m not certain there’s going to be a “Bash hot-keys 2″ post, but there are probably plenty of combinations I’ve forgotten or have yet to learn, so we’ll call this post part 1, and there will most likely be more.

ctrl+l
That’s a lowercase L. This has the same effect of running ‘clear’. It erases everything on the screen and then sticks your $PS1 prompt at the top.

ctrl+r
Perform a reverse search through your bash history. For example, hit ctrl+r then type “grep”, and it’ll pull up the last command you ran that included the string “grep”.

ctrl+s
Essentially pauses the shell, and sticks all keystrokes into a buffer that is sent as soon as you use the un-pause key combo. You can’t delete from the buffer, so if you type a command and hit return, then hold backspace, when you un-pause the shell it will send that command and then send a bunch of backspaces.

ctrl+q
Un-pauses the shell, and sends all keystrokes in the buffer.

ctrl+c
Everyone knows this one. It sends a SIGTERM. In the shell, that has the effect of aborting the current command and giving you a new $PS1 prompt.

ctrl+d
Sends an EOF character. If you’re sitting at an input buffer, it closes that buffer. If you’re sitting at an empty command prompt, it logs out of the session – same as typing ‘exit’ or ‘logout’.

ctrl+v
Basically, ctrl+v holds the ‘ctrl’ button. If you want to type the character ‘ctrl+c’, obviously you can’t just hit ctrl+c – that will just send a SIGTERM. Instead you can hit ctrl+v, then press ‘c’. Instead of seeing a ‘c’ in your shell, you’ll see “^C”, which indicates a ‘ctrl+c’ character.

Of all these, the most useful is definitely ctrl+r. Get comfortable with it and you’ll save an hour of keystrokes every month. The others will save you fractions of a second here and there.