The SYN flood attack sends TCP connections requests faster than a machine can process them.

* attacker creates a random source address for each packet
* SYN flag set in each packet is a request to open a new connection to the server from the spoofed IP address
* victim responds to spoofed IP address, then waits for confirmation that never arrives (waits about 3 minutes)
* victim's connection table fills up waiting for replies
* after table fills up, all new connections are ignored
* legitimate users are ignored as well, and cannot access the server
* once attacker stops flooding server, it usually goes back to normal state (SYN floods rarely crash servers)
* newer operating systems manage resources better, making it more difficult to overflow tables, but still are vulnerable
* SYN flood can be used as part of other attacks, such as disabling one side of a connection in TCP hijacking, or by preventing authentication or logging between servers.

Defensive techniques:

micro blocks
Instead of allocating a complete connection object (which causes the memory failure), simply allocate a micro-record. Newer implementations allocate as little as 16-bytes for the incoming SYN object.
SYN cookies
Instead of allocating a record, send a SYN-ACK with a carefully constructed seqno generated as a hash of the clients IP address, port number, and other information. When the client responds with a normal ACK, that special seqno will be included, which the server then verifies. Thus, the server first allocates memory on the third packet of the handshake, not the first. However, the cryptographic hashing used in SYN cookies is fairly expensive, so servers that expect lots of incoming connections may choose not to use it. (Conversely, newer TCP stacks need to implement secure sequence numbers anyway in order to avoid TCP seqno prediction, so this is not necessarily a problem).
RST cookies
An alternative to SYN cookies, but may cause problems with Win95 machines and/or machines behind firewalls. The way this works is that the server sends a wrong SYNACK back to the client. The client should then generate a RST packet telling the server that something is wrong. At this point, the server knows the client is valid and will now accept incoming connections from that client normally.
stack tweaking
TCP stacks can be tweaked in order to reduce the effect of SYN floods. The most common example is to reduce the timeout before a stack frees up the memory allocated for a connection. Another technique would be to selectively drop incoming connections.