1.TCP建立连接的三次握手过程
先看一下三次握手的大概流程，客户端想要与服务器建立可靠传输通信，需经过三次握手，服务器是一直处于监听状态的，也就是LISTEN状态，用来监听客户的请求。此时客户端想要与服务器建立通信，主动发起请求，也就是三次握手，过程如下。
第一次:建立连接时,客户端发送SYN(syn=j)包到服务器,并进入SYN_SENT状态,等待服务器确认;
第二次:服务器收到SYN包,必须确认客户的SYN(ack=j+1),同时自己也发送一个SYN包(seq=k),即SYN+ACK包,此时服务器进入SYN_RECV状态;
第三次:客户端收到服务器的SYN+ACK包,向服务器发送确认包ACK(ack=k+1),客户端发送完后进入连接建立状态。
！注意，此时服务器还没有进入建立连接状态，服务器在收到客户端发送的ACK包后，进入ESTABLISHED(TCP连接成功)状态,这样三次握手就完成了。
三次握手的过程就是客户端发送两个包，服务端发送一个包，为什么是三次，这里就不详细解释了。
2.TCP三次握手过程在Linux内核中的执行流程
根据老师在课堂上讲的内容，我们知道TCP的三次握手从用户程序的角度看就是客户端connect和服务端accept建立起连接时背后的完成的工作，在内核socket接口层这两个socket API函数对应着sys_connect和sys_accept函数，进一步对应着sock->opt->connect和sock->opt->accept两个函数指针，在TCP协议中这两个函数指针对应着tcp_v4_connect函数和inet_csk_accept函数。tcp_v4_connect触发TCP数据发送过程，进入tcp_transmit_skb函数，该向IP层发送数据包；inet_csk_accept触发数据接收过程，接收数据通过tcp_v4_rcv函数实现，tcp_v4_rcv为TCP的总入口，数据包从IP层传递上来，进入该函数。
大概就是下图这样
通过上面分析，我们知道了TCP在客户端和服务端收发数据大概流程，客户端在tcp层通过调用tcp_transmit_skb向服务器发送带有SYN的TCP报文，服务器在tcp层通过tcp_v4_rcv函数处理接收到的SYN数据包，并调用相关函数向客户发送SYN+ACK包，客户端在tcp层也是通过tcp_v4_rcv函数接收数据进行处理，然后调用相关函数向服务器发送ACK包。
那么三次握手是在内核中是具体怎么执行的呢，又调用了那些内核函数呢，接下来我们通过gdb调试，观察内核栈调用，看看这是怎么样的一个过程。
3.gdb分析三次握手过程在内核中的调用栈
老样子，打开终端输入下列命令打开menuOS，通过在上面运行我们的网络程序，即replyhi/hello来分析Linux内核中TCP三次握手过程。
$ qemu -kernel ~/kernel/linux-5.0.1/arch/x86/boot/bzImage -initrd ~/kernel/linuxnet/rootfs.img -append nokaslr -s
打开另一终端，进入内核调试工具gdb，加载镜像符号表，连接qemu的gdb server,设置断点，这里把断点设置在tcp_v4_rcv处，通过上面分析，我们知道第一次执行tcp_v4_rcv应该是第一次握手的时候，服务端收到客户端发送的SYN包。看看是不是这么回事，里面具体又发生了什么。
(gdb) b tcp_v4_rcv
Breakpoint 1 at 0xffffffff81936870: file net/ipv4/tcp_ipv4.c, line 1782.
(gdb) c
Continuing.
Breakpoint 1, tcp_v4_rcv (skb=0xffff8880069d20e0) at net/ipv4/tcp_ipv4.c:1782
1782 {
(gdb) bt
#0 tcp_v4_rcv (skb=0xffff8880069d20e0) at net/ipv4/tcp_ipv4.c:1782
#1 0xffffffff8190c375 in ip_protocol_deliver_rcu (
net=0xffffffff827d5500 <init_net>, skb=0xffff8880069d20e0,
protocol=<optimized out>) at net/ipv4/ip_input.c:208
#2 0xffffffff8190c545 in ip_local_deliver_finish (net=<optimized out>,
sk=<optimized out>, skb=<optimized out>) at net/ipv4/ip_input.c:234
#3 0xffffffff8190c5bf in NF_HOOK (pf=<optimized out>, sk=<optimized out>,
out=<optimized out>, okfn=<optimized out>, in=<optimized out>,
skb=<optimized out>, net=<optimized out>, hook=<optimized out>)
at ./include/linux/netfilter.h:289
#4 ip_local_deliver (skb=0xffff8880069d20e0) at net/ipv4/ip_input.c:255
#5 0xffffffff8190bcf4 in dst_input (skb=<optimized out>)
at ./include/net/dst.h:450
#6 ip_rcv_finish (net=0xffffffff827d5500 <init_net>, sk=<optimized out>,
skb=0xffff8880069d20e0) at net/ipv4/ip_input.c:414
#7 0xffffffff8190c696 in NF_HOOK (pf=<optimized out>, sk=<optimized out>,
out=<optimized out>, okfn=<optimized out>, in=<optimized out>,
skb=<optimized out>, net=<optimized out>, hook=<optimized out>)
at ./include/linux/netfilter.h:289
#8 ip_rcv (skb=0xffff8880069d20e0, dev=0xffff888007746000,
pt=<optimized out>, orig_dev=<optimized out>) at net/ipv4/ip_input.c:524
#9 0xffffffff818ae917 in __netif_receive_skb_one_core (
skb=0xffff8880069d20e0, pfmemalloc=<optimized out>) at net/core/dev.c:4973
---Type <return> to continue, or q <return> to quit---
#10 0xffffffff818ae978 in __netif_receive_skb (skb=<optimized out>)
at net/core/dev.c:5083
#11 0xffffffff818aea64 in process_backlog (napi=0xffff888007823d50, quota=64)
at net/core/dev.c:5923
#12 0xffffffff818af110 in napi_poll (repoll=<optimized out>, n=<optimized out>)
at net/core/dev.c:6346
#13 net_rx_action (h=<optimized out>) at net/core/dev.c:6412
#14 0xffffffff81e000e4 in __do_softirq () at kernel/softirq.c:292
#15 0xffffffff81c00dba in do_softirq_own_stack ()
at arch/x86/entry/entry_64.S:1027
#16 0xffffffff8109b9c4 in do_softirq () at kernel/softirq.c:337
#17 0xffffffff8109ba35 in do_softirq () at kernel/softirq.c:329
#18 __local_bh_enable_ip (ip=<optimized out>, cnt=<optimized out>)
at kernel/softirq.c:189
#19 0xffffffff819100a3 in local_bh_enable ()
at ./include/linux/bottom_half.h:32
#20 rcu_read_unlock_bh () at ./include/linux/rcupdate.h:696
#21 ip_finish_output2 (net=<optimized out>, sk=<optimized out>,
skb=0xffff8880069d20e0) at net/ipv4/ip_output.c:231
#22 0xffffffff81910dfe in ip_finish_output (net=0xffffffff827d5500 <init_net>,
sk=0xffff88800619f700, skb=0xffff8880069d20e0) at net/ipv4/ip_output.c:317
#23 0xffffffff819125dd in NF_HOOK_COND (pf=<optimized out>,
hook=<optimized out>, in=<optimized out>, okfn=<optimized out>,
---Type <return> to continue, or q <return> to quit---
cond=<optimized out>, out=<optimized out>, skb=<optimized out>,
sk=<optimized out>, net=<optimized out>) at ./include/linux/netfilter.h:278
#24 ip_output (net=0xffff8880069d20e0, sk=0x6 <irq_stack_union+6>,
skb=0xffff8880069d20e0) at net/ipv4/ip_output.c:405
#25 0xffffffff81911d2b in dst_output (skb=<optimized out>, sk=<optimized out>,
net=<optimized out>) at ./include/net/dst.h:444
#26 ip_local_out (net=0xffffffff827d5500 <init_net>, sk=0xffff88800619f700,
skb=0xffff8880069d20e0) at net/ipv4/ip_output.c:124
#27 0xffffffff81912075 in __ip_queue_xmit (sk=0xffff8880069d20e0,
skb=0x6 <irq_stack_union+6>, fl=<optimized out>, tos=<optimized out>)
at net/ipv4/ip_output.c:505
#28 0xffffffff81932f80 in ip_queue_xmit (sk=<optimized out>,
skb=<optimized out>, fl=<optimized out>) at ./include/net/ip.h:198
#29 0xffffffff8192cca3 in __tcp_transmit_skb (sk=0xffff88800619f700,
skb=0xffff8880069d20e0, clone_it=<optimized out>,
gfp_mask=<optimized out>, rcv_nxt=<optimized out>)
at net/ipv4/tcp_output.c:1160
#30 0xffffffff8192ed7d in tcp_transmit_skb (gfp_mask=<optimized out>,
clone_it=<optimized out>, skb=<optimized out>, sk=<optimized out>)
at net/ipv4/tcp_output.c:1176
#31 tcp_connect (sk=0xffff88800619f700) at net/ipv4/tcp_output.c:3535
#32 0xffffffff819351f9 in tcp_v4_connect (sk=0xffff88800619f700,
uaddr=0xffffc90000043df8, addr_len=<optimized out>)
---Type <return> to continue, or q <return> to quit---
at net/ipv4/tcp_ipv4.c:315
#33 0xffffffff8194eac9 in __inet_stream_connect (sock=0xffff888006462500,
uaddr=0x6 <irq_stack_union+6>, addr_len=6, flags=<optimized out>,
is_sendmsg=<optimized out>) at net/ipv4/af_inet.c:655
#34 0xffffffff8194ec4b in inet_stream_connect (sock=0xffff888006462500,
uaddr=0xffffc90000043df8, addr_len=16, flags=2) at net/ipv4/af_inet.c:719
#35 0xffffffff8188b673 in __sys_connect (fd=<optimized out>,
uservaddr=0xffd431ec, addrlen=16) at net/socket.c:1663
#36 0xffffffff818eb977 in __do_compat_sys_socketcall (args=<optimized out>,
call=<optimized out>) at net/compat.c:869
#37 __se_compat_sys_socketcall (args=<optimized out>, call=<optimized out>)
at net/compat.c:838
#38 __ia32_compat_sys_socketcall (regs=<optimized out>) at net/compat.c:838
#39 0xffffffff8100450b in do_syscall_32_irqs_on (regs=<optimized out>)
at arch/x86/entry/common.c:326
#40 do_fast_syscall_32 (regs=0xffff8880069d20e0) at arch/x86/entry/common.c:397
#41 0xffffffff81c01631 in entry_SYSCALL_compat ()
at arch/x86/entry/entry_64_compat.S:257
#42 0x0000000000000000 in ?? ()
emm，这东西也太多了8，看起来有点复杂额，稍稍梳理一下，主要的有这么几个过程：
首先是通过系统调用和socket总入口sys_socketcall，进入__socket_connnect，这个过程上次实验已经分析过了。
接下来
inet_stream_connect-> __inet_stream_connect-> tcp_v4_connect-> tcp_connect-> tcp_transmit_skb-> __tcp_transmit_skb
这个过程主要是干嘛呢，待会儿源码分析一下
紧接着
ip_queue_xmit-> __ip_queue_xmit-> ip_local_out-> NF_HOOK_COND-> ip_finish_output-> rcu_read_unlock_bh-> local_bh_enable
-> __local_bh_enable_i-> ......中间太多了，省略-> ip_rcv_finish
这个过程主要是ip层的内容，主要是数据添加头部，确定路由，数据检查，发送机制，协议信息，分片，将数据发送到网卡或者交给tcp层等等，先暂时不分析它
最后
ip_local_deliver-> NF_HOOK-> ip_local_deliver_finish-> ip_protocol_deliver_rcu-> tcp_v4_rcv
ip层接收到数据之后经过一系列处理，通过tcp层的总接口tcp_v4_rcv将数据传给tcp层。
4.调用过程源码分析
接着上面的内容分析第一次握手在tcp层中的过程
inet_stream_connect-> __inet_stream_connect-> tcp_v4_connect-> tcp_connect-> tcp_transmit_skb-> __tcp_transmit_skb
客户端首先通过connect方法试图与服务端建立连接，通过syscall找到其在内核中的实现__sys_connect，__sys_connect方法对应的内核源码如下，这里sock->ops指向的是inet_stream_ops，因此sock->ops->connect调用的就是inet_stream_connect()。inet_stream_connect()对__inet_stream_connect()做了一个简单的封装。
// net/socket.c
int __sys_connect(int fd, struct sockaddr __user *uservaddr, int addrlen)
{
...
/* __sys_connect调用的主处理函数
* 函数指针指向inet_stream_connect()。
*/
err = sock->ops->connect(sock, (struct sockaddr *)&address, addrlen,
sock->file->f_flags);
...
}
inet_stream_connect()对__inet_stream_connect()做了封装
int inet_stream_connect(struct socket *sock, struct sockaddr *uaddr,
int addr_len, int flags)
{
int err;
lock_sock(sock->sk);
err = __inet_stream_connect(sock, uaddr, addr_len, flags, 0);
release_sock(sock->sk);
return err;
}
EXPORT_SYMBOL(inet_stream_connect);
__inet_stream_connect的主要功能：
- 调用tcp_v4_connect函数建立与服务器联系并发送SYN段；
- 获取连接超时时间timeo，如果timeo不为0，则会调用inet_wait_for_connect一直等待到连接成功或超时
// net/ipv4/af_inet.c
int __inet_stream_connect(struct socket *sock, struct sockaddr *uaddr,
int addr_len, int flags, int is_sendmsg)
{
struct sock *sk = sock->sk;
int err;
long timeo;
...
switch (sock->state) {
default:
err = -EINVAL;
goto out;
case SS_CONNECTED:/* 已经与对方端口连接*/
err = -EISCONN;
goto out;
case SS_CONNECTING:/*正在连接过程中*/
if (inet_sk(sk)->defer_connect)
err = is_sendmsg ? -EINPROGRESS : -EISCONN;
else
err = -EALREADY;
/* Fall out of switch with err, set for this state */
break;
case SS_UNCONNECTED:/* 只有此状态才能调用connect */
err = -EISCONN;
if (sk->sk_state != TCP_CLOSE)
goto out;
if (BPF_CGROUP_PRE_CONNECT_ENABLED(sk)) {
err = sk->sk_prot->pre_connect(sk, uaddr, addr_len);
if (err)
goto out;
}
/* 调用传输层接口tcp_v4_connect建立与服务器连接，并发送SYN段 */
err = sk->sk_prot->connect(sk, uaddr, addr_len);
if (err < 0)
goto out;
/* 发送SYN段后，设置状态为SS_CONNECTING */
sock->state = SS_CONNECTING;
if (!err && inet_sk(sk)->defer_connect)
goto out;
/* 如果是以非阻塞方式进行连接，则默认的返回值为EINPROGRESS，表示正在连接 */
err = -EINPROGRESS;
break;
}
/* 获取连接超时时间，如果指定非阻塞方式，则不等待直接返回 */
timeo = sock_sndtimeo(sk, flags & O_NONBLOCK);
...
/* 发送完SYN后，连接状态一般为这两种状态，但是如果连接建立非常快，则可能越过这两种状态 */
if ((1 << sk->sk_state) & (TCPF_SYN_SENT | TCPF_SYN_RECV)) {
int writebias = (sk->sk_protocol == IPPROTO_TCP) &&
tcp_sk(sk)->fastopen_req &&
tcp_sk(sk)->fastopen_req->data ? 1 : 0;
/* Error code is set above */
if (!timeo || !inet_wait_for_connect(sk, timeo, writebias))
goto out;/* 等待连接完成或超时 */
err = sock_intr_errno(timeo);
if (signal_pending(current))
goto out;
}
...
}
接下来就进入tcp_v4_connect了，主要是功能与服务器建立连接，发送SYN包，函数代码如下
// net/ipv4/tcp_ipv4.c
int tcp_v4_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len)
{
struct sockaddr_in *usin = (struct sockaddr_in *)uaddr;
struct inet_sock *inet = inet_sk(sk);
struct tcp_sock *tp = tcp_sk(sk);
__be16 orig_sport, orig_dport;
__be32 daddr, nexthop;
struct flowi4 *fl4;
struct rtable *rt;
int err;
struct ip_options_rcu *inet_opt;
struct inet_timewait_death_row *tcp_death_row = &sock_net(sk)->ipv4.tcp_death_row;
if (addr_len < sizeof(struct sockaddr_in))//校验目的地址的长度及地址族的有效性
return -EINVAL;
if (usin->sin_family != AF_INET)
return -EAFNOSUPPORT;
nexthop = daddr = usin->sin_addr.s_addr;//将下一跳和目的地址都设置为源地址
inet_opt = rcu_dereference_protected(inet->inet_opt,
lockdep_sock_is_held(sk));
if (inet_opt && inet_opt->opt.srr) {
if (!daddr)
return -EINVAL;
nexthop = inet_opt->opt.faddr;
}
orig_sport = inet->inet_sport;
orig_dport = usin->sin_port;
fl4 = &inet->cork.fl.u.ip4;
rt = ip_route_connect(fl4, nexthop, inet->inet_saddr, //根据下一跳地址等信息查找目的路由缓存项
RT_CONN_FLAGS(sk), sk->sk_bound_dev_if,
IPPROTO_TCP,
orig_sport, orig_dport, sk);
if (IS_ERR(rt)) {
err = PTR_ERR(rt);
if (err == -ENETUNREACH)
IP_INC_STATS(sock_net(sk), IPSTATS_MIB_OUTNOROUTES);
return err;
}
if (rt->rt_flags & (RTCF_MULTICAST | RTCF_BROADCAST)) {//对TCP来说，不能使用多播和组播路由项
ip_rt_put(rt);
return -ENETUNREACH;
}
...
tcp_set_state(sk, TCP_SYN_SENT);
err = inet_hash_connect(tcp_death_row, sk); //绑定IP地址和端口，并将socket加入到连接表中
...
if (likely(!tp->repair)) {
if (!tp->write_seq)//使用加密的方法生成初始序列号
tp->write_seq = secure_tcp_seq(inet->inet_saddr,
inet->inet_daddr,
inet->inet_sport,
usin->sin_port);
tp->tsoffset = secure_tcp_ts_off(sock_net(sk),
inet->inet_saddr,
inet->inet_daddr);
}
...
err = tcp_connect(sk); //发送SYN请求
if (err)
goto failure;
return 0;
...
}
EXPORT_SYMBOL(tcp_v4_connect);
- 此函数前面部分是确定socket的源端口,目的ip及端口。目的IP和目的端口是由connect系统调用的入参指定。
- secure_tcp_seq函数用加密并与时间关联的非法生成初始序列号，以保证一个TCP连接的初始序列号很难被猜到，而且即使先后发起多次是源、目的端口都一样的连接，它们的起始序列号也都不一样。
- tcp_connect函数用于构建并发送一个SYN请求。
继续看看里面的tcp_connect函数
// net/ipv4/tcp_output.c
int tcp_connect(struct sock *sk)
{
struct tcp_sock *tp = tcp_sk(sk);
struct sk_buff *buff;
int err;
tcp_call_bpf(sk, BPF_SOCK_OPS_TCP_CONNECT_CB, 0, NULL);
if (inet_csk(sk)->icsk_af_ops->rebuild_header(sk))
return -EHOSTUNREACH; /* Routing failure or similar. */
tcp_connect_init(sk);//初始化TCP连接信息
if (unlikely(tp->repair)) {
tcp_finish_connect(sk, NULL);
return 0;
}
buff = sk_stream_alloc_skb(sk, 0, sk->sk_allocation, true);//构建SKB用于承载TCP报文
if (unlikely(!buff))
return -ENOBUFS;
tcp_init_nondata_skb(buff, tp->write_seq++, TCPHDR_SYN);//初始化不带数据的TCP SYN报文，SYN包的seq为tp->write_seq
tcp_mstamp_refresh(tp);
tp->retrans_stamp = tcp_time_stamp(tp);
tcp_connect_queue_skb(sk, buff);//将报文放入发送队列中
tcp_ecn_send_syn(sk, buff);
tcp_rbtree_insert(&sk->tcp_rtx_queue, buff);
/* Send off SYN; include data in Fast Open. */
err = tp->fastopen_req ? tcp_send_syn_data(sk, buff) ://如果TFO功能开启，则用tcp_send_syn_data函数发送带数据的SYN
tcp_transmit_skb(sk, buff, 1, sk->sk_allocation);//否则用tcp_transmit_skb发送之前构建好的不带数据的SYN
if (err == -ECONNREFUSED)
return err;
/* We change tp->snd_nxt after the tcp_transmit_skb() call
* in order to make this packet get counted in tcpOutSegs.
*/
tp->snd_nxt = tp->write_seq;
tp->pushed_seq = tp->write_seq;
buff = tcp_send_head(sk);
if (unlikely(buff)) {
tp->snd_nxt = TCP_SKB_CB(buff)->seq;
tp->pushed_seq = TCP_SKB_CB(buff)->seq;
}
TCP_INC_STATS(sock_net(sk), TCP_MIB_ACTIVEOPENS);
/* Timer for repeating the SYN until an answer. */
inet_csk_reset_xmit_timer(sk, ICSK_TIME_RETRANS, //设置重传定时器，如果在超时之前没有收到SYN|ACK，则重传SYN
inet_csk(sk)->icsk_rto, TCP_RTO_MAX);
return 0;
}
EXPORT_SYMBOL(tcp_connect);
总结一下tcp_connect的作用：
- 构造一个携带SYN标志位的TCP头，tcp_init_nondata_skb函数实现
- 发送带有SYN的TCP报文，tcp_transmit_skb函数实现
- 设置计时器超时重发，net_csk_reset_xmit_timer函数实现
tcp_transmit_skb是对__tcp_transmit_skb的封装，继续调用，进入__tcp_transmit_skb发送SYN报文
// net/ipv4/tcp_output.c
static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
gfp_t gfp_mask)
{
return __tcp_transmit_skb(sk, skb, clone_it, gfp_mask,
tcp_sk(sk)->rcv_nxt);
}
__tcp_transmit_skb函数
// net/ipv4/tcp_output.c
static int __tcp_transmit_skb(struct sock *sk, struct sk_buff *skb,
int clone_it, gfp_t gfp_mask, u32 rcv_nxt)
{
const struct inet_connection_sock *icsk = inet_csk(sk);
struct inet_sock *inet;
struct tcp_sock *tp;
struct tcp_skb_cb *tcb;
struct tcp_out_options opts;
unsigned int tcp_options_size, tcp_header_size;
struct sk_buff *oskb = NULL;
struct tcp_md5sig_key *md5;
struct tcphdr *th;
u64 prior_wstamp;
int err;
BUG_ON(!skb || !tcp_skb_pcount(skb));
tp = tcp_sk(sk);
if (clone_it) { //如果还有其他进程使用Socket Buffer，就要克隆Socket Buffer
TCP_SKB_CB(skb)->tx.in_flight = TCP_SKB_CB(skb)->end_seq
- tp->snd_una;
oskb = skb;
tcp_skb_tsorted_save(oskb) {
if (unlikely(skb_cloned(oskb)))
skb = pskb_copy(oskb, gfp_mask);
else
skb = skb_clone(oskb, gfp_mask);
} tcp_skb_tsorted_restore(oskb);
if (unlikely(!skb))
return -ENOBUFS;
}
prior_wstamp = tp->tcp_wstamp_ns;
tp->tcp_wstamp_ns = max(tp->tcp_wstamp_ns, tp->tcp_clock_cache);
skb->skb_mstamp_ns = tp->tcp_wstamp_ns;
inet = inet_sk(sk); //获取AF_INIT协议组套接字
tcb = TCP_SKB_CB(skb); //TCP选项结构体
memset(&opts, 0, sizeof(opts));
if (unlikely(tcb->tcp_flags & TCPHDR_SYN)) //是否是SYN请求数据包
tcp_options_size = tcp_syn_options(sk, skb, &opts, &md5);//构建TCP选项包括时间戳、窗口大小、选择回答SACK
else
tcp_options_size = tcp_established_options(sk, skb, &opts,//构建常规TCP选项
&md5);
tcp_header_size = tcp_options_size + sizeof(struct tcphdr);//tCP头部长度包括选择长度+ TCP头部
/* if no packet is in qdisc/device queue, then allow XPS to select
* another queue. We can be called from tcp_tsq_handler()
* which holds one reference to sk.
*
* TODO: Ideally, in-flight pure ACK packets should not matter here.
* One way to get this would be to set skb->truesize = 2 on them.
*/
skb->ooo_okay = sk_wmem_alloc_get(sk) < SKB_TRUESIZE(1);
/* If we had to use memory reserve to allocate this skb,
* this might cause drops if packet is looped back :
* Other socket might not have SOCK_MEMALLOC.
* Packets not looped back do not care about pfmemalloc.
*/
skb->pfmemalloc = 0;
skb_push(skb, tcp_header_size);
skb_reset_transport_header(skb);
skb_orphan(skb);
skb->sk = sk;
skb->destructor = skb_is_tcp_pure_ack(skb) ? __sock_wfree : tcp_wfree;
skb_set_hash_from_sk(skb, sk);
refcount_add(skb->truesize, &sk->sk_wmem_alloc);
skb_set_dst_pending_confirm(skb, sk->sk_dst_pending_confirm);
/* Build TCP header and checksum it. */
th = (struct tcphdr *)skb->data;//建立tcp头部并检查校验和
th->source = inet->inet_sport;
th->dest = inet->inet_dport;
th->seq = htonl(tcb->seq);
th->ack_seq = htonl(rcv_nxt);
*(((__be16 *)th) + 6) = htons(((tcp_header_size >> 2) << 12) |
tcb->tcp_flags);
th->check = 0;
th->urg_ptr = 0;
/* The urg_mode check is necessary during a below snd_una win probe */
if (unlikely(tcp_urg_mode(tp) && before(tcb->seq, tp->snd_up))) {
if (before(tp->snd_up, tcb->seq + 0x10000)) {
th->urg_ptr = htons(tp->snd_up - tcb->seq);
th->urg = 1;
} else if (after(tcb->seq + 0xFFFF, tp->snd_nxt)) {
th->urg_ptr = htons(0xFFFF);
th->urg = 1;
}
}
tcp_options_write((__be32 *)(th + 1), tp, &opts);//将选项信息写入TCP报文段
skb_shinfo(skb)->gso_type = sk->sk_gso_type;
if (likely(!(tcb->tcp_flags & TCPHDR_SYN))) {
th->window = htons(tcp_select_window(sk));
tcp_ecn_send(sk, skb, th, tcp_header_size);
} else {
/* RFC1323: The window in SYN & SYN/ACK segments
* is never scaled.
*/
th->window = htons(min(tp->rcv_wnd, 65535U));
}
#ifdef CONFIG_TCP_MD5SIG
/* Calculate the MD5 hash, as we have all we need now */
if (md5) {
sk_nocaps_add(sk, NETIF_F_GSO_MASK);
tp->af_specific->calc_md5_hash(opts.hash_location,
md5, sk, skb);
}
#endif
icsk->icsk_af_ops->send_check(sk, skb);//调用tcp_v4_send_check函数计算检验和
if (likely(tcb->tcp_flags & TCPHDR_ACK))
tcp_event_ack_sent(sk, tcp_skb_pcount(skb), rcv_nxt);
if (skb->len != tcp_header_size) {
tcp_event_data_sent(tp, sk);
tp->data_segs_out += tcp_skb_pcount(skb);
tp->bytes_sent += skb->len - tcp_header_size;
}
if (after(tcb->end_seq, tp->snd_nxt) || tcb->seq == tcb->end_seq)
TCP_ADD_STATS(sock_net(sk), TCP_MIB_OUTSEGS,
tcp_skb_pcount(skb));
tp->segs_out += tcp_skb_pcount(skb);
/* OK, its time to fill skb_shinfo(skb)->gso_{segs|size} */
skb_shinfo(skb)->gso_segs = tcp_skb_pcount(skb);
skb_shinfo(skb)->gso_size = tcp_skb_mss(skb);
/* Leave earliest departure time in skb->tstamp (skb->skb_mstamp_ns) */
/* Cleanup our debris for IP stacks */
memset(skb->cb, 0, max(sizeof(struct inet_skb_parm),
sizeof(struct inet6_skb_parm)));
err = icsk->icsk_af_ops->queue_xmit(sk, skb, &inet->cork.fl);//触发协议栈发送数据，调用ip_queue_xmit函数将报文段向下层传递
if (unlikely(err > 0)) { //如果发送失败，则类似于接收到显式拥塞通知的处理
tcp_enter_cwr(sk);
err = net_xmit_eval(err);
}
if (!err && oskb) {
tcp_update_skb_after_send(sk, oskb, prior_wstamp);
tcp_rate_skb_sent(sk, oskb);
}
return err;
}
__tcp_transmit_skb函数的主要任务是向ip层发送数据包，其中包括
- 初始化TCP协议头等数据结构
- 查看clone_it是否要克隆Socket Buffer，应用Socket Buffer可能正被其他进程使用，就要克隆一个份
- 构建TCP协议选项
- 阻塞控制，确定网络上有多少数据包最好
- 构建TCP协议头主要的数据域：源端口、目的端口、数据段初始序列号，计算窗口大小，如果是SYN请求包就不需要计算窗口大小
- 发送数据包到ip层，发送过程状态机切换，发送SYN包之后切换为SYN_SENT
emm，到这里客户端tcp层算是完成SYN包的发送了，经过下层传输到服务器网卡。
服务器此时tcp层inet_csk_accept正处于获取连接请求状态，也就是一直在监听，其执行过程如下
// net/ipv4/inet_connect_sock.c
struct sock *inet_csk_accept(struct sock *sk, int flags, int *err, bool kern)
{...
if (sk->sk_state != TCP_LISTEN)//判断是否是LISTEN状态
goto out_err;
/* Find already established connection */
if (reqsk_queue_empty(queue)) {//如果请求队列为空
long timeo = sock_rcvtimeo(sk, flags & O_NONBLOCK);
/* If this is a non blocking socket don't sleep */
error = -EAGAIN;
if (!timeo)
goto out_err;
error = inet_csk_wait_for_connect(sk, timeo)；//执行此函数等待连接，继续监听
if (error)
goto out_err;
}
EXPORT_SYMBOL(inet_csk_accept);
reqsk_queue_empty函数
// include/net/request_sock.c
static inline bool reqsk_queue_empty(const struct request_sock_queue *queue)
{
return queue->rskq_accept_head == NULL;//判断请求是否为空
}
inet_csk_wait_for_connect函数,等待连接请求，直到reqsk_queue_empty为假，即icsk->icsk_accept_queue不为空
// net/ipv4/inet_connect_sock.c
static int inet_csk_wait_for_connect(struct sock *sk, long timeo)
{
struct inet_connection_sock *icsk = inet_csk(sk);
DEFINE_WAIT(wait);
int err;
for (;;) {//一直等待
prepare_to_wait_exclusive(sk_sleep(sk), &wait,
TASK_INTERRUPTIBLE);
release_sock(sk);
if (reqsk_queue_empty(&icsk->icsk_accept_queue))
timeo = schedule_timeout(timeo);
sched_annotate_sleep();
lock_sock(sk);
err = 0;
if (!reqsk_queue_empty(&icsk->icsk_accept_queue))//icsk->icsk_accept_queue不为空跳出循环
break;
err = -EINVAL;
if (sk->sk_state != TCP_LISTEN)//不是listen状态也跳出
break;
err = sock_intr_errno(timeo);//信号打断
if (signal_pending(current))
break;
err = -EAGAIN;
if (!timeo)//调度超时
break;
}
finish_wait(sk_sleep(sk), &wait);//结束等待状态
return err;
}
inet_csk_accept函数实现了tcp协议accept操作，如果请求队列中没有已完成握手的连接，并且套接字已经设置了阻塞标记，则需要加入调度队列等待连接的到来，inet_csk_wait_for_connect函数完成了这个功能。
等待状态如图
现在我们知道服务器端等待状态是怎么回事了，回到第一次握手，经过之前的分析，数据包经过一些列过程到达服务器的ip层，经过ip层处理后经过TCPv4的入口函数tcp_v4_rcv进入tcp层，emm，这里不详细分析源码了，下次再分析吧，大概流程是：
tcp_v4_rcv调用主处理函数tcp_v4_do_rcv，然后->tcp_v4_hnd_req-> tcp_rcv_state_process-> 接收SYN报文->发送SYN+ACK报文这样一个过程进行第二次握手。
同样，SYN+ACK报文到达客户端，客户端tcp层通过tcp_v4_rcv调用主处理函数tcp_v4_do_rcv，然后经过相关处理分析，调用tcp_rcv_synsent_state_process函数发送ACK包，进行第三次握手，服务器端接收ACK包，三次握手完成，连接成功建立。
先就分析到这里吧，Linux内核中的socket实在是太庞杂了，太深东西太多了，emm。