python - Scapy: Processing partial TLS segments

Question

I am trying to extract TLS meta-data from a pcap using Scapy. I am able to successfully parse the packets and individual messages such as the client-hello, server-hello etc and their fields. What I am having trouble with is when the TLS record is spread across multiple TCP packets/segments. This happens mostly for large TLS messages, such as application data or when server sends multiple TLS messages (server hello, certificate, etc) in one TLS frame. In such cases, scapy says it has the TLS/SSLv2 layer, but I am unable to extract any meaningful information from such frames. When I look at the same pcap in wireshark, wireshark displays

Reassembled TCP segments (T bytes): #X (x), #Y(y), #Z(z)

Where T is the combined total; X, Y, Z are frame-ids and x,y,z are number of bytes in each frame that contributed to the total T. Wireshark is reassembly the entire TLS payload before it parses the TLS frame.

I have something like this in my code:

if pkt.haslayer(TCP) and pkt.haslayer(TLS):
  parseTLS(pkt)
elif pkt.haslayer(TCP) and pkt.haslayer(SSLv2):
  parseSSLv2(pkt)

When it encounters a partial TLS packet, the code always falls through to the SSLv2. Is there a way for me to accurately identify a packet as a partial tls-segment and accumulate and then process the complete TLS frame similar to wireshark? If so, how do I go about it. Any pointers or help is appreciated. Thanks.

标签： pythonssltcptls1.2scapy

Answer 1

Not sure if there is a better solution, but here's what I did to get around my problem. The accumulated frame-sizes seem to agree with what Wireshark does when it reassembles TLS frame from multiple packets.

Note: This solution assumes there are no misordered packets or duplicates. Also, the solution shown here is not complete. The code below only shows how to accumulate TLS frames when they span multiple IP packets for a single TCP stream. Please follow Janus's suggestion (mentioned in the comments below) or come up with your own solution to accumulate different the streams.

def extractDataFromPcap(pcapfile):
  load_layer("tls")

  try:
    reader = PcapReader(pcapfile)

    # tls frame accumulation related variables.
    tls_accumulate = False
    tls_leftover_len = 0
    tls_blist = list()

    for pkt in reader:
      try:
        if tls_accumulate and pkt.haslayer(TCP):
          ip_tcp_hdr_overhead = ((pkt[IP].ihl*4) + (pkt[TCP].dataofs*4))
          pkt_payload_len = (pkt[IP].len - ip_tcp_hdr_overhead)
          tls_leftover_len = (tls_leftover_len - pkt_payload_len)
          tls_blist.append(raw(pkt[TCP].payload))

          if tls_leftover_len <= 0:
            # got complete TLS frame
            tls_raw_bytes = b''.join(tls_blist)

            # parse accumulated frame.
            tls = TLS(tls_raw_bytes)
            #... process tls object

            # reset accumlation variables.
            tls_accumulate = False
            tls_leftover_len = 0
            tls_blist = list()
          else:
            continue

        if pkt.haslayer(TCP) and pkt.haslayer(TLS):
          if not tls_accumulate:
            # Process new TLS frame.
            # Pkt over head: ip-hdr-len + tcp-hdr-len
            ip_tcp_hdr_overhead = ((pkt[IP].ihl*4) + (pkt[TCP].dataofs*4))
            pkt_payload_len = (pkt[IP].len - ip_tcp_hdr_overhead)
            if pkt[TLS].len > pkt_payload_len:
              # partial TLS frame. Start accumulating.
              tls_accumulate = True
              tls_leftover_len = (pkt[TLS].len - pkt_payload_len)
              tls_blist.append(raw(pkt[TCP].payload))
            else:
              # complete TLS frame
              # ... process complete TLS Frame 
      except Exception as error:
        pkt.show()
        print("tcp-processing error: %s", error)
        sys.exit(-1)

  except Exception as error:
    print("packet processing error: %s", error)
    sys.exit(-1)