Analysis

The objective for now is to find a way to get the files that are being transferred.

I struggled a bit with tshark to get those TCP streams out (tshark -r net100.pcap -T fields -e tcp.stream | sort -n | uniq),
eventually settled with wireshark to export them. For this write-up, ill use tcpflow:

~$ tcpflow -r net100.pcap
~$ cat 192* > streams.raw

Now we have a single file containing the 4 TCP streams.

Decoding the data

Before writing our hacky Python script:

yEnc headers include: filename, filesize

yEnc headers mark the beginning of some data belonging to a file

yEnc headers may include an offset/limit range for partial transfers.

For our script we can iterate every line, upon embarking on lines that begin with ybegin record the proceeding data till yend -
decode that data, store it in memory and after everything is done - write the files out.

For the decoding of yEnc data I used the Python library sabyenc.
I used their example script on Github as a starting point.

#!/usr/bin/env python2importre,sys,sabyencdata=open(sys.argv[1],"r").readlines()files={}foriinrange(0,len(data)):line=data[i]ifline.startswith("=ybegin"):# beginning of a new yenc filehas_offsets=data[i+1].startswith("=ypart")# check if the second line contains begin/endheaders={# nntp headers"begin":0,"end":0,"size":int(re.search('size=(.\d+?) ',line).group(1))}ifnotheaders['size']:raiseException("could not parse size=")ifhas_offsets:# try to parse begin/endtry:headers['begin']=int(re.search('begin=(.\d+?) ',data[i+1]).group(1))headers['end']=int(re.search('end=(.\d+?)\r\n',data[i+1]).group(1))except:print"warning: begin/end could not be correctly parsed"ifnotheaders['begin']:# sometimes begin turns into None, reset to 0headers['begin']=0# yencode-decoding using sabyenc - input data includes the 1/2 header lines and the trailing '=yend'decoded_data,output_filename,crc,crc_yenc,crc_correct=sabyenc.decode_usenet_chunks(data[i:],headers['size'])ifnotcrc_correct:raiseException("faulty checksum")print"output_filename:",output_filenameprint"size:",headers['size']print"decoded_data length:",len(decoded_data)ifoutput_filenamenotinfiles:files[output_filename]=[]files[output_filename].append({"begin":headers['begin'],"end":headers['end'],"data":decoded_data})print"sorting chunks and writing files."foroutput_filename,chunksinfiles.items():chunks=sorted(files[output_filename],key=lambdak:k['begin'])data="".join([chunk["data"]forchunkinchunks])open("output/%s"%output_filename,"wb").write(data)print"written %s"%output_filename

I initially did not sort the data chunks which resulted in not having enough par2 recovery blocks.