Project description

wick

wick is a Python command line tool that automatically generates file I/O source code for working with binary data.

Why?

I was working on a project that involved reverse engineering file formats and I found that most data structures tend to be composed of several simpler data structures. I also noticed that the simpler structures were all very boilerplate. So I created this tool to write the simple structures for me, and I can do the more interesting work of composing them into the larger structure.

Supported Target Languages

C#

JavaScript

Python

Installation

$ pip install wick

Usage

$ wick example.h Python

What exactly does it do?

Let's walk through a concrete example.

Say we have binary data that is a sequence of records that are represented by a string name and an integer id. First we create a record.h file that contains a C struct representation of this data:

# record.pyimportstructclassRecord:"""Simple Record object Attributes: name: Record name id: Record id. """format='<64sB'size=struct.calcsize(format)__slots__=('name','id')def__init__(self,name,id):self.name=name.split(b'\x00')[0].decode('ascii')iftype(name)isbyteselsenameself.id=id@classmethoddefwrite(cls,file,record):record_data=struct.pack(cls.format,record.name.encode('ascii'),record.id)file.write(record_data)@classmethoddefread(cls,file):record_data=file.read(cls.size)record_struct=struct.unpack(cls.format,record_data)returnRecord(*record_struct)

Then we can import this code into Python and do work.

Read Data

withopen(path,'rb')asfile:rec=Record.read(file)

Write Data

withopen(path,'wb')asfile:rec=Record(b'name',0)Record.write(file,rec)

Unpack Lots of Data

importstruct# Assuming the file only contains Record datawithopen(path,'rb')asfile:recs=[Record(*chunk)forchunkinstruct.iter_unpack(Record.format,file.read())]

Contributing

Have a bug fix or a new feature you'd like to see in wick? Send it our way! Please make sure you create an issue that addresses your fix/feature so we can discuss the contribution.