tools

projects

design

The rest of this page describes simple tool designs to be implemented in any language or platform. The arguments can be accepted in any manor fitting to the platform, e.g., command-line arguments or HTTP query parameters. The result can be returned as well in any presentation or output of the developer's choosing.

wellrestd

wellrestd is a tool to validate or clean up a restd file.

only a file

If it's only argument is a restd file, it should report

whether or not the file is 100% valid JSON

whether or not the file is in the restd format, i.e., has a header, block section & footer that match the format

all of the keys present in the header & whether or not this version of wellrestd recognizes them

the number of objects in the restd file's data

whether or not each object in the restd file's data is a plain object

file -make

Using the make argument does not change the current blockSize but it does modify the input to be 100% valid restd by forcing the following structure rules.

correct the amount of buffer characters in the header, data and metadata sections

add an ending null in the data array if it does not exist

All of the following commands will also act as though -make were called first.

file -blocksize size

Using the blocksize argument with a numeric value will rebuild a restd file making each object size bytes in length. If the new size is smaller than the largest object, an error should be returned an no harm should come to the original restd file.

file -blocksize tucked

Passing the word tucked with the blocksize argument will read the entire restd file, determine the number of bytes used in the largest object and rebuild the restd file using that number as the new blockSize effectively making the restd file as small as it can be while still having a blockSize set.

file -blocksize fluffed size

Passing the word fluffed with a numeric value after will perform the same optimization as tucked but then buffer the largest object size with the passed-in value. This process can shrink a restd file down while also leaving room for future expansion via services.

file -fluff size

Passing the argument fluff with a numeric value after will increase (or decrease via a negative size) the blockSize by the passed-in value. This includes padding every object inside the restd file with size space characters at the end. Note that this is different than passing fluffed with the -blocksize argument. If decreasing, return an error if the new size would be smaller than an existing object. If the blockSize is currently -1 (unblocked), the -fluff command will not change it.

Also note that the effect of -fluff can be achieved with the -blocksize argument. However, when using -fluff you do not need to look up the current blockSize. This also means -fluff can be used in an automated process.

file -vacuum

Passing vacuum to wellrestd will remove space taken up by deleted objects. This will reorder your objects, giving them new keys. Only do this on restd files for which keys are not stored, e.g., by third parties connecting to services and allowing users to save bookmarks based on the restd keys.

restdman

restdman is a tool to query and modify restd files. It can perform all of the functions of a base service but done via arguments passed in by a user.

file -get

Return all of the objects in the restd file.

file -get key

Return the object at the requested key from the restd file.

file -query terms

Return all of the objects from the restd file that satisfy the query.

file -post json

Add a new object to the end of the restd file.

file -put keyjson

Replace an existing object at the given key with a new one.

file -delete key

Delete the object at the given key, filling the block with space characters.

wget

wget is obviously not a restd-specific tool but since restd tables are just text files, wget is a great way to extract full datasets published by others around the Internet.

diff

diff is also not a restd-specific tool but it is an easy way to tell which objects within two copies of a restd file have changed. Also, saving diffs is a great way to keep track of a change log and replicate changes to copies of restd files hosted elsewhere without downloading the whole restd file.