If you find this post useful, please consider donating in the form of a

If you use OS X or Linux on your desktop/servers, you may be at a point where you have configured a lot of your own settings, configurations, or themes within dotfiles. For the uninformed, dotfiles are files in your home directory that begin with a dot, or full-stop character. This indicates to the operating system that they are hidden files, used to set configuration settings for tools like
vim
, or shells such as
bash
and
zsh
to name a few.

What’s The Point?

If you aren’t convinced it’s worth your time to put your dotfiles into Git version control, consider this:

By storing your dotfiles in a Git repository, you’ll be able to use them on any OS X or Linux machine with Internet access.

This means that in addition to gaining the ability to revert back to a known-working setup should you misconfigure your files, you will also be able to work in an environment you’ve customized yourself. On almost any workstation or server, you’re a simple
git clone
away from the familiarity of your customizations. More on
git clone
later… For now, we’ll begin with an example.

The following is an example of the type of file we would manage with
git
. This is actually an abridged version of my own
.vimrc
. The full version is available for view on my
public Github
:

If you don’t have your own
.vimrc
in your home directory, you’re welcome to use mine. It is intentionally minimal, as it works standalone (you don’t need to do anything other than drop it into your home directory for it to work). That said, we’re going to do things a little differently from here on, so pay attention (if you aren’t already).

The typical location for dotfiles on an OS X or Linux machine would be in a users home directory, e.g.
/home/smalleycreative/.vimrc
. We aren’t typical though, are we? We are trying to be crafty.

For starters, we’ll be putting all of our dotfiles into a folder called
dotfiles
, like so:
/home/smalleycreative/dotfiles/vimrc
. Then, we’ll simply symlink to them from our home directory. To programs like
vim
and
bash
these symlinks are transparent. As far as these programs are concerned, our dotfiles will still appear to exist at the top-level of our home directory, even though they’ll be tucked away in the
dotfiles
directory.

Range iteration latency:
Dr Martens Icon 2216 Safety Shoes Black Size 13 g8jkH9G
’s range iteration is significantly slower than RocksDB’s range iteration, when values are also retrieved from SSD.
We didn’t expect this, and still don’t quite understand it.
We expected some slowdown due to the need to do IOPS on SSD, while RocksDB does purely serial reads. But, given the 100K IOPS
i3.large
instance is capable of, we didn’t even come close to using that bandwidth, despite pre-fetching. This needs further work and investigation.

Range iteration latency:

On the other end of the spectrum,
Kangaroos Men’s KAVU V HiTop Trainers Grau Vapor Grey v0UGv309
’s key-only iteration is blazingly faster than RocksDB or key-value iteration (latency is shown by the almost invisible red bar). This is quite useful in certain use cases we have at Dgraph, where we iterate over the keys, run filters and only retrieve values for a much smaller subset of keys.

Again in a dataset purely containing smaller values, the size of LSM tree would be significantly larger than RocksDB because
Badger
doesn’t run compression on LSM tree. This should be easy to add on if needed, and would make a great first-time contributor project.

1
Recent improvements to SSDs might make B+-trees a viable option. Since WiscKey paper was written, SSDs have made huge gains in random write performance. A new interesting direction would be to combine the value log approach, and keep only keys and value pointers in the B+-tree. This would trade LSM tree read-sort-merge sequential write compactions with many random writes per key update and might achieve the same write throughput as LSM for a much simpler design.

We have built an efficient key-value store, which can compete in performance against top of the line key-value stores in market. It is currently rough around the edges, but provides a solid platform for any industrial application, be it data storage or building another database.

We will be replacing Dgraph’s dependency on RocksDB soon with
Badger
; making our builds easier, faster, making Dgraph cross-platform and paving the way for embeddable Dgraph. The biggest win of using
Badger
is
a performant Go native key-value store.
The nice side-effects are
~4 times faster
Get
and a potential 86% reduction in AWS bills,
due to less reliance on RAM and more reliance on ever faster and cheaper SSDs.

Dr. Bartholomew graduated with a B.A. in Psychology from Harvard University, where he was a three-year letter winner on the varsity football team. He then earned a Ph.D. in Exercise Science with an emphasis in Sport and Exercise Psychology from Arizona State University. He came to The University of Texas in 1996 and lives in Round Rock, TX with his wife Pam and daughters Blair and Emma.

Dr. Bartholomew is the director of the Exercise and Sport Psychology Laboratory. He has two lines of research. His primary interest is in the promotion of physical activity in the elementary school setting and the benefits of physical activity for academic performance. He is also interested in the impact of exercise on mental health, with a specific interest in the use of single bouts of exercise (aerobic and/or resistance) to improve mood and reduce reactivity to stress. His work has been funded by the Texas State Department of Health and the National Institutes of Health.