newsObjectify - News: Generated 50,331,648 swizzle tables in 2009 with no duplicates

Latest News
Admin Name Change posted by redwards, Sun 08 Jul 2018 12:26:12 AM UTC
Remote Storage posted by jsedwards, Sat 28 Apr 2012 06:09:36 PM UTC
Objectify Alpha_31.2 Released posted by jsedwards, Fri 20 Apr 2012 09:40:13 PM UTC
Public Objects 111111-0030 Released posted by jsedwards, Sat 12 Nov 2011 01:59:12 PM UTC
Objectify Alpha_31.1 Released posted by jsedwards, Sun 06 Nov 2011 03:18:33 AM UTC

Generated 50,331,648 swizzle tables in 2009 with no duplicates

Item posted by J. Scott Edwards <jsedwards> on Fri 22 Jan 2010 02:11:06 PM UTC.

Note: I have been working on a new release of Objectify and the public objects. This will be the first one on Savannah. I hope to have it finished in the next week or so. Please check back.

As an extra security measure Objectify uses, what I call, "swizzle tables" to scramble blocks of data after they have been encrypted. Currently these swizzle tables are generated from the pass phrase along with the encryption key.

A couple of years ago I came up with the idea of generating the swizzle tables at random and storing them on a USB stick. That way someone would have to have three things to access your data: access to your Objectify archive, your pass phrase, AND the key (swizzle tables). As it is now, if someone has access to your Objectify archive and somehow obtains your pass phrase, they can access your data.

Each key consists of 4096 swizzle tables.  Each swizzle table is 248 bytes.  Making the key over 1 million bytes (1,015,808 to be exact).

I wanted to make sure the swizzle tables were as random as possible and not likely to be replicated by someone. So in June 2008 I wrote a program called "test_random".  It generates a key of 4096 swizzle tables so I could see how often duplicates occur.

Unfortunately, I got into crunch mode at work and did not have time to really test it at the time. Then I got busy with other tasks and it fell off my radar until last October. It had been so long I couldn't even find the source file and had to search back through a bunch of tarballs for it.

Each invocation of the test_random program generates a full key (4096 - 248 byte tables). On my more modern machines it takes 5 to 6 minutes to generate one key. On the oldest machine I have tried it on, it takes 11.4 hours. (Note: that machine is also running OpenBSD instead of GNU/Linux, so some of the difference is likely because the OS handles usleep differently. The test_random program uses usleep as a way of generating entropy.)

Off and on, from October through December, I ran test_random on all of the machines I have access to. At one point I fired up them all up at the same time and had them all running test_random. It was kind of fun, it was like having my own little supercomputer crunching away. I should note that I ran multiple instances of it on some machines to get better efficiency.

The result was, in 2009, I generated 12288 keys that contained 50,331,648 swizzle tables. I have checked all 50 million and there wasn't a single duplicate.

I have generated another 20 million tables so far in 2010, but I haven't had time to check them for duplicates yet.

I plan to incorporate the random swizzle tables into Objectify this year, so stay tuned.


Back to the top

Powered by Savane 3.12.
Corresponding source code