 | Level: Intermediate Nathan Harrington (harrington.nathan@gmail.com), Programmer, IBM
25 Jul 2006 For the first time, you can hit your computer and get a meaningful response! Using Linux® and the Hard Drive Active Protection System (HDAPS) kernel drivers, you can access the embedded accelerometers on Lenovo (formerly IBM®) ThinkPads, then process the accelerometer data to read specific sequences of "knocking" events -- literally rapping on the laptop case with your knuckles -- and run commands based on those knocks. Double tap to lock the screen, and knock in your secret code to unlock. Tap the display lid once to move your mp3 player to the next track. The possibilities are endless.
In 2003, IBM began releasing ThinkPad laptop computers with integrated accelerometers and associated software
to protect the hard disks when the unit is dropped. Enterprising hackers from IBM and elsewhere have worked to develop modules for the Linux kernel to take advantage of these sensors. On-screen display orientation, desktop switching, even game control and real-time 3D models of the tilt of the laptop are now available. This article presents a new twist -- knock codes -- and a simple program to run commands when specific knock codes are detected.
Using an updated Linux kernel with the HDAPS driver, you can use a simple program called knockAge to generate knock codes. You can also download and use a Perl script to customize your own knocking input environment. See the Downloads and Resources sections at the bottom of this article for links, including links to see knockAge in action.
Hardware requirements
|
Easy does it
As you can see from watching the demonstration video (see the link in Resources, below), the knocking action consists only of light taps with the knuckles. Although the ThinkPad's accelerometers are designed to protect it from accidental impact, too much force can damage it. So be careful out there.
|
|
Many IBM (now Lenovo) ThinkPads manufactured in 2003 or later have the HDAPS hardware. If you are unsure of your hardware configuration, check the technical details on Lenovo's Web site for your model. If you do not have a ThinkPad, this code will not function on your laptop.
This article was written on an x86 architecture. The code in this article was developed and tested on two different models of the ThinkPad T42p. See the Resources section for links to ThinkPad hardware.
If you have an Apple MacBook, you should have the accelerometers and the same general methods available to access them through the kernel. However, the code in this article is untested on Apple hardware.
Software requirements
The HDAPS driver must be included in the kernel to enable access to the accelerometers. Attempts to patch my existing kernel were unsuccessful, so I recommend downloading the latest kernel from your favorite mirror. The newer kernel distributions come with the HDAPS drivers included.
Fire up your kernel configurator of choice, and include the HDAPS drivers in your config. The HDAPS drivers are located under the Device Drivers > Hardware Monitoring Support > IBM Hard Drive Active Protection System (hdaps) option. Further kernel configuration and installation instructions are beyond the scope of this article, but there are plenty of tutorials on the Web for detailed help; see the Resources section for links to help you get started.
This article was developed and tested on Kernel level 2.6.15.1.
Creating a simple knocking sequence
Download the source code repository from the Downloads section and find the knockAge.pl script. This is the main Perl program that allows you to create knock sequences as well as listen for specific knock sequences and run commands. Let's go through the user-space usage and configuration of the knockAge.pl program, and then we'll review its functions.
Run the knockAge.pl program using the following command:
perl knockAge.pl -c
This starts the Perl program listening for knock events and recording their temporal spacing for later use. Once the program is running, impart some percussive impacts upon your laptop's case. You won't need to physically move your ThinkPad for the knock event to register, although some slipping and sliding is likely if your ThinkPad is on a synthetic surface. I recommend holding your ThinkPad on the left side near the hinge with your left hand while knocking with your right hand approximately three inches above the bottom right of the LCD panel on the side of the display frame. Please see the demonstration video linked to in Downloads or Resources for an example of creating a knock sequence.
Experiment with different paces and strengths in your knocking to get a feel for the resolution of events that the knockAge program can capture. This is important for creating more complex knocks.
Your first actual trial knock should be simple, with 0.5 seconds between knocks for a "double tap." Run perl knockAge.pl -c again, and when you see "enter a knock sequence," firmly knock twice on the side of the LCD with one half second delay, then stop. An automatic timeout will occur after 4 seconds (configurable), and your knocking sequence will be printed out similar to the following example:
0 540031 _#_ (command here) _#_ <comments here>
Let's dissect that line -- knock sequence, delimiter, command area, delimiter, and comment area. Your next step is to copy this line into the default configuration file for the knockAge.pl program, {$HOME}/.knockFile, which is probably /home/<username>/.knockFile. Once you have created the .knockFile with the above knocking sequence line, you can modify the line to run a program. Change the (command here) text to /bin/echo "double tap", and modify the comments area to something more descriptive, like so:
0 540031 _#_ /bin/echo "double tap" _#_ Double tap event
Now that you have modified the configuration file to print out a notification, run the knockAge script in daemon mode with the command:
perl knockAge.pl
The program will silently listen in the background for any of the events from the ~/.knockFile listing. Try your double tap with the same temporal spacing, and you will see the text "double tap" printed to the screen. If you want to see the functioning of the knockAge.pl script in more detail, run it in daemon mode with the command:
perl knockAge.pl -v
Locking and unlocking your screen using xscreensaver
Creating a "password" sequence
Run the knockAge.pl program in "create" mode with the command:
perl knockAge.pl -c
You now need to create an unlocking password sequence; I recommend something like "Shave and a Haircut." Make sure you pick something that you can consistently perform accurately. Although you'll be able to modify the parameters that control the precision with which you need to enter your secret knock, it can still be very difficult to match the precise timing. "Shave and a Haircut," in addition to provoking uncontrollable singing in animated rabbits, is a good mix between complexity and simplicity for a screensaver unlocking password. Here is an example knock sequence for "Shave and a Haircut":
0 564025 1185795 621350 516038 960035 444421 _#_ /bin/echo "shave the haircut" _#_ two bits
Before you move on to the next step, you should practice with the above command and the first double tap in your ~/.knockFile configuration file. This will help later when the screensaver is running and it is more difficult to detect if you are knocking correctly.
Command configuration for xscreensaver
The following setup assumes that you are logged into your window manager, and the xscreensaver program has been started by your userid. For example, if you are running Fedora Core 4 and log in to KDE through gdm, xscreensaver is started automatically. So, to activate it, change the double tap command from:
/bin/echo "double tap"
to:
xscreensaver-command -activate &
Now, whenever the "double tap" event is recognized, the xscreensaver program will activate with whatever settings you have specified. Once the screensaver is activated, you can unlock the screen by typing in your password if so configured. What we really want to do, though, is impress our friends with the secret unlocking code to disable the screensaver. So, replace the following command for the "secret password sequence" in your ~/.knockFile:
/bin/echo "shave the haircut"
with:
killall xscreensaver ; nohup xscreensaver -nosplash >/dev/null 2>/dev/null &
This command will kill all of the xscreensaver programs currently running, then restart the xscreensaver in the background. Now you can repeatedly lock and unlock your computer's screensaver just by knocking on the side of the case. Is this faster or better in any way than setting up a custom key combination? Not really. Is it more secure or more convenient than Bluetooth proximity locking? Probably not. Is it cooler? Yep.
More examples
HDAPS sensors and the knockAge.pl program provide an additional user input device that you can use in unique ways. For example:
- If you plan on testing a new X config file on the plane, update the double tap entry to restart your good X server. No more keyboard lockups forcing hard resets.
- Place the location of any shell script you like in the command area, and use a double tap to check your e-mail.
- Knock in the latest break beat from your rave mix, and have the ThinkLight blink out a secret Morse code location of the WWII-era gold storage facility in Kinakuta.
- Tap in Morse code to avoid keyloggers.
See the Resources section for some great examples of reading the "tilt" of the ThinkPad for games, display tools, and more. Or skip right ahead and set the Threshold variable to 15 so when you drop kick your ThinkPad it will automatically reboot.
The knockAge.pl code
History and strategy
The hdaps-gl.c code written by Jeff Molofee is the basis for the knockAge.pl code. Hdaps-gl.c is a great demonstration program of how the tilt sensor can be used to display information about the ThinkPad's orientation in real time. The substantial differences here are the isolation of specific events in time to create a knock, along with the associated code to create and listen for a knock sequence.
Parameter configuration
Let's start at the top of knockAge.pl with the timing and sensor-critical parameters:
Listing 1. Main program parameters
require 'sys/syscall.ph'; # for subsecond timing
my $option = $ARGV[0] || ""; # simple option handling
# filename for hdaps sensor reads
my $hdapsFN = "/sys/devices/platform/hdaps/position";
my $UPDATE_THRESHOLD = 4; # threshold of force that indicates a knock
my $INTERVAL_THRESHOLD = 100000; # microseconds of time required between knock
# events
my $SLEEP_INTERVAL = 0.01; # time to pause between hdaps reads
my $MAX_TIMEOUT_LENGTH = 4; # maximum length in seconds of knock pattern
# length
my $MAX_KNOCK_DEV = 100000; # maximum acceptable deviation between recorded
# pattern values and knocking values
my $LISTEN_TIMEOUT = 2; # timeout value in seconds between knock
# events when in listening mode
|
These variables and their comments are relatively straightforward. Their usage and configuration options are explained later in this article. The following is the remainder of the global variables and their descriptions.
Listing 2. Knock pattern parameters
my @baseKnocks = (); # contains knock intervals currently entered
my %knockHash = (); # contains knock patterns, associated commands
my $prevInterval = 0; # previous interval of time
my $knockCount = 0; # current number of knocks detected
my $restX = 0; # `resting' positiong of X axis accelerometer
my $restY = 0; # `resting' positiong of Y axis accelerometer
my $currX = 0; # current position of X axis accelerometer
my $currY = 0; # current position of Y axis accelerometer
my $lastX = 0; # most recent position of X axis accelerometer
my $lastY = 0; # most recent position of Y axis accelerometer
my $startTime = 0; # to manage timeout intervals
my $currTime = 0; # to manage timeout intervals
my $timeOut = 0; # perpetual loop variable
my $knockAge = 0; # count of knocks to cycle time interval
|
Subroutines
First in our list of subroutines is a simple logic block to check if the accelerometer is available for reading:
Listing 3. Check accelerometer subroutine
sub checkAccelerometer() {
my $ret;
$ret = readPosition ();
if( $ret ){
print "no accelerometer data available - tis bork ed\n";
exit(1);
}
}#checkAccelerometer
|
The hdaps-gl.c code from Jeff Molofee provides a great starting point for all of the code in knockAge.pl. You can see the vestiges of his comments in the readPosition subroutine, below. This subroutine simply opens the file, reads the current accelerometer data, closes the file, and returns the data without the , (comma) characters.
Listing 4. readPosition subroutine
## comments from Jeff Molofee in hdaps-gl.c
#* read_position - read the (x,y) position pair from hdaps.
#*
#* We open and close the file on every invocation, which is lame but due to
#* several features of sysfs files:
#*
#* (a) Sysfs files are seekable.
#* (b) Seeking to zero and then rereading does not seem to work.
##
sub readPosition() {
my ($posX, $posY) = "";
my $fd = open(FH," $hdapsFN");
while( <FH> ){
s/\(//g;
s/\)//g;
($posX, $posY) = split ",";
}# while read
close(FH);
return( $posX, $posY );
}#readPosition
|
getEpochSeconds and getEpochMicroSeconds provide detailed and precise information on the status of the knock patterns.
Listing 5. Time splitters
sub getEpochMicroSeconds {
my $TIMEVAL_T = "LL"; # LL for microseconds
my $timeVal = pack($TIMEVAL_T, ());
syscall(&SYS_gettimeofday, $timeVal, 0) != -1 or die "micro seconds: $!";
my @vals = unpack( $TIMEVAL_T, $timeVal );
$timeVal = $vals[0] . $vals[1];
$timeVal = substr( $timeVal, 6);
my $padLen = 10 - length($timeVal);
$timeVal = $timeVal . "0" x $padLen;
return($timeVal);
}#getEpochMicroSeconds
sub getEpochSeconds {
my $TIMEVAL_T = "LL"; # LL for microseconds
my $start = pack($TIMEVAL_T, ());
syscall(&SYS_gettimeofday, $start, 0) != -1 or die "seconds: $!";
return( (unpack($TIMEVAL_T, $start))[0] );
}#getEpochSeconds
|
Next up is the knockListen subroutine, the first five lines of which read the current accelerometer data values and adjust for the base value readings. The checkKnock variable is set to 1 if the accelerometer magnitude in either dimension is greater than the update threshold value. To adjust the program to only respond to intense knocking events or similar acceleration values, increase the update threshold. For example, you could place your ThinkPad in your car and have it change your mp3 playlist only when hard acceleration (or deceleration!) is detected.
If you knocked the laptop hard enough, and the update threshold has been passed, the getEpochMicroSeconds subroutine is called. The diffInterval variable is then assigned to the duration between knock events. This value is used to compress many rapid acceleration readings greater than the update threshold into one event. Without the interval threshold check, a single hard knock will register as multiple events as the accelerometer continues to issue high magnitudes for an extended time. This behavior is incongruous with the user perception both in sight and touch. A knock is a knock to us, but apparently not to the HDAPS. If the interval threshold has been reached, the knock interval is recorded in the baseKnocks array, and the interval between knocks is reset.
The careful modification of these variables will help tune the program to recognize your particular knocking style. Reduce the update threshold and increase the interval threshold to detect widely spaced soft knocks. Mechanical knocking devices or specific knock methods may require lowering the interval threshold to recognize distinct knock events.
Listing 6. knockListen subroutine
sub knockListen() {
my $checkKnock = 0;
($currX, $currY) = readPosition();
$currX -= $restX; # adjust for rest data state
$currY -= $restY; # adjust for rest data state
# require a high threshold of acceleration to ignore non-events like
# bashing the enter key or hitting the side with the mouse
if( abs ($currX) > $UPDATE_THRESHOLD) {
$checkKnock = 1;
}
if( abs ($currY) > $UPDATE_THRESHOLD) {
$checkKnock = 1;
}
if( $checkKnock == 1 ){
my $currVal = getEpochMicroSeconds();
my $diffInterval = abs($prevInterval - $currVal);
# hard knock events can create continuous acceleration across a large time
# threshold. requiring an elapsed time between knock events effectively
# reduces what appear as multiple events according to sleep_interval and
# update_threshold into a singular event.
if( $diffInterval > $INTERVAL_THRESHOLD ){
if( $knockCount == 0 ){ $diffInterval = 0 }
if( $option ){
print "Knock: $knockCount ## last: [$currVal] curr: [$prevInterval] ";
print "difference is: $diffInterval\n";
}
push @baseKnocks, $diffInterval;
$knockCount++;
}# if the difference interval is greater than the threshold
$prevInterval = $currVal;
}#if checkknock passed
}#knockListen
|
When a knock pattern is created, it is placed in the ~/.knockFile file, and read by the following subroutine
Listing 7. Read knock file
sub readKnockFile {
open(KNCKFILE,"$ENV{HOME}/.knockFile") or die "no knock file: $!";
while(<KNCKFILE>){
if( !/^#/ ){
my @arrLine = split "_#_";
$knockHash{ $arrLine[0] }{ cmd } = $arrLine[1];
$knockHash{ $arrLine[0] }{ comment } = $arrLine[2];
}#if not a comment line
}#for each line in file
close(KNCKFILE);
}#readKnockFile
|
When a knocking pattern is acquired by knockListen, it is compared to the existing knock patterns loaded from readKnockFile. The compareKnockSequences subroutine below performs a simple difference check between the timings of the knocks. Note that the differences between knocks is not compounded: missing the timing on many knocks by a small amount will not accumulate into a total match failure.
The first comparison is between the number of knocks, as there is no point comparing a seven-knock sequence to a two-knock sequence. If the number of knocks matches an existing knock sequence from ~/.knockFile, and the difference between knocks is less than than the maximum knock deviation, the knock is a match. Maximum knock deviation is critical to allowing the matching of knock sequences with accuracy, not precision. You can increase the maximum knock deviation to allow you to be more liberal in your rhythmic timings, but be warned, this can cause erroneously matched patterns. For example, try increasing the maximum knock deviation from 100000 to 500000 microseconds. This will allow your knock patterns to deviate as much as half a second before or after the expected time, and still cause a match. This effectively means that "Shave and a Haircut" can match to "Mary Had a Little Lamb", so be wary of changing this parameter.
If the full pattern is a match, the command specified in the ~/.knockFile is run, and the result printed out if verbose mode is enabled. The next step is to exit the subroutine if no matches are found, or reset the recorded knocks if a match is made. The compareKnockSequences subroutine performs this step:
Listing 8. Compare knock sequences
sub compareKnockSequences {
my $countMatch = 0; # record how many knocks matched
# for each knock sequence in the config file
for( keys %knockHash ){
# get the timings between knocks
my @confKnocks = split;
# if the count of knocks match
if( $knockCount eq @confKnocks ){
my $knockDiff = 0;
my $counter = 0;
for( $counter=0; $counter<$knockCount; $counter++ ){
$knockDiff = abs($confKnocks[$counter] - $baseKnocks[$counter]);
my $knkStr = "k $counter b $baseKnocks[$counter] ".
"c $confKnocks[$counter] d $knockDiff\n";
# if it's an exact match, increment the matching counter
if( $knockDiff < $MAX_KNOCK_DEV ){
if( $option ){ print "MATCH $knkStr" }
$countMatch++;
# if the knocks don't match, move on to the next pattern in the list
}else{
if( $option ){ print "DISSONANCE $knkStr" }
last;
}# deviation check
}#for each knock
}#if number of knocks matches
# if the count of knocks is an exact match, run the command
if( $countMatch eq @confKnocks ){
my $cmd = system( $knockHash{"@confKnocks "}{ cmd } );
if( $option ){ print "$cmd\n" }
last;
# otherwise, make the count of matches zero, in order to not reset
}else{
$countMatch = 0;
}
}#for keys
# if the match count is zero, exit and don't reset variables so a longer
# knock sequence can be entered and checked
if( $countMatch == 0 ){ return() }
# if a match occurred, reset the variables so it won't match another pattern
$knockCount = 0;
@baseKnocks = ();
}#compareKnockSequences
|
Main program logic
With the subroutines in place, the main program logic allows the user to create a knock sequence, or runs in daemon mode to listen for knocks and execute commands. The first section is executed when the user specifies option -c, for create mode. A simple timeout process is used to end the knock sequence. Increase the maximum timeout length variable to permit pauses of more than four seconds between knocks. If you leave the maximum timeout length at four, the program will end and print your currently entered knock sequence.
Listing 9. Create sequence main logic
if( $option eq "-c" ){
print "create a knock pattern:\n";
$startTime = getEpochSeconds(); # reset time out start
while( $timeOut == 0 ){
$currTime = getEpochSeconds();
# check if there has not been a knock in a while
if( $currTime - $startTime > $MAX_TIMEOUT_LENGTH ){
$timeOut = 1; # exit the loop
}else{
# if a knock has been entered before timeout, reset timers so
# more knocks can be entered
if( $knockCount != $knockAge ){
$startTime = $currTime; # reset timer for longer delay
$knockAge = $knockCount; # synchronize knock counts
}# if a new knock came in
}# if timer not reached
knockListen();
select(undef, undef, undef, $SLEEP_INTERVAL);
}#timeOut =0
if( @baseKnocks ){
print "place the following line in $ENV{HOME}/.knockFile\n\n";
for( @baseKnocks ){ print "$_ " }
print "_#_ (command here) _#_ <comments here>\n\n";
}#if knocks entered
|
Section two of the main logic listens for knocks in an infinite loop, sleeping for approximately one hundredth of a second in each loop. A seconds-based timeout is also used in this loop to reset the knock sequences after sufficient delay. Note that in this example, the knock listen timeout is for two seconds, whereas the maximum timeout length is four seconds. This provides for a simple testing setup during the knock creation mode, and a fast resetting option for knock sequence listen mode.
Listing 10. Knock listen main code
}else{
# main code loop to listen for knocking and run commands
readKnockFile();
$startTime = getEpochSeconds();
while( $timeOut == 0 ){
$currTime = getEpochSeconds();
if( $currTime - $startTime > $LISTEN_TIMEOUT ){
$knockCount = 0;
@baseKnocks = ();
$startTime = $currTime;
if( $option ){ print "listen timeout - resetting knocks \n" }
}else{
if( $knockCount != $knockAge ){
$startTime = $currTime; # reset timer for longer delay
$knockAge = $knockCount; # synchronize knock counts
}# if a new knock came in
compareKnockSequences();
}#if not reset timeout
knockListen();
select(undef, undef, undef, $SLEEP_INTERVAL);
}#main knock listen loop
}# if create or listen for knocks
|
Caveats, security
The knockAge program is well suited for providing an additional channel of user input for your system. However, be wary of using knockAge to do anything requiring authentication on your system. Yes, it can defeat key loggers sniffing for passwords, but there are many other variables associated with "knock authentication" suggesting that usage in any serious context is premature at best. The knock sequences are currently stored as 4-9 digit representations of the delay in microseconds in the ~/.knockFile. It is comparatively easy to read this "password" file and simply try and match the knock pattern to gain access to this system. One-way hashes could be used by eliminating some of the precision in the microseconds values, but this exercise is best left to readers wanting to evaluate the risks on their own.
Before deployment in any serious environment, studies should be done to determine whether users have a sufficiently variable and precise knocking apparatus. For example, do we have the spatio-temporal motor skills to create and consistently enter knocking passwords of acceptable strength? Does the average human mind have the capability to intuitively work with knock sequences? Or are we all going to use "Shave and a Haircut" as our password?
Downloads | Description | Name | Size | Download method |
|---|
| Sample Perl scripts for this article | knockAge-0.1.zip | 4KB | HTTP |
|---|
| knockAge xscreensaver demo (QuickTime movie) | knockAge_demo_xscreensaver.zip | 6319KB | HTTP |
|---|
Resources Learn
Get products and technologies
-
Get HDAPS from your favorite Linux kernel download mirror.
-
Order the SEK for Linux, a two-DVD set containing the latest IBM trial software for Linux from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
-
With IBM trial software, available for download directly from developerWorks, build your next development project on Linux.
Discuss
About the author | | | Nathan Harrington is a programmer at IBM currently working with Linux and resource locating technologies. |
Rate this page
| |
