 | Level: Introductory Nathan Harrington, Programmer, IBM
25 Nov 2008 Learn how to apply the open source tools cnee and Perl in mouse-click dynamics to measure the more subtle characteristics of human-computer interaction. Then use the number and hold time of mouse-click events to help authenticate users.
Logging into a computer consists of entering a user ID, password, or on some ThinkPads, reading a fingerprint. But as I show in three earlier articles, it's possible to authenticate a user with other means, too. Think of these articles as a proof-of-concept for authenticating a user independently of other credentials.
In this article, we discuss how mouse-click hold times, or a combination of keyboard and mouse activity can enable new levels of access-requirements obfuscation. Learn how to apply the open source tools cnee and Perl in applications to measure the characteristic attributes of how users manipulate the mouse.
In addition to the many identification options presented by keystroke dynamics discussed in my three earlier articles, mouse usage provides additional data to help determine identity. This article uses example code to demonstrate mouse dynamics for enhancing the security of applications in authentication and continuous usage contexts. With an example login application and using the cnee tool to measure mouse events, learn how to add an additional layer of user-interaction requirements.
Hardware and software requirements
Linux® is required, with a functioning X Window System setup. Any distribution equipped with a graphical desktop released after 1999 should provide an already-configured environment with which to get started. You also need physical access to a computer with a directly connected mouse. Thin clients, VNC connections, and virtualized clients will work in theory, but the various layers of processing and network lag may cause unexpected behavior.
Effective capture and processing of X Window System events is a task handled best by a program like GNU Xnee. Check the Resources to download Xnee (and cnee, the command-line program used here). You'll also need Perl, which is standard on just about every Linux distribution today. Also required are the Tk, Tk::Png, threads, and Thread::Queue Perl modules.
And for generating encrypted hashes of the mouse-click times, you'll need the mkpasswd program.
demoLogin.pl program
Creating a demonstration interface, measuring click times, and eventually comparing them to a stored signature is performed by the demoLogin.pl program. The program presents an interface similar to the GDM face browser using the Tk GUI generation modules. An instance of the cnee program runs in the background to record mouse data precisely. This approach can be used to record attributes for certain applications or for the entire X Window System. This article focuses on demonstrating the monitoring of a single application, and the framework required to generate and compare signatures effectively.
Figure 1 shows the interface created by the demoLogin.pl program. The standard login UI widgets are presented: a highlighted user picture, the password text box, and the Submit button. Each of these is clicked by the user and has its click-hold time measured to create a user identification signature.
Figure 1. Example demoLogin.pl user interface
Listing 1 shows the beginning of the demoLogin.pl program.
Listing 1. demoLogin.pl header
#!/usr/bin/perl -w
# demoLogin.pl - example mouse dynamics login screen
use strict;
use Tk; # GUI handling
use Tk::PNG; # load user images
use threads; # for asynchronous pipe reads
use Thread::Queue; # for asynchronous pipe reads
die "specify record or compare mode " unless @ARGV == 1;
my $mode = $ARGV[0];
my( $salt, $hash ) = ""; # to be read from mouse.signatures
my $checkRng = 5; # fuziness of click time matching
my $userMatch = 0; # click time match found
my @clicks = (); # significant press time for mouse click
|
After including the requisite modules and specifying usage requirements, global variables are defined. Note the $checkRng variable and its description. Modification of this variable is key to determining what amount of variability is acceptable during a click-time signature match. The section demoLogin.pl compare signatures contains more information on this variable and its use. Listing 2 continues with the program and GUI setup.
Listing 2. demoLogin.pl GUI setup
# cnee is used to monitor X windows mouse events
my $cneeCmd = "cnee --record --mouse -e xnee.err 2> raw.err | ";
# open an asynchronous pipe to store events while tk churns
my $pipe = createPipe( $cneeCmd ) or die "cannot create cnee cpipe";
# create a list of images to load (active and inactive)
my @inactiveImages = `ls -1 images/*Inactive.png`; chomp(@inactiveImages);
my @activeImages = `ls -1 images/*Active.png`; chomp(@activeImages);
# create a Tk window and a canvas to overlay user images on
my $mw = Tk::MainWindow->new;
my $can = $mw->Canvas( -width => 648, -height => 226,
-borderwidth => 0, -highlightthickness => 0
)->pack( -fill => "both", -expand => 1);
|
As mentioned, cnee (part of the Xnee package) is started in the background to monitor mouse events while the Tk GUI is running. Two arrays are created that store a list of the active and inactive images. A Tk canvas is then created to provide a background for the inactive and active images. Listing 3 contains the rest of the interface GUI code.
Listing 3. demoLogin.pl inactive image setup
# load all of the inactive images onto the canvas
my $imgCount = 0;
while( $imgCount <= $#inactiveImages )
{
my $tempImg = $mw->Photo( -file => $inactiveImages[$imgCount],
-format => "png");
$can->createImage( ($imgCount * 150) +40, 50,
-image => $tempImg,
-anchor => 'nw',
-tags =>[" $imgCount"] );
my $setVar = " $imgCount"; # this has to be set or variables get passed by
# reference.
# call the active image display on click
$can->bind( $setVar, '<1>' => sub { clickImage( $setVar ) });
$imgCount++;
}#while each image
# left space fill so entry and submit buttons placement ease
my $cantop2 = $mw->Canvas( -width => 50, -height => 150,
-highlightthickness => 0 )->pack( -side => "left");
# asterisk entry box and submit button to run the click time processing
$mw->Entry ( -show => '*' )->pack( -side => "top");
$mw->Button( -text => 'Submit',
-command => sub{ runSubmit() }) -> pack( -side => "top");
MainLoop();
|
Each inactive user image is loaded onto the canvas, and a callback is created to overlay the active image upon user click. After the loop, another canvas, password text box, and Submit button are created and packed into the Tk main window. Listing 4 shows the clickImage subroutine to provide user feedback when an image is clicked.
Listing 4. clickImage subroutine
sub clickImage
{
# overlay the inactive image with the 'active' one
my $imgNum = $_[0];
my $tempImg = $mw->Photo( -file => $activeImages[$imgNum],
-format => 'png' );
$can->createImage( ($imgNum * 150) +40, 50,
-image => $tempImg,
-anchor => 'nw',
-tags => ["activeImage$imgNum"] );
}#clickImage
|
Simply overlaying the active image upon the inactive image is sufficient for this demonstration program. Listing 5 continues with the UI callbacks.
Listing 5. runSubmit subroutine
sub runSubmit
{
# process the recorded cnee events, create signature or attempt match
sleep(1); # wait to ensure tk events bubble up to cnee
processEvents();
if( $mode eq "record" )
{
print "@clicks\n", `echo "@clicks" | mkpasswd -H md5 --stdin `;
}#if in record mode
exit(0);
}#runSubmit
|
If events are extracted immediately after the Tk mouse is pressed on Submit, the mouse-down event (and subsequent release event) will not be picked up by the cnee program. Sleeping 1 second is a simple method for ensuring all of the events have time to process through the cnee event detector and printout stages. In record mode only, the click-timing events are printed to STDOUT, along with the hash signature for those particular events. Listing 6 shows the processEvents subroutine called in Listing 5.
Listing 6. processEvents subroutine
sub processEvents
{
# read all relevant mouse events from cnee
my $lastTime = ""; # store initial mouse press time
while( $pipe->pending )
{
my $line = $pipe->dequeue or next;
# skip line unless it has mouse information ( 7 commas and no colons )
next unless( ( () = $line =~ /,/g ) == 7 && $line !~ /:/ );
my( undef, $button, $x, $y, undef, undef, undef, $time ) = split ",", $line;
if( $button == 4 ){ $lastTime = $time } # mouse down 4, mouse release 5
# only grab the most significant digits of the mouse click-hold time
if( $button == 5 ){ push @clicks, sprintf("%0.0f", ($time-$lastTime)/10) }
}#for each line of data
}#processEvents
|
Mouse position and button information is printed out in the format shown in Listing 7. Subroutine processEvents records the mouse-down and up times, then extracts the most significant bits from that time. For example, the mouse-hold time shown below is 574 milliseconds. The sprintf function call shown in Listing 6 changes this to a click time of 57. This allows for a more-reasonable search space for matching key-hold time signatures.
Listing 7. Example cnee output
0,6,427,141,0,0,0,1319180351
0,4,0,0,1,0,0,1319181256
0,5,0,0,1,0,0,1319181830
0,6,428,141,0,0,0,1319182838
|
Listing 8 shows the createPipe subroutine used to set up a background connection to the cnee program.
Listing 8. createPipe subroutine
sub createPipe
{
my $cmd = shift;
my $queue = new Thread::Queue;
async{
my $pid = open my $pipe, $cmd or die $!;
$queue-<enqueue( $_ ) while <$pipe>;
$queue-<enqueue( undef );
}-<detach;
#detach causes the threads to be silently terminated on exit (sometimes)
return $queue;
}#createPipe
|
Save these code samples as demoLogin.pl and follow the instructions below to begin measuring mouse dynamics.
demoLogin.pl record usage
Unpack the images directory from the source code archive. Run the program with the command perl demoLogin.pl record. You should see a window pop up that looks similar to Figure 1.
Try different click-hold times on the different UI elements. For example, hold the mouse-button-down for nearly 2 seconds on an image, then click the password-entry box and the Submit button with "normal" release times. This will result in the program computing a click signature like that shown below.
Listing 9. Example demoLogin.pl record output
196 4 2
$1$8spGJq5D$Nhz7i/cEfqW0VHfBz31F31
|
Three numbers for three clicks, the first being 196-tenths of a second in hold time. The remaining two numbers represent more "normal" click-hold times of four-tenths and two-tenths of a second.
Salt and hash information is printed on the second line, and should be placed in a file called mouse.signatures. Messages like "A thread exited while 2 threads were running" can be ignored safely.
Depending on your click-hold time repetition skill, you may choose to create a signature based on number of clicks. Simply require the user to click a certain number of times, such as requiring a double-click on the password-entry box. The click signature will then have the requisite number of clicks stored in the mouse.signatures file.
demoLogin.pl compare signatures
With a set of repeatable timings and a mouse signature in place, the demoLogin.pl program can be modified to support signature comparisons. Start by replacing line 90 (}#if in record mode) with the lines shown below.
Listing 10. Replacement signature checking logic block
}else
{
loadSignatureFile();
checkDynamics("", 0);
print "Match found\n" if( $userMatch == 1 );
print "Unknown mouser!\n" if( $userMatch == 0 );
}#if not in record mode
|
Now when the demoLogin program is run with the "compare" option, the signature file will be loaded and the checkDynamics subroutine called. Add the subroutine shown in Listings 11 to the end of the demoLogin.pl file.
Listing 11. loadSignatureFile subroutine
sub loadSignatureFile
{
open(INFILE,"mouse.signatures") or die "no signature file";
my $line =<INFILE>;
die "empty file " unless defined $line;
chomp($line);
( undef, undef, $salt, $hash ) = split '\$', $line;
close(INFILE);
}#loadSignatureFile
|
loadSignatureFile simply reads the salt and hash stored from the record phase. Listing 12 shows the more complex checkDynamics subroutine.
Listing 12. checkDynamics subroutine
sub checkDynamics
{
# recursive 'fuzziness' matching for click time signatures
print "checking level $_[1] instring $_[0]\n";
my $inString = $_[0];
my $level = $_[1];
my $start = $clicks[$level] - $checkRng;
my $stop = $clicks[$level] + $checkRng;
my $curr = $start;
while( $curr <= $stop && $userMatch != 1 )
{
if( $level == 2 ) # deepest level for only three letters
{
my $res = `echo "$inString $curr" | mkpasswd -S $salt -H md5 --stdin`;
chomp($res);
if( $res eq qq/\$1\$${salt}\$${hash}/ ){ $userMatch = 1 }
}else
{
# append to the current 'signature', go to next level
my $tempStr = ""; # temporary signature string
if( length($inString) != 0 ){ $tempStr = "$inString $curr" }
else { $tempStr = $curr }
checkDynamics( $tempStr, $level+1 );
}#if at maximum level
$curr++;
}#while current less than stop
return("");
}#checkDynamics
|
The checkDynamics subroutine recursively calls itself while building signatures encompassing the full range of possibilities defined by the checkRng parameter. Each string passed to mkpasswd is built level by level from a single mouse-click-hold time all the way up to a click-hold time for each recorded click by the user. For example, if the click-hold times are "197 10 10," the checkDynamics subroutine will work through necessary permutations to check "192 5 5," "203 15 15," and everything in between. Loose matching (with a high checkRng value) will drastically increase the amount of time required to check all possibilities.
demoLogin.pl compare usage
Run the program with the command perl demoLogin.pl compare and click the UI buttons. If you match the signature as recorded in the mouse.signatures file, you should see and output similar to that shown below.
Listing 13. Example demoLogin.pl compare output
checking level 0 instring
checking level 1 instring 196
checking level 2 instring 196 -1
checking level 2 instring 196 0
checking level 2 instring 196 1
checking level 2 instring 196 2
checking level 2 instring 196 3
checking level 2 instring 196 4
checking level 2 instring 196 5
checking level 2 instring 196 6
checking level 2 instring 196 7
checking level 2 instring 196 8
checking level 2 instring 196 9
checking level 1 instring 197
checking level 2 instring 197 -1
checking level 2 instring 197 0
checking level 2 instring 197 1
checking level 2 instring 197 2
checking level 2 instring 197 3
checking level 2 instring 197 4
checking level 2 instring 197 5
checking level 2 instring 197 6
Match found
|
Conclusion, further examples
You can use the techniques described in this article to enhance the authentication options of your existing Perl applications or monitor the mouse usage in an existing program.
In addition, mouse dynamics is one of the few fields where biometrics can be used for continuous authentication. Consider modifying your applications to use some of the techniques described in this article to detect and track common click positions, click-hold times, and other mouse usage patterns that are unique to your application's users. Try advancing the captcha arms race through measured click-hold times for your Web authentication applications.
Download | Description | Name | Size | Download method |
|---|
| Sample code | os-userauth-mouse.mouseDynamics_0.1.zip | 185KB | HTTP |
|---|
Resources Learn
- The author of this article wrote three earlier articles for developerWorks on authenticating a user independently of other credentials. See:
- Use Xnee and cnee for monitoring mouse events.
-
To listen to interesting interviews and discussions for software developers, check out developerWorks podcasts.
-
Stay current with developerWorks' Technical events and webcasts.
-
Check out upcoming conferences, trade shows, webcasts, and other Events around the world that are of interest to IBM open source developers.
-
Visit the developerWorks Open source zone for extensive how-to information, tools, and project updates to help you develop with open source technologies and use them with IBM's products.
-
Watch and learn about IBM and open source technologies and product functions with the no-cost developerWorks On demand demos.
-
Learn how to install Perl modules and read more about Perl.
Get products and technologies
Discuss
About the author | 
| | Nathan Harrington is a programmer at IBM currently working with Linux and resource-locating technologies. |
Rate this page
| |
