Sunday, June 23, 2013

Emulating touchscreen interaction with sendevent in Android

I know this has been explained many times before. But I keep getting asked  about it over and over again. So I decided to summarize it in one place I could refer to in the future.

Android uses Linux kernel so it processes input events the same way as any other linux system. In case of touchscreen displays it uses the same multi-touch protocol. There are 2 different versions of the protocol - Type A and Type B. Usually it does not matter which protocol to use for injecting events since the kernel driver supports both protocols at the same time. In my automation scripts I use Type A just because I tried it first and it worked right away and I did not have to try Type B.

Android provides these two convenient tools for dealing with input events:

  • getevent - for dumping input events and providing information about input devices
  • sendevent - for injecting input events

Every sendevent command requires 4 parameters:
  • device_name (string)
  • event_type (decimal int)
  • event_code (decimal int)
  • value (decimal int)

First you need to find the name of touchscreen device. The following command requires busybox to be installed:

getevent -pl | busybox sed -e ':a;N;$!ba;s/\n / /g' | busybox awk '/ABS_MT_TOUCH/{print $4}'

Let's assume it printed out "/dev/input/event0" - this would be the first parameter.

For touch events only 2 event types are used:
  • EV_ABS (3)
  • EV_SYN (0)

Touching the display (in case of Type A protocol) will result in an input report (sequence of input events) containing the following event codes:
  • ABS_MT_TRACKING_ID (57) - ID of the touch (important for multi-touch reports)
  • ABS_MT_POSITION_X (53) - x coordinate of the touch
  • ABS_MT_POSITION_Y (54) - y coordinate of the touch
  • ABS_MT_TOUCH_MAJOR (48) - basically width of your finger tip in pixels
  • ABS_MT_PRESSURE (58) - pressure of the touch
  • SYN_MT_REPORT (2) - end of separate touch data
  • SYN_REPORT (0) - end of report

Let's say we want to emulate a touch down event at the point with coordinates x=300, y=400. We will need to execute the following sendevent commands:

sendevent /dev/input/event0 3 57 0
sendevent /dev/input/event0 3 53 300
sendevent /dev/input/event0 3 54 400
sendevent /dev/input/event0 3 48 5
sendevent /dev/input/event0 3 58 50
sendevent /dev/input/event0 0 2 0
sendevent /dev/input/event0 0 0 0

I used arbitrary values of 5 for the ABS_MT_TOUCH_MAJOR (makes for very small finger tip for high precision) and 50 for the ABS_MT_PRESSURE (a slight tap) which work good enough for most applications.

For most touch screens on the market it takes 20 to 50 milliseconds to reliably register the touch. So I would recommend to wait at least 50 milliseconds before sending the release event report. If you want to emulate the long touch - then you wait longer. You can use busybox usleep command:

busybox usleep 50000

The release report is really simple. To let the input device know that all previous touches have been released - you just send the empty report with ABS_MT_TRACKING_ID = -1:

  • SYN_REPORT (0)

sendevent /dev/input/event0 3 57 -1
sendevent /dev/input/event0 0 2 0
sendevent /dev/input/event0 0 0 0

This is how injecting touch events could be implemented in python:

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.