Thursday, November 13, 2014

How to make the truly universal adb and fastboot drivers for Windows

There are many internet sites out there offering so-called universal adb driver packs for Windows. Usually it's a repacked Google USB Driver package for Nexus devices with its android_winusb.inf file updated to include various USB VendorID and ProductID combinations. Why it is not truly universal? Because if your device's combination is not included - it obviously will not be supported by such driver package.

Yet Microsoft has made it quite easy to make a truly universal driver. When looking for a proper driver for the new device Windows can match not only by the HardwareID (i.e. combination of USB VendorID, ProductID and optionally InterfaceNumber) but also by CompatibleID (i.e. combination of InterfaceClassID, InterfaceSubClassID and InterfaceProtocolID) which is fortunately the same for all android adb and fastboot interfaces for both simple and composite USB devices.

A few years ago I have made such driver and I have been successfully using it with many different Android devices before I completely switched to linux environment for all my development. Here's how you can make one for yourself:

  • Download the latest Google USB driver package from the link above
  • Unzip the usb_driver folder
  • Open the usb_driver\android_winusb.inf file in a text editor (the Notepad will work)
  • Replace the [Google.NTx86] and [Google.NTamd64] sections with the lines below:

  • [Google.NTx86]
    %SingleAdbInterface% = USB_Install, USB\Class_ff&SubClass_42&Prot_01
    %SingleBootLoaderInterface% = USB_Install, USB\Class_ff&SubClass_42&Prot_03

    [Google.NTamd64]
    %SingleAdbInterface% = USB_Install, USB\Class_ff&SubClass_42&Prot_01
    %SingleBootLoaderInterface% = USB_Install, USB\Class_ff&SubClass_42&Prot_03

  • Save the file and you got yourself a truly universal adb driver!


Do not forget to update %USERPROFILE%\.android\adb_usb.ini if needed.

After installing properly modified driver the following command should say WinUSB in the Service line:

powershell "gwmi Win32_USBControllerDevice | %{[wmi]($_.Dependent)} |
?{$_.CompatibleID -like \"USB\Class_ff^&SubClass_42^&Prot_0?\"} | fl Name,DeviceID,Service"


And here is the whole inf file already modified:

Wednesday, November 5, 2014

ADB device detection in Windows

In my previous post I mentioned the command I use to automatically populate the ~/.android/adb_usb.ini file with USB Vendor IDs for all connected ADB devices:

~$ find -L /sys/bus/usb/devices -maxdepth 2 -path "*/modalias" -printf "%h\t" -exec cat {} \; | \
awk -F: '/icFFisc42ip0/ {print $1}' | xargs -i cat {}/idVendor | awk '{print"0x"$1}' | sort -u >> ~/.android/adb_usb.ini



But many people also have found this command to be extremely useful for ADB connection trouble-shooting. The command is obviously linux only. And some of my colleagues who still use Windows asked me if I happened to know a similar command for Windows. I gave it some thought and decided to use powershell for the task and here is what I have come up with:

powershell "gwmi Win32_USBControllerDevice | %{[wmi]($_.Dependent)} |
?{$_.CompatibleID -like \"USB\Class_ff^&SubClass_42^&Prot_0?\"} |
%{write \"0x$([regex]::match($_.deviceid.tolower(), 'vid_(\w+)').groups[1].value)\"} |
sort -u" >> %USERPROFILE%\.android\adb_usb.ini



That was a single line I had to split for better readability.

Also if ANDROID_SDK_HOME is set, adb will use %ANDROID_SDK_HOME%\.android\adb_usb.ini instead.

And the best part of it is that to work (i.e. properly detect ADB devices) it does not need any drivers. You will need to install drivers to be able to use adb but not for the detection!

Here is how to use this command for troubleshooting:

Disconnect all other adb devices you might have other than the one you want to troubleshoot. First thing you want to make sure that adb is properly configured on the device side (i.e. "USB debugging" is enabled, etc). Some manufacturers do not test their devices thoroughly which sometimes results in situations when adb works only when combined with other specific interface. I have seen a device which would only enumerate adb interface when in MTP mode but not in PTP or Mass Storage modes. So check different combinations and run the following command after every change:

powershell "gwmi Win32_USBControllerDevice | %{[wmi]($_.Dependent)} |
?{$_.CompatibleID -like \"USB\Class_ff^&SubClass_42^&Prot_0?\"} | fl Name,DeviceID,Service"



Again that was a single line. The point I am trying to make is that you can not do anything on your PC side to make adb work until this command shows some output. If it shows nothing - the problem is not with your PC software configuration. It is with either your device software configuration or hardware (including PC, android device and everything in between - like USB cables, hubs, etc)

Tuesday, September 23, 2014

The most comprehensive write up on how to make adb work in a debian based linux environment

It is very surprising to me to see how many people are still to this day (almost 7 years since the first public release of Android) confused about proper way of making adb command work in linux environment.

The most common misconceptions about running adb in linux are:

1. You have to install some kind of "adb" driver

2. You have to install full Android SDK

3. You have to run adb as root

none of which are true!

Here are some actual issues to take care of:

1. Where to get the adb binary from?

While there are plenty of third party packages available personally I prefer to use the binary which comes directly from Google themselves - the one included with the official Android SDK. But what if we need just the adb binary alone and do not want to install the whole SDK? The first of the following commands downloads and parses the source code of the latest SDK Manager to find the link for the latest version of platform-tools package containing the adb binary. The second command downloads platform-tools package and extracts the binary into the current directory. The third one obviously makes the binary executable:

~$ eval $(wget -qO - "https://android.googlesource.com/platform/tools/base/+/master/sdklib/src/main/"\
"java/com/android/sdklib/repository/SdkRepoConstants.java?format=text" | base64 -d | tr '\n;' ' \n' | \
sed -e "s/\(NS_LATEST_VERSION\|URL_GOOGLE_SDK_SITE\|URL_FILENAME_PATTERN\)/\n\1/g" \
-e "s/%1\$d/\$NS_LATEST_VERSION/" | tr -d ' ' | grep "^[A-Z_]*=" | sort)

~$ wget -qO - "$URL_GOOGLE_SDK_SITE$(wget -qO - "$URL_GOOGLE_SDK_SITE$URL_FILENAME_PATTERN?format=text" | xml2 | \
grep "/sdk:platform-tool/" | grep "\-linux\.zip$" | tail -n 1 | cut -d= -f2)" | funzip 2> /dev/null 1> ./adb

~$ chmod 755 ./adb



2. I downloaded the binary but it does not run! Why?

The Google adb binary has some dependencies. It requires the following libraries to be installed: libncurses5 libstdc++6. Also the Google adb binary is 32bit and if your linux system is 64bit you need to make sure that 32bit versions of those libraries are also installed. In most debian based systems you can solve those dependencies by running the following commands:

~$ sudo dpkg --add-architecture i386 2>/dev/null
~$ sudo apt-get -qqy update
~$ sudo apt-get -qqy install libncurses5:i386 libstdc++6:i386 zlib1g:i386


After that running ./adb devices command should produce at least these lines:

~$ ./adb devices
* daemon not running. starting it now on port 5037 *
* daemon started successfully *
List of devices attached



3. Running ./adb devices returns an empty list!

In linux no special driver is required for user space programs to access USB devices privided they have permissions and are willing to handle the exchange protocol by themselves. The adb binary uses low level libusb library to access the device. So how does it know to which out of all connected USB devices it needs to talk to? It is looking for devices with a specific interface. Let's examine the lsusb -v command's output on a system which has an android device with enabled "USB debugging" connected to it:

~$ lsusb -v | grep -B 3 -i iInterface
    bInterfaceClass     255 Vendor Specific Class
    bInterfaceSubClass   66
    bInterfaceProtocol    1
    iInterface            4 ADB Interface


You can check any Android device from any manufacturer - its adb interface will always have

  bInterfaceClass=255
  bInterfaceSubClass=66
  bInterfaceProtocol=1

Because this is exactly what adb tool is looking for! But in some misguided optimization effort Google decided to filter out devices with "unknown" USB Vendor IDs first. You can see the latest list of the "Google approved" Vendor IDs by running the following command:

~$ wget -qO - "https://android.googlesource.com/platform/system/core/+/master/adb/usb_vendors.c?format=text" | \
base64 -d | grep "define VENDOR_ID"


What to do if your device's manufacturer did not make the list? If you already know its USB Vendor ID you can add it in 0xffff hexadecimal format (one ID per line if you have multiple IDs) to the ~/.android/adb_usb.ini file. Make sure to run ./adb kill-server command afterwards. Because while being just a single file the adb tool actually consists of 2 distinct parts - adb daemon and adb client. When you ran very first adb command like ./adb devices - it left the daemon part still running. And since the configuration file is only being read by the daemon part when it is starting up - the only way to make it read the changes we just made to the ini file is to kill the daemon and restart again.


4. I don't know my device's Vendor ID or I added it to the adb_usb.ini but ./adb devices still shows nothing!

There is a way to check if any adb devices are connected to the system. And by "adb devices" I mean Android devices with adb interface enumerated (i.e. with "USB debugging" option enabled on the device). The following command does not depend on the local adb configuration. In fact I should have probably started with it before downloading any files or installing any packages:

~$ find -L /sys/bus/usb/devices -maxdepth 2 -path "*/modalias" -printf "%h\t" -exec cat {} \; | \
awk -F: '/icFFisc42ip0/ {print $1}' | xargs -i cat {}/idVendor | awk '{print"0x"$1}'



This command should produce exactly one line with Vendor ID per connected adb device. If you do not see any output - you need to start looking at other possible reasons for the adb interface enumeration failure like always popular:

    "USB debugging" is not enabled on the device
    Defective/disconnected USB cable
    Defective USB hub (if used) or USB port in your system

Let's rule out the hardware issues first. Unplug your Android device and run the following command:

~$ bash -c 'LSUSB=$(lsusb|sort); read; diff -a <(lsusb|sort) <(echo "$LSUSB")' | grep "^<" | awk '{print $7}' | \
xargs -i lsusb -v -d {} | grep -B 3 iInterface


It is not supposed to produce any output or even return back to the command prompt just yet. Plug your Android device back in, wait for like 5 seconds and press [enter] key on the keyboard. Now you should get your command prompt back. But most importantly you should expect to see some lines printed out. If you did not get any output - that means that your android device does not enumerate any interfaces at all. Which is very unlikely and suggests a hardware problem. But if you got some interfaces listed just not the adb interface - this means that the hardware (USB cable/hub/port) is working fine but you have "USB debugging" switched off on the device.


SO STOP WASTING TIME AND TURN "USB debugging" ON!!!


In case of hardware problems replace the USB cable, connect your device directly to the USB port on the PC if you used a USB hub before, try connecting to a different USB port and try again...

So you fixed all problems and the find -L /sys/bus/usb/devices ... command finally started printing some numbers. As I mentioned before those numbers are USB Vendor IDs for all currently connected adb devices. If you had not added them yet to your adb_usb.ini file do it now:

~$ find -L /sys/bus/usb/devices -maxdepth 2 -path "*/modalias" -printf "%h\t" -exec cat {} \; | \
awk -F: '/icFFisc42ip0/ {print $1}' | xargs -i cat {}/idVendor | awk '{print"0x"$1}' | sort -u >> ~/.android/adb_usb.ini
~$ ./adb kill-server



5. ./adb devices command shows ???????????? no permissions!

This is the best news you got all day! Congratulations! This means that adb tool can finally see and recognize your device. It just does not have permissions to access it. Let's give it the permissions:

~$ echo 'ACTION=="add", SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", ENV{ID_USB_INTERFACES}=="*:ff420?:*", MODE="0666", '\
'GROUP="plugdev", SYMLINK+="android/$env{ID_SERIAL_SHORT}"' | sudo tee /etc/udev/rules.d/90-universal-android.rules
~$ sudo udevadm control --reload-rules
~$ sudo udevadm trigger --action=add --subsystem-match=usb
~$ sudo killall -9 adb



6. Finally ./adb devices command shows my device and all other adb commands also work properly! Thank You!!!



You are welcome. You can move the adb binary to a directory in your $PATH now.

Also for your convenience I put all the above commands into a single shell file:

Monday, September 22, 2014

Calling Android services from ADB shell

Many android automation recipes contain references to the "service call" command:

# service
Usage: service [-h|-?]
    service list
    service check SERVICE
    service call SERVICE CODE [i32 INT | s16 STR] ...
Options:
  i32: Write the integer INT into the send parcel.
  s16: Write the UTF-16 string STR into the send parcel.


The command is indeed very useful. But the problem is that the calling "CODES" are android version specific and recipes often get out dated sometimes even after a minor version update. I am frequently being asked to update those codes for my recipes. I decided to share a small bash script I have been using to look up service codes for specific Android versions:


This script checks the Android version of your phone and the java package name of the service you specified as a parameter. Then the script downloads the service AIDL file for your phone's Android version from https://android.googlesource.com and parses it. So obviously it only works for the standard Android services it can find the source code for.

This is how I usually run the script. For example let's find out the calling code for "getCallState()" method of the "phone" service. I usually have multiple phones connected to my system so I need to specify which one to use by setting the ANDROID_SERIAL variable first. If you have just a single phone connected you can skip that part:

~$ ANDROID_SERIAL=XXXXXXXXXX ./get_android_service_call_numbers.sh phone | grep getCallState
ANDROID_SERIAL=XXXXXXXXXX
ro.build.version.release=4.4.4
TAG=android-4.4.4_r1
SERVICE=phone
SERVICE_PACKAGE=com.android.internal.telephony.ITelephony
    32 int getCallState()




Wednesday, July 10, 2013

Installing Android platform tools (ADB) on Ubuntu and Debian systems

Here is a short script I use to install the latest version of Android platform tools on those of my systems I do not need full blown Android SDK on:




The script requires root permissions to install prerequisite packages. It downloads and unpacks adb and fastboot binaries into $ANDROID_SDK_HOME/platform-tools/ folder (or into ~/android/platform-tools/ if $ANDROID_SDK_HOME is not defined). The folder will be added to the current $PATH. And finally the script will create a universal udev rule to take care of device permissions for all Android devices regardless of their VendorID.

The script has been tested on 32 and 64-bit versions of Ubuntu 12.04, 12.10, 13.04, 13.10 and Debian 7.0, 7.1, 7.2.

Sunday, June 23, 2013

Emulating touchscreen interaction with sendevent in Android

I know this has been explained many times before. But I keep getting asked  about it over and over again. So I decided to summarize it in one place I could refer to in the future.


Android uses Linux kernel so it processes input events the same way as any other linux system. In case of touchscreen displays it uses the same multi-touch protocol. There are 2 different versions of the protocol - Type A and Type B. Usually it does not matter which protocol to use for injecting events since the kernel driver supports both protocols at the same time. In my automation scripts I use Type A just because I tried it first and it worked right away and I did not have to try Type B.

Android provides these two convenient tools for dealing with input events:

  • getevent - for dumping input events and providing information about input devices
  • sendevent - for injecting input events

Every sendevent command requires 4 parameters:
  • device_name (string)
  • event_type (decimal int)
  • event_code (decimal int)
  • value (decimal int)

First you need to find the name of touchscreen device. The following command requires busybox to be installed:

getevent -pl | busybox sed -e ':a;N;$!ba;s/\n / /g' | busybox awk '/ABS_MT_TOUCH/{print $4}'



Let's assume it printed out "/dev/input/event0" - this would be the first parameter.

For touch events only 2 event types are used:
  • EV_ABS (3)
  • EV_SYN (0)

Touching the display (in case of Type A protocol) will result in an input report (sequence of input events) containing the following event codes:
  • ABS_MT_TRACKING_ID (57) - ID of the touch (important for multi-touch reports)
  • ABS_MT_POSITION_X (53) - x coordinate of the touch
  • ABS_MT_POSITION_Y (54) - y coordinate of the touch
  • ABS_MT_TOUCH_MAJOR (48) - basically width of your finger tip in pixels
  • ABS_MT_PRESSURE (58) - pressure of the touch
  • SYN_MT_REPORT (2) - end of separate touch data
  • SYN_REPORT (0) - end of report

Let's say we want to emulate a touch down event at the point with coordinates x=300, y=400. We will need to execute the following sendevent commands:

sendevent /dev/input/event0 3 57 0
sendevent /dev/input/event0 3 53 300
sendevent /dev/input/event0 3 54 400
sendevent /dev/input/event0 3 48 5
sendevent /dev/input/event0 3 58 50
sendevent /dev/input/event0 0 2 0
sendevent /dev/input/event0 0 0 0

I used arbitrary values of 5 for the ABS_MT_TOUCH_MAJOR (makes for very small finger tip for high precision) and 50 for the ABS_MT_PRESSURE (a slight tap) which work good enough for most applications.

For most touch screens on the market it takes 20 to 50 milliseconds to reliably register the touch. So I would recommend to wait at least 50 milliseconds before sending the release event report. If you want to emulate the long touch - then you wait longer. You can use busybox usleep command:

busybox usleep 50000



The release report is really simple. To let the input device know that all previous touches have been released - you just send the empty report with ABS_MT_TRACKING_ID = -1:

  • ABS_MT_TRACKING_ID (57)
  • SYN_MT_REPORT (2)
  • SYN_REPORT (0)

sendevent /dev/input/event0 3 57 -1
sendevent /dev/input/event0 0 2 0
sendevent /dev/input/event0 0 0 0



This is how injecting touch events could be implemented in python:


Saturday, April 27, 2013

Universal udev rule for all Android devices

When an Android device is plugged in to a linux system, by default only root has access to the device. In order for regular users to be able to access the device - the device permissions need to be changed. To automate this, Google recommends  adding Vendor ID of every Android device as a separate line per ID.
I thought that there has to be a better way to do it. Here is the rule I came up with:

ACTION=="add", SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", ENV{ID_USB_INTERFACES}=="*:ff420?:*", MODE="0666"


This rule matches all adb and fastboot interfaces for any android device, no matter the Vendor ID.