Eavesdrop with PoKeMon: Position free keystroke monitoring using acoustic data

Yuyi Fang, Zhiwei Zhao, Zi Wang, Geyong Min, Yue Cao, Haojun Huang, Hao Yin

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)


Keyboard input is an important way to enter massive personal information into computing systems and the Internet in our social life. The large amount of acoustic data from the keystroke tapping makes it possible for eavesdroppers to snoop the sensitive and private input remotely. Keystroke monitoring using the acoustic signals, which extracts the user information directly from the keystroke acoustics, is a critical security and privacy issue and has recently attracted much research attention. The existing works use smartphones to collect and infer the keystrokes with various data analysis algorithms. However, most of them rely on the assumption that the phone-keyboard relative position is known in advance, which is often unrealistic for real-world scenarios such as adversary eavesdropping. To address this problem and make keystroke monitoring free to use, in this paper, we propose a Po sition free Ke ystroke Mon itoring scheme using a single smartphone. By exploiting the reliable linguistic accountings and the layout of a certain keyboard, we develop a geometrical approach to estimate the relative positions and angles of the smartphone to the keyboard. Our proposed scheme can self-calibrate the position estimation with the continuous acoustic signals. After that, effective keystroke snooping is established based on the Mel-Frequency Cepstral Coefficient and adaptive -means clustering. We conduct experiments with Samsung S4 smartphones and simulations. The results show that the positioning scheme achieves a high accuracy by 93%, and the keystroke snooping accuracy is improved by 32.6% compared to the existing works.
Original languageEnglish
Pages (from-to)704-711
JournalFuture Generation Computer Systems
Early online date15 Nov 2017
Publication statusPublished - Oct 2018


Dive into the research topics of 'Eavesdrop with PoKeMon: Position free keystroke monitoring using acoustic data'. Together they form a unique fingerprint.

Cite this