Sketch and attribute based query interfaces
Çağlar Tırkaz · Sabanci University · 2015
In this thesis, machine learning algorithms to improve human computer interaction are designed. The two areas of interest are (i) sketched symbol recognition and (ii) object recognition from images. Specifically, auto-completion of sketched symbols and attribute-centric recognition of objects from images are the main focus of this thesis. In the former task, the aim is to be able to recognize partially drawn symbols before they are fully completed. Auto-completion during sketching is desirable since it eliminates the need for the user to draw symbols in their entirety if they can be recognized while they are partially drawn. It can thus be used to increase the sketching throughput; to facilitate sketching by offering possible alternatives to the user; and to reduce user-originated errors by providing continuous feedback. The latter task, allows machine learning algorithms to describe objects with visual attributes such as “square”, “metallic” and “red”. Attributes as intermediate representations can be used to create systems with human interpretable image indexes, zero-shot learning capability where only textual descriptions are available or capability to annotate images with textual descriptions.