Re-architecting web and mobile information access for emerging regions
Lakshminarayanan Subramanian, Jay Chen · 2011
In this thesis, we have proposed a new web architecture to enable information access across the poor connectivity scenarios commonly found in emerging regions: low bandwidth, intermittent, and mostly disconnected. Our web architecture is highly efficient both in terms of network use and end-user time consumed. Our architecture introduces fundamental changes to several layers of the web stack in order to provide an interactive and useful experience regardless of the network environment. Through real-world user studies, we begin by characterizing web user behavior in emerging regions and describe a new Markov model to accurately capture TCP behavior in sub-packet regimes. To address the TCP breakdown problem, our architecture introduces an in-network optimization engine (using middle-boxes) to prevent TCP timeouts and ensures that flows make smooth progress despite congestion introduced by pathological sharing. In the absence of middle-box support, we develop a purely end-host web optimization engine (ELF) that improves web page load times through aggressive caching, prefetching, and offline browsing. At the application layer, our architecture implements three fundamental changes. First, to increase the utility of cached web pages despite intermittent connectivity, we introduce a new vertical caching layer organized around topics. The vertical caching layer maximizes the utility of cached contents by automatically discovering topics of interest and exposing these topics to the end-users. Second, from the user's perspective, we have designed RuralCafe, an asynchronous web browsing system that decouples system interactivity from underlying network conditions and allows users to seamlessly navigate both live web content (as delivered by the network) and cached content (as delivered by the cache). To satisfy the extreme case of disconnected populations, we propose the design of Contextual Information Portals (CIPs) where the goal is to deliver a vertical slice of the web on specific topics as an offline searchable and browse-able repository for users. We have developed CIPs for specific educational subjects based on the school syllabus in Kenya and India and have provided the system as an offline tool for teachers to use web-based resources for enhancing school education. For mobile information access, we show how to build powerful mobile applications despite extremely constrained data channels such as SMS. This is a challenging research problem since the SMS channel is restricted to only 140 byte messages. We have designed SMSFind, an SMS-based search service that allows a mobile user efficiently search the web using one round of interaction where the search response is restricted to one SMS message. Based on real user queries, we show that SMSFind answers 57.3% of queries answered by a human; in comparison, existing SMS search services such as GoogleSMS answered only 9.3% of the queries. To facilitate the development of general SMS based applications, we developed UjU, a mobile application platform that enables users (with no programming experience) to quickly build efficient and reliable applications. To make efficient use of the underlying SMS channel and minimize operational costs, UjU supports a semantic compression layer that achieves a high compression ratio in comparison to standard compression techniques. Our work has had real-world impact and almost every system that we have built has also been deployed and used by real-world users. ELF was deployed in a large peri-urban school in India. RuralCafe, vertical caching, and our in-network optimization engine have been deployed and tested by users in Amrita University in South India and other organizations in Kenya. The CIP system is in use by Strathmore University in Kenya to develop syllabus-specific CIPs for nearly 60 Kenyan schools to enhance education. CIPs have also been experimented with by teachers in India for elementary after school programs. SMSFind has been deployed in an open beta in Kenya, and used by inhabitants of the Haruma slum in Nairobi. UjU is used by a teaching hospital in Ghana and a microfinance institution in Mexico for four applications in daily operational use. (Abstract shortened by UMI.)