Indexing and Analytics
Craig Allen Johnston · Apress eBooks · 2020
A modern, distributed data platform with Kubernetes goes well beyond the collection, storage, and transmission of data. Search, indexing, analyzing, and data science applications are essential elements in data-centric platforms. This chapter focuses on web-scale technologies, an ecosystem that matured outside of the mainline concerns of Big Data. Where Big Data assumes a limited number of simultaneous requests to process nearly unlimited lakes of data, web-scale assumes an eventual unlimited simultaneous demand for data. Web-scale analytics and Big Data are growing closer together and rapidly advancing in combined capabilities. This chapter covers general data indexing, metrics, analytics, and data science built for web-scale architectures. The next chapter covers how Kubernetes fits into the Big Data picture by supporting the development of modern data lakes and warehouses. Kubernetes’s ability to construct robust, distributed clusters brings opportunities to bridge the data and control planes of these two distinct problem domains, Big Data at web-scale.