COSC6339: Big Data Analytics
CARLOS R. ORDÓÑEZ · 2006
This course will cover theory, algorithms, data structures and system programming to analyze big data. Data mining research is being revisited to analyze much bigger data sets, mixing structured and semi-structured content, with more complex statistical and machine learning models and exploiting parallel processing. Topics: I. Big Data overview: 4 Vs, Data Lakes vs Data Warehouses, Data integration, Parallelism, Big Data Analytics problems: ML, graphs, streams. II. R: language, functional programming, environment, runtime, data structures (vectors, matrices, lists, data frames), statistical and machine learning models, graphs. III. Parallel DBMSs: parallel architecture, OLTP versus cubes, Data warehousing and denormalization, query languages: SQL, datalog, AFL, UDFs, row/column/array storage, indexing versus ordering, ETL and pre-processing, advanced SQL (pivoting, horagg, keyword search, recursive queries), data pre-processing and data cleaning, integrating machine learning algorithms. IV. Hadoop ecosystem: HDFS vs Posix, subsystems (Yarn, Storm, Zookeeper, Cassandra, Hive, SPARQL), MapReduce, text versus numeric processing, Spark, Graph systems, Search Engines (IR models, architecture, keywords, page rank, spider); comparing big data analytics technologies (speedup, cost, scalability, fault tolerance, Java vs C++, programming ease). The textbook is [2], complemented by [1]. The course will also require reading important CS research papers and investigating latest research on DBLP and ACM libraries. Pre-requisites: It is encouraged, but not required, that COSC6340 (Database Systems) and COSC6342 (Machine Learning) and COSC6373 (Parallel Computation) were taken before (or concurrently). Courses like Data Mining, Numerical Analysis and Algorithms are also desirable.