VistrailsWiki - User contributions [en]

Course: Advanced Databases

2014-02-20T17:27:06Z

Jsimeon: /* Reading Assignment */

Course: Advanced Databases

2014-02-20T17:17:10Z

Jsimeon: /* Reading Assignment */

Jsimeon: /* Week 5: Monday Oct. 7th - Query Processing on Mapreduce and High-level Languages */

== Fall 2013 ==
'''''This schedule is tentative and subject to change'''''

'''''Make sure to check my.poly.edu for course announcements'''''

== News ==

On September 30th, our class will meet at a different place: 1 Metrotech Center, 19th floor.
Bring your NYU Poly id -- you will need to show it to the security guard.

For frequently asked questions about the course and homework assignments, please check our [[BigDataAnalysisFAQ]].

== Week 1: Monday Sept. 9th - Course Overview ==

* Course overview and introduction to Big Data Analysis
* Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/intro.pdf
* [https://docs.google.com/spreadsheet/viewform?fromEmail=true&formkey=dFdHT3BST2l1TW9KeHYzYjBDaTU0V1E6MQ Student survey] -- to be filled out today!

=== Required Reading ===

* [http://i.stanford.edu/~ullman/mmds/book.pdf Mining of Massive Datasets, Chapter 1]
* [http://lintool.github.com/MapReduceAlgorithms/MapReduce-book-final.pdf Data-Intensive Text Processing with MapReduce, Chapter1]

=== Additional References ===
* [http://dilbert.com/strips/comic/2012-07-29/ Dilbert's BigData]
* [http://www.nytimes.com/2012/08/12/business/how-big-data-became-so-big-unboxed.html?ref=stevelohr New York Time's "How BigData Became so Big"]
* [http://www3.weforum.org/docs/WEF_TC_MFS_BigDataBigImpact_Briefing_2012.pdf World Economic Forum: Big Data, Big Impact]
* [http://www.analytics-magazine.org/november-december-2010/54-the-analytics-journey.html The Analytics Journey]
* [http://practicalanalytics.wordpress.com/2011/12/12/big-data-analytics-use-cases/ BigData Analytics Usecases]

== Week 2: Monday Sept. 16th - Map-Reduce/Hadoop ==

* Introduction to Map-Reduce and high-level data processing languages
* Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/hadoop.pdf
* Hand out AWS tokens. [http://www.vistrails.org/index.php/AWS_Setup Notes on using AWS].
* Introduction to [http://hadoop.apache.org/Hadoop Hadoop]
* The Map-Reduce ecosystem: [http://pig.apache.org/ Pig], [http://hive.apache.org/ Hive], [http://code.google.com/p/jaql/ Jaql], [http://mahout.apache.org/ Mahout], BigInsights

=== Assignment ===

* [[cs9223 Mapreduce Assignment]]
* This is an individual assignment. You may not collude with any other individual, or plagiarise their work.
For more details see http://cis.poly.edu/policies.
* You assignment is ''due on Sun Sept 29th''. '''Make sure you can login and access my.poly.edu!'''
* If you have questions about the assignment, we will hold office hours on Sept 23, 2013 from 2:30-3:30pm at 2 Metrotech, room 10.018

=== Required Reading ===
* [http://infolab.stanford.edu/~ullman/mmds/ch2.pdf Mining of Massive Datasets, Chapter 2]
* [http://lintool.github.com/MapReduceAlgorithms/MapReduce-book-final.pdf Data-Intensive Text Processing with MapReduce, Chapter 2 and Chapter 3]
* [http://research.google.com/archive/mapreduce.html original google map-reduce paper]

== Week 3: Monday Sept. 23rd - Data Management for Big Data ==

* Databases and Big Data: Persistence, Querying, Indexing, Transactions
* Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/paralleldb-vs-hadoop.pdf

=== Related Topics ===
* BigTables and NoSQL stores. Tuple store vs. column stores: [http://hbase.apache.org/ HBase], [http://www.mongodb.org/ MongoDB], [http://cassandra.apache.org/ Cassandra]
* HBase book HBase: The Definitive Guide. Random Access to Your Planet-Size Data: http://shop.oreilly.com/product/0636920014348.do
* HBase book. Chapter 8 Architecture for information about transactional processing, WriteAhead Log notably, and how consistency is being maintained.
* Transactions in NoSQL stores. Google's percolator, [http://research.google.com/pubs/pub36726.html].
* "NewSQL" stores: more on [http://hive.apache.org/ Hive], [http://voltdb.com/ VoltDB], [http://db.cs.yale.edu/hadoopdb/hadoopdb.html HadoopDB],
* Beyond MapReduce: [http://spark-project.org/ Berkeley's Spark], [http://asterix.ics.uci.edu/ UC Irvine's Asterix], Google's [http://code.google.com/p/dremel/ Dremel]

=== Required Reading ===
* [http://cacm.acm.org/magazines/2010/1/55743-mapreduce-and-parallel-dbmss-friends-or-foes/fulltext PDMBS vs. MapReduce]
* http://cacm.acm.org/magazines/2010/1/55744-mapreduce-a-flexible-data-processing-tool/fulltext
* [http://www.cs.arizona.edu/~bkmoon/papers/sigmodrec11.pdf Parallel data processing with MapReduce: a survey. Lee et al, SIGMOD Record 2011]
* [http://database.cs.brown.edu/sigmod09/benchmarks-sigmod09.pdf Benchmark DBMS vs MapReduce (2009)]

=== Additional References ===
* http://www.computerworld.com/s/article/9224180/What_s_the_big_deal_about_Hadoop_
* [http://research.google.com/archive/bigtable.html Bigtable: A Distributed Storage System for Structured Data]
* [http://cs-www.cs.yale.edu/homes/dna/papers/hadoopdb.pdf HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads]
* [http://cs-www.cs.yale.edu/homes/dna/papers/hstore-cc.pdf Low Overhead Concurrency Control for Partitioned Main Memory Databases]
* [http://asterix.ics.uci.edu/pub/ASTERIX-DPD-2011.pdf ASTERIX: Towards a Scalable, Semistructured Data Platform for Evolving-World Models.]
* [http://research.google.com/pubs/pub36632.html Dremel: Interactive Analysis of Web-Scale Datasets]
* [http://research.google.com/pubs/pub36726.html Large-scale Incremental Processing Using Distributed Transactions and Notifications]

== Week 4: Monday Sept 30th - ''Invited lecture by Dr. C. Mohan (IBM)'' ==

* '''Note that we will meet at a different location: NYU CUSP, 1 Metrotech Center, 19th floor'''

* Tutorial: An In-Depth Look at Modern Database Systems

* Abstract: This tutorial is targeted at a broad set of database systems and applications people. It is intended to let the attendees better appreciate what is really behind the covers of many of the modern database systems (e.g., NoSQL and NewSQL systems), going beyond the hype associated with these open source and commercial systems. The capabilities and limitations of such systems will be addressed. Modern extensions to decades old relational DBMSs will also be described. Some application case studies will also be presented. An outline of problems for which no adequate solutions exist will be included. Such problems could be fertile grounds for new research work.

* Presenter: Dr. C. Mohan, IBM Fellow, IBM Almaden Research Center, San Jose, CA 95120, USA.

* Bio: Dr. C. Mohan has been an IBM researcher for 31 years in the information management area, impacting numerous IBM and non-IBM products, the research community and standards, especially with his invention of the ARIES family of locking and recovery algorithms, and the Presumed Abort commit protocol. This IBM, ACM and IEEE Fellow has also served as the IBM India Chief Scientist. In addition to receiving the ACM SIGMOD Innovation Award, the VLDB 10 Year Best Paper Award and numerous IBM awards, he has been elected to the US and Indian National Academies of Engineering, and has been named an IBM Master Inventor. This distinguished alumnus of IIT Madras received his PhD at the University of Texas at Austin. He is an inventor of 38 patents. He serves on the advisory board of IEEE Spectrum and on the IBM Software Group Architecture Board’s Council. More information can be found in his home page at http://bit.ly/CMohan

== Week 5: Monday Oct. 7th - Query Processing on Mapreduce and High-level Languages ==

* Pig Latin and Query Processing:
** [http://www.vistrails.org/images/1-RelationalOnMapReduce.pdf Relational processing over MapReduce]
** [http://www.vistrails.org/images/2-PigOnMapReduce.pdf Queries over MapReduce]
* In-class assignment

=== Required Reading ===

=== Additional References ===
* [http://pages.cs.brandeis.edu/~olga/cs228/Reading%20List_files/piglatin.pdf Pig Latin: A Not-So-Foreign Language for Data Processing]
* [http://www.mpi-inf.mpg.de/~rgemulla/publications/beyer11jaql.pdf Jaql: A Scripting Language for Large Scale Semistructured Data Analysis]
* [http://www.vldb.org/pvldb/2/vldb09-938.pdf Hive - A Warehousing Solution Over a Map-Reduce Framework]

== Week 6: Mon Oct. 14th - Fall Break - No class ==

== Week 7: Monday Oct. 21st - Invited Speaker: Torsten Suel ==

* Big Data and Information Retrieval. Invited lecture by Torsten Suel.
** Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/search-data.pdf

== Week 8: Monday Oct 28th- Statistics is easy - Invited Speaker: Dennis Shasha ==

* Guest lecture by [http://cs.nyu.edu/shasha/ Dennis Shasha]: [http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/stateasy.pdf Statistics is Easy]

=== Required Reading ===
* http://www.morganclaypool.com/doi/abs/10.2200/S00142ED1V01Y200807MAS001 -- book is available for free for NYU students
* Second edition of the book: http://www.morganclaypool.com/doi/pdf/10.2200/S00295ED1V01Y201009MAS008

== Week 9: Monday Nov 5th - EM and Text Processing ==

TODO

=== Readings ===

* Data-Intensive Text Processing with MapReduce, Chapter 6

== Week 10: Monday Nov. 11th - Finding Similar Items and Information Integration ==
* Similarity: Applications, Measures and Efficiency considerations
** Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/similarity.pdf
* Similarity application: Information integration on the Web:
** Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/web-info-integration.pdf
* Homework presentation and demo

=== Required Reading ===
* [http://infolab.stanford.edu/~ullman/mmds/ch3.pdf Mining of Massive Datasets, chapter 3; information integration; entity resolution]

=== Homework Assignment ===
'''Due November 17th'''
Your assignment is in http://www.newgradiance.com/services. Please see http://vgc.poly.edu/~juliana/courses/cs9223 for instructions on how to access this service.

== Week 11: Monday Nov 18th- Frequent Itemsets ==
** Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/assoc-rules1.pdf, http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/assoc-rules1.pdf

=== Required Reading ===
* Mining of Massive Datasets, Chapter 4

=== Homework Assignment ===
'''Due November 24th'''

=== Additional Reading ===
* Mining association rules between sets of items in large databases. Agrawal et al., SIGMOD 1993. http://delivery.acm.org/10.1145/180000/170072/p207-agrawal.pdf?ip=128.238.251.32&acc=ACTIVE%20SERVICE&CFID=198467341&CFTOKEN=23537886&__acm__=1352747519_b80a516e0f5e294b36dc021f13f55bbb
* Fast algorithms for mining association rules. Agrawal and Srikant, VLDB 1994. https://www.seas.upenn.edu/~jstoy/cis650/papers/Apriori.pdf
* An effective hash-based algorithm for mining association rules. Park et al., SIGMOD 1995. http://dl.acm.org/citation.cfm?id=223813

== Week 12: Monday Nov. 25th - Clustering ==

* Lecture notes:
** Graph algorithms: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/mapreduce-graph.pdf
**Clustering: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/clustering.pdf, http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/form-clustering-icde2007.pdf

=== Homework Assignment ===
'''Due Dec 1st'''

=== Readings ===
* Mining of Massive Datasets, Chapter 7
* See readings for previous class
* Web Mining, by Bing Liu. http://www.cs.uic.edu/~liub/WebMiningBook.html
* Information Retrieval. http://nlp.stanford.edu/IR-book/pdf/irbookprint.pdf

== Further Readings ==
* [http://www-users.cs.umn.edu/~kumar/papers/anomaly-survey.php Anomaly Detection: A Survey]

== Week 13: Monday Dec. 2nd - Invited lecture by Enrico Bertini ==
* Introduction to Visual Analytics
** Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/intro-to-visualization.pdf

=== Readings ===
The Value of Visualization. IEEE Visualization 2005. Jarke J. van Wijk. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.78.1138

Visualization Analysis and Design: Principles, Methods, and Practice. Tamara Munzner (Book Draft 2 from Sep. 2012). http://www.cs.ubc.ca/~tmm/courses/533-11/book/vispmp-draft.pdf

== Week 14: Monday Dec. 9th - - Graph Algorithms ==

TODO

=== Readings ===
* [http://infolab.stanford.edu/pub/papers/google.pdf 1998 PageRank Paper]
* [http://lintool.github.com/MapReduceAlgorithms/MapReduce-book-final.pdf Data-Intensive Text Processing with MapReduce, Chapter 4 (Inverted Indexing for Text Retrieval) and 5(Graph Algorithms)]
* [http://infolab.stanford.edu/~ullman/mmds/ch5.pdf Mining of Massive Datasets, Chapter 5 (Link Analysis)]
* Pregel: A System for Large-Scale Graph Processing. Google. [http://kowshik.github.com/JPregel/pregel_paper.pdf]

== Week 15 Monday Dec. 16th - Final Exam ==

== Other topics ==
===Provenance ===
* [http://vgc.poly.edu/~juliana/pub/vistrails-reproducibility2012.pdf Making Computations and Publications Reproducible with VisTrails]
Juliana Freire and Claudio Silva. In Computing in Science and Engineering 14(4): 18-25, 2012.

* [http://vgc.poly.edu/~juliana/pub/freire-cise2008.pdf Provenance for Computational Tasks: A Survey]
Juliana Freire, David Koop, Emanuele Santos, and Claudio T. Silva. In IEEE Computing in Science & Engineering, 2008.

Course: Big Data Analysis

2013-10-02T19:46:14Z

Jsimeon:

== Fall 2013 ==
'''''This schedule is tentative and subject to change'''''

'''''Make sure to check my.poly.edu for course announcements'''''

== News ==

On September 30th, our class will meet at a different place: 1 Metrotech Center, 19th floor.
Bring your NYU Poly id -- you will need to show it to the security guard.

For frequently asked questions about the course and homework assignments, please check our [[BigDataAnalysisFAQ]].

== Week 1: Monday Sept. 9th - Course Overview ==

* Course overview and introduction to Big Data Analysis
* Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/intro.pdf
* [https://docs.google.com/spreadsheet/viewform?fromEmail=true&formkey=dFdHT3BST2l1TW9KeHYzYjBDaTU0V1E6MQ Student survey] -- to be filled out today!

=== Required Reading ===

* [http://i.stanford.edu/~ullman/mmds/book.pdf Mining of Massive Datasets, Chapter 1]
* [http://lintool.github.com/MapReduceAlgorithms/MapReduce-book-final.pdf Data-Intensive Text Processing with MapReduce, Chapter1]

=== Additional References ===
* [http://dilbert.com/strips/comic/2012-07-29/ Dilbert's BigData]
* [http://www.nytimes.com/2012/08/12/business/how-big-data-became-so-big-unboxed.html?ref=stevelohr New York Time's "How BigData Became so Big"]
* [http://www3.weforum.org/docs/WEF_TC_MFS_BigDataBigImpact_Briefing_2012.pdf World Economic Forum: Big Data, Big Impact]
* [http://www.analytics-magazine.org/november-december-2010/54-the-analytics-journey.html The Analytics Journey]
* [http://practicalanalytics.wordpress.com/2011/12/12/big-data-analytics-use-cases/ BigData Analytics Usecases]

== Week 2: Monday Sept. 16th - Map-Reduce/Hadoop ==

* Introduction to Map-Reduce and high-level data processing languages
* Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/hadoop.pdf
* Hand out AWS tokens. [http://www.vistrails.org/index.php/AWS_Setup Notes on using AWS].
* Introduction to [http://hadoop.apache.org/Hadoop Hadoop]
* The Map-Reduce ecosystem: [http://pig.apache.org/ Pig], [http://hive.apache.org/ Hive], [http://code.google.com/p/jaql/ Jaql], [http://mahout.apache.org/ Mahout], BigInsights

=== Assignment ===

* [[cs9223 Mapreduce Assignment]]
* This is an individual assignment. You may not collude with any other individual, or plagiarise their work.
For more details see http://cis.poly.edu/policies.
* You assignment is ''due on Sun Sept 29th''. '''Make sure you can login and access my.poly.edu!'''
* If you have questions about the assignment, we will hold office hours on Sept 23, 2013 from 2:30-3:30pm at 2 Metrotech, room 10.018

=== Required Reading ===
* [http://infolab.stanford.edu/~ullman/mmds/ch2.pdf Mining of Massive Datasets, Chapter 2]
* [http://lintool.github.com/MapReduceAlgorithms/MapReduce-book-final.pdf Data-Intensive Text Processing with MapReduce, Chapter 2 and Chapter 3]
* [http://research.google.com/archive/mapreduce.html original google map-reduce paper]

== Week 3: Monday Sept. 23rd - Data Management for Big Data ==

* Databases and Big Data: Persistence, Querying, Indexing, Transactions
* Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/paralleldb-vs-hadoop.pdf

=== Related Topics ===
* BigTables and NoSQL stores. Tuple store vs. column stores: [http://hbase.apache.org/ HBase], [http://www.mongodb.org/ MongoDB], [http://cassandra.apache.org/ Cassandra]
* HBase book HBase: The Definitive Guide. Random Access to Your Planet-Size Data: http://shop.oreilly.com/product/0636920014348.do
* HBase book. Chapter 8 Architecture for information about transactional processing, WriteAhead Log notably, and how consistency is being maintained.
* Transactions in NoSQL stores. Google's percolator, [http://research.google.com/pubs/pub36726.html].
* "NewSQL" stores: more on [http://hive.apache.org/ Hive], [http://voltdb.com/ VoltDB], [http://db.cs.yale.edu/hadoopdb/hadoopdb.html HadoopDB],
* Beyond MapReduce: [http://spark-project.org/ Berkeley's Spark], [http://asterix.ics.uci.edu/ UC Irvine's Asterix], Google's [http://code.google.com/p/dremel/ Dremel]

=== Required Reading ===
* [http://cacm.acm.org/magazines/2010/1/55743-mapreduce-and-parallel-dbmss-friends-or-foes/fulltext PDMBS vs. MapReduce]
* http://cacm.acm.org/magazines/2010/1/55744-mapreduce-a-flexible-data-processing-tool/fulltext
* [http://www.cs.arizona.edu/~bkmoon/papers/sigmodrec11.pdf Parallel data processing with MapReduce: a survey. Lee et al, SIGMOD Record 2011]
* [http://database.cs.brown.edu/sigmod09/benchmarks-sigmod09.pdf Benchmark DBMS vs MapReduce (2009)]

=== Additional References ===
* http://www.computerworld.com/s/article/9224180/What_s_the_big_deal_about_Hadoop_
* [http://research.google.com/archive/bigtable.html Bigtable: A Distributed Storage System for Structured Data]
* [http://cs-www.cs.yale.edu/homes/dna/papers/hadoopdb.pdf HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads]
* [http://cs-www.cs.yale.edu/homes/dna/papers/hstore-cc.pdf Low Overhead Concurrency Control for Partitioned Main Memory Databases]
* [http://asterix.ics.uci.edu/pub/ASTERIX-DPD-2011.pdf ASTERIX: Towards a Scalable, Semistructured Data Platform for Evolving-World Models.]
* [http://research.google.com/pubs/pub36632.html Dremel: Interactive Analysis of Web-Scale Datasets]
* [http://research.google.com/pubs/pub36726.html Large-scale Incremental Processing Using Distributed Transactions and Notifications]

== Week 4: Monday Sept 30th - ''Invited lecture by Dr. C. Mohan (IBM)'' ==

* '''Note that we will meet at a different location: NYU CUSP, 1 Metrotech Center, 19th floor'''

* Tutorial: An In-Depth Look at Modern Database Systems

* Abstract: This tutorial is targeted at a broad set of database systems and applications people. It is intended to let the attendees better appreciate what is really behind the covers of many of the modern database systems (e.g., NoSQL and NewSQL systems), going beyond the hype associated with these open source and commercial systems. The capabilities and limitations of such systems will be addressed. Modern extensions to decades old relational DBMSs will also be described. Some application case studies will also be presented. An outline of problems for which no adequate solutions exist will be included. Such problems could be fertile grounds for new research work.

* Presenter: Dr. C. Mohan, IBM Fellow, IBM Almaden Research Center, San Jose, CA 95120, USA.

* Bio: Dr. C. Mohan has been an IBM researcher for 31 years in the information management area, impacting numerous IBM and non-IBM products, the research community and standards, especially with his invention of the ARIES family of locking and recovery algorithms, and the Presumed Abort commit protocol. This IBM, ACM and IEEE Fellow has also served as the IBM India Chief Scientist. In addition to receiving the ACM SIGMOD Innovation Award, the VLDB 10 Year Best Paper Award and numerous IBM awards, he has been elected to the US and Indian National Academies of Engineering, and has been named an IBM Master Inventor. This distinguished alumnus of IIT Madras received his PhD at the University of Texas at Austin. He is an inventor of 38 patents. He serves on the advisory board of IEEE Spectrum and on the IBM Software Group Architecture Board’s Council. More information can be found in his home page at http://bit.ly/CMohan

== Week 5: Monday Oct. 7th - Query Processing on Mapreduce and High-level Languages ==

* Pig Latin and Query Processing:
** [http://www.vistrails.org/images/1-RelationalOnMapReduce.pdf Relational processing over MapReduce]
** [http://www.vistrails.org/images/2-PigOnMapReduce.pdf2-PigOnMapReduce.pdf Queries over MapReduce]
* In-class assignment

=== Required Reading ===

=== Additional References ===
* [http://pages.cs.brandeis.edu/~olga/cs228/Reading%20List_files/piglatin.pdf Pig Latin: A Not-So-Foreign Language for Data Processing]
* [http://www.mpi-inf.mpg.de/~rgemulla/publications/beyer11jaql.pdf Jaql: A Scripting Language for Large Scale Semistructured Data Analysis]
* [http://www.vldb.org/pvldb/2/vldb09-938.pdf Hive - A Warehousing Solution Over a Map-Reduce Framework]

== Week 6: Mon Oct. 14th - Fall Break - No class ==

== Week 7: Monday Oct. 21st - Invited Speaker: Torsten Suel ==

* Big Data and Information Retrieval. Invited lecture by Torsten Suel.
** Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/search-data.pdf

== Week 8: Monday Oct 28th- Statistics is easy - Invited Speaker: Dennis Shasha ==

* Guest lecture by [http://cs.nyu.edu/shasha/ Dennis Shasha]: [http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/stateasy.pdf Statistics is Easy]

=== Required Reading ===
* http://www.morganclaypool.com/doi/abs/10.2200/S00142ED1V01Y200807MAS001 -- book is available for free for NYU students
* Second edition of the book: http://www.morganclaypool.com/doi/pdf/10.2200/S00295ED1V01Y201009MAS008

== Week 9: Monday Nov 5th - EM and Text Processing ==

TODO

=== Readings ===

* Data-Intensive Text Processing with MapReduce, Chapter 6

== Week 10: Monday Nov. 11th - Finding Similar Items and Information Integration ==
* Similarity: Applications, Measures and Efficiency considerations
** Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/similarity.pdf
* Similarity application: Information integration on the Web:
** Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/web-info-integration.pdf
* Homework presentation and demo

=== Required Reading ===
* [http://infolab.stanford.edu/~ullman/mmds/ch3.pdf Mining of Massive Datasets, chapter 3; information integration; entity resolution]

=== Homework Assignment ===
'''Due November 17th'''
Your assignment is in http://www.newgradiance.com/services. Please see http://vgc.poly.edu/~juliana/courses/cs9223 for instructions on how to access this service.

== Week 11: Monday Nov 18th- Frequent Itemsets ==
** Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/assoc-rules1.pdf, http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/assoc-rules1.pdf

=== Required Reading ===
* Mining of Massive Datasets, Chapter 4

=== Homework Assignment ===
'''Due November 24th'''

=== Additional Reading ===
* Mining association rules between sets of items in large databases. Agrawal et al., SIGMOD 1993. http://delivery.acm.org/10.1145/180000/170072/p207-agrawal.pdf?ip=128.238.251.32&acc=ACTIVE%20SERVICE&CFID=198467341&CFTOKEN=23537886&__acm__=1352747519_b80a516e0f5e294b36dc021f13f55bbb
* Fast algorithms for mining association rules. Agrawal and Srikant, VLDB 1994. https://www.seas.upenn.edu/~jstoy/cis650/papers/Apriori.pdf
* An effective hash-based algorithm for mining association rules. Park et al., SIGMOD 1995. http://dl.acm.org/citation.cfm?id=223813

== Week 12: Monday Nov. 25th - Clustering ==

* Lecture notes:
** Graph algorithms: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/mapreduce-graph.pdf
**Clustering: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/clustering.pdf, http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/form-clustering-icde2007.pdf

=== Homework Assignment ===
'''Due Dec 1st'''

=== Readings ===
* Mining of Massive Datasets, Chapter 7
* See readings for previous class
* Web Mining, by Bing Liu. http://www.cs.uic.edu/~liub/WebMiningBook.html
* Information Retrieval. http://nlp.stanford.edu/IR-book/pdf/irbookprint.pdf

== Further Readings ==
* [http://www-users.cs.umn.edu/~kumar/papers/anomaly-survey.php Anomaly Detection: A Survey]

== Week 13: Monday Dec. 2nd - Invited lecture by Enrico Bertini ==
* Introduction to Visual Analytics
** Lecture notes: http://vgc.poly.edu/~juliana/courses/cs9223/Lectures/intro-to-visualization.pdf

=== Readings ===
The Value of Visualization. IEEE Visualization 2005. Jarke J. van Wijk. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.78.1138

Visualization Analysis and Design: Principles, Methods, and Practice. Tamara Munzner (Book Draft 2 from Sep. 2012). http://www.cs.ubc.ca/~tmm/courses/533-11/book/vispmp-draft.pdf

== Week 14: Monday Dec. 9th - - Graph Algorithms ==

TODO

=== Readings ===
* [http://infolab.stanford.edu/pub/papers/google.pdf 1998 PageRank Paper]
* [http://lintool.github.com/MapReduceAlgorithms/MapReduce-book-final.pdf Data-Intensive Text Processing with MapReduce, Chapter 4 (Inverted Indexing for Text Retrieval) and 5(Graph Algorithms)]
* [http://infolab.stanford.edu/~ullman/mmds/ch5.pdf Mining of Massive Datasets, Chapter 5 (Link Analysis)]
* Pregel: A System for Large-Scale Graph Processing. Google. [http://kowshik.github.com/JPregel/pregel_paper.pdf]

== Week 15 Monday Dec. 16th - Final Exam ==

== Other topics ==
===Provenance ===
* [http://vgc.poly.edu/~juliana/pub/vistrails-reproducibility2012.pdf Making Computations and Publications Reproducible with VisTrails]
Juliana Freire and Claudio Silva. In Computing in Science and Engineering 14(4): 18-25, 2012.

* [http://vgc.poly.edu/~juliana/pub/freire-cise2008.pdf Provenance for Computational Tasks: A Survey]
Juliana Freire, David Koop, Emanuele Santos, and Claudio T. Silva. In IEEE Computing in Science & Engineering, 2008.

File:2-PigOnMapReduce.pdf

2013-10-02T19:45:55Z

Jsimeon: Pig/lating queries over MapReduce

Pig/lating queries over MapReduce