BigDataHW1 - Revision history

Juliana at 13:34, 2 October 2012

2012-10-02T13:34:55Z

← Older revision		Revision as of 13:34, 2 October 2012
Line 7:		Line 7:

	* Install Hadoop and HDFS on your laptop		* Install Hadoop and HDFS on your laptop
	* The hurricane center data ~~that~~ in ~~the~~ files ~~located at~~: http://vgc.poly.edu/~juliana/courses/cs9223/Homework/HW1~~. File1 (''~~hurricane-center-addresses.csv'') contains the addresses for hurricane centers~~, while~~ File2 ("hurricane-center-coordinates.csv") contains ~~their~~ coordinates.		* The hurricane center data comes in two files:
			** File1 (http://vgc.poly.edu/~juliana/courses/cs9223/Homework/HW1/hurricane-center-addresses.csv) contains the addresses for hurricane centers
			** File2 ("http://vgc.poly.edu/~juliana/courses/cs9223/Homework/HW1/hurricane-center-coordinates.csv") contains the coordinates for the centers
	* Write and run MapReduce programs that:		* Write and run MapReduce programs that:

Juliana at 03:51, 2 October 2012

2012-10-02T03:51:17Z

← Older revision		Revision as of 03:51, 2 October 2012
Line 7:		Line 7:

	* Install Hadoop and HDFS on your laptop		* Install Hadoop and HDFS on your laptop
	* ~~Copy to HDFS the~~ hurricane center data that in the files located at: http://vgc.poly.edu/~juliana/courses/cs9223/Homework/HW1		* The hurricane center data that in the files located at: http://vgc.poly.edu/~juliana/courses/cs9223/Homework/HW1. File1 (''hurricane-center-addresses.csv'') contains the addresses for hurricane centers, while File2 ("hurricane-center-coordinates.csv") contains their coordinates.
	* File1 (''hurricane-center-addresses.csv'') contains the addresses for hurricane centers, while File2 ("hurricane-center-coordinates.csv") contains their coordinates.
	* Write and run MapReduce programs that:		* Write and run MapReduce programs that:

Line 17:		Line 16:
	c) Compute the natural join between File1 and File2, i.e., create a new file that contains both the addresses and coordinates for the hurricane centers		c) Compute the natural join between File1 and File2, i.e., create a new file that contains both the addresses and coordinates for the hurricane centers

	* You should submit your java program source and binary (named hw1.java and hw1.jar). The program should accept as an argument the question letter (a,b, and c) and output the answer for that question.		* You should submit your java program source and binary (named hw1.java and hw1.jar). The program should accept as an argument the question letter (a,b, and c) and output the answer for that question. Your program should read the files from their http location.

Juliana at 03:49, 2 October 2012

2012-10-02T03:49:57Z

← Older revision		Revision as of 03:49, 2 October 2012
Line 4:		Line 4:


	* As a good citizen of NYC, you want to be prepared for potential disaster scenarios and you want to learn about hurricane evacuation centers in NYC: https://www.google.com/fusiontables/DataSource?docid=1BiOUN5JP94FT5pV9UNCQIvFloHqS_NTboOTvVD8		As a good citizen of NYC, you want to be prepared for potential disaster scenarios and you want to learn about hurricane evacuation centers in NYC (https://www.google.com/fusiontables/DataSource?docid=1BiOUN5JP94FT5pV9UNCQIvFloHqS_NTboOTvVD8). You will:

	You will:

	* Install Hadoop and HDFS on your laptop		* Install Hadoop and HDFS on your laptop
	* Copy to HDFS the files in: http://vgc.poly.edu/~juliana/courses/cs9223/Homework/HW1/		* Copy to HDFS the hurricane center data that in the files located at: http://vgc.poly.edu/~juliana/courses/cs9223/Homework/HW1
	* File1 (''hurricane-center-addresses.csv'') contains the addresses for hurricane centers, while File2 ("hurricane-center-coordinates.csv") contains their coordinates.		* File1 (''hurricane-center-addresses.csv'') contains the addresses for hurricane centers, while File2 ("hurricane-center-coordinates.csv") contains their coordinates.
	* Write and run MapReduce programs that:		* Write and run MapReduce programs that:

Juliana: Created page with 'cs9223 - Massive Data Analysis - Homework Assignment 1 Hands-on experience with Hadoop/MapReduce * As a good citizen of NYC, you want to be prepared for potential disaster sce…'

2012-10-02T03:47:40Z

Created page with 'cs9223 - Massive Data Analysis - Homework Assignment 1 Hands-on experience with Hadoop/MapReduce * As a good citizen of NYC, you want to be prepared for potential disaster sce…'

New page

cs9223 - Massive Data Analysis - Homework Assignment 1

Hands-on experience with Hadoop/MapReduce

* As a good citizen of NYC, you want to be prepared for potential disaster scenarios and you want to learn about hurricane evacuation centers in NYC: https://www.google.com/fusiontables/DataSource?docid=1BiOUN5JP94FT5pV9UNCQIvFloHqS_NTboOTvVD8

You will:

* Install Hadoop and HDFS on your laptop
* Copy to HDFS the files in: http://vgc.poly.edu/~juliana/courses/cs9223/Homework/HW1/
* File1 (''hurricane-center-addresses.csv'') contains the addresses for hurricane centers, while File2 ("hurricane-center-coordinates.csv") contains their coordinates.
* Write and run MapReduce programs that:

a) Select the tuples from File1 where zipcode<10030

b) Eliminate duplicates from File1

c) Compute the natural join between File1 and File2, i.e., create a new file that contains both the addresses and coordinates for the hurricane centers

* You should submit your java program source and binary (named hw1.java and hw1.jar). The program should accept as an argument the question letter (a,b, and c) and output the answer for that question.