Course Information
Course description
CS341 (Project in Mining Massive Data Sets) is a project-focused advanced class with access to a large MapReduce cluster. This course is the second part in a two part sequence CS246/CS341.
CS246 discusses methods and algorithms for mining massive data sets. In this class, we will develop large scale data mining techniques and research projects. Students will have access to Amazon EC2 computing cluster. This means we will be able to run massive MapReduce jobs. Because it is challenging to work on algorithms for large scale data mining, we will be able to work with only a small number of students, and enrollment will be limited.
This is a purely project based course. We expect that students are already to some extent familiar with data mining methods. There will be lectures on some advanced data mining algorithm at the begging of the quarter. We also expect to have a good number of industrial guest lecturers discussing big data case studies.
Prerequisites
Knowledge and familiarity with concepts of CS246 or similar class (Hadoop, large scale data mining and machine learning algorithms).
Other courses that might be helpful: CS221, CS224N, CS224W, CS228, CS229, CS276, EE364A.
Enrollment
Doing research in data mining can be challenging! Thus we will only be able to work with a small number of students, and enrollment will be limited.
Course application procedure
To apply to the course follow the following instructions
- Form a team of three students. We strongly discourage teams of 2/4 students. Use Piazza to find a project partner.
- Find a large dataset that you will work on. See list of datasets (and project ideas) that we provide. But feel free to use your own dataset.
- Write a project proposal (there is no page limit but we do not promise to read more than the first 3 pages). Your project proposal should be structured into the following sections (it should concisely answer the following questions):
- What is the problem your team is solving? Give a brief but precise description or definition of the problem.
- What data will you use? Briefly describe the data, its size (number of records, number of GBs) and where will you get the data. It is fine to use your own data but make sure you have it before you propose the project.
- How will you solve the problem? Describe your approach. Tell us what method, algorithm, technique do you plan to develop or use and how will you scale them up to your size of the data. Be as specific as you can!
- How will you evaluate your method? Here we want you to describe how will you measure performance or success of your method. Against what baseline methods will you compare your algorithm or how do you plan to obtain ground-truth labeled data so that you can then measure accuracy, precision, recall or some other metric that will tell us how well is your method really performing.
- What do you expect to submit/accomplish by the end of the quarter?
- Your proposal should also include the following info:
- Project title
- Team member info: name, email address, SUNet id (and if you have also Twitter handle)
- Team member description: give us 2 sentence description of your experience that makes you qualified for this class. Tell us what classes you took in the past (CS 246 very strongly recommended).
- Send us the PDF of your proposal to cs341-spr1516-staff@lists.stanford.edu
If you would like some help or guidance when developing your project idea feel free to contact course staff (Jure or any other instructor). We will help you develop your project idea.
Project proposal submission deadline is March 18 at 11:59pm Pacific Time. Over the weekend we will evaluate the proposals and notify you whether your team got accepted in the class.
Project writeups
The result of the project is a 5-10 page paper. We will not accept longer reports.
- Writeup should address :
- Problem description: Give a brief but precise description or definition of the problem or hypothesis you set to evaluate.
- Data: What data did you use?
- Experiments:
- Setup: Describe how did you setup your experiments, how the training/testing data was prepared, what your evaluation methods are, and what baseline methods for comparison are you using.
- Results: Describe your experimental results.
- Impediments: What were some of the issues you encountered in your experiments? How did you deal with them?
- Future Work: What remains to be done?
- Brief conclusion
- Send us the PDF of your final writeup to cs341-spr1516-staff@lists.stanford.edu
Course materials
Lecture notes and/or slides will be posted in the handouts section of the website.
The book Mining of Massive Datasets by Anand Rajaraman and Jeff Ullman serves as a comprehensive reference for the background required for this course. Also you will find Chapter 20.2, 22 and 23 of the second edition of Database Systems: The Complete Book (Garcia-Molina, Ullman, Widom) relevant.
Requirements
Students will be required to successfully complete a substantial data-mining project. There will be mid-quarter project milestone submission/present ion in addition to the final project presentation and report submission.
Recitation sessions
Recitation sessions will be held to guide the students on the use of Amazon services.
Course work
The coursework for the course will consist of:
- Midterm in-class presentation
- Final project report and in-class presentation
Communication:
General course questions should be posted Piazza.
Piazza requires @stanford.edu email address to register. If you do not have @stanford.edu address, send us email with your email address and we will add you to Piazza.
If you need to reach the course staff, you can reach us at cs341-spr1516-staff@lists.stanford.edu (consists of the professors and the TA).