Capstone: Retrieving, Processing, and Visualizing Data with Python

About this course: In the capstone, students will build a series of applications to retrieve, process and visualize data using Python. The projects will involve all the elements of the specialization. In the first part of the capstone, students will do some visualizations to become familiar with the technologies in use and then will pursue their own project to visualize some other data that they have or can find. Chapters 15 and 16 from the book “Python for Everybody” will serve as the backbone for the capstone. This course covers Python 3.

Created by:  University of Michigan

• Taught by:  Charles Severance, Associate Professor

  School of Information

Basic Info Course 5 of 5 in the Python for Everybody Specialization Commitment 6 weeks of study, 2-4 hours/week Language English How To Pass Pass all graded assignments to complete the course. User Ratings 4.5 stars Average User Rating 4.5See what learners said Coursework

Each course is like an interactive textbook, featuring pre-recorded videos, quizzes and projects.

Help from your peers

Connect with thousands of other learners and debate ideas, discuss course material, and get help mastering concepts.

Certificates

Earn official recognition for your work, and share your success with friends, colleagues, and employers.

University of Michigan The mission of the University of Michigan is to serve the people of Michigan and the world through preeminence in creating, communicating, preserving and applying knowledge, art, and academic values, and in developing leaders and citizens who will challenge the present and enrich the future.

Syllabus


WEEK 1


Welcome to the Capstone
Congratulations to everyone for making it this far. Before you begin, please view the Introduction video and read the Capstone Overview. The Course Resources section contains additional course-wide material that you may want to refer to in future weeks.


4 videos, 4 readings expand

1. Video: Introduction: Welcome to the Class
2. Reading: Capstone Overview
3. Reading: Help Us Learn About You!
4. Reading: Python Textbook
5. Reading: Coming from Python 2 - Encoding Data in Python 3
6. Video: Unicode Characters and Strings
7. Video: Office Hours in Den Haag, Netherlands
8. Video: Interview: John Resig and Pam Fox - Khan Academy

Graded: Using Encoded Data in Python 3

WEEK 2


Building a Search Engine



This week we will download and run a simple version of the Google PageRank Algorithm and practice spidering some content. The assignment is peer-graded, and the first of three required assignments in the course. This a continuation of the material covered in Course 4 of the specialization, and is based on Chapter 16 of the textbook.


6 videos, 1 reading expand

1. Reading: Building a Search Engine - Introduction
2. Video: Page Rank Overview
3. Video: Worked Example: Page Rank - Spidering (Chapter 16)
4. Video: Worked Example: Page Rank - Computation (Chapter 16)
5. Video: Worked Example: Page Rank - Visualization (Chapter 16)
6. Video: Office Hours Detroit, Michigan
7. Video: Interview: Anil Jain - Image Processing

Graded: Peer Grade: Page Rank

WEEK 3


Exploring Data Sources (Project)



The optional Capstone project is your opportunity to select, process, and visualize the data of your choice, and receive feedback from your peers. The project is not graded, and can be as simple or complex as you like. This week's assignment is to identify a data source and make a short discussion forum post describing the data source and outlining some possible analysis that could be done with it. You will not be required to use the data source presented here for your actual analysis.


2 videos, 2 readings expand

1. Reading: Identifying Your Data Source - Introduction
2. Reading: List of Data Sources (Instructional Staff Curated)
3. Discussion Prompt: Identifying a Data Source
4. Video: Dr. Chuck's New Kitten - Sakaiger
5. Video: Interview: Bruce Schneier - The Security Mindset

WEEK 4


Spidering and Modeling Email Data
In our second required assignment, we will retrieve and process email data from the Sakai open source project. Video lectures will walk you through the process of retrieving, cleaning up, and modeling the data.


5 videos, 1 reading expand

1. Reading: Spidering and Modeling Email Data - Introduction
2. Video: Gmane Introduction
3. Video: Worked Example: Gmane / Mail - Retrieval (Chapter 16)
4. Video: Worked Example: Gmane / Mail - Model (Chapter 16)
5. Video: Office Hours Baltimore, MD
6. Video: Interview: Bruce Schneier - Building Cryptographic Systems

Graded: Loading and Modeling Mail Data

WEEK 5


Accessing New Data Sources (Project)
The task for this week is to make a discussion thread post that reflects the progress you have made to date in retrieving and cleaning up your data source so can perform your analysis. Feedback from other students is encouraged to help you refine the process.


1 video, 1 reading expand

1. Reading: Accessing New Data Sources - Introduction
2. Discussion Prompt: Analyzing a Data Source
3. Video: Office Hours: Dr. Chuck Pretends to be Anthony Bourdain

WEEK 6


Visualizing Email Data
In the final required assignment, we will do two visualizations of the email data you have retrieved and processed: a word cloud to visualize the frequency distribution and a timeline to show how the data is changing over time.


3 videos, 1 reading expand

1. Reading: Visualizing Email Data
2. Video: Worked Example: Gmane / Mail - Visualization (Chapter 16)
3. Video: Office Hours, Montreal, Canada
4. Video: Interview: Nathaniel Borenstein - The Father of MIME

Graded: Visualizing Email Data

WEEK 7


Visualizing new Data Sources (Project)



This week you will discuss the analysis of your data to the class. While many of the projects will result in a visualization of the data, any other results of analyzing the data are equally valued, so use whatever form of analysis and display is most appropriate to the data set you have selected.


2 videos, 2 readings expand

1. Reading: Visualizing new Data Sources - Introduction
2. Discussion Prompt: Data Analysis and Visualization
3. Video: Office Hours - Dr. Chuck's Office - Ann Arbor, Michigan
4. Video: Video: Steve Jobs, NeXT and the Internet
5. Reading: Post-Course Survey