{"id":3395,"date":"2012-04-12T16:07:54","date_gmt":"2012-04-12T20:07:54","guid":{"rendered":"http:\/\/www.dr-chuck.com\/csev-blog\/?p=3395"},"modified":"2012-04-12T16:43:37","modified_gmt":"2012-04-12T20:43:37","slug":"crawling-page-rank-and-visualization-in-python-for-si301","status":"publish","type":"post","link":"https:\/\/www.dr-chuck.com\/csev-blog\/2012\/04\/crawling-page-rank-and-visualization-in-python-for-si301\/","title":{"rendered":"Crawling, Page Rank and Visualization in Python for SI301"},"content":{"rendered":"<p><a href=\"http:\/\/www-personal.umich.edu\/~csev\/courses\/w12\/networks\/search3\/force.html\" target=\"_new\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/www.dr-chuck.com\/csev-blog\/wp-content\/uploads\/2012\/04\/search3.jpg\" alt=\"\" title=\"search3\" width=\"250\" height=\"237\" class=\"alignright size-full wp-image-3396\" \/><\/a>I have been hacking up some sample code for my SI301 course the past few weeks.  The course is about Networks, Crowds ,and Markets and so I wanted to build a rudimentary Python web crawler that would retrieve a web site, run a page rank algorithm on it, and then visualize the page rank and the links.<\/p>\n<p>If you click on  the image, you will see an interactive version of the visualization and be able to play with the visualization of some pages on <a href=\"http:\/\/www.sakaiproject.org\/\" target=\"_new\">www.sakaiproject.org<\/a>.  You can hover over a node to see the URL, or click and drag a node around, or double click on a node to launch the actual web page.<\/p>\n<p>Here is the <a href=\"http:\/\/www-personal.umich.edu\/~csev\/courses\/w12\/networks\/search3.zip\" target=\"_new\">Source code<\/a> in Python.<\/p>\n<p>It uses the completely cool <a href=\"http:\/\/mbostock.github.com\/d3\/\" target=\"_new\">D3 Data Driven Documents<\/a> to perform the visualization.<\/p>\n<p>Comments\/bug fixes welcome.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I have been hacking up some sample code for my SI301 course the past few weeks. The course is about Networks, Crowds ,and Markets and so I wanted to build a rudimentary Python web crawler that would retrieve a web site, run a page rank algorithm on it, and then visualize the page rank and [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3395","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/www.dr-chuck.com\/csev-blog\/wp-json\/wp\/v2\/posts\/3395","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dr-chuck.com\/csev-blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dr-chuck.com\/csev-blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dr-chuck.com\/csev-blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dr-chuck.com\/csev-blog\/wp-json\/wp\/v2\/comments?post=3395"}],"version-history":[{"count":11,"href":"https:\/\/www.dr-chuck.com\/csev-blog\/wp-json\/wp\/v2\/posts\/3395\/revisions"}],"predecessor-version":[{"id":3407,"href":"https:\/\/www.dr-chuck.com\/csev-blog\/wp-json\/wp\/v2\/posts\/3395\/revisions\/3407"}],"wp:attachment":[{"href":"https:\/\/www.dr-chuck.com\/csev-blog\/wp-json\/wp\/v2\/media?parent=3395"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dr-chuck.com\/csev-blog\/wp-json\/wp\/v2\/categories?post=3395"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dr-chuck.com\/csev-blog\/wp-json\/wp\/v2\/tags?post=3395"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}