This is a repost from the IMS Developer’s forum – I forgot to put this in my own blog.

Possible issues with the OAuth C# library. I was working with some folks on C# Basic LTI and came across a seeming interoperability. Here is the discussion:

I looked at the Java Implementations and the PHP implementation and both seem to go to some length *to* double URL-encode the signature string. The PHP code interoperates with the Java code as distributed.

This results in the base signature string having %2520 where there are spaces. Which looks like a mistaken double-encode – but I think that it is an intentional double encode.

If you look at this page (broken into multiple lines to enhance readiblilty)

http://hueniverse.com/2008/10/beginners-guide-to-oauth-part-iv-signing-requests/

After you pick “Non-URL Safe Parameter” expand the little + signs and follow it through – it ends up with a base string that includes a %2520:

GET&http%3A%2F%2Fphotos.example.net%3A8001%2FPhotos&oauth_consumer_key
%3Ddpf43f3%252B%252Bp%252B%25232l4k3l03%26oauth_nonce
%3Dkllo~9940~pd9333jh%26oauth_signature_method%3DHMAC-SHA1
%26oauth_timestamp%3D1191242096%26oauth_token%3Dnnch734d
%25280%25290sl2jdk%26oauth_version%3D1.0%26photo%2520size%3D300%2525
%26title%3DBack%2520of%2520%2524100%2520Dollars%2520Bill

As I look at the sample Java code below – it is effectively double encoding very much on purpose.

So far, this suggests that it is *correct* when spaces turn into %2520 in the pre-signature string instead of %20. So I am thinking that the the sample VB.NET code from oauth.net *might* be wrong.

Sample Java code

public static String getBaseString(OAuthMessage message)
            throws IOException, URISyntaxException {
  List<Map.Entry<String, String>> parameters;
  String url = message.URL;
  int q = url.indexOf('?');
  if (q < 0) {
    parameters = message.getParameters();
  } else {
    // Combine the URL query string with the other parameters:
    parameters = new ArrayList<Map.Entry<String, String>>();
    parameters.addAll(OAuth.decodeForm(message.URL.substring(q + 1)));
    parameters.addAll(message.getParameters());
    url = url.substring(0, q);
  }
  return OAuth.percentEncode(message.method.toUpperCase()) + '&'
    + OAuth.percentEncode(normalizeUrl(url)) + '&'
    + OAuth.percentEncode(normalizeParameters(parameters));
}

protected static String normalizeParameters(Collection<? extends Map.Entry> parameters)
  throws IOException {
  if (parameters == null) {
    return "";
  }
  List<ComparableParameter> p = new ArrayList<ComparableParameter>(parameters.size());
  for (Map.Entry parameter : parameters) {
    if (!"oauth_signature".equals(parameter.getKey())) {
      p.add(new ComparableParameter(parameter));
    }
  }
  Collections.sort(p);
  return OAuth.formEncode(getParameters(p));
}

public static void formEncode(Iterable<? extends Map.Entry> parameters,OutputStream into) 
  throws IOException {
  if (parameters != null) {
    boolean first = true;
    for (Map.Entry parameter : parameters) {
      if (first) {
        first = false;
      } else {
        into.write('&');
      }
      into.write(percentEncode(toString(parameter.getKey())).getBytes());
      into.write('=');
      into.write(percentEncode(toString(parameter.getValue())).getBytes());
    }
  }
}

I logged in to YouTube today and was greeted by the following message, “You have videos that may contain content that is owned by someone else.” My first thought was, “Yikes – I am not intentionally a bad guy – nor am I intent on ripping anyone off – what had I done wrong?” It gave me a link to the offending video:
CHUM 2006 Drill Team Video (http://www.youtube.com/watch?v=vM6drBOzVjo)
Then it dawned on me – we had used “You Raise Me Up” by Selah as the background for the drill team. My thoughts raced – this was non-profit and this was about handicapped children riding horses and showing what they could do in front of their friends family and fans. How could YouTube be so heartless as to take away the background music from such a lovely and poignant moment. As the cameraman, I even sniffle part-way through the video, it was so moving for me personally. The music was what made it so poignant and without the music there would be no point. What hoops would I have to jump through to get the so important music back? Could I pay money to license the music? Probably there would not even be anyone to beg to restore the music. How could big powerful YouTube be so heartless and not notice the emotion in this video and cut us some slack?
With all these negative thoughts in my mind, I clicked on the video, expecting to hear silence and the dreaded “copyrighted material removed” message. Without the music, the video was meaningless – so I knew I had no other choice but to take it down. And it had 1800+ views and from the comments, it also moved some of those viewers to tears as well.
The video started out silent, but about five seconds in, the music was there! And about 2 seconds later an ad popped up for “You Lift Me Up”. The Drill Team Video was still intact!
So I went back to the YouTube page where it said “Copyright Info” and this is what it said:
“Your video, Drill Team 2006 Video – Dedicated to Chum, may include content that is owned or licensed by these content owners: WMG Type: Audio content. No action is required on your part. Your video is still available worldwide. In some cases ads may appear next to your video.”
Woah – the right thing just happened. I wonder if a human was involved in this – if so – thanks! Perhaps they saw that the video was very respectful of the original work and lovingly added value to the work without any attempt on my part to make money off the original work and instead my work hopefully in some small way added value to their work by honoring Selah’s work rather than ripping it off.
It is nice when things work out in a win-win for everyone. Thanks YouTube.

I thought this one was funny so I wrote it in my blog. I was engaging in E-Mail reparte when I thought this one up :)
Folks who bring up vague “security holes” in competing products as their advantage are usually squirting squid ink.
I won’t say who I was talking to or what we were talking about..

I spent the last week significantly revising “Think Python” in preparation for use in my course SI502 – Networked Programming that starts January 6.
I am at the point where I have reasonably polished text for the first 10 chapters of the book – which is what I will give to the students at the beginning of the semester and will be the first half of the course.

https://source.sakaiproject.org/contrib//csev/trunk/pyinf/tex/book.pdf

Once the course starts, I will write three more chapters (likely the night before lecture) when the topics come up in the class (Chapters 13, 14, and 15) in the extended table of contents in this version of the book.

https://source.sakaiproject.org/contrib//csev/trunk/pyinf/tex/book_full.pdf

I don’t where I will go next after that, I may write the visualization chapter (18) – we will see how that course goes. In a sense the book_full.pdf is where I would like to see the book before I publish it for anything other than use in my class.

Of course, books are always smoothed by use in classes and I expect that by using this book in SI502 I will get solid feedback from my students and grad-student instructors.
Of course any comments are always welcome.

I am pretty geeked since it took less than 10 days to produce this book. Now I spend the rest of the break, writing some code, preparing for class in January, and starting that *other* book I keep meaning to write but keep getting distracted from.

Well it has taken a week to get to the first rough draft of my new SI502 text book titled, “Python for Informatics”. Thanks to Allan B. Downey and Cambridge University Press and their forward looking open copyright, I already have 162 pages done. So far I have written a preface, changed Chapter 1, changed the order of the chapters, removed all text regarding recursion, removed object oriented and graphics content that was too advanced, and ran it all through LaTeX.

Here is a link to the most recent draft: Python for Informatics: Exploring Information. (This will always point to the latest draft.)

Here is the text from the preface of the book:

Remixing an Open Book

It is quite natural for academics who are continuously told to “publish or perish” to want to always create something from scratch that is their own fresh creation. This book is an experiment in not starting from scratch, but instead “remixing” the book titled Think Python: How to Think Like a Computer Scientist written by Allen B. Downey, Jeff Elkner, and others.

In December of 2009, I was preparing to teach SI502 – Networked Programming at the University of Michigan for the fifth semester in a row and decided it was time to write a Python textbook that focused on exploring data instead of understanding algorithms and abstractions. My goal in SI502 is to teach people lifelong data handling skills using Python. Few of my students were planning to be professional computer programmers. Instead, they planned to be librarians, managers, lawyers, biologists, economists, etc., who happened to want to skillfully use technology in their chosen field.

I never seemed to find the perfect data-oriented Python book for my course, so I set out to write just such a book. Luckily at a faculty meeting three weeks before I was about to start my new book from scratch over the holiday break, Dr. Atul Prakash showed me the Think Python book which he had used to teach his Python course that semester. It is a well-written Computer Science text with a focus on short, direct explanations and ease of learning.

The overall book structure has been changed to get to doing data analysis problems as quickly as possible and have a series of running examples and exercises about data analysis from the very beginning.

Chapters 2–10 are similar to the Think Python book, but there have been major changes. Number-oriented examples and exercises have been replaced with data-oriented exercises. Topics are presented in the order needed to build increasingly sophisticated data analysis solutions. Some topics like try and except are pulled forward and presented as part of the chapter on conditionals. Functions are given very light treatment until they are needed to handle program complexity rather than introduced as an early lesson in abstraction. Nearly all user-defined functions have been removed from the example code and exercises outside of Chapter 4. The word “recursion”1 does not appear in the book at all.

In chapters 1 and 11–16, all of the material is brand new, focusing on real-world uses and simple examples of Python for data analysis including regular expressions for searching and parsing, automating tasks on your computer, retrieving data across the network, scraping web pages for data, using web services, parsing XML and JSON data, and creating and using databases using Structured Query Language.

The ultimate goal of all of these changes is a shift from a Computer Science to an Informatics focus is to only include topics into a first technology class that can be useful even if one chooses not to become a professional programmer.

Students who find this book interesting and want to further explore should look at Allen B. Downey’s Think Python book. Because there is a lot of overlap between the two books, students will quickly pick up skills in the additional areas of technical programming and algorithmic thinking that are covered in Think Python. And given that the books have a similar writing style, they should be able to move quickly through Think Python with a minimum of effort.

As the copyright holder of Think Python, Allen has given me permission to change the book’s license on the material from his book that remains in this book from the GNU Free Documentation License to the more recent Creative Commons Attribution — Share Alike license. This follows a general shift in open documentation licenses moving from the GFDL to the CC-BY-SA (e.g., Wikipedia). Using the CC-BY-SA license maintains the book’s strong copyleft tradition while making it even more straightforward for new authors to reuse this material as they see fit.

I feel that this book serves an example of why open materials are so important to the future of education, and want to thank Allen B. Downey and Cambridge University Press for their forward-looking decision to make the book available under an open copyright. I hope they are pleased with the results of my efforts and I hope that you the reader are pleased with our collective efforts.

I would like to thank Allen B. Downey and Lauren Cowles for their help, patience, and guidance in dealing with and resolving the copyright issues around this book.

Charles Severance
www.dr-chuck.com
Ann Arbor, MI, USA
September 9, 2013

This is the first section of the first chapter of a new book project I am playing with called “Python for Informatics” which is a remix of the book “Think Python” by Allen B. Downey. Comments welcome.

Writing programs (or programming) is a very creative
and rewarding activity. You can write programs for
many reasons ranging from making your living to solving
a difficult data analysis problem to having fun to helping
someone else solve a problem. This book assumes that everyone needs to know how to program and that once
you know how to program, you will figure out what you want
to do with your newfound skills.

We are surrounded in our daily lives with computers ranging
from laptops to cell phones. We can think of these computers
as our “personal assistants” who can take care of many things
on our behalf. The hardware in our current-day computers
is essentially built to continuously as us the question,
“What would you like me to do next?”.

Our computers are fast and have vasts amounts of memory and
could be very helpful to us if we only knew the language to
speak to explain to the computer what we would like it to
“do next”. If we knew this language we could tell the
computer to do tasks on our behalf that were reptitive.
Interestingly, the kinds of things computers can do best
are often the kinds of things that we humans find boring
and mind-numbing.

For example, look at the first three paragraphs of this
chapter and tell me the most commonly used word and how
many times the word is used. While you were able to read
and understand the words in a few seconds, counting them
is almost painful because is is not the kind of problem
that human minds are designed to solve. For a computer
the opposite is true, reading and understanding text
from a piece of paper is hard for a computer to do
but counting the words and telling you how many times
the most used word was used is very easy for the
computer:

python words.py
Enter file:words.txt
to 16

Our “personal information analysis assistant” quickly
told us that the word “to” was used ten times in the
first three paragraphs of this chapter.

It is this very fact that computers are good at things
that humans are not so good at is why you need to become
skilled at talking “computer language”. Once you learn
this new language, you can delegate mundane tasks
to your partner (the computer), leaving more time
for you to do the
things that you are uniquely suited for. You bring
creativity, intuition, and inventiveness to this
partnership.

It is fun when things move quickly. On Monday morning while reading Twitter on my inbound commute, I saw a tweet from Martin Dougiamas about Moodle 2.0 design being different than Moodle 1.0 since things are very different in 2009 than in 2003. He saw things like Ning, etc as part of the mix these days. Back in 2003 things were much simpler so Moodle 1.x could be much simpler.
So I thought to myself, “If Martin is thinking about Ning – maybe I should take a look at it…”. I asked a few friends if they knew anything about Ning. It turns out Michael Korkuska knows some folks at Ning – so I asked Michael if Ning is interested in education. He said there was some interest in education (I was sending E-Mail to Michael Monday night while driving home).
So on the last 40 minutes of my commute Monday night, I idly wondered if there was a way to plug learning tools into Ning using IMS Basic LTI (my obsession).
Tuesday morning, I figured I would work through the Ning developer documentation and make a Ning App. So I started to look at Ning Apps. Woah! Ning Apps are OpenSocial apps. Well that *is* pretty cool excuse to learn OpenSocial coding. Sure it is Javascript, but they give you little pre-provisioned APIs to so stuff like set and get preferences (kind of like SCORM). And they tell you if the current user is the Owner of the widget or just a viewer of the widget. Cool! This will be easy – I was “almost done” before I even started.
Then I realized that this is all Javascript and as such there is zero security. Crap! People can run the whole thing in a Javascript debugger and call any API they like – grrr.
But then I found signed requests and how to validate requests – and in Python even! We are back in business:
http://wiki.opensocial.org/index.php?title=Introduction_To_Signed_Requests
http://wiki.opensocial.org/index.php?title=Validating_Signed_Requests
Oh yes… And then you can preload the data talking server to server to do this in parallel – and make these preloads signed.
http://wiki.opensocial.org/index.php?title=Remote_Data_Requests
Oh and one more thing – it uses OAuth. It does use a public/private key signing pattern instead of shared secret – but hey, it works and has some cool potential for simpler bootstrapping than shared secret. (And it is Tueday lunch time – about 24 hours since I saw Martin’s Tweet).
Going forward – I think that I cannot build an OpenSocial Widget that directly implements IMS Basic LTI because JavaScript is a GIANT security hole that I think is unsolvable (I asked Noah if it could be solved and he said “no” and I never question Noah on matters of breaking JavaScript security).
But I think I can make a *lovely* little App Engine Application that exports a Widget out one end and connects to IMS Basic LTI out the other end seamlessly. While it would be nice to do it inside the widget completely, it appears that here is a *LOT* of variation between OpenSocial containers and that both the spec and vendor level of support is a moving target, the safest bet is to use as little of the OpenSocial API as one can get away with and do as much as possible in our own server to maximize interoperability and the ability to put in vendor-specific hacks as needed. With that in mind, a simple and elegant approach is pretty obvious and will take a few days of hacking to build. Of course I don’t have those days right now since I want to write two books over the break – perhaps this will be an independent study next semester :).
I will end thinking, “what a difference 24 hours can make” when you have people who help you come up with new ideas and people who can help you think. To reward you for reading this rambling tome this far, here is a screen shot of IMS Basic LTI inside of Ning:
.
This is not real code yet – but it does send the signed form to the tool with the auto-submit within the OpenSocial widget. Here is source code
http://www.dr-chuck.com/ning/bltihack.xml
Please do not laugh too much – because I stopped in mid-hack to work on final exams for SI502 and SI539. I stopped once I realized that my in-widget approach was pointless and that a server would need to be involved.

I am thinking seriously about adopting “Think Python” by Allen B. Downey as the SI502 textbook. I like this book in particular because it is open and as such, I can alter it towards my ultimate goal of a book titled “Python for Informatics” which is focused on data analysis in Python. Think Python is a good starting point and saves me from writing the basic stuff about variables, loops, etc.

It is written in LaTeX and so I need to learn LaTeX – here are my notes so far for getting LaTeX up and running under Mac OS/X and getting the ThinkPython book to render.

Install the MacTeX-2009 Distribution
http://www.tug.org/mactex/2009/
It needs a style file which you install in the main book directory:
LATEX to HTML - http://hevea.inria.fr/
curl -O http://hevea.inria.fr/distri/hevea.sty
Then I run these commands manually since I don't
have make installed because it is my
Air is always short on disk space :(
latex book
makeindex book
latex book
dvips -t letter -Ppdf -o thinkpython.ps book
open thinkpython.ps

Then the issue is how to make new figures – Allen uses the classic XFig – I am looking for an alternative – I found two.

I found jPicEdt described on a Forum on MacRumors. But I simply could not figure out how to change Font size – so I kept looking.

Then I found LaTeXDraw – and while the UI confuses me – I did figure out how to put up a text box and change the font in about 45 minutes – urrg. So I think I can make new EPS figures if I want to add a chapter or two to the book.

So it looks like I can write text, make figures and off I go. I might even learn to like LaTeX… Many thanks to Allen B. Downey for having a great book with a nice copyright. Off I go remixing your stuff… :)

I just recorded a web briefing on IMS Basic LTI – it is 30 minutes long.

http://www.vimeo.com/8073453

Here is the abstract:

This is a recorded presentation about the IMS Basic Learning Tools Interoperability standard from imsglobal.org. This talk gives a high-level technical overview of Basic LTI and talks about how Basic LTI connects to IMS Common Cartridge (CC) and IMS Learning Information Services (LIS). This also talks about the status of market adoption amongst the LMS vendors and tool providers.

I am having trouble typing this blog post because I have tears streaming down my face and my throat hurts because I have been laughing so much. I had to stop looking at the page for five minutes so I could calm down enough to type this blog post.

This is a real product on Amazon. Look at the customer images, customer tags, and the reviews of the product. Just trust me – but make sure you are in a safe place where you cannot hurt yourself.

Laptop Steering Wheel Table

You have been warned. My laptop screen has tear drops on it because I laughed so hard. Wisdom of crowds – Oh My!