I am really pleased with the new Basic LTI and IMS Common Cartridge 1.1 support in Sakai 2.9. If you want to skip the boring detail below – just watch this video and send it to your friends:
http://vimeo.com/27113903
Here is the back story.
Back on July 2, I decided to build a whole new capability to support Basic LTI and IMS Common Cartridge 1.1 in Sakai. I kicked it off with the following blog post:
Coming Back Home to Sakai 2.9 (July 2)
I talk about how Sakai started out with the first shipping support for IMS Basic LTI over two years ago, but the support from the other vendors had matured very nicely and our support was looking a little oversimplified and creaky. With my desire for Sakai 2.9 to be a nice, fresh relaunch of Sakai with its new Portal and Rutger’s Lesson Builder, it seemed a perfect time to dive into building a state-of-the-art Basic LTI support for Sakai and prepare Sakai to support additional new LTI features coming from the IMS LTI working group thanks to Lance Neumann of Blackboard and Greg McFall of Pearson.
I had a deadline of August 1 because Chuck Hedrick and Eric Jeney of Rutgers were madly working on improving Lesson Builder for a Fall rollout at Rutgers and inclusion in the Sakai 2.9 release. Chuck built IMS Common Cartridge 1.0 support and all of IMS CC 1.1 support except the Basic LTI part. Without Basic LTI support in Lesson Builder, we could not claim compliance to IMS CC 1.1.
So I needed to build a new capability for Sakai, plug it into Site Info, build a new administrator tool, and then plug it all into Lesson Builder’s authoring environment and import code. And it all had to be done by August 1 to give Chuck enough time to test it for production at Rutgers.
This resulted in a sprint for the past four weeks while I coded on planes, in my home office, on a camping trip, at the joint Sakai/Jasig meeting in New York, during cab rides to and from airports, during the Blackboard Developer conference. You can even see a mess up at the end of my recorded BBDevCon presentation because I forgot to test a demo that John Fontaine and I were doing because I was coding on the new Sakai features right up until five minutes before my talk. It was a 24×7 code sprint for four weeks.
Here is my presentation at BbDevCon11 – the disaster is towards the end. It all finally worked and the source of the problem was filtering of port 8080 – but it had me flustered for a while thinking all Sakai servers in the world were down. You can hear Fontaine in the background giving me a good-natured ribbing as I struggled to find a Sakai server that worked. I eventually just did the launch of CourseSites from Michigan’s CTools production server (because it did not use port 8080):
You can also see a demo of the Fontaine/Severance hack wthout a disaster in the middle here:
I do need to apologize to everyone who I interacted with during July if I seemed distracted and itchy to get back to my laptop. I was not just writing code, I was also building a UI and persistence framework and then using that framework to build my tools.
What We Accomplished
I am mixing the work of Chuck and Eric with my own work.
- Built a new “External Tools” extension to Site Info that lets you manage your tool installations (Full LTI coming soon) as well as make tool configurations. These tool configurations have URLs that can be used elsewhere in Sakai like in Resources, Wiki, Melete, anywhere that you can put a URL. You can spread the URLs around and still manage all the tools (change keys settings, etc) from one central location in Site Info.
- Added a new tool for System Administrators to monitor usage of external tools across the system, and reach in and tweak settings on a tool installation or a tool configuration. They can create site-wide tool installations that appear in all sites.
- Lesson Builder now has the ability to install, configure and launch external tools right from within the Lesson Builder UI as another resource type. In particular, instructors can set per page values for LTI launches like custom parameters very easily.
- Lesson Builder imports Basic LTI items from IMS 1.1 Common Cartridges. Since cartridges do not have keys and secrets, the import either hooks the imported items to an existing installed tool with a key and a secret or makes a new tool with an un-configured key and secret. There are several workflows for getting a key / secret set on a tool ranging from using the Administrator tool, Site Info, or even when an instructor launches a partially configured tool. The flow was designed to make it as natural for the instructor as possible.
Everything is working and demonstrated in the above video. I am sure that there are little rough edges with our flows and screens UI wise and we can tweak those as more people look at the code (now sitting in the trunk of Lesson Builder, Site Info, and Basic LTI). We are very much open to suggestions as folks review the work. But hurry because code freezes are coming up quickly.
Technical Details
First, I had to decide on which display technology and persistence to use to build my new tool in Sakai. My preference for a new tool is by far to build a JSR-168 portlet using JSP and JSTL as the presentation layer. My Basic LTI portlet avoided the persistence problem by using Sakai’s Placement properties. But this tool needed to be a frame-based helper because it needed to work with Site Info and Lesson Builder as a helper. So a portlet was not possible. Also since this needed lots of tools independent of a particular placement, I had to break down and make some database tables – so choosing a persistence layer was necessary for this project.
I don’t like heavy frameworks like JSF, RSF, and Wicket because they force tons of extra Java and XML just to print out “Hello World”. I was going to have a very complex data model and very dynamic UI because the administrator gets to control what fields (like popup and custom parameters) the instructor can configure on each LTI tool placement. I did not feel like encoding it all into a bunch of getters, setters, and bean properties, having to code everything 4-5 times when I wanted to change something.
So I decided to use Velocity augmented by my own presentation layer. I had built a really cool presentation layer when I wrote the Basic LTI code for ATutor where I made it really easy for myself to make a complex dynamic UI. I called that notion the “Form Oriented Object Relational Model”. I documented an early PHP version of FOORM in this blog post:
ATutor with Basic LTI is Released
I decided instead of writing thousands of lines of PHP to implement the views for ATutor, that I would make a library to build the HTML for the sets of fields I needed. This is also a bit of a riff on the way Moodle handles forms generically. Moodle provides a set of form handling routines that enforce consistency of look and feel and keep the details hidden behind an abstraction. It worked but was a little too chatty with too many lines needed for each form field. The Moodle model is similar to Glenn’s Ambrosia approach.
I took the opportunity to rewrite my FOORM code in Java and clean it up and make it work more generally.
FOORM Presentation and Persistance
In FOORM you build a model using an array of strings that look as follows;
static String[] CONTENT_MODEL = {
"id:key",
"tool_id:integer:hidden=true",
"SITE_ID:text:maxlength=99:label=bl_content_site_id:role=admin",
"title:text:label=bl_content_title:required=true:maxlength=255",
"frameheight:integer:label=bl_frameheight",
"newpage:checkbox:label=bl_newpage",
"debug:checkbox:label=bl_debug",
"custom:textarea:label=bl_custom:rows=5:cols=25:maxlength=1024",
"launch:url:hidden=true:maxlength=255",
"xmlimport:text:hidden=true:maxlength=16384",
"created_at:autodate",
"updated_at:autodate" };
This is kind of like a Rails model in colon-separated form. One thing that FOORM assumes is that you will be internationalizing your UI and so it even encodes the field labels..
You can build an input form with the following lines of code:
String formInput(Object previousData, String [] modelInfo, Object resourceLoader)
This returns a set of input tags, one for each field, repopulated from the previous data (can be Properties, Map<String,Object>, or ResultSet) and using the provided resource loader.
You embed the form tags in some Velocity as follows:
<form action="#toolForm("")" method="post" name="customizeForm" >
$formInput
<input type="hidden" name="sakai_csrf_token" value="$sakai_csrf_token" />
<input type="submit" accesskey ="s" class="active" name="$doAction"
value="$tlang.getString('gen.save')" />
<input type="submit" accesskey ="x" name="$doCancel"
value="$tlang.getString('gen.cancel')"
onclick="location = '$sakai_ActionURL.setPanel("Content")';return false;">
</form>
There are utilities to validate incoming data, and extract it into the right objects and type for inserting into a database. with a very few lines of code. Here is an extract from request properties into newMapping and insert into the database:
HashMap<String, Object> newMapping = new HashMap<String, Object>();
String errors = foorm.formExtract(requestProperties, formModel,
rb, true, newMapping, null);
if (errors != null) return errors;
String[] insertInfo = foorm.insertForm(newMapping);
String makeSql = "INSERT INTO " + table + " ( " + insertInfo[0] +
" ) VALUES ( " + insertInfo[1] + " )";
final Object[] fields = foorm.getInsertObjects(newMapping);
m_sql.dbInsert(null, sql, fields);
It supports a very REST-style property bag approach to data models except that all the fields are properly and fully realized as database columns so they can be searched, selected, grouped, and ordered with as sophisticated query as you want to create.
FOORM is designed to be extended so you have the generic Foorm and you extend it to make a SakaiFoorm that captures all the Sakai-specific rules for generating fields, handling internationalization, etc.
I even dispensed with the “CREATE TABLE” scripts – since Foorm knows the entire model, it can make the tables and sequences automatically. It know about the various rules for fields in MySql, HSQL, and Oracle. The AutoDDL code looks as follows:
String[] sqls = foorm.formSqlTable("lti_content",
LTIService.CONTENT_MODEL, m_sql.getVendor(), doReset);
for (String sql : sqls)
if (m_sql.dbWriteFailQuiet(null, sql, null)) M_log.info(sql);
I could make this more succinct with an overridable method in Foorm. Foorm still needs to learn about changes to data models and automatically generate “ALTER TABLE” commands. I will write that code when I need to do a conversion when I release a future version of DBLTIService is released and I need to expand the model.
I think that Foorm has a lot of potential. It is kind of a weird combination of a partial presentation layer and partial object relational mapper. It is focused on providing useful library utilities that let the developer get as tricky as they want, making sure that the nasty repetitive common tasks take as little effort as possible.
You can ask me “Why not Hibernate?” or “Why not Spring JDBC?” at a bar sometime. Be prepared to hear my voice raise a few notches as I rail about “Hibernate’s is an overbearing over-engineered steaming pile of…(oops did I say that out loud?)” or “Spring JDBC makes SQL portable in Java only to the level that they solve the trivial problems and punt on anything mildly interesting or complex…”. If I have had a few beers before the conversation starts, it will be more (or less) interesting.
Oh by the way, Foorm provides a nice, generic way to handle paged SQL queries. I still need to build advanced WHERE clauses, and support ORDER BY and GROUP BY operations.
This is why Foorm is still a prototype and I don’t want anyone else using it for a while. I feel that any library / framework should be used by its creator for a long time, solving lots of real-world problems before they claim to have anything truly reusable. But I figured I would show folks what I am thinking.
I have been experimenting with finding the right combination of power and convenience for both display layers and persistence. Before FOORM there was OMG-ORM (http://omg-software.com/) – it too was an exploration of the balance between powerful capabilities and intrusiveness as well as trying to make things intuitive. FOORM is much simpler than OMG and I find it more natural to use.
Summary
In short, it has been a heck of a month of pure code to the exclusion of nearly everything else. I built a UI framework and persistence layer and then built a tool on top of my new frameworks. Chuck Hedrick and Eric Jeney of Rutgers sprinted on Lesson Builder (pausing along the way to take the “Upload web site” code from Adrian Fish of Lancaster). And In the last few days, Chuck, Eric, and I spirited to connect it all together to make it so Lesson Builder has great, extensible Basic LTI support in both its authoring and import.
It means that Sakai 2.9 really catches up in many important ways to Blackboard, Desire2Learn, Learning Objects, Jenzabar, Instructure, and OLAT in their support of Basic LTI and Common Cartridge. Blackboard is still the only CC 1.1 exporter – so they have the lead on that one and I don’t think we will have that until next year sometime in Sakai. Export is hard to make import easy – my hat is off to Blackboard for making it work. Blackboard’s CC 1.1 export is why they were the first in the IMS ring of tattoos.
But for me export is lower priority than the next round of innovations for LTI coming from the IMS LTI Working Group led by Greg McFall of Pearson and Lance Newmann of Blackboard. This work is ready to pop and we should see some real progress on adding auto provisioning and run-time services to LTI implementations in the field. The spec leads have done a great job and the new Site Info work is ready to be extended to add these new features as we get the standards out the door.
It might be a good time to start to watch IMS closely. You only have a few more days to make the IMS meeting in Redmond, WA at Microsoft’s HQ. You have to pre-register – so hurry!