Twas the night before Sakai 2.2. integration week and all through the house…

As we start integration week for 2.2 tomorrow, it seems like a good time for a status update. It seems like yesterday when we were releasing 2.1. The time between 2.1 and 2.2 went very swiftly, probably because the biggest goal right after 2.1 was to being OSP tools into Sakai – that effort started in December and is still ongoing but finishing up.
It looks like if all goes well next week, the OSP tools will be provisional tools in Sakai 2.2. That is great news as it seals the connection between Sakai and OSP going forward and allows pretty much every single Sakai site to run OSP pilots this Fall if they chose so by simply enabling the tools.
For me, the API name change is a big deal for 2.2. The reorganized API naming convention really allows us to start writing some detailed developer documentation without worrying about some wholesale name change coming up in the future. These kinds of things are best done as early as possible with as few tools in the mix as possible. Sakai’s tool set is expanding rapidly and we do not need another year’s worth of tools that need to go through an API change. The APIs will continue to evolve but now they will evolve slowly and evolve as new functionality comes out.
Another big thing that has taken a bunch of effort is the source tree reorganization for 2.2. This effective is a pre-requisite to two important capabilities – multiple Sakai distributions and a far more flexible branching scheme. We already have three Sakai “distributions” – Sakai “mini”, Sakai “training” and Sakai “jumbo”. The key is that by breaking Sakai into component pieces and separating the “assembly” from the pieces, we enable a wide range of distributions. I see a Sakai-Research distribution, a Sakai-K12 distributions, a Sakai-HigherEd distribution, and who knows how many different commercial distributions. Each distribution can pick and choose a different set of components and different tags/branches/revisions of each component. This is a very very powerful flexibility that will hold us in good stead going forward.
In terms of branching, we now can branch individual elements rather than branching “sakai-jumbo” all at once. Also branching and branch maintenance can be delegated to the project teams. They can create a branch as necessary for what ever purpose – and with the multi-distribution capability, we can have some more conservative and some more aggressive distributions. Now even though we have the capability to have 1000 different distribution variants, we still need to “stick together” so we can continue to share QA and production experience.
However, 2.2 may mark the beginning of the “real maintenance branch” for Sakai. Historically we have had a pretty aggressive maintenance branch approach where we put stuff beyond just bug fixes into maintenance branches. Unfortunately with the ability to branch all or nothing, we had to stick together as some elements of Sakai (Resources, OSP, and Samigo) experienced significant development and bug fixes in between releases. This meant that while 85% of the maintenance branch was really “maintenance”, the other 15% of the maintenance branch was “chasing the trunk” somewhat. We can debate over beer if this was a good approach or not – but I think that we were making the best of a complex situation for 2.1.
But for 2.2 we can and should change. We need to switch to a convention where maintenance branches are really maintenance branches. For the 85% of Sakai that is pretty mature and evolving slowly – this will be no big deal. For the other elements (this will be different for each release), we will just ask the project teams to do maintain a fix-only branch and do their evolutionary work in other branches.
It is always dangerous for the Chief Architect (a worthless bureaucrat) to brainstorm about branching, but it seems like the right way to go for slow-moving tools is to have a trunk and maintenance branch. For fast moving tools, perhaps we need a trunk, stable, and maintenance branch. Stable is that code aimed at the next release but viewed as solid by the project team.
Most sites then can take maintenance with a pretty high degree of confidence – and other sites needing new features can take stable for some elements as needed. In effect we could produce a maintenance distribution and a stable distribution. Stable would be more like what we called “maintenance” for 2.0 and 2.1.
This will be a burden on the project teams who are doing active development – especially the further we get past any major release. So for this to work, we will need to find as many bugs as possible before the final release – especially in these fast-moving elements like Resources, OSP, and Samigo *before* the final 2.2 release.
The 2.2 release will also contain some new functionality beyond OSP although it has been kind of quiet. Probably the biggest end-user improvements will be group-awareness in more of the tools – this is a nice follow-on to Sakai 2.1.
In terms of the Foundation – things are moving smoothly there as well. We have a number of staff who have some or all of their salary paid by the Foundation. Part of the transition was to “staff up” the foundation to cover the key bases. The foundation staff are:
Wende Morgaine – PSU – Conference Coordinator – This used to be a rotating position but for the time being – Wende is it. She is doing an amazing job on a very complex and tiring task.
Peter Knoop – UM – Project Coordinator – Peter has developed and maintains our project tracking site which keeps track of *everything* going on in the Sakai community. Peter also works closely with the requirements group and in general is the “grease” that makes the little things work out.
Megan May – Indiana – QA Director – Megan is doing a wonderful job on a complex and challenging task. Being in QA is like playing goalie. Even if someone else made the mistake that led to the goal, every goal is always your fault :). In the past six months, Megan has transitioned us from a project-style QA to a QA where the servers are distributed around the world as well as the QA teams.
Anthony Whyte – UM – Technical Support – Anthony works with sites in production, deals with little fires at member sites, and all around fills gaps.
Lance Speelmon and Glenn Golden are not paid by the foundation at all, but they are both spending a great deal of time on Foundation work thankfully contributed by IU and UM.
This is an excellent core staff and very nice to have a pretty stable situation only six months after the start of transition. During and after the transition, we have not lost any inertia in terms of releases, etc.
Looking forward, I see a number of challenges that we should keep in mind:
Samigo is the largest single element in Sakai and one of the most important aspects of Sakai. Stanford has been very committed to Samigo development and has always been there when any Sakai site has needed help in Samigo – major kudos to Lydia, Marc, and Daisy. However we need to not simply keep Samigo as “Stanford’s responsibility”. Since it is so important to Sakai, the whole community needs to pitch in and make sure Samigo is solid. The first thing folks can do is really and truly participate in the QA of Samigo in 2.2 – finding bugs before the release is far better than finding them afterwards. We need to QA Samigo heavily on Oracle and MySql as well as QA the conversion scripts including the conversion of production data. In addition to QA, we need people to become familiar with the Samigo code so they can help in finding and diagnosing problems. There is no rule that says that only Stanford staff can work on Samigo – they have repeatedly said they welcomed help from anyone.
I have felt that our import and export has been a weakness for a long time. One of the most annoying things is that not all tools participate in the import and export. This has historically not been the fault of the tools as much as the complexity of working with the import and export mechanisms within Sakai. This has been cleaned up somewhat in Sakai 2.2 so as we move toward Sakai 2.3, we will need to start connecting all of the tools to the improved import/export. At the same time, we need to make our import and export be IMS Content Packaging Compliant – and while we are at it, we need to support the IMS Common Cartridge. There is a scheduled demo at alt-i-lab in June where Sakai is supposed to import a IMS Common Cartridge. Last year Sakai was part of the IMS Tool Interoperability demo (http://www.dr-chuck.com/media.php?id=49) based on some great work by Anthony and Lydia. So far we have zero resources who have worked on preparing for this year’s demo. I just don’t want to be in a situation where Angel, WebCT, and Blackboard all implement the Common Cartridge standard which Sakai helped caused to get started and Sakai cannot implement it for the demo. (Sorry – I was ranting there…)
Another really weak area for Sakai is the Site Setup – nothing against what we currently have – it is cool and gloriously simple. But it is way too inflexible to be seen as a neat tool by innovative teachers or instructional designers. Most LMS systems allow far more flexibility in placing and ordering tools and content when constructing a course. Over and over as I go see folks and watch them demonstrate what they like about their current commercial or open source LMS – it is the site construction that Sakai simply cannot compete with. I think that the solution for us is Melete – Melete already allows the nice construction of ordered elements, with dates, pre-requisites, etc. But it is very limited and lives only inside of a single tool window. Melete cannot place a Samigo quiz or a Message Center thread or a wiki page. Vivie of course knows all this and is increasing her team size by adding folks from the community to work on building the next generation of Melete – at the same time, for Melete to cleanly place other Sakai elements, it will require significant improvement to the Sakai Entity bus. So we need to think about this for the long term – figure 6-9 months at least before we start to see the combination of new architecture capabilities and the new Melete which can use them.
Another long-term issue is Resources and ContentHosting. There are literally a zillion things that people want to do here (see requirements) – all of the ideas a re very good and we need them all right away. But we have the “air traffic control problem”. We cannot just let folks go in and chop away at ContentHosting and expect to keep the code nice and solid. We need to form a community of folks who can work with this code for the long-term – we need folks who will be there long after something works – when it turns out that Feature X that they added breaks performance and we need a critical bug fix in a few days. We cannot afford to have this code backslide into a situation where it does not perform well or becomes buggy due to rampant new features dropped in willy-nilly. It looks like a cross-institutional group is indeed forming around this area – this is a great thing. Patience and commitment is what will bring us outstanding results in the next 6-12 months in this critical area.
We need to understand and take the next steps around the “Requirements” process. We have a long list of what folks “want” – with some form of priority on the “wants”. But we need to be very clear that there are no “magic fairies” that read the requirements list while we are all sleeping and just put that stuff into the next release. The word “requirements” mistakenly might seem to imply that the magic fairies *must* do these as their top priority. Unfortunately we have met the magic fairies and they are “us”. We are already seeing benefit from the requirements process in that as groups get together and attack problems they start with a nice common vocabulary REQ-375 – but lets not kid ourselves by assuming that just having something on the requirements list will cause it to get done. I think that this will all work out as long as people don’t assume that once they put in a requirement – all they have to do is sit back and wait for someone else to do it. If we are patient, the process will work itself out and we will learn the “right way” to do this. The resources folks (above) are really doing a great job in experimenting with how to tackle requirements – we need to watch their patterns and learn what works well.
Probably the greatest disappointment for me is Sakai’s non-delivery on a nice clean presentation framework. I hoped JSF would solve this. JSF has been nice in some areas, but a heavy burden in other areas. With the loss of Jon Andersen to grad school almost a year ago, we have not had anyone who has taking it on themselves to improve the framework bits around JSF. One of the Sakai goals was originally to have things like a date-picker or WSYWIG editor appear only one place deep in some framework and then use an abstraction to get at that functionality. If you look at Stephen Marquad’s recent set of Calendar format JIRAs or look at how many places must be patches to change WSYWIG editor, or add a resource picker to the WSYWUG editor, clearly to date, we have failed (or perhaps it should be, “Chuck failed when he chose JSF in February 2004”). It is bad enough that we have too many different presentation approaches – with tools coming from a number of past histories, this is to be expected – but what is doubly galling for me is that right now we hardly even have a “best practice” for newly developed tools to tell folks to use. I tell folks to use JSF but kind of apologize in advance for our poor state of affairs in JSF. Now that Ray Davis has volunteered here we at least have a leader for the area who might help guide some investment in improving JSF to the point where it is worthy to be our “best practice” – but this will take some investment, energy, and decisions. I hope that we can find a way out of the presentation wilderness. RSF certainly is interesting and some folks have given up on JSF and switched to RSF. Perhaps RSF will be the answer – I generally worry about Sakai using a presentation technology that no one else uses as its best practice. To me RSF needs uptake outside of Sakai to be our best practice. For now, RSF is just one of the many things that people *can* use in Sakai – not the Sakai best practice.
Probably my biggest strategic concern is where we will find “fungable” resources. These are people in the community who are highly talented and very skilled at all things Sakai who are available to work on little things whenever they come up and make some progress on those things. Right now we are a community of developers who all are very overloaded and running as fast as they can, and often working on local issues at the same time as participating in Sakai. These folks are stretched to the limit and generally not available to take on a project of a month or two that is important and requires deep skills to accomplish. The lack of these type of resources keeps us working on short-term things and keeps sliding the long-term things forward. The problem is that unless we take care of the “short term”, the “long term” is irrelevant. So we cannot be faulted for our short-term focus – it is just something that we hopefully will catch up with someday.
So enough of the doom and gloom on the strategic front. This stuff will all work itself out in time – I just write it down to get it into the back of folks minds going forward. I will probably try to write up a document that sumarizes what I see as Sakai’s strategic issues and send it around in the next few weeks.
Sakai 2.2 looks like it is shaping up to be another very cool release which moves us forward in exactly the directions we need to go to move Sakai to the point where we are ready for an installed base of very large production sites of 200 and beyond and a developer base in the hundreds.