Ross Harms who is formerly of Flickr and currently at Etsy, published a memo he sent around Yahoo! in 2009 explaining the relationship between developers and operations at Flickr:

http://www.kitchensoap.com/2012/01/05/convincing-management-that-cooperation-and-collaboration-was-worth-it

Here is a quote from the post:

I did this in the hope that other Yahoo properties could learn from that team’s process and culture, which we worked really hard at building and keeping. The idea that Development and Operations could: (1) Share responsibility/accountability for availability and performance, (2) Have an equal seat at the table when it came to application and infrastructure design, architecture, and emergency response, (3) Build and maintain a deferential culture to each other when it came to domain expertise, and (4) Cultivate equanimity when it came to emergency response and post-mortem meetings.

My Comment To the Post

Very nice post and all quite obvious to folks with enough experience across multiple real-world situations. Usually when organizations don’t structure their ops / dev relationships as you describe, it is often in an obsessive attempt to “eliminate risk”.

The basic (incorrect) premise is that everything the developers do increases risk and that ops have the job of reducing that risk to zero. Developers are the “problem” and Ops is the “solution”. Or as you say above, Developers are the “Arsonists” and Ops are the “Firefighters”. Casting the relationship in this way leads to ops wanting to limit change and the devs naturally want the product to move forward so the organization can better serve its stakeholders.

Uninformed ops feel the need to do large tests with complete instances of the product and frozen “new versions” and as the product gets more complex, these test phases take longer and longer and so more and more features end up in each release.

Again, ops is trying to eliminate risk – but in reality because each release is larger and larger there is a super-linear likelihood that something will go wrong. And when there are a lot of features in a package upgrade, folks cannot focus on the changes because there are too many – they hope it is all OK or sometimes it is all declared “bad” as a package without looking for the tiny mistake and everyone goes back to the drawing board which further delays the release of functionality and insures that the next release attempt will be even larger and even more likely to fail. It is a vicious circle that your approach nicely avoids.

The gradual approach you describe allows everyone to focus intently on one or a few changes at a time and do it often enough that you avoid the risk of a large change consisting of lots of details.

I like to think of the way you describe as “amortizing risk” – where there is always a small amount of risk that everyone understands but you avoid the buildup of accumulated risk inherent in large package upgrades. Again, thanks for the nice description.

I wrote a summary report for the SI curriculum committee describing of how I saw the various technology courses in SI fitting together. I figured I would share the report more broadly to help students decide which courses to take.

In the future SI572 is likely to be renumbered SI672 – but for this post, I use the old numbering scheme.

Nothing here is official – it is just my own opinions.

Course Summaries

These are the courses we have that are technical:

SI502 – Networked Computing: Storage, Communication, and Processing

SI502 is a very introductory course that is required for all students unless they pass the SI502 place out test. It is very much a survey course covering Python, Networks, HTML, CSS, Database, Search, and Security. The place out test insures that students with even a moderate amount of prior technical expertise bypass SI502. At this point about 1/5 of our incoming MSI’s bypass SI502. SI502 is taught at a very moderate pace and students have small discussion sections to insure everyone gets the attention they need to master the course material. SI502 is taught to about 140 students each Fall. There is a book “Python for Informatics” – www.py4inf.com written specifically for SI502.

SI539 – Design of Complex Websites

SI539 uses Python, HTML, CSS, Javascript, and JQuery to build a simple web application and deploy it on Google App Engine. There are no discussion sections. There are typically about 80 students in SI539 each semester. There are no discussion sections, but a GSI provides extensive office hours and assistance to allow struggling students to get help with the technical assignments in the course. About 60% of the students work pretty much on their own throughout SI539. There is a book written specifically for SI539 published by O’Reilly titled, “Using Google App Engine”.

SI543 – Programming I

SI543 is a traditional “first programming class” in the Java language. While it starts “at the beginning”, it moves quickly through quite a bit of material with the students building desktop-style applications including simple graphics and real-time (i.e. game-like) interactions with their applications.

SI572 – Database Application Design

SI572 covers PHP and SQL and prepares students to write real-world web applications in one of the more popular and widespread programming environments. The course features a significant group project where a real, significant web application is developed starting from the requirements and going through building and deploying a working application with a polished user interface. SI572 expects students can program independently, but does not specifically demand prior PHP experience. SI572 uses a book titled, “Learning PHP, MySql, and JavaScript”. We teach SI572 to about 50 students each semester.

How these Four Courses Fit Together

SI502 and SI539 are designed as courses for people with little or no programming background. SI502 in particular is designed for students who are “nervous” about computing and technology. The pace of both courses is sufficiently slow to insure that we don’t lose students. After SI502, students should have a good understanding of programming and technology but not be independent programmers. By the end of SI539 they should be well on the way toward being an independent programmer. Both SI543 and SI572 expect that incoming students function at about the level that students at the end of SI539. It would not hurt to add a very suggestive loose pre-requisite to SI543 and SI572 that says “Suggest an introductory programming course like SI539 or equivalent”.

Sample Sequences

In this section, I show several paths taken through these courses by different types of students.

Just Starting out In Technology

These students come in dreading SI502 and we work very hard in SI502 to get them excited about technology enough to want to take SI539 at least. These students are the target audience of SI502 and we make that very clear. Their sequence is either:

SI502
SI502 -> SI539 (if they get a little confidence)

Solid Programing But No Web Experience

Some of our incoming students have a very narrow Computer Science background where they were not exposed to the web at all but are solid “pure-programmers”. They tend to place out of the SI502 course and start in SI539

SI539 -> SI572

Self-Taught Web Programmers on the Web

Some of our incoming students have little formal training in programming or their formal training was a long time ago, but they have been playing with web technologies all along but their knowledge is spotty and they want better coverage in terms of their skills. They also need to see how programming is done on desktops. There is usually no point in them taking SI539 – it is too easy and too slow to keep their interest.

SI572 -> SI543 (EIther order works actually)

Interest In Data Analysis

Sometimes students are uninterested in building web applications and just want to do data analysis.

SI502 -> SI601 -> SI618

Traditionally SI601 (Data Manipulation) and SI618 (Exploratory Data Analysis) have been perl courses because that is how the teachers in those courses had developed their assignments. But there is a possibility with someone new teaching these courses they might switch to Python. That would make them articulate nicely with SI502 – particularly given SI502 is very data-oriented and includes a module on regular expressions.

Skilled Web Master

These students start with SI572 and move through this set of courses:

SI520 – Graphic Design
SI572 – Database Applications
SI634 – Application Platform Configuration (Drupal)
SI635 – Application Platform Customization (Drupal)
SI658 – Information Architecture

This is a great set of courses that exposes the students to a wide range of applicable technologies and techniques.

Gaps and Opportunities

Given the demands and interests that I am hearing from the students in these classes, I get a sense there is a demand for these topics – we may only need 7 weeks on each topic:

– A course in mobile computing (Java/Android and/or Objective-C/Apple) is taught.

– A course where advanced JavaScript (i.e. JQuery is taught)

– A course where we cover advanced data manipulation – would help SI508, SI601, SI618 and similar courses get started in their topics with less remedial instruction in each course

– A course that combines Graphics and CSS to make sophisticated, gorgeous and functional web sites

Conclusion

Over the past four years, we have gotten to a pretty good understanding of our four core programming courses and where they fit in our curriculum to serve our entire range of MSI students that range from nervous beginners to students with computer science undergraduate degrees and work experience in web programming. Probably the biggest key fact is to understand the SI502 and SI539 are very different courses than SI572 and SI543. SI502 and SI539 are broad courses designed to gently build confidence and competency whereas SI572 and SI543 are real, solid programming courses that produce graduates with real programming skills relevant to the marketplace.

At the University of Michigan School of Information we have this mailing list we call “si.all.open” that is prone to interesting long threads about everything from traffic tickets to lost laptop power supplies to where the GIF has moved to since the first bar got too full.

Recently there was a thread titled ” [Ann Arbor Mexican Food] Amazing Discovery” where it started out with one person talking about a restaurant they particularly liked. This triggered many follow on comments that made me very hungry.

I decided to turn the thread into a personal gastronomic TODO list. It may take a while before I get to all these places – but now I have a plan. Just Google these strings – you will find them quickly enough.

Taqueria La Fiesta, in Ypsilanti, MI

Taqueria La Fiesta at 4060 Packard, Ann Arbor, MI

Evie’s Tamales, Detroit, MI

Taqueria Lupita’s, Detroit, MI

Nuevo Leon, Detroit, MI

El Zocalo, Detroit, MI

TMAZ Taqueria, 3182 Packard Rd, Ann Arbor, MI 48108

Xochimilco, Detroit MI

I was going through my blog doing some cleanup and noticed that I had about 12 posts that never got published. So I am going through and publishing them. So you will see some out-of-date-out-oforder posts for a week or so.

Sorry.

This Friday I am giving a lecture as part of the

Using game theory, we can formally analyze how people react to choices—specifically choices in which significant payoffs are at stake. We will describe game theory, look at classic games (the Prisoner’s Dilemma, the Hawk-Dove Game), and learn about finding the Nash Equilibrium for various games.

Luke Fernandez of Weber State posted a cool review of my Sakai book:

http://itintheuniversity.blogspot.com/

Here is a quote:

As the Soviet’s used to say (and as historians often still profess), “the future may be certain but the past is always contested territory.” Which is another way of saying that if Chuck has offered up an intriguing story, I hope it doesn’t end up being the authoritative history of Sakai. The sub-title, after all, is a “retrospective diary” rather than a history, which would suggest that many other stories are worth telling.

Here is my reaction:

Luke thanks for such an insightful review of my book. You hit so many of the themes of the book perfectly. I would amplify that I absolutely do not intend for this to be the definitive history of the Sakai project from 2003-2007. Others have completely valid perspectives and I wish others would write their own (perhaps contradictory) views of the events and I would love to be able to let folks assemble he “real” history from all those perspectives. You are also right in that my primary motivation is not simply to “stick it to the man” – the thing I fight for is for the creative types and management types to function as peers rather than the typical structure where management is “above” the creative types. I am fighting for the freedom of the creative types to take part in the decision making.

During my talk at the Blackboard DevCon, I was explaining why the development of standards seems quite dull – but is essential. I likened it to building a sewer system in a new subdivision long before any homes are built and long before any people live in the subdivision. And once the people live there, all the fine workmanship in building the sewer is deeply underground and never seen again (hopefully).

Here was my quote:

“If the toilet does not flush, no one will live in that house. That is why I am so excited about plumbing.”

Here are the slides for the talk:

http://slidesha.re/qqYy0P

Here is a video of the Demo/HACK of Blackboard as a Basic LTI Provider:

http://www.vimeo.com/26310497

John Fontaine did most of the work of the demo by hacking up a Basic LTI Provider Building block roughly modeled on the ProviderServlet in Sakai that makes Sakai a Provider. I contributed by checking code into his building block that broke it and triggering John to fix it.

Here is that code on www.oscelot.org :

http://projects.oscelot.org/gf/project/oscelotblti/

I must repeat that this is *not* a product direction – it was just a fun five-hour hackathon result.

It is like my Christmas present came early this year!

After a successful IMS quarterly meeting in Commerce, Texas in the second week of November where we demonstrated prototype interoperable implementations of LTI v1.1 with Moodle 2.2, Sakai 2.9 and SPV Software’s Building Block and PowerLink, the spec passed the threshold test for going to Public Draft.

The primary new feature for LTI v1.1 is the ability to send grades back from an external tool to an LMS.

In the IMS process, when a specification reaches Public Draft it is very mature and nearly complete and folks should review it quickly to see if there are any issues because it is very likely to be final in the next 30-60 days.

Documentation: You may download the public draft at: http://www.imsglobal.org/lti/index.html

Public Discussion Forum: http://www.imsglobal.org/community/forum/categories.cfm?catid=44

Open Source Implementation Code: http://code.google.com/p/ims-dev/

IMS Members–Only Benefits

Testing: Only Members may review and begin to practice for the LTI v1.1 Conformance Certification at: http://www.imsglobal.org/developers/alliance/LTI/cert-v1p1/

Feedback and Support: Only Members receive personalized support. Please post any comments/questions in the CC/LTI Alliance at: http://www.imsglobal.org/community/forum/categories.cfm?catid=138&entercat=y

Note: I am compensated for my work in IMS as a consultant and work extensively on IMS LTI.

Joshua Kim of Dartmouth University had a provocative post in Inside Higher Education that has provoked a lot of great comments:

http://www.insidehighered.com/blogs/technology-and-learning/please-push-back-my-d2l-sakai-biases

I’m hoping that the Desire2Learn (D2L) and Sakai folks among our IHE community will tell me that I don’t know what I’m talking about. When it comes to the LMS (or really anything in life), where you stand is where you sit. I’ve been on Blackboard for 10 years now (across 2 different institutions and in roles ranging from instructor to learning designer to program manager), so naturally I’m biased toward what I know best. I tend to see the LMS race as falling out between 3 competitors (and 3 models): Blackboard, Moodle and Instructure.

There are lots of great comments and I encourage you to read them all.

Here is my comment:

Fun discussion and fun comments.

I would say that Sakai is different and distinct from all the other products mentioned in terms of the amount of choice you are offered.   With Sakai you can choose self-hosted or cloud hosted.   With Sakai you can be involved with the community anywhere from becoming a core contributor (takes some effort and commitment) to silently downloading and using the product or even purchasing a branded version that barely calls itself “Sakai”. 

Sakai is by far the most customizable LMS in the marketplace because the primary developers are from higher education institutions that need unique solutions based on how different schools, different countries, and different cultures teach.    Sakai can be highly customized and still upgraded.  If you looked at the University of Michigan Sakai and then looked at the Oxford University Sakai, you might be amazed that under the highly customized navigation and authorization these two LMS’s share 99.5% of their code.  We are *not* just talking skins, images and color here – we are talking about different functionality and philosophy matching the pedagogy of each school.  This flexibility overwhelms small schools that want to have one way to do things and that is OK – those schools should buy a cloud hosted version of something generic and leave the upgrades and choice of functionality, look and feel to some cloud-hosted product manager somewhere.

But perhaps the most important difference is that the core people moving Sakai forward are not motivated by money or profits.  There are over 20 commercial vendors in the Sakai ecosystem that do want to make money.   But the core leadership of Sakai and its product direction and contribution is overwhelmingly led by Higher Education.  There are great contributions from some of our commercial partners, but when they participate in core activities, they participate as individuals and not members of their companies.  There are meetings where decisions get made in Sakai by a core set of great people – but the people making the decisions are mostly the *customers/adopters of the product*.  There is no separation between those who make the product and those who use the product – they are the same people.   No other LMS in the marketplace is as open in its governance than Sakai.

One of the advantages of Sakai not being venture capital funded or dominated by commercial concerns is that if the marketplace changes dramatically, Sakai will still be there for you even if you never invested a minute of your time or a penny of your money.  Venture Capitalists can pull the plug if a company does not progress to profitability rapidly enough.  A company can have a giant VC-paid-for Educause booth one year and then sold for pennies on the dollar a year later.   A product that is making money and paying its development and sales team is vulnerable to market shifts.  If OpenClass is successful or Instructure gets significant market share, that market share comes from someone else.   If there is too much of a market shift and revenues drop, then companies need to cut their development teams and then the product kind of deflates.  I won’t mention any names, but there is one product in the marketplace that seems likely to me to have a lower market share this time next year because of some combination of Instructure and OpenClass taking business away.   If the market share drop and revenue drop is significant, that might be a little scary to a product that has enjoyed a number of years of very solid and increasing revenue.

If of course you were at a Sakai school, you could sit back and smile, knowing that you controlled your own destiny and that while the world of Sakai is not rolling in cash to do tons of stuff for you, we have passion, heart, dedication, and you are working arm-in-arm with people from some of the most brilliant schools in the world in the form of Oxford, Cambridge, Capetown, ANU, Rutgers, Michigan, Indiana, Stanford, Duke, UNC, Berkeley, Nagoya, Georgia-Tech, and many others large and small.  And our commercial ecosystem is rich and competitive and does not rely on sweetheart deals, proprietary IP, trademark licensing, or revenue sharing to keep somebody’s business model propped up.

As I try to point out in my book, Sakai is about freedom, not low price.   And in particular, whether you or Dartmouth contributes or not, we are as committed to *your* freedom as much as we are committed to our own freedom.

So sit back and relax in your cloud-hosted, vanilla-flavoured, revenue driven, product-manager controlled LMS and smile knowing that you have traded freedom for having no responsibility.   Like in the Matrix, the real world is a little more raw.  When/If your organization decides to take the the steering wheel of the LMS and decides to be a driver instead of a rider, Sakai will be here waiting with open arms.   We miss you, where have you been?  :)

IMS Learning Tools Interoperability (LTI) 1.1 has moved from the working group to “Internal Draft” / “Member Draft”. This means that we consider the specification development 100% done and locked down.

The scope of IMS LTI 1.1 is the addition of grade return from a tool to an LMS using web services. It is simple and easy to implement and we expect pretty rapid market uptake.

IMS LTI is already supported in the trunk of Sakai 2.9 in the BasicLTI portlet, tracked under

https://jira.sakaiproject.org/browse/BLTI-123

Designating an IMS spec as “Member Draft” triggers the implementation activity of our members so we can make sure that there are no little “gotchas” in the spec. Having developers actually implement the spec is a great way to cause them to read it *very carefully*.

If you are not an IMS member, there will soon be a public draft of the specification once we have developed the certification and seen demonstrations of two interoperable products. The standard is pretty simple and mature so I hope that the public draft is ready in two months or less.

You can wait until public draft to see the spec or join IMS to be part of the member draft evaluation. The cost is not all that great to on the Alliance and it can be done online.

http://www.imsglobal.org/joinims.html

If you already are an IMS member – the discussion of the LTI 1.1 Member Draft is here:

http://www.imsglobal.org/community/forum/categories.cfm?catid=138&entercat=y

Here is an Educause presentation about LTI from this past week:

http://www.slideshare.net/csev/educause-ims-update-learning-tools-interoperability

Let me know if you have any questions.

This is a great time to publicly thank the co-Chairs of the IMS LTI Working Group for their hard work and dedication: Greg McFall of Pearson Education and Lance Neumann of Blackboard. They have put an amazing amount of work into this specification and we all benefit from their knowledge and skill. It has been a great personal learning experience and I consider it a privilege to have worked with them on the preparation of this specification.

Note: I am a member of the IMS staff part-time – but my compensation is not linked to changes in IMS memberships.