A Recent Tsugi Talk I gave at Oxford University

This is a recording of a talk I gave at Oxford University about Tsugi, Sakai, Canvas, and IMS Content Item back in June:

Doing a certbot renew when a site is behind CloudFlare

I have lots of my web sites behind CloudFlare – which is nice because I get free auto-updated SSL certs and all the other benefits of CloudFlare.

But in case I want to bypass CloudFlare, I like to keep a solid SSL cert on the original server. So I logged in and ran a:

sudo certbot renew --dry-run

And got this error message:

Attempting to renew cert from /etc/letsencrypt/renewal/www.tsugi.org.conf produced an unexpected error: Failed authorization procedure. www.tsugi.org (tls-sni-01): urn:acme:error:tls :: The server experienced a TLS error during domain verification :: remote error: tls: handshake failure. Skipping.

The solution was to log in to CloudFlare and reconfigure my tsugi.org to temporarily bypass the proxy and then re-run:

sudo certbot renew --dry-run

Then the renewal worked just fine and afterwards – I restored Cloudflare as my proxy.

Of course you might need a few minutes whilst the DNS changes propagate.

New Coursera Specialization – Web Applications For Everybody

Now that we are well into the rollout of the Python 3.0 version of Python for Everybody. It is a good time to let you know about our next big thing beyond PY4E.

We are preparing a new specialization called “Web Applications for Everybody” (WA4E) that covers HTML, CSS, PHP, MySQL, JavaScript and JQuery to develop web applications. WA4E assumes that you have a basic understanding of programming at least through Chapter 10 of Python for Everybody (Course #1 and #2 in the Specialization).

We teach both back end and front-end Web Application development. There are extensive auto-graders just like in PY4E and all of the course material is freely reusable under Creative Commons so you can use the material to teach your own classes if you like. There is a preview web site at www.wa4e.com where you can get a sense of what will be contained in the specialization.

We expect that the specialization will be available for registration in a few weeks and the first session should be starting by October 1 (subject to change).

If you have completed Python For Everybody and want to prepare for the new Web Applications for Everybody (WA4E) specialization, I highly recommend that you take Colleen van Lent’s Web Design for Everybody (WD4E).

https://www.coursera.org/specializations/web-design

Colleen and I have been planning a coordinated track of three specializations that can take you from knowing no programming at all through being ready to work as an entry level web developer. It has taken us two years to cover this this multi-specialization curriculum for beginning developers:

Internet History, Technology, and Security (IHTS)
Python for Everybody (PY4E)
Web Design for Everybody (WD4E)
– Web Applications for Everybody (WA4E)

Colleen and I are both personally committed to making technology education available to anyone who wants to put in the time and effort.

I will do a quick review/summary of HTML and CSS at the beginning of the WA4E specialization but I highly recommend taking Colleen’s WD4E specialization since she covers essential front-end skills like UI layout, responsive design and accessibility. If you want to valuable as a professional web developer, you should be able to build solid front-end (Colleen’s WD4E) code and solid back-end (WA4E) code.

I need to thank all of the people who have helped make this new specialization possible. Building the new specialization has been very much a team effort including the staff at the University of Michigan, the current mentors in Python for Everybody, and the staff at Coursera. It is an honor to work with such a talented and dedicated team.

Stay tuned for further announcements and I hope to see you online.

Sakai-12 ContentItem in the Rich Text Editor

The Sakai ContentItem Support in the text editor is now in master and on nightly:

https://trunk-mysql.nightly.sakaiproject.org/portal/

It features a very pretty integration between Sakai and Tsugi using Content Item.

Here is a video demo:

Here is the JIRA:

https://jira.sakaiproject.org/browse/SAK-32656

Here is the test plan:

https://github.com/sakaiproject/sakai/blob/master/basiclti/basiclti-docs/resources/docs/ContentItem_TestPlan.xls

And if you have five hours you can watch the most boring video in the world – I live streamed much of the work last week to make this work in Sakai:

It is super boring. It is an experiment I am playing with in live coding. I rarely narrate because I am concentrating but you can see what I do. Sometimes I swear – be warned.

Experiment: Live Streaming Coding – SakaiJuly 2017

I am going to do a weird experiment in July. Sakai-12 is code freezing at the end of the month and I am going to put some work into adding some pretty cool features into Sakai-12 in terms of standards compliance. The first bits will be around ContentItem and if I have time I will work on the Outcomes and Membership services.

But one thing I am going to do is live stream my coding efforts. I will announce the streams on Twitter @drchuck.

I also have a PlayList where the recorded streams will be stored. I have a few there already.



The Best Meeting Ever (Sakai 12 Planning)

IMG_6792In the Sakai community we regularly have open meetings of the sakai governance group – the Sakai Project Management Committee (PMC) to plan upcoming releases and strategize about how to best meet user requirements with Sakai. Our most recent open Sakai PMC meeting was June 8, 2017, the day after the end of the Open Apereo 2017 Conference in Philadelphia, PA.

As the current Chair of the Sakai PMC, it was one of the best meetings I have ever participated in my life. I was amazed at how our community has grown and matured since we became part of Apereo in 2012.

While nearly all of the PMC members were present and participating, this was a meeting of the community – not the PMC. Some of the people in the room have over a decade of expertise in Sakai software development, others in the room were customers of Sakai commercial affiliates, others were technical support specialists, we had faculty members, and a number in the room were first-time participants in the meeting.

Neal Caidin, the Sakai Community Coordinator ran the meeting and kept us moving through our topics but made sure to have a very inclusive discussion all along. We had a lot of big issues to cover like:

– When would we code freeze the Sakai 12 release?
– What would be the scope of the Sakai 12 release?
– Where did the NoodleTools contributions fit in the Sakai 12 release?
– Could we remove the legacy Grade Book UI from Sakai 12?
– What were the community hot buttons in terms of “things to fix” before Sakai 12?
– What were the major “holes” that could be filled by Sakai 12?

We had the advantage that we were coming off a great Open Apereo conference and there had been lots of discussion about Sakai in presentations, in BOFs, at Karaoke, and in hallways – so in many ways the post-conference meeting was bringing a lot of threads together to form a plan of action.

We had about four hours to talk so we did not need to rush and could allow lots of people the time they needed to talk. Virtually everyone in the room talked on one or more topics regardless of seniority, role, or experience. The most senior developers spent a lot of time listening and reflecting and helping to add a sense of resource costs / availability to each of the ideas as they moved throughout the room.

Results

Here is some notes from the meeting:

https://confluence.sakaiproject.org/display/REL/Sakai+12+Straw+person

Among many small decisions and ideas, we decided to freeze Sakai 12 at the end of July, and that the old grade book UI would be removed, and that we would try very hard to get the Noodle Partners work into core by Sakai 12 and others.

But what impressed me is how much fun we were having together while discussing these very significant and consequential product decisions that might tie a company product manager in knots. Neal would bring up a topic, the conversation take its time and wander around the room gaining lots of diverse perspectives and wind its way to a decision and then we would move on the the next topic and the pattern would repeat.

All very relaxed and enjoyable – and at the end we had an impressive roadmap for Sakai 12 that was responsive to the community needs and practical in terms of timing and resources. In a way it was the “iron triangle” done properly – together we collectively balanced, time, resources, and features. No one got everything they wanted but we were able to arrive at a balanced plan for the next year.

An Open Invitation

What bothers me as we move into the LMS Vendor Conference season, is how little mainstream commercial vendors actually talk to their users and give the members of the community of adopters and users a seat at the table where final and ultimate decisions are made in regards to the product they are paying 100’s of thousands of dollars for. This is one of the aspects of “Sakai is 100% Open” that makes me most proud.

The commercial LMS vendors are great at throwing parties with an open bar and musical entertainment that runs long into the night. What I would hope customers of these vendors try to do a little bit is see through the parties that are distracting you from the fact that you and your school has very little involvement in the direction of your most important software product on their campuses.

What I would hope is that sometime, that technical support people from a few of these Canvas or Blackboard customers would come to a Sakai meeting and see how it can be done differently and in the open. Just come and sit in the back and listen – or participate if you like. You would be welcome even if you are a Canvas or Blackboard customer. You could come to our Summer Open Apereo meeting or even our Sakai Camp. Sakai Camp is in January and is two days dedicated to the kind of collective meeting to plan for Sakai. You need to register early for Sakai Camp because we have a size limitation. Sakai Camp is more fun because it is 3 days of nothing but Sakai planning – just like the meeting above.

If you come to Sakai Camp you will sit next to and talk the leading lights in the Sakai Community, from our QA leadership, support experts, pedagogy experts, UI/UX experts, end users, and core developers. There is no “hierarchy”, there is no “pecking order”, everyone is encouraged to participate, talk, and share your opinions.

What you would learn from those few days is how a community that cares deeply about each other can work together to produce a world-class learning management system with the best strategy and path towards a truly next-generation learning environment.

Tsugi DB Upgrade Required Upgrade in Master (start of 0.4.x)

Attention Tsugi Production folks.

If you have been following along and doing database upgrades when I told you – my Tsugi commits this morning:

https://github.com/tsugiproject/tsugi-php/commit/cd3bfe8431a6035e4be702714ca281c36f75e832

https://github.com/tsugiproject/tsugi-php/commit/e725fe86a4a41ec85b623cb0bb5c0547daef1ba9

Will not even be noticed. These commits are the first that *depend* on the recent database upgrades being in place.

If you have been ignoring my messages and upgrade from a master a month ago to current master with a git pull, your LTI launches will break because there is now SQL in LTIX in master that depends on:

lti_user.image
lti_context.ext_memberships_id
lti_context.ext_memberships_url
lti_context.lineitems_url
lti_context.memberships_url

As well as “deleted” columns in all of the core tables.

If you upgrade and “brick” your Tsugi (which I highly doubt will happen), there are two ways to unbrick a Tsugi:

(1) Go straight to the admin UI

https://www.py4e.com/tsugi/admin/

This code specifically tries to limit how much of Tsugi is working (i.e. there are no LTI launches involved in this page) for the admin interface to work. So go to admin, enter the password and do the database upgrade and things should pop back to normal. This is the preferred unbrick trick and works almost all the time.

(2) Log in and drop back to a tag if you are checking out my Tsugi – I made a tag/release with the code that adds all the needed fields, but does not depend on those fields. It can work before or after a database upgrade:

https://github.com/tsugiproject/tsugi/releases/tag/0.3.2

A simple

git checkout 0.3.2

Should do this trick. Launches should work even with a pretty old non-upgraded DB. Log in, upgrade the database, and then re-checkout master and all should be well. You might need a tiny DB upgrade after going back to master – but it does not affect the LTIX runtime.

All my servers py4e.com, wa4e.com, tsugi.com and my Coursera auto grader are all running master solidly as of an hour ago.

If you get stuck, let me know – I will be watching my email.

Sakai 11.4 released!

This message is from Neal Caidin, Sakai Community Coordinator.

Dear Community,

I’m pleased to announce on behalf of the worldwide community of participants that Sakai 11.4 is released and available for use [1] !

Sakai 11.4 has 144 improvements, mostly bug fixes, [2] including:

18 fixes to the interface (aka Morpheus)
15 fixes for the Lessons tool
14 fixes for Samigo Test and Quiz tool
8 fixes for Assignments tool
7 fixes for the Gradebook tool
6 fixes for Accessibility issues (a11y)

Improvements (features or enhancements) added in this maintenance release include:
Support for Melete Import (LSNBLDR-311)
New property to override the system default and reset tools during navigation (off by default) (SAK-31389)
Improved display option for subset menus (SAK-31824)
Email sent by Messages tool links directly to the specific message sent in Sakai, making it easier to respond to the message (SAK-32240)
Drag and Drop upload enhanced to allow overwriting of existing files (SAK-30924)
Searching on scores improved in Samigo Tests and Quizzes (SAM-3130)

Other areas improved include (in alphabetical order):

Admin
Announcements
Calendar
Citations Helper
Delegated Access
Email
External Tools (LTI)
Forums
Internationalization
Kernel
Membership
Messages
PA System
Podcasts
Portal
Profile
Resources (Content)
Roster
Signup
Site Archive
Site Info
Sites Admin
Syllabus
Wiki

Three security issues were fixed in 11.4 (details will be sent shortly to the Sakai Security Announcements list)

[1] Downloading information available at – http://source.sakaiproject.org/release/11.4/

[2]
2a. Improvements in 11.4

2b. or look directly in the Jira filter – https://jira.sakaiproject.org/issues/?filter=16015

2c. Github issues (for GradebookNG only)

The path to Tsugi 0.4.x (Roadmap for the next month or so)

I just wanted to let you know about the roadmap for the next month or so that will finish the 0.3.x release and move master to 0.4.0. There will be non-upwards compatible changes in 0.4.0 as the first step will be to do some cleanup after the Tsugi-Koseu split. At this point certain things can be used the old Tsugi way or the new Koseu way – once we go to 0.4.x we will remove the backwards compatibility stuff. More below.

Tasks to finish 0.3.x:

– Add code to the LTIX runtime that handles new data model items (url fields for memberships and outcomes, an user image field, and others). The data fields are mostly there already (please upgrade your database).

This will wrap up and make the file 0.3.x release. Some people will want to switch from master to the 0.3.x branch to avoid or delay the 0.4.x changes.

Major 0.4.x changes:

(1) Remove the vendor folder from

https://github.com/tsugiproject/tsugi

This means that to upgrade Tsugi, you will need to do both a

git pull
composer update

And when tsugi-php or koseu-php are upgraded – you will need to do a

composer update

I know folks think that having vendor in github violates the separation of church and state

https://github.com/tsugiproject/tsugi/issues/30

I think that after the switch, you will realize it was kind of nice to *only* need to do a git pull and leave the “composer updates” to me.

(2) Make it so that the Koseu tools that organically grew in Tsugi will only exist in Koseu. As an example in my www.py4e.com web site, the following URLs all work

https://www.py4e.com/lessons
https://www.py4e.com/tsugi/lessons
https://www.py4e.com/tsugi/lessons.php

Post 0.4.x, only the first (Koseu-style) url will work – the old code will be removed from /tsugi – so the best way to smooth the transition is to switch to the Koseu-style urls. See these files for examples of how to switch:

https://github.com/csev/py4e/blob/master/.htaccess
https://github.com/csev/py4e/blob/master/koseu.php
https://github.com/csev/py4e/blob/master/nav.php

Comments / questions welcome.

Developing a Tsugi Tool (New Videos)

I use the attendance tool as my “basic” Tsugi tool. I recorded five videos showing how this tool works and showing how you can write a tsugi tool several ways:

– A single PHP file with the controller and view
– A Silex+Twig application
– A JQuery + Handlebars application
– A single page application using JSON and tmpljs

https://www.tsugi.org/lessons/devintro

The code is in github:

https://github.com/tsugitools/attend

Each of the variations is in a different permanent branch – cool to be able to switch implementations with a

git checkout handlebars