Author Archives: Charles Severance

Sakai Development moving to Java 11

Since Sakai has moved to Java 11 for the Sakai 22 and later releases, we will need to update our developer environments.  I also made the following changes.

  • Use the org.mariadb database connector and Hibernate dialect (even if I am using MySQL 5.7).  The connector is free and automatically deployed by Sakai so there is no extra fuss downloading the com.mysql connector and hand-installing it in Tomcat.
  • Upgrade Maven to 3.8.1 or later
  • On my fresh-install Linux environments I am switching to MariaDB as my dev database server.  If I have an existing MySQL (i.e. MAMP on Mac or Aurora on AWS) I will just keep using it but switch to the MariaDB connector.

I have setup and tested four different environments in the post-JDK11 environment:

  • Matt Jones’ docker-build solution.  This is very cool – especially for Windows.  Matt mounts folders from your computer’s file system into his (build/database/sakai) docker containers so you can edit files outside of the docker containers and then build / run all inside docker containers with a zero footprint install on your computer other than docker itself.  Also your database persists even when you have no docker containers running.  The container and mounted files slows this down a bit – but if you have a super fast Windows or modern MacBook box with a lot of memory and SSD, it is plenty quick.
  • My MacBook running natively installed Java, Tomcat, and MAMP using my sakai-scripts code.  For Mac development it is faster than Matt’s docker solution especially for my circa-2013 MacBooks that I refuse to upgrade.
  • A simple self contained Ubuntu image running under docker using my sakai-scripts code.  I use this to make sure I can quickly test my scripts or Sakai’s build under ubuntu without spinning up an EC2 node or DigitalOcean Box.  This is different from Matt’s docker approach because everything (including the MariaDB server and data) is in the docker – it is nice for quick tests because it is a zero-install other than docker on your main computer.
  • An ubuntu server running on Digital Ocean / EC2 using my sakai-scripts code like https://dev1.sakaicloud.com

If you want to figure out things like shell settings or other configuration to adapt your own setup, you can take a look at some of the files in my sakai-scripts repo.  If you used these scripts before, there was a complex step of getting all the environment variables set in your profile (mac or linux).  Now each script sets its environment variable and does its thing – much better and easier to fix / upgrade when things change a little.

You can examine these scripts and extract bits to use in your environment.

  • Start with the README.md it contains the overall outline of manual and automated steps.  This shows how to setup (a) a MackBook, (b) ubuntu in a stand-alone docker, and (c) ubuntu on a stand-alone server.
  • Take a look at ubuntu.sh to see what prerequisites are needed on ubuntu.  Because I use sdman the installs are much simpler – thanks Matt and Earl for teaching us about this.  Using sdman also allows you to switch back to JDK-8 if you need to.  It is the logical equivalent of Python virtual environments.
  • Take a look at qmv.sh to see things like needed environment variables to make maven work.  Without these maven will run out of memory whilst building.
  • Take a look at start.sh to see the relatively complex set of environment variables needed to set up Java to run Sakai.  These variables are adapted from the Matt / Earle recommendations.  Matt designed these to be tweak able and there are some examples of things like setting up a debugger if you want that.

I hope you find this helpful.  I would also like to explore moving these scripts and Matt’s docker stuff into the core repository in a “dev” folder.  It is kind of silly to have these in hard-to-find locations in contrib and my repo.

 

 

 

Sakai 22 moving to Java 11

(This was originally from Earle Neitzel on the Sakai dev-list)

The time has finally come for Sakai to adopt and move to Java 11.

A big part of this decision is inline with the libraries that Sakai uses, once all the significant libraries have transitioned to running in a newer JDK Sakai itself becomes eligible for the same. Sakai software (like most software) is like lego’s and the blocks all need to have the same shape and size in order to work together. So once all the fundamental blocks (like spring, hibernate, elasticsearch, ignite, server side UI frameworks) support a newer JDK, then Sakai can look at adopting that JDK.

History
JDK 5 Dec 14 2006 (approx)
JDK 6 Feb 20 2012 SAK-20908
JDK 7 Oct 27 2014 SAK-28016
JDK 8 March 29, 2015 SAK-25903
JDK 11 Sept 24 2021 SAK-33595

In the past, it was customary to limit newer language syntax in order to preserve compatibility when merging changes to previous versions. I propose we do this for at least 1 Sakai version which would mean once the 23 branch is cut then we could allow newer syntax. I would also revisit this at that time to see if it should be extended to 2 Sakai versions. (Most IDE’s will allow you to set the syntax level to help prevent you from using newer syntax)

Jakarta package name
So in conjunction upgrading to jdk 11, jdk 9+ removed much of the javax namespace (aka JEE) it means that many of these packages are now separate libs which need to be included. On top of that Oracle has given the Java EE to Eclipse with the restriction that they cannot use the word “Java” hence the need to rename the packages to Jakarta. Given many of our libs are still using the javax package names we can use jakarta libs that keep the package name space (like activation and validation) but newer jakarta packages are changing the package names as well this last part is causing some issues as some of the libs are still on versions using the javax package name. I propose that until Sakai is on newer libs that support the newer package names that we stay on the javax namespace. This means that i reverted a recent contribution SAK-45950 by David Horrowitz to stay on the javax package name, I apologize but believe it’s the right thing to do at this time.

Building Sakai with JDK 11
The source can only be built using JDK 11 moving forward. So you will need to download and use a respective JDK. I propose that Sakai standardize on the OpenJDK project as all other projects use that project as a base and should ensure our compatibility with other JDK flavors.

This project https://sdkman.io/ makes that task simple.

Running Sakai with JDK 11
Here will list the options we will need to use to run Sakai in JDK 11.
In your setenv.(sh|bat) you will need to add the following lines to JAVA_OPTS or CATALINA_OPTS:
–add-exports=java.base/jdk.internal.misc=ALL-UNNAMED \
–add-exports=java.base/sun.nio.ch=ALL-UNNAMED \
–add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED \
–add-exports=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED \
–add-exports=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED \
–add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED \
–illegal-access=permit

Next you will want to select a Garbage Collector to use this may depend on the JDK that your using. See the following page for some info https://docs.oracle.com/en/java/javase/11/gctuning/available-collectors.html

 

New Amazon AMI – Tsugi – Ubuntu 20 / PHP 8.0 – ami-04dd1d5da2d25fe43

I have finished testing a new AMI for Tsugi + Ubuntu 20.04 and PHP 8.0:
tsugi-php-prod-2021-08-16-ubuntu20.04-php8.0 -  - ami-04dd1d5da2d25fe43
This is a public AMI and can be used according to the instructions at:
Ubuntu 16.04 is already out of support (they only stick around five years) – so we need to keep moving forward.  This is the first AMI that includes Node and NPM.
If you want to update your own build process – or review what is being installed – the bulk of the software that is being installed is here:
AMI versions are released about 1-2 times per year unless a problem is identified in an AMI.
Questions / comments / testing welcome.

Sakai 20.4 Released

This is a note from Wilma Hodges – the Sakai community coordinator

Dear Community,

Congratulations to our amazing worldwide community!

I’m pleased to announce that Sakai 20.4 is released and available for downloading [1]!

Sakai 20.4 has 88 improvements [2] including

  • 12 fixes in Assignments

  • 9 fixes in Gradebook

  • 12 fixes in Lessons

  • 26 fixes in Tests & Quizzes

Other areas improved include:

  • A11y

  • Announcements

  • Basic LTI

  • Calendar

  • CKeditor

  • Content Review

  • Dropbox

  • Feedback Tool

  • Forums

  • Internationalization

  • Messages

  • Message Bundle Manager

  • News

  • Profile

  • Resources

  • Rubrics

  • Site Info

  • Statistics

  • Syllabus

There was 1 security issue fixed in 20.4 (details will be sent to the Sakai Security Announcements list).

Please also note the upgrade information page [3] for important notes related to performing the upgrade.

[1] Download information- http://source.sakaiproject.org/release/20.4/

[2] 20.4 Fixes by Tool – https://confluence.sakaiproject.org/display/DOC/20.4+Fixes+by+tool[3] Upgrade information – https://confluence.sakaiproject.org/display/DOC/Sakai+20+Upgrade+Information

Happy Friday!

Sakai 21.1 Released

This is from the sakai-dev list – Wilma Hodges

Dear Community,

Congratulations to our amazing worldwide community!

I’m pleased to announce that Sakai 21.1 is now available for downloading!

Sakai 21.1 has 171 improvements including

  • 24 fixes in Assignments

  • 7 fixes in Gradebook

  • 17 fixes in Lessons

  • 9 fixes in Rubrics

  • 41 fixes in Tests & Quizzes

Other areas improved include:

  • Accessibility

  • Announcements

  • Basic LTI

  • Bullhorns

  • Calendar

  • CKEditor

  • Content Review

  • Dropbox

  • Feedback

  • Forums

  • Internationalization

  • Login

  • Mail Sender

  • Message Bundle Manager

  • Messages

  • Polls

  • Portal

  • Profile

  • Quartz Scheduler

  • Resources

  • Roster

  • Site Info

  • Statistics

  • Syllabus

There was 1 security issue fixed in 21.1 (details will be sent to the Sakai Security Announcements list).

Please also note the upgrade information page for important notes related to performing the upgrade.

Wilma Hodges, Ed.D.

Sakai PMC – Community Coordinator

Lecture Recording Setup (Basic)

When I record my lectures for my online courses I am using a DIY studio where I am trying to mimic a high end studio the best I can and I have described the technologies that I have used in a blog post I reference below But my studio building hobby is more than most folks need for good lecture recordings so I decided to write down my advice for a less complex way to produce good quality lectures.

Good Quality Lectures

When I am not trying to be all fancy – I find that Camtasia is a great tool that can get the work done.

https://www.techsmith.com/video-editor.html

A good camera like the Logitech StreamCam is nice.

https://amzn.to/3wyhxne

Microphones are important.  The built-in Macintosh microphone is as good as any setup I have used. If you are Mac person – then use the Mac microphone – I don’t know for Windows – audio is very important so do some testing including recording a few words and then be silent for 10 seconds and a few words and some silence. Cheap microphones don’t do well with silence and have problems when transitioning from silence to sound.

Then use two ring lights offset is good for lighting – like these:

https://amzn.to/3zvz4Os

Ring lights are always shown to be set up with the camera in the center – but if you have classes then you have two ring reflections that look really bad. Just get two lights and put them at 45 degree angles and you have nice lighting and no circles in your glasses.

The Fancy Version of Lecture Recording

How I Record My MOOC Lectures (Updated)

 

Lemons Build 2006 Mustang Details

After three years of running the Sakai Racing Team with Generation 1 (1995-1999) Neons, we have reached the point where parts are getting harder and harder to come by so we are starting work on building up a a 2006 Mustang for our net season 24 Hours of Lemons campaign.  This blog post will keep track of things like purchased parts in case we want to build another one.

Notes

  • When you remove the seats take off the wiring harnesses and seat belt hardware including the passenger seat sensor if you want to avoid airbag lights.  You can just keep this if you don’t want to install it.  Just don’t throw it away.

Parts

  • October 2021 – Passenger Seat Bracket for MOMO / NRG / Sparco / Recaro / Bride / OMP – Ford Mustang (2005+) – Part # SB039PA  – https://amzn.to/3BwYVoY
  • June 2021 – Seat Slider – Sparco Slider Seat Track Set – Amazon – https://amzn.to/3coCSar
  • June 2021 – Driver Seat Bracket for MOMO / NRG / Sparco / Recaro / Bride / OMP – Ford Mustang (2005+) – Part #SB039DR – Amazon – https://amzn.to/2SdT73k
  • June 2021 – OMP HC/733E Seat Mount Bracket – Amazon – https://amzn.to/3vXhWiL
  • June 2021 – Ford Side Airbag Light Eliminator Plug Plastic Mustang 2005-2007
    Ford: XF2Z-14B022-AA – https://www.cjponyparts.com/ford-side-airbag-light-eliminator-plug-plastic-mustang-2005-2007/p/HW4709/
  • June 2021 – 1 HA/781E TRS-E XL USD$ 409.00 Color:Black, Size:XL – https://www.ompamerica.com/
  • May 2021 – 3-INCH SFI 16.1 CAM-LOCK RACE HARNESS – CUSTOMIZE – SKU: BUILD A BELT 3IN-CAM-CUSTOM-30298-B-18-H-W-06-BW-B – https://www.racerdirect.net/

Sakai 21.0 Released!

This is from our wonderful community coordinator – Wilma Hodges.  I just pasted it into a blog post for the record.

I’m pleased to announce that Sakai 21.0 is now available!

Many thanks to everyone involved with the release! It wouldn’t happen without all of you!

Our community wiki has both functional [1] and technical [2] release notes.

Download instructions are also available [3].

New in Sakai 21

  • Dark Mode
  • Dashboard – New tool!
  • Lessons improvements, such as revamped Add/Edit Dialog, new Add Layout template options, improved Reorder and Date Release Indicators, and more color and formatting options for headings, buttons, and borders.
  • LTI Advantage Improvements, including autoconfigure option and new LTI Assignment Type in the Assignments tool.
  • Gradebook enhancements for exporting category averages and messaging students from the gradebook
  • Rubrics are now searchable and can contain weighted criteria
    And more! [4]

[1] Functional release notes – https://confluence.sakaiproject.org/display/DOC/Sakai+21+Release+Notes

[2] Technical Release notes – https://confluence.sakaiproject.org/display/DOC/Sakai+21+Technical+Release+Notes

[3] Download instructions – http://source.sakaiproject.org/release/21.0/

[4] Complete Feature Summary – https://confluence.sakaiproject.org/display/DOC/Sakai+21.0+Complete+Feature+Summary

Wilma Hodges, Ed.D.
Sakai PMC – Community Coordinator
wilma.hodges@apereo.org

Open Is As Open Does

The recent announcement of the OpenLMS making its extensions to Moodle available as Open Source generated a bit of a dust up in the EdTech journalism space.  Here are several great articles:

  • Phil Hill wrote a piece titled “Moodle’s Dispute with LTG and its Growing Suite of Former Moodle Partners” that focused on the relationship between Moodle and OpenLMS.  OpenLMS keeps buying Moodle commercial partners as soon as they get a decent number of customers and switching the customers from the Moodle community edition to the OpenLMS edition.  It highlights the difficulty of making a 100% open LMS sustainable (i.e. getting sufficient funding to support a team of 50 or so).
  • Michael Feldstein wrote a piece titled, “Moodle’s Sanctimony on Openness is Moot“.  It is a fun read as it wanders around the well-known fact that virtually all open source products are available from commercial vendors and those vendors add a bit of glue so the products run cheaply and reliably at scale in hosting environments like Amazon.  Michael takes the somewhat narrow position that if a little bit of code is not open then the whole “claim to be open” kind of falls apart (greatly simplified – ed.) – if there is a single crack in the “perfect open” – then there is no point in loudly arguing subtle differences.

Please read these before reading my post – as my post is very much additive to the conversation and I will not repeat what both of the above posts have accurately and elegantly said about the situation.

Also before reading further, understand that this is not me grinding an axe on my enemies or competitors.  I am friends with Martin Dougiamas (Moodle),  Phil Miller (Open LMS),  a number of Canvas employees like Karl and Melissa as well as friends of Phil Hill and Michael Feldstein.   I cherish those friendships so this post is not to bash anyone – it is just looking at the current situation through my personal lens.

Open is as Open Does

I think that the major flaw in both of the above pieces is that they look at the situation through very literal non-gray-area lenses like.  “What is the license?” or “Does Martin have a legal right to be upset about OpenLMS?” or “Where is that hidden code?” or “Is there a moral right or wrong when something is claimed to be open?”  These excellent “in depth” pieces are flawed because they are “in depth” and as such miss the bigger picture and don’t explore the all important gray area that matters most.

The bigger picture is that the purpose of “open” is to empower others to fully participate in a software ecosystem and contribute to its directions without requiring a commercial relationship.

I will look at the major “LMS” projects that will provide a Git repository with an OSI-certified license – that is the pure definition of “de-jure open”.    But that is only a small part of the “de-facto” definition of open.  I see a number of factors that are very important to non-paying adopters where I will grade each LMS in the grade book below:

  • How many non-paying servers have been installed?  This is where the rubber meets the road and the analysis could almost stop here.  Ultimately the other factors in this grade book lead to better or worse adoptability and that drives installation count.
  • Is there a structured community where non-paying participants have equal standing as paying participants.  For example, are there conferences?  Are these “paying customers only” or are they open?  Are the conferences about building community or herding customers into rooms with free drinks and snacks once per year.  Are there active developer lists, IRC’s or Slacks where folks can and do participate?
  • Is there a real way for non-paying adopters to get support for their product?  If they install a new version and see a weird message during database conversion, where do they go? Who will respond?  If they find a bug, can they submit a patch?  If they submit a patch will it ever make it into the core source code and a released version of the product?
  • Can non-paying adopters be part of the core team of developers?  Can they be part of the core decision making process?  Do they have access to the leadership team and can they make their case for a feature to the actual person who will build the feature?

All of these are variations on an organizational value of “welcoming, celebrating, and appreciating non-paying adopters”.

Since I am a teacher, I gave four LMS’s with some aspect of “open” in their nature grades in these categories.

The Grade Book

Non-paying participants
LMS License Server Count Community Support Leadership Access
Moodle GPLv3 A+++ (> 100K) A A+ B-
Sakai Apacheish B (> 100) A A A++
Canvas AGPL D (Some) D F F
OpenLMS GPLv3 0 / TBD C / TBD TBD TBD

Comments on the grades

  • Moodle is the best / “most de-facto open” in this analysis.  They “do open” better than anyone else.  They are three orders of magnitude better than all of the others combined in the most important metric of non-paying servers installed.  Their community is wonderful – their idea of teacher-centered MOOTS is *brilliant*.  If you have a problem with Moodle you post to the list and will likely get an answer from almost any time zone in the world and in almost any language.  #Jealous The only knock I have on Moodle is the insular nature of their core team.  They are treated as celebrities at Moodle events but then run back and hide in the walled garden of HQ and make all the decisions.  Fixing a bug or making a small contribution to Moodle requires you to become friends with folks on the inside and leave your offering at the door and hope someone comes out and picks it up.
  • Sakai (my entry in the list) is much smaller than Moodle in terms of servers installed.  Sakai’s events and other community activities are an even split between teacher and developer focus.  We have many examples of large and small contributions from relatively unknown non-paying community members.  If the code is good and follows the style guide and they fill out the contribution agreement – it goes in.  It is sometimes a little challenging to know the right way to do something in a nearly 20 year old code base with 1.4M lines of code. Sakai has a 1.5 hour online core meeting *every week* with open access that anyone can attend where our most core, senior, and talented engineers, UI experts, and accessibility experts will review github PR requests from anyone on the planet and help the person re-work their changes so we can get them in.  Sakai’s strongest column is “Leadership Access” – you can monitor core meetings by just joining online – Sakai has over eight meetings per week that are open to the public.  These are the only meetings where things get done and decisions get made.  “Sakai is 100% open – all the way to the core”.
  • Everything about Canvas derives from the AGPL license and “Open Core” approach.  I won’t nick them on Open Core because frankly the core is enough of a decent LMS for a smaller organization to use that the non-core bits are not all that important.  And the bits to host at scale are not very reusable anyways.  Canvas does not encourage non-paying adopters or small commercial companies hosting Canvas – but it happens.   I have no idea the extent or the number of schools and companies that run the free Canvas.  I think this is more prevalent in places like China and India where (a) Canvas does not have much presence, and (b) the countries have a wealth of technical talent and can handle a Ruby-On-Rails application without asking a developer list for help.  Canvas does a good job of maintaining the instructions on how to install the open version but does not really welcome non-paying adopters into “the family” with open arms.  Also the core Canvas staff (who are awesome by the way) are super insulated from any customers (paying or otherwise).   Their “vote on new features” web site is a laughingstock – but consistent with a core value that non-paying adopters are only useful as a marketing gimmick to assure the infinitely more important paying customers.  But kudos to Canvas – their minimalist approach to openness has made a decent LMS available to countries and communities that can make use of the product.
  • And now we come to OpenLMS.  To be fair, OpenLMS has only been open source for a month.  Almost all their grades are TBD.  There is a nascent community at https://community.openlms.net/ – there is an Open LMS Open Roadmap: What To Expect blog post you should read.  That is all we get.

Conclusion

Since OpenLMS is only a few weeks into their Open Source experiment, OpenLMS / LTG is mostly “potential” and they will be making some decisions along the way.   They can look at this blog post (and its grades) as one possible roadmap to “open greatness”.  I think back to the “score card” that Ray Henderson did at a few Blackboard meetings – OpenLMS could use my grades and categories as their starting point and goals.

OpenLMS could track and regularly publish how many non-paying adopters use their product.  They could have a series of online forums and Slack channels staffed by OpenLMS staff to help those not paying any money for the product with online support.  They could even open source some of their “scaling magic”.  They could have their core design and developer meetings open to the public.  They could be the most open open source LMS ever!

But… (there is always a but)  LTG is a public for-profit UK company.  Companies are not known for their largess.  Phil Miller is a great friend and really cares about his paying customers.  He loves to build clever software to keep his development and hosting costs low (some of the best in the industry by the way) and pass some of the savings on to his customers.  And it is why LTG loves (and should love) Phil Miller.  I have learned a lot about how to build inexpensive scalable systems (like Tsugi) from having beers with Phil and his teams over the years.

My opinion is that LTG/OpenLMS will evolve to follow the Canvas path – in essence the high level goal is compete with Canvas head-to-head.  Release it open source – don’t worry too much if no-one bites – and if there are some markets that OpenLMS is not interested – some tiny companies can use OpenLMS and sell services to a few local schools.  But with Moodle available and a known commodity it would be hard to find that much extra value in  OpenLMS.  If I were a small school and interested in OpenLMS after I downloaded it and ran it on my desktop – I would probably just pay LTG for hosting and let them deal with all the complexity.  Oh wait,  LTG just acquired a new paying customer who was attracted to the free version as bait.

Only time will tell.  But at least this blog post will be available in the Internet Archive and we can all come back and see if my predictions were accurate :)

P.S. Oh yeah – in EdTech I far prefer to deal with non-US companies and for those companies to be public – not private.   I am quite unhappy about the LMS market trend towards large privately held US companies.  So I like the fact that LTG is UK and public.

Should Sakai Evolve Towards MySQL 8.0 or Maria DB 10?

With the recent release of Sakai 21.0 (Thanks to all involved). We need to do some thinking about our direction forward w.r.t. how we approach our MySQL support in the Sakai community.

We have been supporting MySQL and Oracle since 2002 (19 years). In the old days, MySQL was the 100% open alternative for open source applications. But then in 2010, Oracle gained control of MySQL by buying Sun Micro-systems. I wrote a pretty fierce blog post about the Oracle purchase of Sun and my concerns for the future. At that point in time MariaDB was forked from the last non-Oracle version of MySQL. Some in the Sakai Community experimented with MariaDB from time to time – in the early 2010’s if things went wonky we ran back to MySQL.

Oracle has released MySQL 5.6 and 5.7 and MariaDB kept pace – even to the point of binary interoperability. We along with the rest of the open source world found this equivalence comforting and felt it was OK to just keep using MySQL and let the “other folks” use MariaDB.

But support for MySQL 5.6 just expired (February 2021) and support for MySQL 5.7 expires in October 2023.

The version of MySQL after 5.7 is MySQL 8.0. This is the first version that is heavily influenced by Oracle and it feels to me like MySQL 8 and MariaDB may begin to diverge. Hibernate (as of 5.3) supports the org.hibernate.dialect.MariaDBDialect separately from its MySQL dialects. Hibernate also supports the org.hibernate.dialect.MySQL8Dialect.

The reasonably high likelyhood is that Sakai will be able to simultaneously support the various versions of MySQL and MariaDB without too much effort. About 25% of the SQL in Sakai is hand-constructed and 75% of the SQL is constructed by Hibernate. Our hand-constructed SQL does not make use of intricate features of MySQL so it is likely to keep working in MySQL 8 and MariaDB 10. And you can pick your Hibernate dialect and support all the databases.

So the question is less about *not* supporting MySQL 8 that it is about deciding what our “preferred” / “first choice” database will be. For this preferred MySQL compatible database we can encourage developers to test using the preferred database and then run our nightly servers using that preferred database.

For example for Sakai 21 on our nightly page, we have eight MySQL 5.7 servers and one Oracle server.

For Sakai 22, we could keep things the way they are or switch to any the following:

* Run eight MariaDB 10 servers, 1 MySQL 5.7 server, and 1 Oracle Server
* Run eight MySQL 5.7 servers, 1 MariaDB server, 1 MySQL 8 server, and 1 Oracle Server
* Run eight MariaDB 10 servers, 1 MySQL 5.7 server, 1 MySQL 8 server, and 1 Oracle Server
* Run eight MySQL 8 servers, 1 MariaDB server, 1 MySQL 5.7 server, and 1 Oracle Server

You get the picture. By leaning towards either MySQL 8 or MariaDB 10 – we are signaling our “preference” / “first choice”.

We of course would fix problems that emerged in any of those databases. We would not want to break Sakai and MySQL 8 just to do something cool in MariaDB 10 or vice versa. We need to write conservative SQL and accept fixes if some hand-written MySQL breaks 5.7, 8.0 or MariaDB 10.

The question is where the majority of us are going to go so we can stick together and protect each other’s flanks by using a common approach wherever practical.

We can delay this decision until we release Sakai 22 – but the decision is easier and safer now. We get a year to experiment with the different databases and then based on what we learn this year – we can make a more informed decision next year. Also by next year we might get some signals from the market and other open source projects as to where they will be going w.r.t. the “MySQL 8 – to be or not to be” question.

Once we chat a bit about this on the lists, I will do a survey to poll the community.

My Opinion

At a minimum, I do *not* want to go “all in” with MySQL 8 – I want to move toward MariaDB if a year of experiments reveals no issues.

I would like a trunk master running on MySQL 5.7, MySQL 8.0, MariaDB 10, and Oracle. I will run my Smoker process against all four every night, to probe for really bad regressions. If we could somehow make them have identical sites and data – I could compare them click for click with smoker.

Earle Nietzel’s Opinion (from email)

I do think it’s time to start more formally adding MariaDB to the mix, however I think our database setup on nightly should reflect that which is being used by our users meaning:

For example:
80% of Sakai installs use MySQL (where 20% of installs make up 10% Oracle and 10% MariaDB – hypothetical numbers). For the other DB variants that make up the minority lets simply have a version of master representing that DB.

So an infrastructure might look like this:

master – 1 mysql / 1 mariadb / 1 oracle
experimental master – 1 mysql
21.x – 1 mysql
20.x -1 mysql
19.x – 1 mysql

Also the SQL ratio according to tables is: ~300 total tables of which 75% are Hibernate managed and the other 25% represent SqlService and Spring jdbc template.

Another thing to be thinking about with all these instances of Sakai is whether or not we have the resources to test on them.

I am not in favor of expanding instances if we don’t have resources to even test on them. This is the reason I suggest having only master instances for all of the databases we support and other instances are the db that is used most by users of Sakai.

Matthew Jones’ Opinion (from email)

I don’t know if that accurately represents the community or not. I feel many use AWS Aurora, another fork that is not compatible with MySQL 8, and only goes as far as 5.7, similar to MariaDB. So do we need an Aurora instance too? Probably not as much as if we had MariaDB.

I like the idea of going MariaDB first as it’s the most open, 5.7 compatible and it’s also the easiest for localhost. Though MySQL 8 isn’t too big of a problem to continue supporting as long as we don’t do much MySQL 8 specific stuff, which we likely won’t do. If we’re running MariaDB, I don’t see any point to leaving a separate MySQL 5.7 around, especially since it’s going to be EOL in 2023.

I think the hard decision about supporting multiple databases is if we want to take advantage of any special features of a database, we can’t while still supporting the others. For instance MariaDB supports a number of storage engines that aren’t available on the other databases and features like system versioned tables, that could be “nice to have” if we were able to use them. We’re basically stuck forever at the lowest common denominator, which is going to be MySQL 5.7 in this case.