Commit, Commit, Who Gets Commit?

There is a delightful debate on the dev list about grandfathering in folks with repo-wide commit into K1. Here is my thoughts – it certainly follows in the vein of my rant a few days back. But this post is more reasoned (i.e. less of a rant).
I am a person who ‘lost commit” to a lot of code that I care about, monitor, and work on regularly. I have not asked for commit nor to I intend to ask for commit for those areas in K1 – at least for now.
Given that Ian is very responsive now, and given that I do believe that changes to this most critical bit of our code base need more eyes and more thought than the rest of our code base – I think of not having commit as a *benefit* – not a problem. I wait 1-2 days usually – but as a result I always have a second set of eyes looking at my code and doing a bit of testing of my code and a smart person – looking over my shoulder.
Thinking back to how things worked when this code was spread across the repository and quite a few folks had broad commit – there are several problems that pop out – that are (for the moment) solved in K1:
– Many people got broad commit for some specific task and then we never took it back after the task was complete – often people agreed in the short term to not use their broad power without coordinating with whomever was the lead in the area – but over time they just decided that they knew enough about one or another area to just appint theselves as a committer and start fixing stuff on their own here and there. I remember a time when we would have a discussion about whether someone should have commit to authz – an usually the answer was “yes” – but now – so many people have commit on the whole repo that they get to make the decision about committing to authz them selves. And yes – we are lax about project leads – but frankly it is better to slow down and find a project lead on something than just declare open season with a bunch of repo-wide committers. SO the hypothesis that someone who had repo-wide commit should just get kernel-wide commit is a very bad one IMHO. Perhaps we should revoke a few folks repo-wide commit because their task is completed.
– Some people, (myself included) have commit everywhere because we are SVN administrators – but we never do anything other than what would be called “maintenance” unless we are really part of a project team – things like making branches, changing copyright, etc etc – are very different than fixing/improving code bits here and there. Those people with broad maintainer commit have been pretty good about knowing when not to use their commit. Again – I *technically* can trivially commit to Kernel – but have only done so to change the text of copyrights. Everything else – I go through the lead using the process the lead has asked folks to do.
– We have a number of different technical opinions about the overall direction of our code base and approach – we have competing architectural ideas and we have some pretty strong disagreements between some of our lead technical people. Far too often these conversations reach an “agree to disagree point” and when the conversation is between two committers with repo-wide powers – the two folks just start doing “both” things in the repo and trying to gain customers to “their way” of thinking over time – they get their particular approach into folks dependencies and then they can slowly win the day – frankly what happens is that we never get back to one solution – we just end up with two way of doing something for ever and lame compatibility between the two approaches over time. Our core architecture gets less and less clean as we have many approaches scattered across the SVN – each with a different level of maturity and different resources applied to them – and little cliques forming around approach X or approach Y – slowly splitting our thinking. Instead of working through the debate – folks with repo-wide commit just put the code in the repo and let us all figure it out. This is not a good approach to the most core bits of our code base.
I just think that it is a good idea for folks to have to justify their idea to a review team at a minimum before they just get to drop something into the kernel.
I guess the simplest formulation is this: Before K1 – if people disagreed with Ian on how something should happen – they could just say “piss-off” and commit the code that they wanted. After K1 – people who disagree with Ian do not have the alternative of just committing their opinions to the SVN and bypassing him.
So if you are advocating giving kernel-wide commit to all the folks with sakai-wide commit – what you are saying is that you want a continuation of anarchy in our approach to technical direction for the core code base – you are making sure we continue to be able to make major changes in technical direction with absolutely no review – if a person with svn wide commit feels strongly about some technical direction for Sakai in the morning – by evening – whether we know it or not, we are all stuck going in that direction.
I am sure that the folks who want this commit are the folks who have disagreed with Ian over the years – these folks are all smart folks and they have great ideas – and yes some chaotic organic improvement can be a good thing – but do you want chaotic improvement in the kernel??? Do we want the chaotic approach at this point in Sakai 2.x code base? Do you want to have alternative technical approaches explored in the code base and in your production instances? My feeling from the broad community (not the core 10 bright folks with great ideas) is that people want stability – not continuous change in our source tree.
This is where folks keep saying – “move the creative energy” to K2 – get involved – dig in – and fight things through with a group of people where we can try two alternatives as an experiment and then pick the best one without messing up production and a whole library of contrib developers – and a bunch of vendor branches.
The current K1 approaches insures that a technical disagreement will remain a disagreement until it has been discussed long enough and by enough people to make a community wide decision.
I am sure that folks will bring up the “I have a lot of bugs that need fixing” argument – for me that is not a good argument – bug fixing is *easily* done with patches – and a forced code review step is a great thing for bug fixing – and a great idea for the long term health of the code.
I know we want more unit tests – so perhaps the commit access to the unit test are should be opened up a bit – to folks who can make a long-term commitment to writing *and* maintaining all of the unit tests. This seems like a legitimate request – but getting commit in one area does not give one commit everywhere – folks can “ask for this commit” and when there seems to be long-term commitment to the code area in question – then commit can be granted.
Another excuse that folks want broad commit is “everything in Sakai’s current code base sucks – and I am here to find and fix all the suckiness wherever the suckiness is hiding – and given that I have a very small amount of time – I cannot afford to do anything other than committing the fixes to the sucky bits right to the code base”. This statement is wrong on its face – Sakai certainly has room for improvement in lots of places – but at the same time – what we have mostly works solidly and at scale. Those people who do not respect the value in the current code base – are *not* the people you want fixing it – because they don’t respect what is there – they don’t even bother to figure out what something does before “fixing it” – and then they will be off to the next sucky thing that pops into their mind and go “fix” that.
Everyone brings up “Apache” as the inspiration for this approach – actually this is somewhat un-Apache like. For simple projects like Pluto, generally when you are accepted as a committer – you get full commit to the whole source base. Common sense and community culture keeps you from doing stupid things in places you should not touch and as things evolve – perhaps you become the technical lead this time next year – you end up committing all over the place or even trigger a major refactor that you lead.
The reason that this works is that Apache has ways to resolve technical disagreements – strong disagreements are allowed in Apache and these disagreements are worked through in discussion to a resolution – these discussions are allowed to get heated and intense when necessary. If someone loses, they either get on board or (rarely) to fork the whole project (this happened in Pluto). When the person is so sure that they are right but lost the argument – they fork the whole code base – this way they are taking responsibility for *everything* – not just the little are they care about – this way they will learn the full impact of their decision before *anyone else* is affected. Some forks die – some forks live. My guess is that the most common case is that the person starts playing in the fork – realizes they took on a lot of responsibility and then comes grudgingly back into the fold – folks had choices – and with enough effort they could force their choice – and when their branch works – they are the new technical lead for a while – and everyone starts bitching at them :)
Our problem is that we never resolve the discussion and we never make a “fork” – the person who loses the argument – just uses their “svn-wide commit” to slide their alternative solution into the common code base – and all of our users, developers, and system admns are stuck with yet another confusing and inconsistent approach.
I have talked too long (as usual) – we need to slow things down in K1 – we need to stabilize the core bits of sour source base – the 2.X line needs to move toward “enterprise thinking”. Innovation and fixing the “perceived ills of Sakai” need to be done in K2.
Switching to K1 is an opportunity to bring a bit of discipline to ourselves that we are sadly lacking. Lets not throw that out immediately – lets see if we can make it work.
Interestingly this very argument is a place-holder for all those technical arguments – one group is advocating for a calm, conservative approach and the other group wants freedom to directly implement their ideas in the code base. As a developer – I like the idea of causing change through commit access – but as a community leader – trying to represent the broader stakeholder interests – I understand the need for limitations on my own actions – I understand that may I need some review before I go in and re-do some bit of kernel based on a brilliant idea I had in the shower this morning. And maybe the answer to my brilliant idea should be “no” – or perhaps the answer needs to be “next summer”.
P.S. If it gets to the point where Ian and his folk take more than about a week to apply my trivial patches – I will join the clamor to get commit for me. But for now with Ian responding super-fast – I am loving the new process.