Designing Collaborative CyberInfrastructure Scientific Analysis Software

I am writing some draft text for a grant having to do with Cyber Infrasturcture. This text will be massively edited before it goes into the final proposal – I am just putting it up here as draft and thoughts – comments welcome.

Design User-Centered Computational Tools
The goal of the technical team associated with this project will be to build simple tools that allow our users to collaboratively explore our data, annotate that data, form models and hypothesis, and exchange information with others working on the data. Our approach to this is to create a very agile technical team that can gather user requirements and develop software and get it into production and in the hands of the end-users in a matter of weeks, to keep the user’s deeply engaged in the design processes. The team will always take user input and evolve the software to meet the emerging user needs as the software is being used. In order to achieve this level of flexibility, the project will not use any particular “framework” or existing software package. While starting with a well-developed software as the core can bring a lot of capabilities to bear on the problem quickly – standardizing on foundational software early in the process also brings a “pre-baked” perspective or point of view which limits flexibility.
The initial software will be kept simple and focus on the most important use cases as seen by the users. By working together and co-evolving the software both the users and technical team develop a deep understanding appreciation of the real user requirements. Because the process focuses on the act of co-creation (rather than trying to bend existing software into usable shape) – users and developers are energized to look very deeply into the problem space and unearth fine detail about the use cases.
The project will make use of readily available compute and data resources including commercially available offerings for hosting as appropriate. We do not want to invest any project time or energy in building technical infrastructure. We will make sure that the technical approach requires very little training effort by doing our work in widely used languages such as PHP and Java and using MySql as the database. This way we can easily bring a variety of new developers onto the project with little or no training time.
Advancing an agile and user-centered approach to designing Cyber Infrastructure
The approach to design and development is a classic user-centered approach with technical staff participating in all of the brainstorming and idea generation phases of the projects. The design sessions need a diverse set of participants ranging from domain experts, to design experts, to management stakeholders, and technical developers. One of the ground rules is that technical developers are generally not allowed to “veto” design ideas based on how difficult the ideas appear on first glance. The goal is to capture the user’s needs in terms of the kind of functionality needed in the collaborative data analysis tools.
Often in a project like this, the technical team worries about performance, scalability, or maintainability far too early in the design processes. Because our project will scale slowly over time, we will be able to address issues such as performance and scalability in later versions of the software. And often once domain expert users have a chance to work with a feature they will come up with a new and often simpler (and more technically feasible) design. The key is to avoid saying “no” to an idea in the design phase – but instead to try to make it work as the users conceive of it and explore and adjust the design as experience is gained.
The technical team needs to be very careful not to “fall in love” with any single technical approach. It is entirely possible that we will end up starting over or switching directions several times through the life of the project. As such, each of the software components should be developed and deployed quickly with a minimum of effort and energy so that the technical staff will be willing participants in any new directions.
We expect to produce two general categories of software in this project: (1) domain specific tools to search and work with learning science data and (2) a set of collaborative research widgets (rating, annotation, messaging, etc.) that are not particularly about learning data. We will use a simple JavaScript based approach (see and for ideas) to make all the capabilities available to our users. By “mashing up” functionality, we allow each bit of functionality to be written by a different technical team, in a different programming language, and hosted independently. The approach will initially focus on building of the domain specific-tools and as we see general patterns emerge, we will re-factor functionality into general-purpose tools. The following mockup and whiteboard photo are taken from a project in the University of Michigan Medical School that is using the same development methodology to develop software to support introducing first-year medical students to clinical experiences:

You can see the “background” is a collaborative document developed specifically for one of seven course activities, dealing with all of the workflow and stakeholders in this particular activity. At the bottom of the screen is a Facebook-like toolbar that contains the common/reusable elements that is layered on top of the background document using embedded JavaScript. You will note that during the design session we don’t even represent the reusable widgets in the design because we assume that they will always “be there if needed”