Daily Archives: July 4, 2010

Adding Data Loading to Our Learning Feature in Shindig

Now that we know and love the way of the asynchronous batch request in Shindig, it is time for us to add our own data retrieval for the course-related information and provide that information for Learning Gadget. If you look at the existing pattern in the “Social Hello World” gadget, you see the pattern where the gadget simply takes the data that is returned as part of each request and uses it. In my gadget, I want to add a bit of an abstraction layer and let the user use a set of accessor methods so the pattern is a little different.

You might want to grab a copy of the completed code for reference as we go forward:

The first thing we need to do is build a getInfo method and add it to our osapi.learning service. Our code gets a little larger and we need some more imports.

java/social-api/src/main/java/org/apache/shindig/social/opensocial/service/LearningHandler.java

package org.apache.shindig.social.opensocial.service;

import java.util.Map;
import java.util.HashMap;
import java.util.concurrent.Future;
import org.apache.shindig.auth.SecurityToken;
import org.apache.shindig.protocol.DataCollection;
import org.apache.shindig.protocol.Service;
import org.apache.shindig.protocol.Operation;
import org.apache.shindig.common.util.ImmediateFuture;

@Service(name = "learning")
public class LearningHandler {

  @Operation(httpMethods = "GET")
  public Future<DataCollection> getInfo(SocialRequestItem request) {
    SecurityToken token = request.getToken();
    System.out.println("Owner="+token.getOwnerId()+" viewer="+token.getViewerId());

    // This data *should* come from an SPI ...
    Map<String, Map<String, String>> results = new HashMap<String, Map<String, String>>();
    Map<String, String> data = new HashMap<String, String>();
    data.put("context_label","SI124");
    data.put("context_title","Network Thinking");
    results.put("info", data);

    DataCollection dc = new DataCollection(results);
    return ImmediateFuture.newInstance(dc);
  }

  @Operation(httpMethods = "GET")
  public void setOutcome(SocialRequestItem request) {
    System.out.println("Param = "+request.getParameter("outcome","default"));
    // Do something clever here like call an SPI ...
  }
}

We need to construct a DataCollection which is a map of maps. This gets turned into JSON or REST by magic code that is calling us. We name the top map info and put two fields into the second-level map. There is a bit of clunkiness for all of this but the JSON/REST symmetry is probably worth it.

Again, since we are in a hurry, we will just call the service directly at the moment of gadget startup:

   gadgets.window.setTitle('Social Hello World');
   osapi.learning.getInfo().execute(function(result) {
        if (result.error) {
            alert('Error on retrieval');
        } else {
            alert('Name '+result.info.context_label);
        }
    }) ;
    var hellos = new Array('Hello World', 'Hallo Welt', 'Ciao a tutti',

You see the asynchronous pattern and we simply pull the info object apart and directly look up the context_label that was placed in there by the service.

But this is not batch-friendly. So lets do this in a more batch friendly manner by making the following changes after undoing the above changes.

     function render(data) {
       alert('Render Label ' + data.learningData.info.context_label);
       var viewer = data.viewer;
       allPeople = data.viewerFriends.list;
    ...
     function initData() {
       var fields = ['id','age','name','gender','profileUrl','thumbnailUrl'];
       var batch = osapi.newBatch();
       batch.add('viewer', osapi.people.getViewer({sortBy:'name',fields:fields}));
       batch.add('viewerFriends', osapi.people.getViewerFriends({sortBy:'name',fields:fields}));
       batch.add('viewerData', osapi.appdata.get({keys:['count']}));
       batch.add('viewerFriendData', osapi.appdata.get({groupId:'@friends',keys:['count']}));
       batch.add('learningData', osapi.learning.getInfo());
       batch.execute(render);
     }

Now we have piggybacked the retrieval of our learning information along with all of the other OpenSocial requests that our gadget needs to do. So we make one request, get one response, and have access to the learning data along with the Open Social data when we are making our initial render of the widget.

Now if you like, you can stop now as you have seen how to retrieve data from the server and do so in concert with the rest of a gadget’s batch requests.

But for me, I wanted a little more abstraction and I wanted my gadget to be provisioned so that a tool could use my learning feature over and over whenever it liked and anywhere in the gadget code. So I make a few changes to my feature to make this possible.

First I put in an instance variable, a setter to store the info from getInfo and changes to my accessor methods getContextLabel and getContextTitle as follows:

learning/learning_client.js

gadgets.learning = (function() {

    var info = null;

    // Create and return our public functions
    return {
        ...
        setInfo : function(learninginfo) {
            info = learninginfo.info;
        },
        
        getContextLabel : function() {
            if ( info ) {
               return info.context_label;
            } else {
               return null;
            }
        },
    
        getContextName : function() {
            if ( info ) {
               return info.context_title;
            } else {
               return null;
            }
        },
    };

})(); 

I also then make the following changes to the “Social Hello World” gadget:

     function render(data) {
       gadgets.learning.setInfo(data.learningData);
       alert('Gadget Label ' +  gadgets.learning.getContextLabel());
       var viewer = data.viewer;
       allPeople = data.viewerFriends.list;
    ...
     function initData() {
       var fields = ['id','age','name','gender','profileUrl','thumbnailUrl'];
       var batch = osapi.newBatch();
       batch.add('viewer', osapi.people.getViewer({sortBy:'name',fields:fields}));
       batch.add('viewerFriends', osapi.people.getViewerFriends({sortBy:'name',fields:fields}));
       batch.add('viewerData', osapi.appdata.get({keys:['count']}));
       batch.add('viewerFriendData', osapi.appdata.get({groupId:'@friends',keys:['count']}));
       batch.add('learningData', osapi.learning.getInfo());
       batch.execute(render);
     }

I still add the osapi.learning.getInfo to the batched request, but instead of using the returned info data directly, I call setInfo to pass it into my learning feature to provision it and then call the getContextLabel accessor method.

This has the advantage that now any accessor method for my learning feature can be called anywhere in the gadget including much later in the processing since the gadget is fully provisioned.

If you look at the full source code, you will see another provisioning pattern called loadInfo I included it for completeness but I only think it would be useful if a gadget was not going to retrieve any other data except for learning data from the server at startup. A normal gadget will likely need plenty of OpenSocial data from several services so the batch pattern will be the right pattern.

So this brings us to the conclusion of this little tutorial series on how to add a feature and service to Shindig. I have tried to keep the sample code and length of the examples to the absolute minimum to give you a skeleton to hang your own code on.

I also have not explored the System Program Interface (SPI) pattern at all here. If I were to develop the LearningHandler to be real code, I would immediately build a Learning SPI interface to make the Handler be reusable code with a number of different LMS systems.

So, if you have gotten this far, thanks for taking the time to read all of this. If you are a Shindig wizard and I missed something obvious – please drop me a note and tell me where I missed the boat. I am truly a Shindig beginner having only downloaded the code two weeks ago – so the patterns might have been lost on me.

I do think that it would be a good idea in Shindig to make these kinds of extensions possible without code hacking. Perhaps this outline shows a pattern where we can use Guice to find and register both client features as well as serve-side elements. But that will be for another time. I need to prepare for the July 4 barbecue today.

— End of Shindig Post Series —

Batch / Asynchronous Loading of Data in Shindig

In this post we won’t actually write any new code – we will look at the “Social Hello World” gadget and get an understanding of the asynchronous data loading pattern.

The first thing to understand and accept is that all requests are asynchronous. You could so do something evil with setTimer() in JavaScript to fake synchronous requests – but if you did that, my guess is that you would be chided as not knowing the “way of the Gadget”.

The problem is once you go asynchronous, you need to delay real work of making markup and making UI until the message comes back much later. And once you accept the fact that UI change is effectively “event driven”, you really want to batch up all your requests into a single request, send in one large multi-call request and then wait once for all of it and then when it all comes back, put up the UI.

Here is the documentation for osapi.BatchRequest – study it – it is your friend as a gadget writer.

Up to now, we have been doing our hacks to the “Social Hello World” gadget right at the moment of start up. For example in all of the screenshots you will notice that the gadget UI is not present behind the alert box when our alert boxes come up.

This is because the gadget has not started yet – we jumped in right after the title was set and sneaked in our alert boxes that have the effect of (a) showing us if our stuff is working and (b) pausing the code before the gadget has a chance to retrieve its data and generate its markup.

So lets look through the “Social Hello World” gadget. (preferably a clean version without our hacks).

vi ./target/work/webapp/samplecontainer/examples/SocialHelloWorld.xml

If we look down for the following code:

     function initData() {
       var fields = ['id','age','name','gender','profileUrl','thumbnailUrl'];
       var batch = osapi.newBatch();
       batch.add('viewer', osapi.people.getViewer({sortBy:'name',fields:fields}));
       batch.add('viewerFriends', osapi.people.getViewerFriends({sortBy:'name',fields:fields}));
       batch.add('viewerData', osapi.appdata.get({keys:['count']}));
       batch.add('viewerFriendData', osapi.appdata.get({groupId:'@friends',keys:['count']}));
       batch.execute(render);
     }

     gadgets.util.registerOnLoadHandler(initData);

This is making a nice batch call and adding a number of service requests to the call which includes all of the data needed to build the initial UI. When it calls batch.execute, it is requesting that the server (in one request) make all the service calls in the order specified, take all the return data, and send it back to us as a single response and when that response is complete call the method render.

If we look at render we see that it takes a response object and starts pulling it apart, setting the local data needed by the Gadget and then later making the UI markup and later putting it in an empty div so the UI appears to the user.

     function render(data) {
       var viewer = data.viewer;
       allPeople = data.viewerFriends.list;

       var viewerData = data.viewerData;
       viewerCount = getCount(viewerData[viewer.id]);
       …
       html += '<div class="person">';
       html += '<div class="bubble c' + count % numberOfStyles + '">' 
         + hellos[count % hellos.length];
       html += '<div class="name">' + allPeople[i].name.formatted 
         + ' (' + count + ') ' + allPeople[i].gender;
       html += '</div></div>';
         …
       document.getElementById('helloworlds').innerHTML = html;
       gadgets.window.adjustHeight();

It pulls out the data (never checking the data.error status), builds some HTML from the data, and puts it in a div and adjusts the height of the div, and voila! there is a user interface.

The batch pattern makes it so we only have to wait for one request and then we do all our work when we receive the “event” that indicates that our data is back from the server and ready to process. The code in the sample gadget is a bit light on error handling but that should not be too hard to imagine.

So now that we understand the batch/asynchronous/render-on-event pattern, we can do some data retrieval of our own in the next post.

Next post in series

Sending Data to a Server-Side Service with Shindig

We will break the talking to the server bit into two pieces. First we will send / set some data on the server and then in the next post we will retrieve data form the server.

The thing that we need to understand is that server requests are asynchronous. As much as we RPC loving server dudes want to make synchronous calls – resist it. Accept and embrace asynchronous calls – in Ajax it is the only way. And as we will see in the next post, batched Ajax is the only way. Since widgets are tiny and they will be on lots of screens, for performance reasons and decent user experience, batched asynchronous requests are essential.

I will use code snippets throughout the next few blog posts so you might want the entire files right away. Understand that these are the complete files and will have the complete solution in them that will be explained in the next few posts.

In this exercise, we add a new server-side service to Shindig to be accessed using the osapi feature. I may have jacked-in at the wrong place – but this will get you started.

java/social-api/src/main/java/org/apache/shindig/social/opensocial/service/LearningHandler.java

package org.apache.shindig.social.opensocial.service;

import org.apache.shindig.protocol.Service;
import org.apache.shindig.protocol.Operation;

@Service(name = "learning")
public class LearningHandler {

  @Operation(httpMethods = "GET")
  public void setOutcome(SocialRequestItem request) {
    System.out.println("Param = "+request.getParameter("outcome","default"));
    // Do something clever here like call and SPI...
  }
}

Now of course, we would do something more clever than just printing out out parameter – but that detail is up to the container. But this short bit of code is enough to see the round-trip to the server.

Then modify this file:

java/social-api/src/main/java/org/apache/shindig/social/core/config/SocialApiGuiceModule.java


import org.apache.shindig.social.opensocial.service.PersonHandler;
import org.apache.shindig.social.opensocial.service.LearningHandler;

   protected Set<Class<?>> getHandlers() {
     return ImmutableSet.<Class<?>>of(ActivityHandler.class, AppDataHandler.class,
       PersonHandler.class, MessageHandler.class, LearningHandler.class);
   }

This makes sure our service and methods are included in osapi as osapi.learning.setOutcome. And yes, it would be nice if there were a way of doing this without jacking in at a code level. Perhaps there is such a way that I missed or perhaps it is simply ‘yet to be invented’.

Good Comment from Michael Young Instead of modifying SocialApiGuiceModule you can extend it (ie LearningGuiceModule) and replace this module in web.xml.

Just because we are in a hurry, we will compile this and see if our server call works before we alter our learning feature. So compile and start Jetty:

mvn
mvn -Prun

And navigate to http://localhost:8080/samplecontainer/samplecontainer.html

You should see the “Social Hello World” gadget. Now lets edit this file:

vi ./target/work/webapp/samplecontainer/examples/SocialHelloWorld.xml

And add these lines:

   gadgets.window.setTitle('Social Hello World');
   osapi.learning.setOutcome({'outcome' : '123456'}).execute(function (result) {
        if (result.error) {
            alert('Error, unable to send outcome to server.');
        }
    } ) ;
     var hellos = new Array('Hello World', 'Hallo Welt', 'Ciao a tutti',
...

Actually you will note that to do this we do not need the learning feature because we have fully provisioned the server-side learning service into the osapi helper. When the gadget starts up, osapi pulls down all its services from the server and registers them. This is independent of gadget registration which the Require accomplishes.

When you press refresh (or whatever you need to to force a full reload) on the container and watch the log on the server, you will see a cute litle line scroll by in your server log:

Param = 123456

Very simple – but very cool.

Now lets alter our learning feature to call the service on our behalf in the setOutcome method. We will give the user the option to provide a handler or let the learning feature do the handling.

We edit the setOutcome method in learning_client.js from the last post as follows:

        setOutcome : function(data, handler) {
            if ( handler === 'silent' ) handler = (function (result) { } );
            if ( handler === undefined ) handler = (function (result) {
                if (result.error) {
                    alert('Error, unable to send outcome to server.');
                }
            } ) ;
            osapi.learning.setOutcome({'outcome' : data}).execute(handler);
        },

It is pretty simple stuff, the user can give us the handler, or we provide a simple alert on error, or we can provide a completely silent handler at the user’s request.

We also need indicate that we want access to the osapi service:

taming.js

var tamings___ = tamings___ || [];
tamings___.push(function(imports) {
  ___.grantRead(gadgets.learning, 'getContextLabel');
  ___.grantRead(gadgets.learning, 'getContextName');
  ___.grantRead(gadgets.learning, 'setOutcome');
  caja___.whitelistFuncs([
    [osapi.learning, 'setOutcome']
  ]);
});

This makes sure that we have access to our service call when running through Caja. (I think I am saying this properly).

Once we have done this and recompiled Shindig, started Jetty and started the container, we make the following changes to the “Social Hello World” gadget. Now lets edit this file:

vi ./target/work/webapp/samplecontainer/examples/SocialHelloWorld.xml

And add two lines:

   <Require feature="osapi"></Require>
   <Require feature="learning"></Require>
   <Require feature="settitle"/></Require>
…

   gadgets.window.setTitle('Social Hello World');
   gadgets.learning.setOutcome('0.97');
     var hellos = new Array('Hello World', 'Hallo Welt', 'Ciao a tutti',
...

We are just using our learning gadget method to send the outcome to the server. By omitting the second parameter, the learning feature will give us a little alert if it has trouble sending data to the server.

Again, we press refresh and in the log we see:

Param = 0.97

So that completes our look at a simple call to the server to send some data. In the next post, we will get a little deeper into how to retrieve data from the server. The bit that gets complex is the requirement that things be done asynchronously and if at all possible with multiple batched requests in a single request.

So the code will initially look a little obtuse – at its core it is simple – but the asynchronous pattern takes a little getting used to. And since I only figured it out in the last 24 hours, I might have missed a bit in the pattern as well. Of course comments and improvements are welcome.

Next post in the series